Article critique
The authors to the study did not cite any research question.What is its importance?
The study sought to predict rater elevation using the Five-factor model (FFM) of personality. Elevated ratings cause numerous problems for operational-performance management systems such as employee dissatisfaction with appraisal and reward systems and difficulties in justifying termination decisions. The researchers thus sought to elucidate on the problem of rater elevation by identifying individual variables that contribute to the behavior. What is/are the hypothesis/hypotheses?
The authors hypothesized that Agreeableness (A) scores were positively linked to rating level and conscientiousness (C) scores were positively associated with rating levels. They also predicted that persons high on A and low on C would have the most elevated ratings. MethodsDescribe the participants in the study.
The study participants were undergraduate students who undertook a course in human resource management as a requirement of their course. They were 111 in number. They all partook in 1 of 5 sections of an undergraduate class taught by the same instructor over a period of 2 years. Each class had 20-32 students. What is/are the independent variable(s)? Dependent variable(s)? If this is a correlational studyand there are no IVs/Dvs, what are the variables being studied?
The independent variables in this study were the big five personality factors that neuroticism, extraversion, openness, agreeableness, and conscientiousness. The dependent variable was the student’s average peer rating level.How did the authors measure the variables in question?
The authors measured the big five personality factors using the NEO Five Factor Inventory (NEO-FFI) which is a 60-item revised version of the NEO-personality inventory. It provides a brief but comprehensive measure of the five domains of personality. Peer ratings were made on other students during group exercises.
What was the procedure?
All the students completed the NEO-FFI during the first two weeks of the semester. The participants were required to prepare a written report in which they were to take a definite position on an issue on human resource management. The students were then to present their report to other group members. The group members were then to rate each other’s performance on 8 performance competencies that is behavioral flexibility, analytic thinking, leadership, decision making, personal impact, planning and organizing, oral and written communication. They made these ratings on a 7-point scale that ranged from 0 (poor) to 7 (Outstanding). Each student participated in 4-6 groups. The rating level of a student was measured as the student’s average rating of other students across the various exercises and dimensions. The written student products and the manner in which the students appraised their peers were evaluated by their professor. ResultsHow was the data analyzed?
Correlation coefficients were calculated for the Big Five personality factors. The hypothesized relationship between A and C was analyzed in two stages. The data on average peer ratings was analyzed via a hierarchical moderated regression analysis following which they were analyzed via an alternative method for analyzing priori ordinal interactions that was first proposed by Bobko (1986). The authors postulated that the outlying group mean was what would be responsible for the hypothesized interaction. They divided the sample into four clusters based on their A and C scores. A t-test was then used to compare the various groups because their means of the three remaining groups were found to be highly equivalent. This procedure was repeated with the difference scores obtained by deducting the mean professor’s rating from the mean peer rating of every rater.What did the authors find?
On the correlation of the Big Five factors, the authors found that the scores for A and C were independent. The scores for extraversion were positively correlated with the scores for openness (r=. 19) and conscientiousness (r=.40) and negatively correlated with the scores for neuroticism (r=-.36). All the means were not significantly different from those reported for college students in the NEO-FFI manual. The relationship between ratings of A and rating level was found to be positive as well as statistically significant (r=.33, p<.01). They also found a negative statistically significant association between C and the rating levels of the students. The average peer ratings on a student were unrelated to the professor ratings on the same. They also established that ratings by students high on A and low on C were more elevated than those made by other groups of students.Were the hypotheses supported?
The findings of the study supported the two hypothesis made by the authors that is, Agreeableness (A) scores were positively linked to rating level and conscientiousness (C) scores were positively associated with rating levels and that persons high on A and low on C would have the most elevated ratings.DiscussionWhat are the implications of the study?
The findings of the study imply can be applied in a number of ways in practice settings. Personality inventories or evaluations can be used to aid in the selection of supervisors in instances where performance in the appraisal process are considered to be essential functions of the position. Rater scores on the FFM may also be used to identify raters who need self-efficacy training to make them more proficient at rating. FFM measures dispositional constructs and can be combined with attitudinal constructs to diagnose rating elevation. What are the limitations of the study?
The generalizability of the findings of this study to other appraisal situations is not known because it used undergraduate students who acted as both raters and rates. Do the authors discuss any future directions? What are they?
The authors describe future directions for the study. The authors recommend that future research studies explore the boundary conditions that affect trait-behavior relationships in appraisal contexts. Their recommendation is informed by the conditional view of dispositional variables by Kane et al. (1995) which posits that the predictive power of traits is influenced by the specific context. They further recommend that future researches evaluate whether the predictability of elevation of ratings may be increased by measurements of more specific aspects of the elements that compose the Big Five factors. CritiqueWhat new questions does this research trigger?
The findings of the study trigger the questions of whether boundary conditions affect trait-behaviors relationships in appraisal contexts, whether the predictability of increased ratings can be improved by measurements of more concise elements of the Big Five Factors and whether personality interacts with different treatments to influence ratings. Is there a better way to answer the research question?
The methodology utilizes by the authors was appropriate for the purposes of the study.Was this research worthwhile (worth the time, money, effort)?
Rating elevations are endemic problems in most organizations. They portend serious consequences for instance, in cases where employees’ contests termination decisions in courts. Therefore, the efforts of the authors to this study to investigate the individual differences that contribute to the behavior are worthwhile. Does your reading of the article agree with Levy""?s use?
References
Bernadin, H. J., Cooke, D. K., & Villanova, P. (2000). Conscientiousness and agreeableness as predictors of rating leniency. Journal of Applied Psychology, 85(2), 232-234.
