Research Paper Critique
Research Paper Critique
This paper seeks to critique the case study: “Effects of Rater Goals on Rating Patterns: Evidence from an Experimental Field Study. “In this research, the researcher sought to examine and provide evidence of goal rating causality. The researcher used a sample of 104 undergraduate students in his study. He manipulated rater goals of graded group reports. The researcher concluded that pursuing harmony goals not only increased the mean rating but also decreased discriminability. The researcher also found out that pursuing fairness goals increased the mean rating while it decreased the discriminability if a group project has not been finished. However, if a group project has been finished; pursing fairness goals only increased the mean rating.
The first research problem the researcher sought to address was the influences that rater goals had on the rating patterns used. The researcher acknowledged that raters normally use more than one ratee. He indentified three main rater goals; harmony, fairness and motivation goals. The rating pattern normally comprises of two main rating features; mean and discriminability ratings. The second problem the researcher sought to address was the differential influences of the different rater goals. The last problem the researcher sought to address was if there is a causal relationship between performance rating and rater goals. These research problems are relevant to organizational psychology.
The researcher formulated five research questions and hypotheses. There were two research questions and hypotheses under the harmony and fairness goals respectively and there was one research questions hypothesis under the motivation goal. Considering the research problems, the research questions and hypothesis were appropriate to adequately cover the matters under consideration. However, five research questions and hypothesis are too many to be covered in depth in a single case study. The researcher should have narrowed down to utmost three research questions and hypothesis.
The study had two depend variables; mean rating and discriminability rating. Mean rating was taken to be the average rating obtained by all group members while the discriminability rating was taken to be the standard deviation of the scores obtained by all group members. The independent variables were the different rater goals. These variables were appropriate for this study.
The researcher used a sample of 104 undergraduate students however data from one participant was excluded from the analysis because the participant did not complete all the evaluations. They were placed in 14 groups of between seven and eight members.
In research, privacy refers to the research participants directly disclosing information to the researcher while confidentiality refers to the extent to which a researcher safeguards information that has been disclosed by participants. The researcher did not consider privacy and confidentiality. The participants were required to fill evaluation forms in a large lecture hall in the presence of all other participants. In the evaluation forms, the names of all group participants were listed and the participants were required to rank the contribution of each member on a scale of zero to seven. The researcher did not make any attempts to conceal the identity of the participants or the scores given to them by other group members. To maintain privacy the researcher should have ensured that the participants complete the peer evaluation separately. The researcher should have assured the participants that the private information will not be disclosed to anyone and will only be generalized in the case study.
The researcher used a true experimental design for this study. Each of the 14 groups was required to submit a project proposal which was to be approved by their instructor, make oral presentation and later submit a written report. The oral presentations and written reports were to be graded. The participants were then required to complete peer evaluations; at mid semester after project proposal approval and at the end of the semester after submitting the written report. The students were informed that the peer evaluations will be used to adjust their grades. Peer evaluation was an appropriate means of collecting data because the students were familiar with it.
The researcher used regression analysis to identify if there were any relationship between performance rating and rater goals. The rater goals were the independent variables and the performance ratings were the dependent variables. The researcher used ANOVA for the mid semester and end semester data to test for interaction effect effects between the factors. At a confidence level of 99 percent, the researcher concluded that pursuing fairness goals increased the mean rating while it decreased the discriminability if a group project has not been finished. However, if a group project has been finished; pursing fairness goals only increased the mean rating.
The research was exempt because of several reasons. First, the research was conducted in an educational setting; that is in a university. Secondly, the research involved the use of educational test; that is oral presentation and written reports which were graded and peer evaluations which were used to adjust the grades. Lastly, the participants were adults of majority age.
Gravetter, F. J., & Forzano, L.-A. B. (2011). Research Methods for the Behavioral Sciences (4 ed.). London: Cengage Learning.
Wong, K. F., & Kwong, J. Y. (2007, May 20). Effects of Rater Goals on Rating Patterns: Evidence From an Experimental Field Study. Retrieved July 4, 2012, from The University of Phoenix: http://web.ebscohost.com.ezproxy.apollolibrary.com/ehost/pdfviewer/pdfviewer?sid=f4a65bfe-6a2b-4e70-8e76-d05a64cb015a%40sessionmgr113&vid=2&hid=105
