Hall, Ward, and Comer’s (1988) “Published Educational Research”
Abstract
In this educational research analysis, the topic is about the quality of published educational research articles. The authors’ main research question is about the quality of previously published articles based on the evaluations of 178 volunteer judges. The quantitative research methods consisted of integrated literature review, random selection of judges, and evaluation instruments. In the data analysis, chi square value of 11.28 (df = 6) suggested that the distribution of quality among the three types of evaluated journal articles were significant at alpha = .02, which suggested that primarily research and related professional articles were generally of higher quality than primarily non-research articles. Specifically, the MANOVA resulted to a non-significant F statistics, which corresponded to the Wilks’ lambda of 1.54; df = 2, 125), suggesting a lack of difference across journal categories on the set of rating subscales. Consequently, the results showed that the most frequently cited shortcomings of 23 (43%) out of the 54 sample research articles were in procedural or methodological area. In other words, the researchers reported that more than forty percent (40%) of the journal articles that were expertly evaluated contained serious or complete methodological flaws (i.e., in research methods and statistical techniques). The researchers thus suggested that many of the methodological shortcomings are associated to researchers’ poor methodological training. Since the authors employed expert judges, the conclusions arrived at were valid and reliable concerning the previous journal studies’ publishable or non-publishable qualities. The implications of the findings are that they are not generalizable in other population; thus, further research is needed.
1. Your first reaction to the article.
My first reaction to Bruce Hall, Annie Ward, and Connie Comer’s (1988) article entitled “Published Educational Research” is the manner in which it evaluated the quality of primarily research, primarily non-research, and related profession articles in educational research. It mentioned well the random selection of sample from a population of volunteer judges from the American Education Research Association (AERA). Hall and his colleagues asked for the voluntary participation of 178 judges from the said professional organization considering the nature of their research. The researchers likewise justifiably presented their results and discussions from an evaluation of 128 articles from 38 journals using various measures of central tendencies and other statistical techniques (e.g., Wilks’ lambda of 1.54; df = 2, 125), which suggested “a lack of difference across journal categories on the set of rating subscales” (Hall, Ward, & Comer, 1988, p. 186). Further, all judges who are from the US and elsewhere (e.g., Canada, Australia, Venezuela, Taiwan, etc.) were all expert judges in the field of educational research. Hence, even when the authors have not explicitly stated all the limits and weaknesses of their study, it still showed as I read thoroughly their paper.
2. How is this article related to your experience as a teacher?
The article is related to my experience as a teacher because I can use Hall and his co-authors’ findings in improving my teaching pedagogy. For example, I should rely only on quality peer-reviewed journal articles’ findings for me to be better informed in teaching my specialty subjects or courses. In other words, I should use the findings of studies that are of high quality standard when teaching or performing my duties in a public educational institution.
3. What is your analysis of the article?
In my analysis, Hall, Ward, and Comer’s (1988) article was well-researched, well-written and credible because they included a specific portion where they discussed the study’s scope, limitations, and delimitations as they made inferences and conclusions concerning their study results. Moreover, the researchers’ quantitative research methodology was of good quality itself since they patterned their study to a previous research that they conducted (that is, excluding Comer in place of Schramm) more than a decade ago. Consequently, as reported in their conclusions, more than forty percent (40%) of the journal articles that the volunteer judges expertly evaluated contained serious or complete methodological flaws; hence, the implication to train better future researchers about rigor in research methodologies.
4. What did you find interesting, useful, or suspicious?
In the lens of public education, Hall, Ward, and Comer’s (1988) article signaled the need to publish only credible journal articles with rigorous research methodologies. Articles, prior to their publication in journal articles, should meet the highest standard of quality by passing through expert peer reviews. Indeed, Hall and colleagues’ article was useful because they set the criteria for quality, publishable educational research papers. On the contrary, the article sounded suspicious because its authors mentioned to the judges that the articles they rated were already published articles; hence, the potential confirmation bias. Likewise, the authors only mentioned that any of the judges were not authors of the articles that they received and will critique, but did not divulged whether they might be former co-authors or peer reviewers of the articles under evaluation.
Reference
Hall, B., Ward, A., & Comer, C. (1988). Published Educational Research: An Empirical Study of Its Quality. Journal of Educational Research, 183-189.
