Article Evaluation
Article Evaluation
A. Introduction
- Does investigator report on relevant literature?
- How many sources were used and how many were recent (i.e. within five years of publication date of the article)?
There was an apparent use of relevant literature in the study, which is evident in the linking of theoretical underpinnings of the study itself and those of the literatures being used. For example, the study establishes the relationship between academic, first-semester academic achievement and psychological attitudes. The relationship in these variables constitutes understanding of resilience among first year college students, which was reinforced by the findings in the literatures used in the study. However, all of the resources used in the study are out dated except for one that is dated in 2007. The majority of the resources were published in the 90’s with a few nine of other resources dated from 2004 to 2007. In total, the study utilized 42 sources varying from peer-reviewed journals to book chapters and other publications.
2. Is there evidence of bias in the investigator's language?
- If yes, give examples.
There was no evidence of bias in the investigator’s language duly because of the use of measurable variables that encompasses quantifiable outcomes.
B. Hypothesis
How clear are the hypotheses?
The hypotheses presented in the study are not clear for one reason, and that is because they were not prominently stated in the study.
- What types of hypotheses are listed? – Scientific hypothesis was stated, but not listed. Since hypothesis includes null and alternative, the investigator chose to make an educated guess by generalizing assumption by stating that several factors influence first year college students’ success such as student characteristics, environmental and psychological factors.
- If no hypotheses are listed, then what information was given? – A thesis statement was given instead of hypothesis. The difference between hypothesis and thesis is that thesis states the problem that the study tends to resolve while hypothesis is probable guest to the given problem. Therefore, the investigator stated the problem directly instead of making educated assumptions.
- List all the independent variable(s) and dependent variable(s)
- Independent variables
- Race
- Motivation
- Dependent variables
- Resilience
- Self-efficacy
- Specify the measurement scale of independent and dependent variables (i.e. nominal scale, ordinal scale, or interval/ratio scale).
Dependent and independent variables were measured using a nominal scale employing the Likert scale specific to each variables. Academic motivation uses 7-point, self-efficacy inventory 8-point, university environment 7-point, and college resilience 5-point Likert scale respectively.
C. Sample
- Is the sample appropriate for the study? Discuss. (The sample chosen to participate in the study was an appropriate match to the population the study was examining.)
The study focuses on first year college students. Therefore, the use of samples from RU/VH university first year students is the most appropriate. The chosen sample to participate in the study matches the population that the study was examining.
2. Is the sample large enough? Discuss. (Give specific numbers, if available, and discuss the appropriateness of them.)
There is a problem with the sample size used in the study. Primarily, the sampled population was obtained from only one university and the selection process was not randomized. Determining the outcome of the given problem in the study encompasses a need for more robust sample size. Demographic factor as mentioned by the investigator as a potential variable influencing the outcome, the investigator could have taken the study into more than one location in order to establish a larger implication of the findings. Although 164 samples are big enough, only 29% of the targeted sample has completed the administered survey, which is likely to yield narrow results.
- Do the sample procedures result in possible biases in the sample?
If yes, give examples.
There is a possible marginal bias in the sample given that the size of the sample obtained is very limited (only 29% of the 164 student target). Furthermore, it was stated that the sample acquisition was not randomized, which means the investigator could have manipulated the outcome by pre-selecting samples that are assumed to yield responses fit to establish the thesis claims.
- Discuss the sampling procedure used – The sampling procedure used the study is the administration of surveys to samples in which the responses were measured using the Likert scale. Participants were recruited from a course that was designed to improve success rate of first-year student. URL links were provided to the sampled students where the survey was administered together with online studies about academic and psychological attitudes.
D. Methodology
- What type of research design is used? In case it is experimental design, name and describe the type of experimental design used in the article (e.g. pre-experimental, true-experimental, quasi-experimental, and factorial design)
The research design used in the study is correlation, which means that two or more factors were tested for apparent relations. In the article, the objective encompasses establishment of correlation between academic attitudes and psychological attitudes. Covariation is evident in which the alleged cause vary depending on the effect assumptions.
- Measurement(s)/instrument(s)
- What measurement(s) are used in the study? – The measurement used in the methodology is Linkert scale.
b. Is evidence of validity and reliability of the measurement(s) given? If yes,
- What type(s) of reliability was discussed? – There were no direct discussion of reliability in the article, but the methodology suggests average inter-item correlation. It is a subtype of reliability that determines internal consistency in which the correlation coefficient is being measured and finally taking the means of all the correlation coefficients. Since the study encompasses comparison test and the methodology entails determination of averaged responses. Therefore, even without an apparent statement, the study still shows employment of inter-item correlation.
- What type(s) instrument validity was discussed? - In terms of validity instrument, the study evidently has used construct validity, which ensures that the measure is covering the intended factors that need measurement. In the article, the variables were measured separately and specifically according to the need of study.
- Report and discuss any coefficients (i.e., reliability coefficient, validity coefficient) that are listed.
The preliminary analysis of the study involves computation of the alpha coefficient along with the means, standard deviation, and range. In addition, correlation coefficient was computed to determine the primary relation among all the scales. For the self-efficacy, it was found that student’s confidence was significantly related to resilience level as measured by (r=.68, p<.001) as academic engagement and (r=.67, p<.001) as social engagement. All the resulting computations are listed on separate tables to show results variation.
- Are there weaknesses in the measurement procedures or the measures themselves? If yes, give examples. – No apparent weakness found in the use of measurement or procedures because the computational models used were found to have constituted accurate results.
- What are strength and weaknesses in the overall implementation procedure used – The perceived strength in implementing the procedure is the effective use of methodology and quantitative measures that enables higher accuracy of collected data. On the other hand, the study also encompasses weakness in terms on the size of the sample due to the inability of the survey administration approach that yielded marginal completion of the questionnaire.
- When applicable, name and describe two threats to internal validity. – One, of the possible threat to internal validity is selection bias due to non-randomization of samples. Second is repeated testing, which may also result to bias since the questionnaires were administered online, there is a possibility of repetition on the part of the participants, which may render data inaccuracies.
- When applicable, name and describe two threats to external validity. – Two of the threats to external validity are the Rosenthal effect and reactivity, which impedes generalization of cause-consequence relationships to other investigations pertaining to similar area of study.
F. Results and Conclusions
1. Are results reported in clear, understandable terms? Explain your answer.
The results were delivered in understandable terms because of the clear interpretation of the gathered data and statement of correlation between them. For example, in explaining the relation between race and attitude towards university (r=−.17, p<.05), the report explains that white students appears to have higher mean score of 5.16 as compared to other students of color garnering a mean score of 4.82. This conclusion was justified and was clearly interpreted based on the results of the computation.
- Does the investigator relate his results to his hypothesis? Explain your answer.
The investigator was able to relate the results to his hypothesis by explaining the underlying influences that affect academic success and motivation among first year college students, which is highly important in establishing effective support programs for the students. The hypothesis of the study states that psychological and student attitude are factors attributed to student college freshmen retention rate. The investigator was able to relate the results by concluding that self-efficacy and resilience are detrimental in elevating motivation level of first year college students.
G. Overview
- What are the most serious deficiencies/limitations of this research?
The most serious deficiency found in the research is its tendency of the findings for limited generalizability due to the marginal size of the samples, which yields limited results. In addition, the research follows a quantitative model, which could have been made more compelling by listing null and alternative multiple hypotheses to test.
- In what way and to what extent are these deficiencies likely to affect the soundness of the research findings?
The limitation perceived in the ability of the findings to be generalized into a fundamental knowledge affects the soundness of the research on various levels. For one, a knowledge that cannot be established and as a theoretical foundation for further research means that the findings does not encompasses strong relevance on its intended area of purpose. In addition, the study is very typical of its kind and the findings are predictable, which demonstrates lack of complexity.
References
Reynolds, A. L., & Weigand, M. J. (2010). The relationships among academic attitudes, psychological attitudes, and the First-Semester academic achievement of First-Year College Students. Journal of Student Affairs Research and Practice, 47(2), 175–195. doi:10.2202/1949-6605.6004
