“An intervention program to promote health‐related physical fitness in nurses” by Yuan et al. (2009) is a quasi-experimental study that aimed to assess the impact of an exercise intervention on the nurses’ health-related physical fitness. Although quasi-experimental designs can be used to test causal hypotheses and share certain assumptions with experimental designs, quasi-experimental studies do not assume random assignment and control over extraneous variables, so they tend to have low internal validity (Trochim, 2006). Despite potential threats to internal validity, quasi-experimental studies are effective designs for investigating causal effects outside of controlled settings, so they often have good external validity (Campbell & Stanley, 1963).
The study by Yuan et al. (2009) appears to have good test validity because the Laborer Physical Fitness Test Method was used to assess the participants’ physical fitness, and it is a standardized tool developed by the Taiwan Institute of Occupational Safety and Health, so it is expected that the tool has good criterion and construct validity. Because the measurement method was consistent at pre-test and post-test data collection points, it is possible to suggest that the internal validity threat of instrumentation was addressed properly (Polit & Beck, 2008). Selection bias was also eliminated because the participants’ demographic data was compared between the control and intervention group, and no significant differences between the groups were found. The independent variable in this study was an exercise intervention, but the logistic regression model was used to adjust for four confounding variables (i.e. marriage, workload, work hours, and exercise habits), so confounding was also addressed.
Although the internal validity of this study is adequate given the quasi-experimental design, contamination of the results and lack of randomization are two significant limitations. This was a single-center study, so it is highly likely that the control and intervention groups were in contact and could imitate each other, and workplace policies and procedures specific to that center could have affected the results. Convenience sampling was used, so it is not possible to determine whether the sample is representative of the entire population because probability theory does not apply to non-probability sampling strategies. The lack of randomization could also explain low pre-test scores in the experimental group (Yuan et al., 2009), which indicates the possibility of a selection-regression threat to internal validity.
Both limitations could have been addressed by using an experimental study design because it is a feasible design in this case. An experimental study design should have been used to randomly select the institutions and participants to minimize the selection-regression threat. Consequently, this design would also further minimize confounding by making it possible to determine whether workplace differences are a significant confounding variable. External validity would also be improved because it would be possible to estimate whether the sample is representative, and it would also be possible to generalize the findings to multiple settings. These changes would not affect the measurement methodology used or the purpose of the study, so the study’s face validity and test validity would most likely remain unchanged.
Failing to consider the validity of the study is dangerous because researchers could make inaccurate causal inferences if they fail to address validity threats. For example, this was a single-center study, so the researchers cannot generalize their findings to different healthcare institutions. Therefore, the findings of this study should not be considered when developing exercise interventions focused on the nursing staff because the same intervention in a different setting may prove ineffective or even hazardous to the staff’s health.
References
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-experimental designs for research. Boston, MA: Houghton Mifflin.
Polit, D. F., & Beck, C. T. (2008). Nursing research: Generating and assessing evidence for nursing practice. Philadelphia, PA: Lippincott Williams & Wilkins.
Trochim, W. M. K. (2006). Quasi-experimental design. Retrieved from http://www.socialresearchmethods.net/kb/quasiexp.php
Yuan, S. C., Chou, M. C., Hwu, L. J., Chang, Y. O., Hsu, W. H., & Kuo, H. W. (2009). An intervention program to promote health‐related physical fitness in nurses. Journal of Clinical Nursing, 18(10), 1404-1411.
