Evaluation is the process of systematically collecting data pertaining to different aspects of a project for the purpose of judging effectiveness and making decisions about the project’s future (CDC, 2012). Especially for projects that utilize funding, evaluations take the form of formal research typically by external evaluators to establish both effectiveness, cost-effectiveness, and other attributes. On a smaller scale, an evaluation can be formative or summative. A formative evaluation focuses on the process of planning and implementing the project while a summative evaluation looks at the outcomes.
In the study by McNabney et al. (2009), the researchers employed a summative method to evaluate a series of educational activities after it was concluded. In addition, the assessments after each individual session qualify as formative evaluation as they were conducted while the program was ongoing. On the other hand, the study by Miake-Lye et al. (2011) is purely a formative evaluation employing qualitative data. The researchers considered the implementation strategies of the project, communications during meetings, feedback of stakeholders, adherence to the plan, barriers, facilitators, and data in the patients’ medical health records.
This writer employed both formative and summative evaluations to determine the impact of a mentorship program. Because the program was conducted to address nursing staff retention, the evaluation took into account retention rates as a direct outcome indicator. Indirect or secondary measures were recruitment and job satisfaction indicators. The evaluation focused on structures and processes, specifically leadership, staff education and training, and management support. Nursing staff engagement and overall feedback regarding their participation in change are also part of the formative evaluation.
Following reflection and obtaining information from the literature, this writer found more effective ways to evaluate a mentorship project. Escallier & Fullerton (2009) implemented a retention strategy in a school through workforce diversity and mentorship. Their evaluation focused on the utilization of the mentorship program. As such, the researchers elicited the experiences of the mentors and mentees with particular attention to barriers and facilitators. There was suboptimal use of the mentorship program because it was set in an online environment. Many of the mentees were not comfortable with this set-up. On the contrary, this writer’s project entailed mentorship via face-to-face interaction. In the future, this writer will include an evaluation of mentorship utilization although this will be conducted a few months following implementation to allow mentors and mentees sufficient experience under the program.
Further, this student also learned more robust methods of carrying out an evaluation. Clauson et al. (2010) evaluated a mentorship program in a hospital setting utilizing senior nurses aged 55 years and over as mentors by virtue of their knowledge and experiences. Surveys, interviews, and focus groups were employed for the evaluation. The researchers used a validated survey tool. The structure and purpose of the interviews and focus groups were also developed following a workshop. The use of a combination of quantitative and qualitative data is similar to this writer’s evaluation. In the future, however, this writer will make sure that instruments are validated based on literature, testing, or peer feedback.
References
Centers for Disease Control and Prevention (2012). Introduction to program evaluation for public health programs: A self-study guide. Retrieved from http://www.cdc.gov/EVAL/guide/introduction/index.htm
Clauson, M., Wejr, P., Frost, L., McRae, C., & Straight, H. (2010). Legacy mentors: Translating the wisdom of our senior nurses. Nurse Education in Practice, 11(2), 153-158. doi:10.1016/j.nepr.2010.10.001
Escallier, L.A., & Fullerton, J.T. (2009). Process and outcomes evaluation of retention strategies within a nursing workforce diversity project. Journal of Nursing Education, 48(9), 488-494. doi:10.3928/01484834-20090610-02.
McNabney, M.K., Willging, P.R., Fried, L.P., & Durso, S.C. (2009). The “Continuum of Care” for older adults: Design and evaluation of an educational series. Journal of the American Geriatrics Society, 57(6), 1088-1095. doi: 10.1111/j.1532- 5415.2009.02275.x.
Miake-Lye, I.M., Amulis, A., Saliba, D., Shekelle, P.G., Volkman, L.K., & Ganz, D.A. (2011). Formative evaluation of the telecare fall prevention project for older veterans. BMC Health Services Research, 11(119), 1-9. Retrieved from http://www.biomedcentral.com/1472-6963/11/119
