Critical Review of Program Evaluation Study: Family Life-Skills Program
A Critical Review of Program Evaluation Study: Family Life-Skill Program
Introduction
There is sufficient evidence suggesting that the Temporary Assistance to Needy Families (TANF) program in New Hampshire, USA is giving increased attention to women empowerment by introducing the Leadership for Employment, Achievement, and Purpose (LEAP) curriculum that involves a welfare-to-work process. The program evaluation authored by Wichroski et al. (2000) seeks to determine the impact of the program in relation to the program objectives and expected outcomes. On the other hand, this discussion will determine the adequacy of the evaluation by looking into the methodology and the areas of analysis highlighted in the evaluation. More importantly, this discussion will determine whether the program evaluation have demonstrated effective measures in assessing the outcome of the TANF program. Lastly, the review of the evaluation offers recommendation on the areas of the evaluation perceived as inadequate or needs improvement. It is apparent that the success of programs is measurable through evaluation of its outcome, but to evaluation process itself should also demonstrate accuracy and coherence, which can be determined through a critical analysis of evaluation results.
Program Evaluation Study Overview
In addition, the evaluation encompasses identification of the program size in terms of participants and the extent of the TANF program’s coverage and duration. More importantly, assessing the effectiveness of the said program was measured in the evaluation by employing both qualitative and quantitative method. For example, a quasi-experimental design including a pre-test and post-test deigns were used to measure the participant’s desirable practices and knowledge gained from attending the LEAP classes. One member of the evaluating team attended the first and last session of the meeting allowing the participants to self-assess their progress in the program using a 5-point Likert scale. On the other hand, social support was measured using 10 selected items in the Social Provision Scale while self-esteem was measured using Rosenberg’s Self-Esteem scale (Wichroski et al., 2000). A total of 127 women participating in the welfare-to-work program were recruited for the program evaluation ranging from age 19-62 years of age. The result of the TANF life-skill program showed that women made significant gains from the program, which evidently suggests that the program allowed participants to gain relevant competencies.
Program Scope
The family life-skill program was initiated in New Hampshire to address the women disempowerment due to life stressors exacerbated lack of social and self-improvement support. TNAF was created with the scope of providing women with an opportunity to participate in curriculums such as LEAP, which were designed to improve women’s self-esteem and social skills that will enable them to be independent of welfare by becoming more employable. This program objective was discussed in the evaluation, but did not clearly identify important areas such as program boundary, size, duration, complexity, and innovativeness. These criterions encompasses effectiveness of evaluating a program, otherwise the lack thereof limits the findings in terms of assessing the program scope. For instance, evaluating whether a program intended to improve the competitiveness of the targeted population. Furthermore, programs are normally structured to provide results and aggregate change to address. In order to realize the intended change, the program has to demonstrate a considerable level of effectiveness, which is often, measured trough evaluation. On the other, if the evaluation has failed to measure determine the adequacies of the factors such as boundary, size, duration, complexity, and innovativeness, there would be no clear measure of the extent of the program success in achieving its objectives.
Evaluation Methodology
In terms of methodology, it is apparent that the program evaluation employed the quasi-experimental design, which concerns about internal validity. This is because treatment of a control group may vary in comparison to the baseline. In addition, the methodology encompasses random assignment of study participants in order to have an opportunity to conduct testing in different comparison groups and intervention groups. In the evaluation of the family life-skill program, one of the issues that the evaluators may encounter is that the treatment of intervention group varies across the duration of the program. In order to have baseline data, a pretest is necessary to have a comparative baseline at the end of the intervention period, which was done in the evaluation of the life-skill program. The challenge of variability across the intervention group was effectively addressed by having a baseline data that can be used as basis in examining the changes before and after the intervention. Furthermore, the evaluation also conducted a post-test, which is also necessary in measuring the program outcome by comparison to the pretest results.
Evaluation Outcome
The described findings are indications of prior knowledge before participating in the program, which was identified through the pretest. On the other hand, to determine if further development in terms of more complex knowledge and skills acquired during the program, the post-test was conducted. The results have determined that the program was successful in improving the participants’ perception about social support. The same results can be observed on self-esteem and self-efficacy, as indicated on the Rosenberg Scale wherein the mean score of participants at 27.8 from a scale of 10-40 suggest significant increase in self-esteem. The perception about improves in self-efficacy and self-confidence are indicators of the effectiveness of the program, which was captured in the evaluation through a statistical analysis using correlations. It is apparent that the use of quantitative method in the evaluation process improves the internal validity of the program outcomes, which the evaluation has clearly emphasized.
Recommended Improvements to the Evaluation Study
Although the evaluation showed a coherent approach to program measure, certain areas of the evaluation still needs further improvement. For one, the evaluation showed limitations in terms of qualitative measure. This is because the evaluation relied heavily on the qualitative side of the measurement and lacks focus on the descriptive findings that would also help in the determining the program success. For example, there were no instrumentalities used to capture participants’ perception about the program in narrative context. Although scaling responses are indicative of the participants’ perception, having substantial descriptive evidence to support the quantitative findings would significantly improve the result of the evaluation. For example, the self-esteem factor showed positive impact on the participant’s self-efficacy, but there was no descriptive evidence present to support such finding. It can be recalled that the program evaluation also seeks to determine the level of motivation that participants gained from participating in the program. However, motivation could have been measured more specifically by incorporating a qualitative feature in the evaluation such as an interview to through a feedback survey. In addition, effectively measuring the success of the program should also involve taking feedback from the participants about what they feel is inadequate or missing in the activities, which are types of responses essential for improving the future implementations of the said program.
Conclusion
Other than the aspects of the evaluation discussed in the recommendation, the overall assessment of the family life-skill program showed considerable adequacy in terms of measuring the program outcome by creating a baseline data for comparison. The use of qualitative methodology was also effective in terms of quantifying the correlations between the described variables, which makes the evaluation plausible in terms of assessing program effectiveness.
References
Wichroski, M. A., Zunz, S. J., & Forshay, E. (2000). Facilitating Self-Esteem and Social Supports in a Family Life-Skills Program. Affilia, 15(2), 277-293. doi:10.1177/08861090022093868
