- A
Thomas D. Cook is affiliated with the northwestern university. He is also a faculty fellow at the institute for policy research . At the Northwestern University, Thomas Cook is a professor in the fields of psychology, sociology, education and social policy. Thomas Cook’s major area of concentration is the methodology of program evaluation. He has written many books and articles in his stellar career and for his efforts and contributions, he has been awarded several awards. Some of the awards he has received are the Myrdal prize in 1982, the Rossi Award in 2012 and the Campbell prize in 1988. It is his contributions towards methodology of program evaluation that this paper will concentrate on.
- A
Thomas D, Cook regards describing what works in an evaluation as the most important part of an evaluation program. In his analysis, an evaluation program should use knowledge and a lot of experience to come up with solutions and advice on what parameters the program or concept can operate. Certain factors affect the outcome of an evaluation, and these are budget constraints, the ability and training of the staff carrying out the evaluation, and the sponsor’s sophistication about the whole evaluation program. He is of the opinion that these factors will influence important decisions such as what is selected for evaluation, the role of the evaluation and the method in which the evaluation is to be carried out (Cook, 1986).
The other purpose of evaluation is to improve social programs. To do this evaluation has taken three major approaches; identifying manipulable solutions, identifying generelizable explanations and lastly providing stakeholder service. According to cook, the first approach mentioned in the previous sentence is concerned more with the extent of which a particular solution to a social problem can be manipulated. The extent to which the solution can be manipulated is therefore, more important in an evaluation as compared to what solution works. To get the best manipulable solution, all the solutions, or several best solutions, to a social problem are evaluated and the most effective is selected. The most manipulable solution is one which can best handle the key manipulable parameters of an evaluation, thus, sophistication of the sponsor of the evaluation, the budget constraint and the skill set of the staff carrying out the evaluation. By doing this, the best manipulable solution replaces the initial solution, assuming it has been found not to be the most manipulable, and the social program is consequently improved (Cook, 1986).
The second approach, which involves identification of generalizable solutions looks to make great use of our ability to come up with social theories that relate the inputs, the processes and the outcomes of the program. This approach banks on the transfer of knowledge to improve social programs. It is also evaluator friendly as the evaluator only has to explain rather than describe the relationships between the above mentioned variables.
The third approach emphasizes on putting the stakeholder of the program at the center of the whole evaluation process. The stakeholder is the person who is directly or indirectly affected by the program under evaluation. His interests should be put first and so he should be consulted before any changes are made to a program. The proponents of this approach have therefore criticized the proponents of the above mentioned approach for being inconsiderate of the stakeholders of programs and instead putting too much attention to science and methodology. The proponents of the 1st two approaches have not taken this criticism lying down and have also criticized the stakeholder approach, arguing that there may be more than one stakeholder in a program and each of these stakeholders may have totally different interests in the program and so it will be very hard to recommend any changes to a program (Cook, 1986).
B
The theorist, Thomas D, Cook, uses a method focus in carrying out evaluation purposes. He has specifically been credited with bringing quasi experiment designs into use. This is a slightly different approach than the normal random experiments in that instead of using statistical tools and measures to control the results or to test a program, quasi experiments use design to create the best possible approximation to the missing counterfactual (Cook, 2002). The theoretical approach also involves purposeful sampling which serves to increase heterogeneity of irrelevancies. There is also the involvement of stakeholders, through the stakeholders approach to social program evaluation, so as to generate relevant questions that must be answered by the evaluation and also which must be considered before carrying out the evaluation. There is also the use of meta- analysis, which helps in comparison between several other studies and evaluations already done, so as to deduce trends and patterns (Cook, 2010).
C
RCT’s have been widely used before, but cook is skeptical about it and criticizes it. He points to the easy possibility to manipulate the technique, citing an example that the RCT may not be exactly random as described. He also points out the problem of differential attrition, though he says it is much less of a problem nowadays because we are much more aware of it and know how to minimize it (Cook et al, 2010). Quasi experimental approach, as already explained before in this paper, has been adopted as a better way to carry out evaluation. There is also more consideration of the context in which evaluation is being carried, as well as effect size variation where the question now is under which conditions the evaluation program works and for whom, as opposed to just if the program works (Cook, 2010).
- C
The evaluator plays a crucial role in accommodating the political class. Politicians are driven mostly by the desire to remain in power, thus get re-elected, rather than follow evidence research in making decisions. Cook is of the opinion that if these evaluation research results are presented to politicians in a timely manner rather than just being given incontrovertible advice. Cook cites the example of the worker profiling and re-employment services program which showed that an in depth job search assistance rapidly got the unemployed back to work. It is safe to assume that the government clearly thought that implementing such a program would be beneficial to it, because it will give it the good image from the public. The public would be convinced that their government is doing something about the unemployed and would hence re-elect them back to office when the next elections come around (Nathan, n.d). The government also is one of the bodies that funds most evaluation programs that deal with social policy and thereby making it a stakeholder. In the United states, the executive government funds most of the evaluations on social programs, but they only do this after they have received an outline of the type of evaluation being carried out and the whole evaluation design that includes specifics like the evaluation questions. In doing this, the government clearly decides what is to be evaluated and the type of response desired, that would not harm its chances of getting re-elected. The government does not want an evaluation report that would catch it unawares and put a blot on its record. As Cook (1998) puts it the U. S. funding system helps government agencies get evaluation reports that reduce the chances of surprises to them. In other continents like Europe, evaluation has been used as a scapegoat to manage crises. Evaluation results have been, for example, been cited as the basis for cutting jobs during a contraction period.
F
Thomas D, Cook, in one of his approaches to program evaluation, proposes the involvement of stakeholders in the program evaluation design. All those affected by the program are the stakeholders of that program. These stakeholders have to be involved in the construction of program evaluation goals and questions. Failure to do this may lead to the evaluation addressing the wrong issue, or an issue which is not of the utmost importance at that particular time. Decision makers also have to be involved because mostly they are the ones funding or sponsoring the program evaluation. Evaluating the wrong program or addressing the wrong question would therefore make their funds go to waste. The results of the evaluation program would also be wrong and therefore of little or no value at all to the sponsor. Evaluators and stakeholders, therefore, need to work together so as to avoid wastages. Although all stakeholders must be involved in the evaluation design, it is important to note that the more stakeholders there is, the harder it will be for the evaluators and the stakeholders to agree on the evaluation questions, because different stakeholders have different needs and preferences. One action may also affect different stakeholders differently (Cook, 1986). Cook is also of the opinion that nowdays, the criterion for evaluation has been left to politicians, whose main aim is re-election, and also to lobby groups who he says articulate self interests (Cook, 1998).
G
According to this theorist an evaluator has to have certain specific qualities. He must have a strong methodogical background and strong quantitative and qualitative skills. The evaluator has to be aware that different evaluations require different evaluation designs and so the evaluator needs to be equipped with different methodologies that can fit the context of the evaluation being carried out. Cook (1998) states that “, the method preferences evaluators espouse need to be very broad and reflect the reality that evaluators work in different substantive areas and have been trained in disciplines with their own method predilections.” The evaluator needs to understand the culture and the standards of operation of the particular program one is evaluating. This will be important in helping avoid making wrong conclusion at the end of the evaluation. The evaluator also needs to be able to do meta- analysis so as to acquaint himself with existing patterns and trends as may be applicable to the type of evaluation he is undertaking. Cook is also of the opinion that more and better evaluation theory has to be done. He opines that in the past 10 years or so, there existed a lot of robust debates about evaluation methodologies than there exists today. The big picture theory of the past has been overlooked and in its place theories related to issues of epistemology adopted. Cook sees the need to involve a lot of different professionals in the evaluation, as different settings and results make more sense to a certain type of profession than another.
References
Cook D, T. & Shadish, R W. (1986). PROGRAM EVALUATION: THE WORLDLY
SCIENCE. Ann Rev Psychol 37, 193- 232
Cook, D T. (2002). Educational Evaluation and Policy Analysis. Educational Evaluation and
Policy Analysis Fall, vol 24, No. 3, pp. 175-199
Cook, D T, Scriven, M., Coryn, L. S.C & Evergreen, D.H. S (2010). Contemporary Thinking
About Causation in Evaluation: A Dialogue With Tom Cook and
Michael Scriven. American Journal of Evaluation, 31 (1), pp. 105-107
Cook, D T & Wittmann, W W. (1998). Lessons Learned about Evaluation in the United States
and Some Possible Implications for Europe. European Journal of Psychological Assessment, Vol. 14, Issue 2, pp. 97–115
Nathan, P.R (n,d). Point/ Counterpoint.
