Background
Assessment of staff is an important task for the success of any organization (Bob, 2007, p. 82). In a world that is always growing and developing new skills, every organization ought to train its staff to adapt to changes in their field (Mathis & Jackson, 2010, p. 261). In the medical field, change is the only constant aspect. Large amounts of money are invested every year by a multitude of organizations into research of all kinds of medical fields. Better treatments, procedures, medications and equipment are coming into operation every day. There has been an increase in sensitivity to this issue by many hospitals in Oman and staff training has been going on for more than a decade now. This has seen an improvement in the quality of health services with better trained doctors and paramedics and better equipments to handle all kinds of ailments and medical emergencies. As this practice continues, there is an increase in number of trainees. This research will be examining whether the methods used to evaluate the progress of the trainees are effective in achieving high standards of staff assessment. The research will take Khoula Hospital in Oman as a case study to evaluate the utility of assessment of staff development to the whole process. It will examine whether or not the hospital and staff benefit from the methods used to assess the staff.
Khoula Hospital is one of the major hospitals in the Sultanate of Oman. It is acclaimed to have developed from a 20 bed health facility to among other functions, being the only trauma center in Oman. The hospital is a national training centre and is among the major teaching institutions in the country. The hospital, through its Training and Staff development department is responsible for training a myriad of medical staff in many fields and continued development of already qualified staff in the hospital and elsewhere. The training department works in collaboration with the other 20 departments in providing training and staff development in courses that run throughout the year in different bouts of time depending on the program. Courses run weekly, monthly or in 2-3 month packages. The department (T&SDD) uses both multiple choice question (MCQ) and itemized checklist methods to assess the development of staff under their program. Training and staff development optimizes the capability of staff to learn new things and protects the public from incompetent services by medical workers (Riccardi, Lilly, Cardinal, Nandini & McCusker, 2005, p. 546). Assessment also helps to chose staff suitable for further training and development (Vinten, 2000, p. 320). Generally the training and development of staff improves the quality of services provided by the staff to the public and in the case of the health sector, helps save more lives (Noe, 2006).
There have been many written papers and books on the best ways to carry out assessments of staff especially those in school (Forysth, 2009, p. 115). Many suggest different ways of carrying out assessments in the work place. Many medical professionals are not comfortable using academic methods of assessments to assess their peers or students working under them (Epstein, 2007). Such methods include filling in forms that have pre-described attributes of the staff under review that one requires to assess. This kind of method gives a scale to which the medical professional is supposed to rate the staff member by. There have been concerns about whether assessing a person through such means is effective since the assessment is done based on often quick observations and personal feelings of the person carrying out the assessment (Chappuis, 2009, p. 22). Also, many are concerned about how their reviews affect the students’ professional and academic development. As a result of these and other concerns, some institutions have adapted other forms of assessment such as written examinations which give an objective score and rate the students’ understanding of certain areas, observation and video reviews (Arter & Stiggins, 2009, p. 72). These help the supervising doctors and other medical staff an opportunity to clearly examine an individual’s capabilities and limitations in the work place, and use of clinical simulations which examine an employee’s response to varied situations which might not occur at the time of the assessment. All these assessment methods, though modern, all have flaws that make them vulnerable to failure in effective assessment (Cox, 1997, 148). It has been suggested that the use of many methods collectively as assessment tools may help improve the effectiveness of assessment procedures for medical personnel (Fisher, 2007, p. 33). Assessment of staff and student development in the medical fields involves planning for the assessment in terms of how the assessment will take place, where it will take place, who will be involved in the assessment and what are the procedures for the assessment. The second step is execution of the planned assessment which involves managing the process, taking care of the logistics, collecting the information after the assessment is done and carrying out analysis of the information (Stiggins, 2011, p. 94). After the information for all the staff is consolidated and analyzed, presentation of the general information is necessary to the appropriate authorities. In extreme cases negative assessments the authority that assessed the student or staff and the subject themselves are called in for follow up. In most institutions, this has been the cycle of assessments. However, the addition of an evaluation step would benefit many institutions (Klein, 2010, p. 62). Where a process has been in place for a long time, assessment becomes irrelevant to the development of staff (Thornton, 2005, p. 159). Constant evaluation of the effectiveness of the assessment methods used is necessary to streamline assessment with current staff development policies and practices.
Effective assessment ought to be a continuous process of planning, execution, presentation and evaluation of the assessment procedures (Fowell, 1999). Evaluating assessment procedures helps the institution to come up with better methods of undertaking assessments that will be more effective to the greatest number of students in the training program (Gupta, 2007, p. 226).
Specific and Achievable Research Questions
The research will seek to answer the following questions:
1. Does the assessment method used in the course align with the learning outcomes and the teaching and learning approaches?
2. Are the assessment methods used in the course assessing high cognitive levels?
3. What are the trainees views of the assessment methods employed in those course?
4. What are the instructors and experts view of the assessment methods used in those courses?
Methodology of the Study
The research will be carried out with an action oriented approach design. This approach seeks to bridge the gaps often found between practice and research. This means that the ultimate purpose of this research is to integrate the findings into the practice of assessment in the field of medical training. The research will seek to come up with practical conclusions to the problem at hand that can thereafter be used by medical professionals in assessing students and by institutions to assess their staff’s development. The research will use both quantitative and qualitative methods to collect data from the sample selected at the Khoula Hospital. Quantitative method will be the main method used. They will seek to collect information about the present and past assessment procedures at the hospital. Figures collected can then be used to develop models describing the findings of the research. This kind of data is important since it is easy to manipulate into whatever kinds of presentation methods are appropriate and to generalize the findings into data distributions (Johnson & Christenses, 2008, p. 125).
Qualitative method will be used in the research to collect information that the researchers cannot think of alone, such as opinions about the existing assessment procedures of which the respondents are more familiar with. These methods will help the research to understand the personal feelings, understanding and suggestions of respondents involved in the research (Johnson & Christenses, 2008, p. 132). In drawing conclusions, qualitative information will be important as it will guide the kind of practical inferences made. This is because, when coming up with conclusions that will be effective on other people, there will be need to consider the opinions of the persons involved in the assessment process, both the students and the personnel responsible for undertaking assessments. If the views of the respondents are not considered, the suggestions made by the research will have minimum effect on the assessment procedures used since the people involved will not be forth coming in implementing them.
The research will use combined data collection methods such as administering questionnaires, conducting interviews and making observations. It will take place in the Khoula Hospital under the instruction and guidance of the Training and Staff development department of the hospital which will help coordinate other department and ease the process of data collection. Since the hospital is a teaching institution, there are always a number of students and staff undertaking a training program at the hospital. The training and practical sessions will be the base of the research since this sessions provide the basis by which these persons are assessed by their trainers and supervisors. Carrying out the research in the hospital will help the researchers to socialize with respondents involved in their usual environment which will ease their willingness to share information concerning the hospital’s assessment policies (Walliman, 2010, p.87). Carrying out the research in the hospital will also reduce the financial burden of the research, by reducing costs of transportation, materials, personnel and even security, since the hospital will be providing all of these.
The research will involve sensitive information provided by the respondents. This means that the personnel chosen to conduct interviews and administer questionnaires must be trustworthy and safe to talk to. There will be a small number of researchers to reduce the sense of invasion whenever a respondent is approached. Before a questionnaire is administered or interview started, the researcher will first get the permission from the respondents to continue with the procedure.
The sample population for the research will consist of personnel directly involved in assessment as well as students and staff who are currently being assessed or have in the past been assessed at the Khoula Hospital training program. This pool of respondents is relevant to the purpose of the research, to find out how effective and useful the assessment procedures of the program are. This pool will include trainers, non-teaching personnel from the Training and Staff development department, hospital executives responsible for this department, other department heads, current students as well as staff who have completed the program and medical personnel who assess the students while in practice.
Data Source and Collection Procedures
Data will be collected from the selected sample through a couple of methods as described earlier. The data collection will center around the questions that the research seeks to answer with its outcomes. Questions asked to the respondents will be those relevant to the research’s purpose of coming up with practical conclusions and suggestions.
Research Question 1: Does the assessment method used in the course align with the learning outcomes and the teaching and learning approaches?
Under this question, the research will seek to identify the course content of the courses offered and comparing them with the expected outcome of the course. The question also evaluates the relevance of the models of assessment used in comparison to the content of the course and to the expected learning outcome for the students and staff.
The research will use the ten questions proposed by Harden (1986) to ask when planning a course as a framework. These questions will gather data from the respondents about the course and help describe the course content for the research. These questions are:
i. What problem is the course trying to solve?
This question provides a basis for analysis of the course content.
ii. What are the aims and objectives of this course?
iii. What is the main subject of the content of this course?
The question will help develop a sense of how the content was sourced and whether it was relevant to the objectives of the course.
iv. How is the content organized?
Content organization is important for learning and for assessment as well. The question will help understand what methods were used and issues considered in the development of the course structure.
v. What educational strategies are used in teaching the content of the course?
vi. What teaching and training methods are used in the program?
vii. How is assessment carried out in the program?
Assessment being the main focus of this research, this question will be subdivided into various categories to help come up with a broad idea as to how assessment is carried out in the course. Questions under this will focus on the techniques used in assessment, the timing observed in assessment, the personnel involved and the standards of assessment.
viii. How is information regarding the curriculum communicated?
ix. What educational environment is fostered at the program?
x. How is the course processes managed?
The researcher will analyze data collected from this section and then another person from the Khoula Hospital to be identified later will come in to validate the analysis from the data collected. Only courses with Multiple Choice Questions and itemized checklist assessment will be surveyed.
Are the assessment methods used in the course assessing high cognitive levels?
Under this question, the research will use Bloom’s Taxonomy to evaluate and analyze the outcomes of the courses selected. The research will compare the cognitive level of tested for each course and the level of performance required. The research will look for and record any mismatches between the cognitive level tested and the level expected.
What are the trainees views of the assessment methods employed in those course?
The main purpose of this question is to collect the views of the trainees about the methods used for assessment at the institution. The research will use open and close ended questions with choices on a scale ranging from “strongly agree” to “strongly disagree”. The research will ask questions to identify whether, in the view of the students, the assessment methods used test high cognitive levels, whether the contents of the courses have relevance and to test the quality of the feedback from the participants.
The research will send invitation letters to students of the training department for participation in this survey. Only persons who have undertaken courses in the last three years will be eligible for the survey. The research will take a minimum of 60 % from each course. The data collected will be analyzed by the researcher with the help of the hospital’s statistician.
What are the instructors and experts view of the assessment methods used in those courses?
This research part will use stratified sampling to identify 50 faculty members for the initial survey which will broadly seek to determine the faculty’s perception of the assessment methods used to assess students and staff of the hospital. The researcher will then use purposive sampling to form a second sample of 20 faculty members for a second round of survey. Semi-structured interviews will be conducted for these 20, seeking to collect more in-depth data on the methods of assessments used in the institution. The researcher will ask the faculty members for opportunities to conduct follow-up research later on. The main purpose for this 2nd round is to prioritize suggestions for improvements of the assessment methods, if any.
Ethics
There will be few ethics concerns while carrying out this research because of its small nature and localized area of study. However the research will observe appropriate ethical rules set by the hospital. The research will also seek to maintain the anonymity and confidentiality of the participants to prevent any victimization due to their views. The research will, for this reason, not collect participant’s personal information such as names and photos.
The research will seek approval from the Directorate of Research and Studies at the Ministry of Health, Oman. This will give the research legitimacy and credibility, and may mean more exposure if widely adopted by the government.
Logistics
The research will be localized and focus on the Hospital’s staff and students alone therefore the logistics involved will be minimal. The researcher will work with two other personnel of the Training and Staff development department and utilize the hospital’s statistician for data analysis. The budget for this research will be minimal since the all local and can be easily accessed by the researcher. Research materials such as the questionnaires will also be produced by the hospital.
There is little to none risk in conducting this research. It touches on one issue, so the possible risk would be that the outcomes of the research are not what the Hospital’s executives that give approval for the research want. Since the research does not require funding from outside, the loss of funds would not be a worry of the researcher.
Significance and Impact
This research is a first of its kind in Omen. There have been gaps in research in the assessment of medical education. The findings of this research will be used in the future formulation of policies concerning how medical students are assessed. The research will also attract attention from relevant authorities and institutions and lead to the development of faculty staff development programs in Khoula Hospital and other hospitals in Oman.
References
Arter, J. & Stiggins, R. (2009). Classroom Assessment for Student Learning. Boston: Allyn & Bacon
Bob, F., (2007). Integrating learning outcomes and assessment into the curriculum: a case study. On the Horizon 15(2), 80 – 88
Chappuis, J. (2009). Seven Strategies of Assessment for Learning. Boston: Allyn & Bacon
Cox, N. (1997). Practical Pointers for Quality Assessment. New York: Routledge
Creswell, J. (2008). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. New York: Sage Publications
Epstein, R.M. (2007). Assessment in Medical Education. New England Journal of Medicine, 356:387-396
Fisher, D. (2007). Checking for Understanding: Formative Assessment Techniques for your Classroom. New York: Pearson
Forsyth, I. (2009). Evaluating a Course. New York: Routledge
Fowell, S. (1999). Evaluating Assessment: The missing link? New Jersey: Blackwell Science Limited.
Gupta, K. (2007). A Practical Guide to Needs Assessment. North Carolina: Pfeiffer
Harden, M. (1986). Ten Questions to ask when planning a Course or Curriculum. Dundee: University of Dundee
Johnson, B. & Christenses, L. (2008). Research Methods, Design, and Analysis. Boston: Pearson
Klein, T. (2010). Creating Interdisciplinary Campus Cultures. New Jersey: Jossey-Bass
Mathis, R. & Jackson, J. (2010). Human Resource Management. Massachusetts: South Western College Publishers
Noe, R. (2006). Employee Training and Development. London: Irwin
Riccardi, M., Lilly, K., Cardinal, L., Nandini, D. & McCusker, J. (2005). Assessment of the Work Environment of Multidisciplinary Hospital Staff. International Journal of Health Care Quality Assurance 18 (7), 543-551
Stiggins, R. (2011). Introduction to Student-Involved Assessment FOR Learning. Boston: Addison Wesley
Thornton, G. (2005). Assessment Centers in Human Resource Management: Strategies for Prediction, Diagnosis, and Development. New York: Lawrence Erlbaum Associates
Walliman, N. (2010). Research Methods: The Basics. New York: Routledge
Vinten, G. (2000). Managing Staff Selection and Assessment. Managerial Auditing Journal 15 (6), 319 - 320
