In my research design, a classroom of students taking an Arabic second language acquisition course at the Defense Language Institute Foreign Language Center will be evaluated in terms of a specific intervention program intended to increase oral and language proficiency for the students, mostly American, who take it. At the end of the 24-week course, the Oral Proficiency Test (OPI) will be conducted and the cumulative scores collected and analyzed. At the same time, interviews will be conducted with the participants in order to gauge their collective feelings about their classroom experience while taking the course. This mixed methods data will be compared with the same evaluations performed on a classroom experiencing the normal curriculum. The aim of this is to test the strength of the hypothesis that the classroom with this intervention will have a marked improvement in OPI scores, as well as overall enjoyment and comfort level of the material within the classroom.
Action research is the overall goal of the research design I have proposed. Donato (2003) states that action research "is any systematic inquiry conducted by teacher researchers to gather information about the ways that their particular school operates, how they will teach, and how well their students learn" (p. 1). Action research typically involves planning an intervention, monitoring the effects of its implementation within the classroom setting, and studying the results. Methods of data collection include interviews and standardized test scores, which fall squarely into the methods suggested by my research design - this makes it a pure and strong example of action research (Donato, p. 1).
Typically, action research is performed in order to determine what changes need to be made to a classroom or other setting. In my research, however, the need for change will already be assumed; the desire for improvement drives the impetus for the intervention in order to glean still-better scores out of the students taking Arabic language acquisition courses. An established trend of challenges experienced by American students as they take Arabic courses leaves sufficient call for interventions to attempt to remedy the situation. Performing action research will provide the systematic and practical approaches that are required to fully investigate this intervention and this classroom issue (Nolen & Putten, p. 401).
The research design I have created for this second-language acquisition intervention carries with it a tremendous number of strengths. Among these is the overall benefit of a comprehensive data set that would be gleaned from the mixed methods design - both quantitative and qualitative data would be acquired over the course of the experiment. With the quantitative data, measurable, comparable and discussable data would be gleaned regarding the rote, pure performance of the students who undergo language learning courses through this set of interventions.
Quantitative data can be defined as research which places the emphasis "on facts and causes of behavior...the information is in the form of numbers that can be quantified and summarized...the mathematical process is the norm for analysing the numeric data, and...the final result is expresssed in statistical terminologies" (Galafshani, p. 598). The Oral Proficiency Test (OPI) would provide a means by which to discern language proficiency in spoken language and understanding, which is the primary context by which learners entering a military setting would find its greatest use. By offering concrete numerical data that can be analyzed, the objective usefulness of these interventions can be attained.
In terms of the qualitative data gained through interviews with those undergoing the interventions, a generalized, yet detailed sense of the overall classroom experience can be gained and included in the research. By providing a more comfortable, effective environment with which to learn, the alternative structure and activities of the intervention may be able to improve not only the amount learned, but the effectiveness of the learning process. Action research typically includes 'dialogic action research,' wherein participants engage in open communication with the research regarding what could be done better about an intervention or possible program (Dick, p. 450). This would help to determine whether the students had more fun, felt more at ease with the material, and developed better study habits. This data is important to determine the overall experience of the intervention method - the goal is to find the most cumulatively improved second language acquisition experience possible, and so comprehensive quantitative and qualitative data is required in order to determine the entirety of the experience. With these things in mind, a mixed methods approach is the most pertinent one to use for this research.
Despite the inherent strengths to this study, there are some potential weaknesses to the mixed methods approach. There are some threats to validity, including the possibility of students not fully assimilating or engaging with the material. It is entirely possible that some students will not participate at all in the class; while this would likely lead to low marks anyway, it also prevents objectively discovering the possibility of the intervention's effectiveness on that student's education. In other words, other factors besides the curriculum itself could affect the child's learning or performance, such as outside family and personal stresses and the like.
There is also a decided amount of uncertainty in regards to the way in which the quantitative and qualitative data would be combined and synthesized to form a coherent result, or answer to the hypothesis. The often subjective nature of qualitative data can make it difficult to scale or gauge levels of participant reaction to intervention; one person's idea of 'great' may be different than another's. In terms of the qualitative data, issues of reliability and validity may arise. "Although the researcher may be able to prove the research instrument repeatability and internal consistency, and, therefore reliability, the instrument itself may not be valid" (Galafshani, p. 599). Researchers becoming involved in the research process also contributes to a test becoming less valid, due to issues with construct validity. The primary concerns regarding the qualitative portion of my research is whether or not it can be replicated, and determining the accuracy of my responses (Galafshani, p. 600).
In response to those concerns, I believe that the settings and conditions will be firmly established by my overall written work, with the goal of making the experiment as reliable and replicatable as possible. Furthermore, appropriate research and researcher conduct will be performed in order to increase validity as much as possible; the researcher will remain objective, provide fair, neutral questions, and do as little as possible to affect the qualitative answers elicited from the participants interviewed.
Another weakness includes the ethical issues involved; provided that the experimental group's intervention provides a better learning experience and higher grades, it can seem unfair to the control group, who would receive no intervention. However, this could be possibly prevented if the control group were allowed to participate in the intervention group at a later date. Though this may be far from feasible, given the resources required to run it again and the fact that these students would already be done with the class, that may not work as well as hoped.
Other weaknesses include the problem of informed consent of participants. Informed consent is meant to provide participants information "about the likely risks involved in the research and of potential consequences for participants" (Nolen & Putten, p. 402). If the students sign consent forms, however, they will be well aware of the consequences of performing this study. Since it is merely for a classroom intervention, little physical harm is to be expected; the only real hazard is the sacrifice of their grades, provided the intervention works far worse than expected.
In spite of these weaknesses, it is my belief that the mixed methods approach is the best way in which to address the problem statement and research questions. Its combination of quantitative and qualitative data provides a multifaceted investigation of the effect of this language intervention on a classroom, as well as its applicability in other settings. Not only with the definitive language learning improvements be determined through thorough analysis of the OPI results, the interviews with the participants will show whether or not the intervention crafted an effective and interesting classroom experience throughout the course.
While action research is the blanket term for the type of research this study pertains to, there are subtypes of action research that would be wholly inadequate for this research. School-wide research involves the changes in activity of an entire school; in this case, it would mean applying the same intervention model to all classes, regardless of its applicability to that subject (Ferrance, p. 4). A research design involving no intervention, simply to see what is wrong with the course, would provide little workable solutions; we would see what is specifically wrong with the existing course to warrant the challenges that are being faced, but the specific hypothesis would not be answered.
One alternative would be to merely do a literature review, a library project where previous studies on the subject would be examined to determine whether or not these kinds of interventions would work. Firstly, it would not work due to the relative lack of literature on this specific subject; the concept of Arabic second language acquisition courses experiencing difficulty in a military setting is not one that has received sufficient study to warrant a mere literature review. Secondly, this specific intervention calls to be evaluated in this specific setting; it is hoped that the same positive results in its initial application would be experienced in this same application. None of these types of research apply to the tenets of action research, which involves "people working to improve their skills, techniques, and strategies. Action research is not about learning why we do certain things, but rather how we can do things better" (Ferrance, p. 3).
There are three different models of action research that are typically used; the Sagor Model, which provides collaborative action research through problem formulation, data collection, analysis, results reporting and action planning, is the model that most closely resembles my mixed methods design (Brown, p. 11). The problem of challenging Arabic classes is formed, data will be collected and analyzed, the results will be reported, and action will potentially be taken to implement the intervention model on a larger scale, provided it yields satisfactory results.
When collecting qualitative data, I may use partially closed surveys during the exit interview, so as to provide each participant with the same set of questions regarding the course's fairness, ease, and effectiveness of learning. By providing a partially-closed survey, the participant has a chance to write in their own answer based on their own experiences (Cengage Learning, 2005). This will permit the researcher to have as comprehensive and accurate a set of data regarding the student's feelings toward the intervention course as possible.
In conclusion, a mixed methods approach to action research is the preferred method to determine the result of my hypothesis. The quantitative data will provide real, measurable data on the performance of the students during their OPI; qualitative data will come from interviews and partially-closed surveys to gain insight into the overall classroom experience. Alternative methods of research would not provide the accurate type or breadth of data required to gauge the overall effectiveness of the intervention; as a result, the mixed methods approach I have detailed in my research design is the best option.
References
Brown, B. L. (2002). Improving teaching practices through action research (Unpublished doctoral dissertation). Virginia Polytechnic Institute and State University.
Dick, B. (2006). Action research literature 2004-2006: Themes and trends. Action Research, 4 (4), 439-458.
Donato, R. (2003, December). Action research. ERIC Digest.
Ferrance, E. (2000). Action research. Brown University.
Golafshani, N. (2003, December). Understanding reliability and validity in qualitative research.
The Qualitative Report, 8 (4), 597-607.
Nolen, A. L., & Vander Putten, J. (2007, October). Action research in education: Addressing
gaps in ethical principles and practices. Educational Researcher, 36 (7), 401-407.
CITI: Course in the Protection of Human Research Subjects. Retrieved from http://www.citiprogram.org/default.asp.
The Psychology Research Handbook. SAGE. Retrieved from http://www.sage-reference.com.proxy1.ncu.edu/view/hdbk_psychrsch/SAGE.xml
Wadsworth Cengage Learning: Surveys – Research Methods Workshops. (2005). http://www.wadsworth.com/psychology_d/templates/student_resources/workshops/res_m
ethd/surveys/surveys_01.html
