Both articles, analyzed within the limits of this paper are dedicated to the issues, related to arbitrary metrics in psychology and the accuracy of results obtained through questioners and, thus, applied in quantitative research.
Schwarz and Oyserman (2001) in their article make a point saying, that a number of researchers tend to avoid developing their own questioners, as a reasonable approach to developing such has not yet been worked out. The types of evaluation questions used in modern questioners take the results very far from the expectations of the evaluators. Basically, the thesis of the article is that the questioners do not work the way they are supposed to work. And here are the arguments the authors provide in support of their position.
First of all, the process of recalling a piece of information like “how many cigarettes do you smoke a week” may take a significant period of time. The respondent does not necessarily know this information right away. Meanwhile, questioners limit the respondents in the time for the answer. One minute, which is, according to the authors, an average period of time provided to the respondent to cope with one question, is not enough to properly recall all relevant information and put it into the questioner.
Another argument is that while remembering the information requested, the respondent analyzes significant volumes of contextual information. It is in many ways related to the piece of information requested; moreover, it may influence the answer. Thus, this background information would be very handy to analyze for the evaluator. Meanwhile, the evaluator does not have a chance to get a hold of such information or analyze it.
One more important argument, provided by the authors is that the respondent is put into the limits of the questioner hinted options. He usually does not have an option of putting the answer into the questioner in their own words. Thus, suggested options may also influence their final responses.
Lastly, even after the respondent has got an answer to the questions, it is quite possible, that they would reconsider their answer in accordance to social reasonability. They are likely to “edit” their final answer in accordance to the expectations of the society, to look good in the light of the answer provided.
Firstly, the authors recommend testing each and every question by answering it. If the question appears to be difficult to the evaluator, it will be impossibly difficult for the respondent.
The second advice given by the authors of the publication is to keep the information, which the respondents may withdraw from the questioner consistent with the intended purpose of the study. To make sure it is so it is important to analyze the questions, suggested options and other information, which the respondents may use to make their conclusions.
Thirdly, prior to developing one’s own questioner, the authors recommend addressing a good questioner, to look at its structure and the model of its questions. Definitely, it will require adjustments for the purposes of the new questioner, but it may serve as a base, as a sort of a pattern.
The sixth recommendation is reminding the respondents of the importance of providing accurate answers. The authors are sure, that once informed about such importance the respondents are likely to react properly and accurately.
Another recommendation provided by the authors is to consider the context when developing the questioner. One of the most promising formats for that would be events history calendar format.
Lastly, the authors strongly recommend making sure the respondents to understand the actual meaning of the questions and the type of answers they are supposed to provide.
In his article Jaccard (2006) also underlines the importance of understanding the relative nature of the accuracy of the results, provided by psychological questioners. However, he particularly underlines that the accuracy of results is particularly important for any psychological research. The point is that there is no mechanism of direct observation and direct calculation. And this is why it is important to take measures in order to provide more accuracy to the answers, provided by the respondents. In order to obtain such results it is critically important to work on the accuracy of the results, which are obtained through psychological questioners. In psychology it is particularly important, the author underlines, to not blindly use the results of any questioner, but rather test it with common sense in order to see, whether the results are reasonable and only after such an examination should the results be trusted.
One of the arguments, provided by the author to support his stand point that the results of such questioners need further interpretation is that researchers attempt to measure certain phenomena in psychology with precise scales. Meanwhile, such phenomena cannot be measured with such a precise scale. It is impossible to measure the likelihood of some behavioral pattern in real figures. It is really hard of a task to express one’s mental state within read figures as well. This is why; it is important to keep in mind that the figures, shown in the results are of only approximate exactness and can only be referred to as relative figures.
When conducting my own quantitative research, I would first of all make sure, my questions are very straight-forward and easy to understand. I would make sure, all the respondents are aware of the importance of providing accurate answers. On the other hand, I would consider making the questions, the answers to which are likely to be edited for the sake of social reasonability, not very obvious, so that to avoid such “editing”.
Definitely, prior to developing my own questioner and making a research based on such a questioner, I would definitely consider studying a number of similar questioners. I would make such a research in order to find a few good questioners and apply them, if possible, as a pattern for my own questioner. I would definitely make sure, that my respondents have got enough time for memorizing the data, requested by the questioner. It is, however, very important, that the respondents do not have too much time, as such time may be used for hesitation, reconsidering and, lastly, editing their answers to meet the social standards in which the respondent believes.
Reference List
Jaccard, James (2006). Arbitrary Metrics in Psychology. Hart Blanton University of North Carolina at Chapel Hill.
Schwarz, Norbert and Daphna Oyserman (2001). Asking Questions About Behavior: Cognition, Communication, and Questionnaire Construction. American Journal of Evaluation. 22; 127 DOI: 10.1177/109821400102200202
