Introduction
The current essay involves a critical review of the research process and development of two research reports on nursing care and shared care services and evidence based practice in the nursing profession. Understanding the research process and development may furnish important insights on how evidence based practice be implemented effectively in aid of better patient care in both traditional hospital nursing care and shared care services involving nurses as useful members in the healthcare staff.
Parameters for the research process critique involve important aspects in a research report: research approach or design, sampling method, methods of data collection, validity and reliability (trustworthiness and credibility), ethical considerations, analysis, results, discussion, limitations of the study, strengths, and implications of the study for practice.
Two research reports will be comparatively analyzed and critiqued: Burston and Stichler’s (2010) correlational study between the nursing environment/nurse caring and selected motivational factors; and Iliffe, Wilcock, and Haworth’s (2006) thematic analysis of themes surrounding shared care protocol for dementia management.
Research Approach
Both Burston and Stichler (2010) and Iliffe, Wilcock, and Haworth (2006) used a descriptive research design. However, Burston and Stichler (2010) used a correlational method in establishing a causal relationship between variables, while Iliffe, Wilcock, and Haworth (2006) followed a non-causal, thematic approach. There is inherent technical limitation in Burston and Stichler’s method to establish causality. Mahoney (2001: 576), citing a review by statistical researcher Charles Ragin (Fuzzy-Set Social Science), noted that the problem rests on its strong assumptions about “unit homogeneity” as basis for the linear causation objective. In Ragin’s observation, even if the time order of variables is well-established, the method had limited representation of the causal process. Conversely, Iliffe, Wilcock, and Haworth’s approach fits very well with its study objective. Oftentimes, thematic analysis like this had to be conducted first, especially in largely unexplored study areas, before any quantitative study becomes feasible to conduct.
Sampling
Burston and Stichler (2010) used convenience sample method primarily for the convenience of the researchers, using power analysis to determine the minimum sample to reduce chances of a type II error (failure to reject a bad model: MacCallum, 1996: 131). However, Iliffe, Wilcock, and Haworth (2006) used purposive sampling in this study to specifically determine the research sampling area, not for researcher or participant convenience, but to focus on areas where qualified samples are supposedly abundant.
Convenience sampling, however, is inherently a non-representative sampling method (Anderson, 2001: 1294), and allowing only weak statements about some features of the sample. Consequently, it cannot make inductive inference about the population of interest. Nonetheless, despite the low return rate of 28 percent, the actual sample size of 126 meets the minimum sample size of 88 as required in power sampling analysis, indicating an acceptable level of confidence amidst the inherent imprecision (to some degree) of a point estimate of fit as a model of fit in the interest population (MacCallum, 1996: 130).
Meanwhile, Iliffe, Wilcock, and Haworth’s sampling method provided no rational bases for selecting the specific sampling areas they had over other alternative areas. To choose study areas on the basis of reflecting the “different populations of inner city, urban, suburban and rural areas” (354) was inadequate as the aim of the study presumed the experience of respondents in handling dementia cases. In this case, the participants appeared to have no such experience. Coyne (1997: 624; cf. Patton, 1990) insisted that the power of purposeful sample lies in the selection of “information-rich” areas (i.e. experience in handling or managing dementia cases). The logical objective for such preference is to learn a great deal of information about the aim of the study (i.e. dementia management and shared care). If this had been so, the authors would have improved disclosure by stating that these centers had been recognized as rich sources of dementia cases and provided an estimated number of cases managed annually.
Furthermore, it is fairly reasonable to presume that these sampling areas or institutions have unknown number of dementia cases under management, if ever there had. In this regard, the authors would have conducted beforehand a prevalence study of dementia serviced in the candidate sampling areas to ensure “information-rich” situation. However, the inclusion of an area in Scotland was appropriate over its distinction as an atypical of the population (Weiers, 2014: 126); that is, having been involved in “a long tradition of collaboration” between the Dementia Care Centre at Sterling University and local general practices. However, the rest of the sampling locations appeared to have no previous history of contact with dementia cases. If there had been, those cases remained unknown at the beginning of this study, making the use of purposeful study without basis and failed the requirement of “information-rich” sampling areas. This inability to rule, with certainty, that the sampling areas had information-rich or not, could partly explain the uninformative or vague answers about dementia experience from the respondents; that is, these were not the right samples. The authors’ impression that the respondents had no experience-based information about dementia management or shared care could due to the fact they never had well-diagnosed cases of dementia. This problem could have been avoided had a preliminary prevalence survey for dementia cases had been conducted before this study commenced.
Method of Data Collection
Burston and Stichler (2010) used a paper-and-pencil method, involving four sets of standardized questionnaires [the Mueller McCloskey Satisfaction Scale (MMSS); the Professional Quality of Life Scale (ProQOL); the Stress in General Scale (SIG), and; the Caring Behaviors Inventory (CBI-24)] to cover the variables identified in the aim. Meanwhile Iliffe, Wilcock, and Haworth (2006) used semistructured interviews, which is fundamentally a survey method using a guide questionnaire through a face-to-face interview (auditory channel).
The Burston and Stichler study had only one instrument of strong validity and reliability (CBH-24); two having problems with low sample sizes (ProQOL and SIG), and one with unverifiable strength (MMSS; Tourangeau, 2006). ProQOL had been validated in 1130 samples (α = 0.88) for compassion satisfaction, in 976 samples for burnout (α = 0.75), and in 1135 samples for compassion fatigue (α = 0.81) (Higson-Smith, Hudnall and Stamm, 2014). Using a one-factor study of distinct samples, SIG had been validated in two sample sizes (N1 = 589; N2 = 4322) (Yankelovich, Broadfoot, Gillespie and Guidroz, 2011; Stanton, Balzer, Parra and Ironson, 2001: 866). The study’s sample size (126) was too low for an acceptable reliability and validity.
Conversely, CBI-24 showed high internal consistency for small samples (patients = 64; nurses = 42; α = >0.95), covers the same four major dimensions as that in the 42-item CBI (Wu, Larrabee and Putman, 2006: 18). Its test-retest reliability was also high for patients (α = 0.88) and nurses (α = 0.82). The use of CBI-24 in the Burston and Stichler (2010) study was appropriate as the use of semistructured interviews appropriate for the Iliffe, Wilcock, and Haworth (2006) study. It provided convenience for the participants and works well in soliciting more information compared to plain paper-and-pencil survey questionnaires. It is also the least burdensome method of administering a questionnaire-based survey (Bowling, 2005: 283). However, the current study failed to maximize its edge, and miscarried in verifying directly or indirectly the answers of doubtful accuracy. The authors failed to anticipate wary answers.
Validity and Reliability (Trustworthiness and Credibility)
The Burston and Stichler (2010) had only strong validity and reliability in CBI-4. While the Iliffe, Wilcock, and Haworth (2006) was hounded with issues resulting from the inadequate use of semistructured interview, which generated largely euphemistic answers and indicating lack of experiential basis. Bowling (2005: 282) noted two of the requirements in semistructured interviews: “probe for responses” and “clarify ambiguous questions.” Data materials that are not experience-based will fail to credible represent reality in the practice shared care in their respective institutions, and thus invalid information to use.
Ethical Considerations
Both studies appeared to have covered ethical issues before their commencement. The Burston and Stichler (2010) study received permission to waive the option for documented consent letter. Instead, participate consent had allowed with their completed questionnaires and the demographic survey. In addition, all potential participants were given full freedom to participate or not. The Iliffe, Wilcock, and Haworth (2006) also received permission from the ethics committees in Camden & Islington, Barnet, Bradford, North Cumbria and Forth Valley to conduct the study in their respective jurisdictions. The invitation letter, the conduct of the interview, and the recording of the interview implied presumed consent. Moreover, due to lack of danger to life, signing of informed consent is unnecessary (U.S. Food and Drugs Administration [FDA], 2010).
Analysis
Both studies used appropriate analytical methods to understand their data. The Burston and Stichler (2010) study used software capable of analyzing for analysis of variance (ANOVA), the Pearson Product-moment, the Levene’s test, and a four-step hierarchical multiple regression analysis. The Pearson Product-moment measures the linear relationship between the variables (AcaStat, 2014), reinforcing the strength of correlational analysis. The Levene’s test ensures homogeneity of all factor’s standard deviations against alternative standard deviations (Heckert, 2003). The ANOVA allows the comparison of the means of more than two populations (Statistics Solutions, 2009). The four-step hierarchical multiple regression analysis helps avoid multi-co-linearity between highly correlated independent variables (Jong, 1999: 98; cf. Cohen and Cohen, 1983), and succeeded in measuring the amount of variance shown by the independent variables. Iliffe, Wilcock, and Haworth (2006) used thematic analysis, allowing them to classify meanings through themes (Braun and Clarke, 2006: 77; cf. Holloway and Todres, 2003: 347). It facilitated the determination of important themes useful in achieving the aim. Although it is not a specific method (Braun and Clarke, 2006: 77; cf. Boyatzis, 1998), it is highly useful as a foundational method for any qualitative studies.
Results
Both studies have had been well-presented and in sufficient details, adequate statistical information to be satisfactorily understandable. However, the results in Burston and Stichler (2010) are detailed and well-presented. The weakness in their sampling instruments created a condition wherein only the data from CBI-24, and subsequently their interpretation, valid and credible. These data covered four dimensions of caring, such as assurance, knowledge and skill, respectfulness, and connectedness. The resultant themes in Iliffe, Wilcock, and Haworth (2006) appeared consistent with the analytical tool used and shed informative light according to the aim of the study. It succeeded in identifying themes and categorizing them into a small number of categories. Nonetheless, the validity issues hounding their implementation of the data collection method introduced serious uncertainty in the usefulness of the data gathered (Bowling, 2005: 282). Consequently, such unreliability of information may have caused misrepresentation or erroneous ranking of certain themes either false positively or negatively.
Discussion
The mechanics of the discussion in both studies had been handled very well. Burston and Stichler (2010) described the results thoughtfully, well-presented and understandable. Iliffe, Wilcock, and Haworth (2006) skillfully interpreted certain implications of each identified themes in relation to the aim of the study (shared care for dementia patients). They even managed to read through information gaps and intentional euphemisms to cover lack of actual experience in managing dementia cases. However, the defective selection of instruments and implementation of their data gathering strategies impaired seriously the value of the discussion. Burston and Stichler (2010) suffered weaknesses in three of the four data collection tools they employed, eliminating the usefulness in much of data, except those on assurance, knowledge and skill, respectfulness, and connectedness. Iliffe, Wilcock, and Haworth (2006) also suffered from inadequate verification of data provided, which put a serious but unresolved credibility question. For instance, the impression of lack of actual experience in managing dementia appeared accurate on the bases of limited details in their answers as well as preponderance of euphemisms. However, such inadequate information resulted from errors in sampling selection; thus, making it inadmissible as credible data for this study.
Limitations of the Study
Both studies included as limitations weaknesses, both directly affecting the credibility of data that should not have been allowed to stay so during the study due to the critical roles of these weaknesses in the empirical value of their investigation. Burston and Stichler (2010) noted the expressly the problems with the small sample size they obtained. This was an issue that they should have resolved first during the study, instead of relying at including it as a limitation. This error of judgment had seriously impaired whatever admissible data they gathered, except for the four CBI-24 factors. Recognizing this neglect as a limitation will correct the validity and credibility problems of the data. Iliffe, Wilcock, and Haworth (2006) admitted their being “not able to verify shared care protocols were ostensibly operational, or simply in development not always able to say consistently and with certainty whether respondents were speaking from experience, or in anticipation of shared care arranges” (360). Its inclusion as a limitation was also inappropriate due to the central importance of quality data in any empirical study. Checking it against a ‘true value’ is imperative for data quality (Bowling, 2005: 284). Their negligent inaction irreparably destroyed the value of their data. Instead of including it as a limitation, extra effort should have been done to verify, or increase certainty with, such information first.
Strengths
The strengths of both studies rest upon the methods of statistical analyses used, which had been well-selected. Burston and Stichler (2010) followed a rigorous analytical plan while Iliffe, Wilcock, and Haworth (2006) chose a fundamental analytical approach very informative for the study design.
Implications of the Study for Practice
The Burston and Stichler (2010) study had the potential determine the positive and
negative motivation factors involved in the dynamics of patient care in the professional perspective of nurses. An expanded understanding on the impacts of these motivational factors can provide opportunities for nurses to reinforce positive motivators and set up strategies to control and manage the negative motivators. Conversely, the Iliffe, Wilcock, and Haworth (2006) study had potential to open up a special focus in shared care programs for patients with dementia and open up awareness from stakeholders on the urgency and importance of providing shared healthcare for dementia patients, and slow down the mental deterioration associated.
Conclusion
The process review managed to isolate two errors of research development planning and design that had resulted to widespread problems in drawing reliable findings. The first error involved the selection of adequate sample size in relation to the data gathering instruments used (Burston and Stichler (2010). Inadequate sample size injected negative issues about validity and reliability of data gathered, resulting to a smaller number of motivational factors that passed the test. Related to this error is the failure to prequalify potential sampling area, which impaired the correct implementation of the sampling method (Iliffe, Wilcock, and Haworth, 2006). The second error involved the failure to clarify and verify information that respondents provided, casting doubts on the reliability of such information to the study. These two errors provide evidence that any error in sampling design and its implementation would result into a serious impairment of the study conducted. Indications show that the authors of these studies had been aware of these errors, and still insisted on proceeding with the study. To correct this problem in the future, researchers must response actively to any threats to the integrity of the research outcome, and correct observed problems right away.
Reference List
AcaStat (2014) “Pearson’s Product Moment Correlation Coefficient.” Applied Statistics
Handbook. Available at: http://www.acastat.com/Statbook/correlation.htm
Anderson DR (2001). “The Need to Get the Basics Right in Wildlife Field Studies.” Wildlife
Society Bulletin 29(4): 1294-1297. Available at: http://web.unbc.ca/~michael/courses/413/Anderson_JWM_2001.pdf
Bowling A (2005) “Mode of Questionnaire Administration Can Have Serious Effects on Data
Quality.” Journal of Public Health 27(3): 2891-291. DOI: 10.1093/pubmed/fdi031.
Boyatzis RE (1998) Transforming Qualitative Information: Thematic Analysis and Code
Development. Thousand Oaks, CA: Sage Publications; cited in: Braun V and Clarke V (2006) “Using Thematic Analysis in Psychology.” Qualitative Research
in Psychology 3(2): 77-101. DOI: 10.1191/1478088706qp063oa.
Braun V and Clarke V (2006) “Using Thematic Analysis in Psychology.” Qualitative Research
in Psychology 3(2): 77-101. DOI: 10.1191/1478088706qp063oa.
Burston PL and Stichler JF (2010) “Nursing Work Environment and Nursing Care: Relationship
among Motivational Factors. Journal of Advanced Nursing 66(8): 1819-1831. DOI: 10.1111/j.1365-2648.2010.05336.x
Cohen J and Cohen P (1983) Applied Multiplied Regression/Correlation Analysis for the
Behavioral Sciences, Second ed. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.
Coyne IT (1997) “Sampling in Qualitative Research. Purposeful and Theoretical Sampling;
Merging or Clear Boundaries? Journal of Advanced Nursing 26: 623-630.
De Jong PF (1999) “Hierarchical Regression Analysis in Structural Equation Modeling.”
Structural Equation Modeling 6(2): 198-211. Available in: http://dare.ubvu.vu.nl/bitstream/handle/1871/33537/118530.pdf?sequence=1
Food and Drug Administration (2010) “Exception from Informed Consent for Studies Conducted
in Emergency Settings: Regulatory Language and Excerpts from Preamble – Information Sheet.” U.S. Food and Drug Administration 4 November 2010 (last updated). Available at: http://www.fda.gov/RegulatoryInformation/Guidances/ucm126482.htm
Heckert A (2003) “Leven Test” Data Plot 1. Available at:
http://itl.nist.gov/div898/software/dataplot/refman1/auxillar/levetest.htm
Holloway I and Todres L (2003) “The Status Method: Flexibility, Consistency and Coherence.
Qualitative Research 3(3): 345-357; cited in: Braun V and Clarke V (2006) “Using Thematic Analysis in Psychology.” Qualitative Research in Psychology 3(2): 77-101. DOI: 10.1191/1478088706qp063oa.
Iliffe S, Wilcock J, and Haworth D (2006) “Obstacles to Shared Care for Patients with Dementia:
A Qualitative Study.” Family Practice 23: 353-362. DOI: 10.1093/fampra/cmi116.
Mahoney J (2001) “Beyond Correlational Analysis: Recent Innovations in Theory and Method.”
Sociological Forum 16(3): 575-593. DOI: 10.1023/A:1011912816997.
Patton MQ (1990) Qualitative Evaluation and research Methods, Second Ed. Newbury,
California: Sage Publications; cited in: Coyne IT (1997) “Sampling in Qualitative Research. Purposeful and Theoretical Sampling; Merging or Clear Boundaries? Journal of Advanced Nursing 26: 623-630.
Stamm BH, Higson-Smith C, Hudnall AC and Stamm HE IV (2014) “Professional Quality of
Life Scale (ProQOL).” Statisticssolution.com. Available (on 17 November 2014) at: http://www.statisticssolutions.com/professional-quality-of-life-scale-proqol/
Stanton JM, Balzer WK, Smith PC, Parra LF and Ironson G (2001) “A General Measure of
Work Stress: The Stress in General Scale.” Educational and Psychological Measurement 61(5): 866-888. DOI: 10.1177/00131640121971455.
Statistics Solutions (2009) “Analysis of Variance. Statisticalsolutions.com. Available at:
http://www.statisticssolutions.com/analysis-of-variance/
Tourangeau A (2006) “Psychometric Properties of the McCloskey Mueller Satisfaction Scale”
(Conference Presentation). WIN Nursing Research Conference. Available at: http://www.nursinglibrary.org/vhl/handle/10755/157881
Weiers RM (2014) Introduction to Business Statistics, 7th Ed. Singapore: Cengage Learning
Asia Pte Ltd.
Wu Y, Larrabee JH and Putman HP (2006) “Caring Behaviors Inventory: A Reduction of the 42-
Item Instrument.” Nursing Research 55(1): 18-25. Available at: http://journals.lww.com/nursingresearchonline/Abstract/2006/01000/Caring_Behaviors_Inventory__A_Reduction_of_the.3.aspx
Yankelevich M, Broadfood A, Gillespie JZ, Gillespie MA and Guidroz A (2011) “General Job
Stress: A Unidimensional Measure and Its Non-Linear Relations with Outcome Variables.” Stress and Health 0(0): 000-000. DOI: 10.1002/smi.1413.
