Introduction
Part I of the organizational change plan focused on the; aims of the EMR project, identified the factors potentiating the need for the EMR system, organizational and individual barriers to the planned change, and the external and internal resources available to support the implementation of the EMR system. The factors influencing the proposed change as well as those influencing the organization’s readiness for the change were also identified. Part I concluded with the application of Kurt lewin’s change model to the change process.
Meanwhile, the change strategies to be utilized in implementing the change as far as monitoring of the project and communication of issues that may arise during the go-live phase project are concerned were addressed in part II of the organizational change plan. Further, part II identified the organization’s related workflows and their potential impact on the proposed change.
Part III of the organizational change plan will elaborate on the process for determining the effectiveness of the EMR system once it has been implemented. Further, it will analyze the various outcome measurement strategies related to the EMR system. Ultimately, the processes for evaluating quality, cost and satisfaction outcomes will be discussed.
Effectiveness of the EMR
Evaluation of the effectiveness of the EMR system will be done annually throughout the implementation phase and 5-yearly after completion of the project. The project team will be mandated to conduct these evaluations. Effectiveness of the EMR system will be determined via a systematic process that will evaluate the effectiveness of EMR in regard to a host of individual outcomes relating to patient care, record-keeping, reporting and accessibility of patient information amongst others. The evaluation strategy will borrow from aspects of the CDC evaluation generic evaluation protocol for EMR systems. A number of key goals to be achieved by the EMR system for each expected outcome will have been identified by the steering committee (PEPFAR, 2007). Further, it will identify the attributes to be measured, components of the attributes to be evaluated as well as the method to be used in evaluating the various attributes for each identified outcome as shown in the example for outcomes related to record-keeping in the table below.
Evaluation of the effectiveness of EMR in record-keeping
- Attribute
- Component to be measured
- Method
- Indicator
- Usefulness
Is the EMR system able to identify issues with data entered?
Survey of staff
Observation
% of staff who will feel that the EMR system is adequate as far as identification of issues is concerned.
It will be noted whether the EMR system is able to detect errors that may occur during data entry.
Simplicity
Whether the staffs believe that data entry into the EMR system is easy.
Staff survey
% of staff who believes that entry of data into the EMR system is easy.
Flexibility
Whether the EMR system can be modified when entering data.
Observation
EMR system modifiable or not modifiable.
Data quality
Whether data is entered correctly.
Accuracy of reported parameters such as BMI and age.
Availability of a system to do control checks of entered data.
Observation
Staff survey
Absence or presence of system of control check for data.
% of data entered correctly in comparison to the paper system.
% of data not captured during data entry in comparison to the paper based system.
Stability
How frequently the computers malfunction every week.
How often the computers are unavailable for use due to power shortages every week.
Staff survey
% of staff who rely totally on the EMR system.
Average number of times the computers malfunction in a week.
Average number of power shortages in a week.
Timeliness
The average length of time it takes to enter patient data.
Survey of staff
Time-flow analysis
% of staff who think they the time available is enough for data entry into the EMR system.
Average time taken to enter data per chart.
Acceptability
The perception of the members of the staff in regard to the EMR system.
Staff survey
Percentage of staff who think that the EMR system suits their needs.
% of patient records entered into the EMR system.
Source: CDC, UCSF: Evaluating Electronic Patient Monitoring Systems for HIV Treatment and Care: Guidance
Document (http://www.globalhivevaluation.org/media/doc/EPMSEvaluationGuidance.doc) (as cited in PEPFAR, 2007, pp.56-57).
Outcome measurement strategies
Observational, qualitative and quantitative designs will comprise the major outcome measurement strategies which will be used in the evaluation phase of the EMR project (PEPFAR, 2007). Qualitative data will be gathered via questionnaires and EMR usage parameters which will be collected on a continuous basis by the remote monitoring software. Quantitative data analysis will be applied because it will minimize biases that may result from confounding between variables which will be eliminated via multivariate modeling during data analysis (Kaelber et al., 2005).
Data from the analysis of identified EMR outcomes will be compared with data of the same outcomes analyzed from the pre-EMR phase. For example, for patient cycle times, number of medication errors or instances of missing patient records. Observation on the other hand will require very few resources and will also rely on pre-existing data for comparison purposes (PEPFAR, 2007). Qualitative data will be collected via questionnaires and observation and will subsequently be analyzed via qualitative synthesis (Kaelber et al., 2005).
Determination of quality, cost and satisfaction outcomes
Quality of patient care will be determined by evaluating and comparing the incidences of medical errors prior and after implementation of the EMR. Meanwhile, the quality of data collected will be evaluated through control checks and observation to determine the accuracy of reported parameters such as BMI and age as well as the correctness of data entered (PEPFAR, 2007).
Satisfaction as an outcome will be evaluated via a web-based survey of the primary users of EMR including physicians, nurses, laboratory technicians, nutritionists, physiotherapists and patients (Samoutis et al., 2007). The survey will be conducted one-year following the implementation of the EMR and will be repeated yearly throughout the implementation of the EMR. Purposive sampling will be used to determine the study sample (Kaelber et al., 2005). Satisfaction related attributes to be evaluated using the web-based survey will be identified through a comprehensive literature review and will comprise level of satisfaction in regard to EMR training, functionality of EMR systems both software and hardware, fostering better patient care, detection of errors amongst others.
Participants of the study will be required to rate their level of satisfaction on the identified satisfaction outcomes on a likert-scale of 1-5 with one being very dissatisfied and 5 very satisfied (Kaelber et al., 2005). Data from the survey will be analyzed via SPSS and the results disseminated to all the stakeholders for appropriate action.
The cost of the EMR project will be determined using the Excel spreadsheet developed by Valancy (2002) for estimating the long-term costs of an EMR system. The costs of the EMR systems will be determined annually and compared to the original projections made at the beginning of the project. Details on the costs of the various items will be obtained from the proposal prepared by the EMR vendor and further clarifications sought from the vendor when the need arises. The Valancy (2002) spreadsheet captures information pertaining to vendor costs relating to software, interface and training; total costs of hardware such as those of database servers, printers, laptops and desktop computers; other costs associated with an EMR project such as remodeling and wiring; costs for remuneration of the project manager and the project team and staff training costs.
Conclusion
In conclusion therefore, evaluation of the effectiveness of the EMR system will be based on measurements of individual attributes like usefulness of the EMR system related to the goals of the EMR system in regard to components of patient care, record keeping amongst a host of others. Qualitative, quantitative and observational measurement strategies will be used to measure the outcomes of the EMR project following its implementation. The level of satisfaction with the EMR project will be determined through a web-based survey of the various stakeholders. The cost of the project will be determined via the Valancy (2002) excel spreadsheet for determining the cost of an EMR project. Quality of patient care on the other hand will be measured via the determination and subsequent comparison of the incidences of medical errors prior to and after implementation of the EMR project. Quality of data will be evaluated via observational and check controls that will determine the correctness of data entered as well as its level of accuracy.
References
Kaelber, D., Greco, P. & Cebul, R.D (2005). Evaluation of a Commercial Electronic Medical Record (EMR) by Primary Care Physicians 5 Years after Implementation. AMIA Annu
Symp Proc., 1002. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/20090083.
PEPFAR (2007).Developing an Electronic Medical Record (EMR) system implementation plan. Retrieved from
http://www.healthmetricsnetwork.info/.../fetch.php?...emr_implementation.
Samoutis, G., Soteriades, E.S., Kounalakis, D.K., Zachariahdou,T., Philalithis, A. & Lionis, C. (2007). Implementation of an electronic medical record system in previously computer-naïve primary care centers: a pilot study from Cyprus. Informatics in Primary Care, 15(4), 207-216.Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/20090083.
Valancy, J. (2002). How much will that EMR system really cost? Fam Pract Manag., 9(4).Retrieved from http://www.aafp.org › ... › Journals › fpm.
