The Introduction of Computerized Physician Order Entry and Change Management in a Tertiary Pediatric Hospital
http://www.100md.com
《小儿科》
Surgery Quality and Care Management Cardiology, Children’s Hospital of Pittsburgh, Pittsburgh, Pennsylvania
ABSTRACT
Objectives. The objectives of this review were to document the introduction of computerized physician order entry (CPOE)-centered changes in an academic tertiary care center and to review the CPOE-focused literature.
Design. We performed a systematic literature review of CPOE-related articles indexed on Medline, with particular emphasis on pediatric applications. We focused our commentary around the concepts involved in the implementation process at a tertiary pediatric hospital.
Results. In 2001, the Children’s Hospital of Pittsburgh (CHP) embarked on the process of CPOE design and implementation. We determined that CPOE is a tool for improving pediatric care. The CPOE implementation process is more than a technologic change; it involves an organizational cultural transformation. Although the complete transition to CPOE was little more than 1 year ago, CHP has overcome the typical obstacles of CPOE implementation to begin to realize its many benefits. The early success of CHP was achieved by creating a realistic, positive, work environment, which fostered hospital-wide participation and integration.
Conclusion. CPOE is an invaluable resource for supporting patient safety in health care settings. The successful implementation of CPOE requires a paradigm shift in hospital policies and processes.
Key Words: computerized physician order entry patient safety medical errors adverse events
Abbreviations: CHP, Children’s Hospital of Pittsburgh ADE, adverse drug event IOM, Institute of Medicine CPOE, computerized physician order entry
Heightened focus on improving health safety standards developed with the 1999 report by the Institute of Medicine (IOM), To Err Is Human, which estimated that between 44000 and 98000 persons per year die as a result of adverse events in hospitals.1 The landmark report by the IOM noted that computerized physician order entry (CPOE) is a crucial safety measure and should be implemented widely in all US hospitals. Since this recommendation, only a handful of US hospitals have instituted partial or full implementation of CPOE.2 Some of the factors limiting the widespread implementation of CPOE are internal institutional resistance, considerable financial costs, and technology limitations.3–6
CPOE implementation is much more than an information technology change. Rather, CPOE involves a major health care delivery paradigm shift in administrative and clinical policies and functions. Administratively, as noted by a leading CPOE expert group, "successful CPOE is not a technology implementation but a redesign of a complex clinical process, integrating the technology at key points to enhance and optimize ordering decisions."5 In other words, a paperless trail that makes sense from inventory management to frontline notification of critical drug orders must be designed and checked. Clinically, the introduction of CPOE requires a new level of integration among all aspects of the hospital. In addition, CPOE algorithms can be designed to move clinicians from expert opinion-based to evidence-based clinical decision-making through computer-supported knowledge management.5,6 For example, antibiotic choices can be tailored through the CPOE environment. While ordering antibiotics, clinicians can view the appropriate treatment guidelines and references online during the ordering process, which can assist them in making better therapeutic decisions.
Hallmarks of CPOE implementation include a high level of leadership, widespread employee commitment to the project, availability of resources, access to technology, comprehensive training, and communication.3,5–8 This review presents the pertinent features of CPOE and illustrates the importance of these factors by examining the implementation process at a tertiary pediatric hospital. The case study depicts the changes during the prelive, live, and postlive time periods (Fig 1) and outline the CPOE implementation strategy and the management of organizational change at the Children’s Hospital of Pittsburgh (CHP).
CPOE BACKGROUND
CPOE Design
In response to the IOM report, a consortium of Fortune 500 companies formed the Leapfrog Group, with the goal of promoting greater safety standards in hospitals. The IOM and the newly formed Leapfrog Group identified systems such as CPOE as one of the important technologies for reducing adverse events and promoting standardization in health care institutions.2,9 In pediatric settings, the CPOE effects are very relevant, as reported in a 2001 pediatric study that focused on adverse drug events (ADEs). Kaushal et al10 reported that, among nearly 11000 medication orders, there were 5.7% medication errors, 1.1% potential ADEs, and 0.24% definite ADEs. They also determined that a significant increase in potential ADEs occurred in NICUs and, more importantly, the majority of potential ADEs occurred at the ordering stage; ADEs included incorrect dosing, antibiotics, and intravenous medications. Kaushal et al10 concluded that a CPOE system could have prevented 93% of the potential ADEs. Many other reports offered compelling evidence for the potential benefits of CPOE in both adult and pediatric settings.
Fundamentally, CPOE is a system in which physicians enter orders directly into a computer interface instead of writing them out on paper. The actual utility of CPOE is in the capacity for specific edits or alerts applied automatically by the system. The algorithms use current specific patient information (eg, weight or allergies) to prevent possible errors.2–6 In some instances, the drug dose is calculated automatically on the basis of the current patient weight. The system is designed to augment physicians’ decision-making process by providing edits and relevant information at the time of order entry.2,5,11
The introduction of a CPOE system requires a shift in everyday practices of all members of a health care team. Change is daunting for many physicians, nurses, and health administrators, which is why a clear understanding of CPOE and its benefits, possible drawbacks, and obstacles is necessary. An institution must overcome these barriers for a successful transition. Organizational change is a cyclical process that requires perpetual evaluation. CPOE implementation is no different, and hospitals should anticipate this requirement and plan for periodic previews and reviews of system upgrades (Fig 2). Figure 2, adapted from the software life cycle, provides a model for the "life cycle" of organizational change. First the leaders analyze the needs and goals of the organization. Then the leadership team assesses the risks and benefits and identifies the necessary resources for a successful change. Next, the design team details the project specifications and crafts a coordinated plan. The plan is executed and the results are validated and compared with stated objectives. Subsequently, the leadership team evaluates the overall project impact, reassesses organizational needs, and starts the process again.
Many issues affect the deployment of CPOE throughout a health care institution. User satisfaction is a major determinant in the success of the new system. Several studies have analyzed these factors and determined that most user satisfaction is correlated strongly with end-user efficiency, and satisfaction ultimately affects the long-term success of CPOE.12 Therefore, identifying barriers to user satisfaction in general and points of contention in a particular institution is an important exercise in preparation for CPOE. Barriers are linked frequently to poor preconceptions held by health care providers regarding the computer order system.13 Common fears of CPOE held by physicians include the following: (1) CPOE will "deintellectualize" care with protocols; (2) CPOE will promote "cookbook" medicine; (3) CPOE will increase time at computers at the expense of patient interactions; (4) edits (alerts) will interrupt workflow and thought flow; and (5) cultural traditions and practice routines will be altered.3,6,11,13–16 To combat such negative notions and obstacles to implementation, early and consistent clinician involvement is critical in the development and maintenance of CPOE.5,13
CPOE Efficiency
Most CPOE benefits include process improvement, cost-conscious decision-making, clinical decision support, and efficiency.3,5,13,17–19 Time efficiency incorporates all of these factors and is a high priority in hospitals.20 In fact, often user satisfaction is correlated strongly with the efficiency aspects of CPOE systems, rather than the particular technologic innovations or decision-support features of a particular system.11,12,21 For instance, clinicians immediately recognize the value of having laboratory reports and current medications on one computer screen as they are making decisions, but the fact that it takes a tremendous amount of software manipulation to make this happen does not matter to them. Multiple studies indicate that CPOE decreases the overall time required to process orders, compared with traditional, paper-based methods. Consequently, physicians have more time to spend on unique aspects of a particular case and student or resident education.3,18,22 Furthermore, CPOE allows the flexibility for orders to be entered from essentially any location in the hospital, which again promotes work efficiency and unaltered workflow.6 One impressive example of the time-saving possibilities of CPOE systems came from a study involving information retrieval. The average time required for a computerized antibiotics management program to retrieve the same information as an infectious disease specialist was 3.5 seconds, compared with 14 minutes for the specialist.22 Also, with standardization of orders, calls between physicians and the pharmacy have been shown to be reduced drastically, which again saves time and money and prevents errors.23 These time-saving measures all address the fears of deintellectualization and excess time spent at a computer, at the loss of direct patient care.
CPOE Effects on Workflow
Studies claim that CPOE interrupts workflow and alters practices, including multitasking, decision support, and error prevention. An important underlying theme emphasizes the intrinsic limitations of human knowledge, which substantiate the use of clinical decision-support programs and the standardization of ordering algorithms that augment existing expert knowledge.3,24–26 In reality, CPOE systems are meant to enhance clinicians’ decision-making and not to control or to enforce decisions.14,24,27,28
As many studies suggest, the CPOE threat of "cookbook" medicine is unfounded and CPOE scarcely alters established practice traditions. The positive aspect is that clinicians, armed with more information, can spend more time on problem-solving and direct patient care. With less reliance on rote memorization, which can lead to sensory or factual overload, clinicians can provide more error prevention and consistent prescribing patterns. In a study involving an antibiotics management program, the number of drug allergy alerts was reduced from 405 to 87 and the number of adverse events decreased from 28 to 4 over time.22 In a previous study performed by the same investigators, a survey indicated that physicians applied knowledge from CPOE to similar patients, which suggests that the clinicians "learned" from the CPOE system.28 Early in the CHP experience, we demonstrated similar decreases in serious ADEs.29 These studies provide additional evidence for many of the positive outcomes of CPOE. Furthermore, study results offer cogent evidence for constructing project goals for future CPOE implementation.
Technology Integration
Although there are many fears regarding CPOE, a smooth transition can be achieved with proper orientation and implementation strategies. Although a primary consideration in the development of CPOE is the technologic requirements of the system, clearly human considerations and benefits, such as improved workflow efficiency, quality care, error prevention, and enhanced clinical decision-making, are crucial to the adoption of CPOE. Institutional leaders and managers must consider the following questions. (1) Is the CPOE software compatible with current computer systems and resources (2) What is the complexity of the system (3) Is the introduction of the system feasible in the current environment (4) Should we implement a packaged product or develop a "home-grown" system6,19,30 For instance, a hospital system may have an established laboratory information management system that is the industry standard, and the leaders must consider the costs of "scrapping" the laboratory system and building another laboratory information system unique to the CPOE system. Similar considerations arise with accounting, radiology, and pharmacy systems that may be in place already. The needs and requirements of the organization determine the answers to these questions, which should be reflected in the operational strategy for the project.31
CHP IMPLEMENTATION ISSUES AND STRATEGIES
Development
In 2002, the CHP launched an effort to implement a hospital-wide introduction of CPOE. The CHP system went "live" in October 2002, but the preparation process started 1 year before the first computerized order was placed. Although the intense preparation seemed exhaustive, it was critical to the success of the CPOE phase-in at CHP. CPOE, like many other global organizational changes, can be uncomfortable and onerous at times, but the transformation is justified by the eventual outcome of improving patient safety and health care quality.
In the introductory phases of CPOE implementation at CHP, an information technology firm introduced system components, past system integration issues, and, most importantly, the promise of a customized CPOE system to meet the needs of a pediatric hospital. The entire process started >1 year before the actual live implementation, through organization of stakeholders, leadership workshops, committee formation, employee education, and readiness assessments. From the outset, involvement in the project included a wide range of hospital personnel, from top executives to frontline providers. Vertical integration within the hospital project team set the stage for the successful hospital-wide implementation.
The tenets of the CPOE implementation strategy included evaluating environments, reducing risks, and continuing maintenance. When practice environments are evaluated, identification of potential trouble areas is useful. For example, the CHP respiratory department already used a computer system for data management. Therefore, the new CPOE environment requires integration between the old department laboratory and administrative data system and the CPOE system. Another factor to consider in the analysis of the current setting is the context in which orders are placed. For instance, in the PICUs, the volume of orders, including a substantial number of verbal orders, posed possible barriers to a smooth CPOE transition. Therefore, an upgraded policy regarding verbal orders and communication with ICU stakeholders before the implementation of CPOE were crucial to the success of the new system. With careful analyses, new software and technology supporting system application risks can be reduced.31 Lastly, software maintenance is incredibly important in the overall project success. Common software maintenance factors include corrective, adaptive, and perfective changes.31 Corrective changes refer to the identification and elimination of computer system errors; adaptive responses involve the accommodation of the computer system to organizational changes; and perfective maneuvers involve the ability to modify the computer program over time, with improved efficiency in the system.31 All of these factors are important in constructing and maintaining a CPOE design and implementation strategy.
Leadership Orientation
The first integrated gathering at CHP was a leadership workshop conducted by the hospital leadership and the CPOE firm. The goal of the workshop was to provide background on information technology integration, to discuss common problems related to information technology changes, and to allow participants to organize their thoughts regarding the goals of CPOE implementation. The workshop message centered on the correlation between implementation success and change management. The facilitators cautioned leaders not to focus only on technology-centered changes. The 3 goals of this session were for participants (1) to articulate goals and objectives that this project would enable; (2) to identify challenges or lessons learned from recent enterprise-wide information technology changes; and (3) to identify design activities required for transformation. Major stakeholders used this exercise to focus the agenda and to form a committee structure. This type of meeting was ceremonial in some ways, but it served as an identifiable starting point for the multidisciplinary team.
Project Governance
Committee assignments included policy development, workflow changes, specific end-user requirements, and guidelines for rule creation and training (Fig 3 and Table 1). One key committee, the physicians’ advisory committee, was formed 1 year before the live date and continued after implementation. The committee consisted of clinicians (largely physicians but also pharmacists) and technical support staff members. The committee’s overall strategy was aimed at facilitating physician transition and providing vigilance regarding system-wide medical issues. The physician-led committee catered to physician workforce needs and customized the ordering software for pediatric hospital requirements (eg, weight-based dosing). Recommendations from the physicians’ advisory group were implemented and monitored by a physician leader, with the assistance of the technology support staff, on a daily basis. Recommendations included downtime protocols, error reporting, software updates, and pharmacy protocols. Physician access to system design changes on an on-going basis is critical to the continued success of the CPOE system.
Rules Management
A major function of the physicians’ advisory committee is rules management. A rule is a computer algorithm designed by the programmers to alert or to remind clinicians about a particular clinical parameter. A synchronous order entry rule fires in response to a triggering event, ie, the order. For example, when an order is placed for an aminoglycoside (the trigger), a rule can fire that alerts the user to the patient’s elevated creatinine level. Another synchronous rule can fire to recommend best practices and to remind the user to order a serum creatinine assay if there is no laboratory result from the previous 48 hours. Similarly, a rule can fire that alerts the user to order a peak or trough blood level assay at the time of the medication order. An asynchronous order entry rule is triggered in response to an event that occurs after the order has been written and executed. For the patient described above who is receiving an aminoglycoside, the receipt of an elevated creatinine blood level (the trigger here) would fire an asynchronous rule that notifies a caregiver that an abnormal blood level has been reported. The clinician is thus alerted to modify the existing order because of the new occurrence of the elevated creatinine level. There can be many different kinds of events that cause a rule to fire. In addition to when an order is placed, triggers can occur when a patient is admitted, transferred, or discharged. Some of our rules include reminders about standard medication times, pregnancy indicators when a radiograph is being ordered, and a rule that requires documentation of weight and allergies before any medication order can be written. The goal of rules management is to create safety reminders that protect patients from the most deleterious outcomes while not overwhelming end-users with insignificant warnings. Good rules fire occasionally, under appropriate circumstances, and when there is the greatest potential for impact on patient treatment. The physician advisory committee must anticipate the behavior modifications associated with implementing rule sets and create support tactics that support end-users. One tactic used by CHP is to test rules against the current ordering environment and to demonstrate the impact of the new rules. Finally, regular maintenance of rules is an important task for the physicians’ advisory committee.
Another committee central to planning the CPOE integration was the process redesign committee. The overall goal was to incorporate CPOE into the everyday hospital inpatient work environment. The process redesign committee examined inpatient unit processes and workflow issues. It also identified a number of activities that would be changed by CPOE, as well as some that would be eliminated. Ultimately, many of the changes identified were related to nursing care; therefore, the impact of CPOE on nursing workflow would be substantial. For example, the CPOE system was designed to link nursing notations with orders, so that items such as vital signs (eg, patient weight) and other care notes could be observed by all care providers through the same CPOE patient screen. Such integration reduced the amount of paperwork but also altered the way the staff functioned. Therefore, the committee not only needed to pinpoint changes in function attributable to CPOE but also coordinated education activities about workflow-related changes for the nursing staff.
Readiness Assessment
After several months of committee work, a group was formed to create, to disperse (7 months before the live date), and to analyze a readiness assessment survey. The objective was to determine the committee effectiveness and problem areas. Four hundred five employees responded to the CPOE survey, generating trends of opinion regarding personal preference, job impact, effectiveness of training, change imperatives, and perceived rewards or recognition attached to implementation. A second survey, titled stakeholder analysis, was given to a smaller control group (n = 46) in various departments (eg, pharmacists, residents, respiratory therapists, health unit coordinators, and nurses). Recognizing stakeholder concerns is key to preventing many obstacles to transition. The survey revealed some insights into the change process but, surprisingly, most respondents were satisfied with the preparation; therefore, the work continued as planned previously. In summary, the evaluation/reevaluation process through surveys is key to understanding and gaining participation from critical participants in the organization.
Advertisement
Near the end of the prelive period, an extensive network of committees was formed explicitly for the purpose of supporting and easing the anxiety of change. Figure 4 shows a detailed transition map, facilitated by the integration of all committees. The next important phase in the introduction of CPOE was advertising the go-live event and the critical changes that would occur. The committee work was relayed to the general hospital through newsletters, e-mails, and posters. General employee meetings were held once per week for 1 month before the live date. These meetings reviewed CPOE implementation status, departmental readiness, go-live strategy, and organizational downtime procedures. The open forum allowed important questions to be answered, so that all hospital employees were informed and knowledgeable regarding CPOE.
GO-LIVE PHASE (OCTOBER 2002)
Probably the most important and fundamental activity necessary for a smooth transition to CPOE is staff CPOE training. Training systems must be widely accessible and encompass a comprehensive approach to system functioning. In other words, training programs should match the end-user profile and introduce end-users to real but hypothetical problems. Poor training may lead to a lack of system understanding, which can result in frustration, poor acceptance, and a lack of full utilization.2,5,6,16,32 The CPOE training team, led by the education department, was formed in the prelive period and continued into the live and postlive periods. Training strategies were designed for optimal effectiveness with minimal work interference. The team set a lofty goal of educating 3000 end-users before the live date. Formal instruction was initiated 3 months before the live date, when the go-live system was essentially operationally complete. Early training proved to be a useful support tactic to quench change anxiety. Users had ample time to practice using the system and to gain comfort before the stressful live period.
One key feature of the training strategy was to select, to train, and to acknowledge a cadre of super-users in the hospital. A super-user is an individual who is expected to know more about the CPOE system and to assist regular end-users with problems on the floors. Training was intended to provide familiarity and guidelines for use of the system. The pretraining regimen included hands-on, in-house sessions, after which a training manual was distributed. The manual included a user guide, a quick reference book, and a CD-ROM for take-home training in order entry and viewing. The user guide was created by the hospital and software vendor specifically for the CHP CPOE system. The very extensive guide provided an overview of the system, common troubleshooting solutions, CD-ROM installation procedures, and detailed sections regarding specific uses of the system. Chapters included patient access lists, multi-patient task lists, Kardex, documenting vital signs, allergies, results viewing, physician in-box, batch charge entry, and a charge viewer. Each section was accompanied by pictures of pertinent viewing screens and instructions on use. The guide also allowed for interaction with the CD-ROM through viewing scenarios and assessments. Furthermore, every individual was given an identification number and password, to ensure continued, flexible, monitored training. During the live period, hands-on, in-house training was available 24 hours per day, 7 days per week. By the time go-live was reached, 2261 clinical staff members, including 493 physicians, had been trained successfully; there were 223 super-users.
Extensive training in the prelive and live periods is a beneficial method for reducing anxiety. To continue with this support in the postlive period, attention to necessary modifications and prompt responses to complaints are vital. The leader of the physicians’ advisory committee continued to be an important clinical liaison during the postlive period. Everyday concerns regarding CPOE were directed to this physician, who was able to facilitate modifications if necessary. This provided a tangible point of action for employees and aided in supporting change.
HOW CPOE HAS CHANGED "THE SYSTEM"
Successful CPOE design and implementation require strong executive and clinical leadership, with a focused but integrated committee structure. The role of executive and cabinet council leadership must be pervasive throughout the change process but should not overshadow and diminish the contributions of frontline personnel. The primary role of leadership is to articulate and to promote strategic change, to commit human and financial resources, to critique global decisions, and to ensure budget and timeline objectives. All committees should report their findings and disseminate the results to the group. Finally, problems should be categorized according to the degree of risk posed to the overall project timeline, to focus resources in solving the problem. Through the integration of all parties, the barriers to both development and diffusion of CPOE are reduced dramatically.
Analysis of the process of CPOE implementation at CHP provides a model for other institutions interested in CPOE. The described barriers to CPOE implementation were overcome, and the initial benefits of CPOE are coming to fruition. CPOE introduction has created improved clinical integration and increased efficiency. Most importantly, however, safety indicators suggest that the most significant types of medical errors are decreasing.29 Figure 5 provides an example of the effects of CPOE on the clinical processes of CHP. The lessons learned from the entire CPOE implementation process are many; however, a few stand out as general guidelines for other institutions considering CPOE implementation. Table 2 provides summaries of the lessons learned, as well as points to avoid when undertaking CPOE implementation.
CONCLUSIONS
The beneficial hallmarks of CPOE include increased work efficiency and, most importantly, improved patient safety. However, realization of these benefits is the product of shifts in hospital culture and practices. Careful management of the systematic changes accompanying introduction of CPOE is crucial to successful CPOE implementation. This management includes integration of, and cooperation between, various hospital departments. CPOE is an important innovation that requires total organizational commitment and sustained support. Consequently, our patients appreciate the improved safety and care that CPOE supports.
FOOTNOTES
Accepted May 25, 2005.
No conflict of interest declared.
REFERENCES
Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Healthcare System. Washington, DC: National Academy Press; 1999
Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994;1 :108 –123
Metzger J, Turisco F. Computerized Physician Order Entry: A Look at the Vendor Marketplace and Getting Started. New York, NY: First Consulting Group; 2001:1–36
Drazen E, Kilbridge P, Metzger J, Turisco F. A Primer of Physician Order Entry. Oakland, CA: First Consulting Group; 2000:1–43
Doolan DF, Bates DW. Computerized physician order entry systems in hospitals: mandates and incentives. Health Aff (Milwood). 2002;21 :180 –187
Ahmad A, Teater P, Bentley TD, et al. Key attributes of a successful physician order entry system implementation in a multi-hospital environment. J Am Med Inform Assoc. 2002;9 :16 –24
Dick RS, Steen EB, Detmer DE, eds. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: National Academy Press; 1997
Soumerai SB, Avorn J. Crossing the Quality Chasm: A New Health Care System for the 21st Century. Washington, DC: National Academy Press; 2001
Kaushal R, Bates DW, Landrigan C. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285 :2114 –2120
Lee F, Teater J, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc. 1996;3 :42 –55
Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001;8 :499 –509
Massaro T. Introducing physician order entry at a major academic center, II: impact on medical education. Acad Med. 1993;68 :25 –30
Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med. 1996;124 :884 –890
Chisholm CD, Collison EK, Nelson DR, Cordell WH. Emergency department workplace interruptions: are emergency physicians "interrupt-driven" and "multitasking" Acad Emerg Med. 2000;7 :1239 –1243
Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003;326 :314 –318
Chisholm CD, Dornfeld AM, Nelson DR, Cordell WH. Work interrupted: a comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38 :146 –151
Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory care internal medicine practices. J Am Med Inform Assoc. 2001;8 :361 –371
Benson T. Why general practitioners use computers and hospital doctors do not, part 2: scalability. BMJ. 2002;325 :1090 –1093
Massaro T. Introducing physician order entry at a major academic medical center, I: impact of organizational culture and behavior. Acad Med. 1993;68 :20 –25
Tierney WM, Overhage JM, McDonald CJ, Wolinksy FD. Medical students’ and housestaff’s opinions of computerized order-writing. Acad Med. 1994;69 :386 –389
Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other anti-infective agents. N Engl J Med. 1998;338 :232 –238
Ogura H, Sagara E, Iwata M, et al. Online support functions of prescription audit in an integrated hospital information system. Med Inform (Lond). 1988;13 :161 –169
McDonald C. Protocol-based computer-reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295 :1351 –1355
McDonald C. Physician response to computer reminders. JAMA. 1980;244 :1579 –1581
Greenes R. Knowledge management as an aid to medical decision making and education: the Explorer-1 system. Medinfo. 1986;88 :895 –899
Kawahara NE, Jordan FM. Influencing prescribing behavior by adapting computerized order-entry pathways. Am J Hosp Pharm. 1989;46 :1798 –1801
Evans R, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP. Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154 :878 –884
Upperman JS, Staley P, Friend K, et al. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg. 2005;40 :57 –59
Dick RS, Steen EB, Detmer DE, eds. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: National Academy Press; 1997
Orthner HF, Blum BI. Implementing health care information systems. In: Orthner HF, Blum BI, eds. Implementing Health Care Information Systems. New York, NY: Springer-Verlag; 1989:3–21
Barnett GO. The application of computer-based medical-record systems in ambulatory practice. In: Orthner HF, Blum BI, eds. Implementing Health Care Information Systems. New York, NY: Springer-Verlag; 1989:85–99(Jeffrey S. Upperman, MD, )
ABSTRACT
Objectives. The objectives of this review were to document the introduction of computerized physician order entry (CPOE)-centered changes in an academic tertiary care center and to review the CPOE-focused literature.
Design. We performed a systematic literature review of CPOE-related articles indexed on Medline, with particular emphasis on pediatric applications. We focused our commentary around the concepts involved in the implementation process at a tertiary pediatric hospital.
Results. In 2001, the Children’s Hospital of Pittsburgh (CHP) embarked on the process of CPOE design and implementation. We determined that CPOE is a tool for improving pediatric care. The CPOE implementation process is more than a technologic change; it involves an organizational cultural transformation. Although the complete transition to CPOE was little more than 1 year ago, CHP has overcome the typical obstacles of CPOE implementation to begin to realize its many benefits. The early success of CHP was achieved by creating a realistic, positive, work environment, which fostered hospital-wide participation and integration.
Conclusion. CPOE is an invaluable resource for supporting patient safety in health care settings. The successful implementation of CPOE requires a paradigm shift in hospital policies and processes.
Key Words: computerized physician order entry patient safety medical errors adverse events
Abbreviations: CHP, Children’s Hospital of Pittsburgh ADE, adverse drug event IOM, Institute of Medicine CPOE, computerized physician order entry
Heightened focus on improving health safety standards developed with the 1999 report by the Institute of Medicine (IOM), To Err Is Human, which estimated that between 44000 and 98000 persons per year die as a result of adverse events in hospitals.1 The landmark report by the IOM noted that computerized physician order entry (CPOE) is a crucial safety measure and should be implemented widely in all US hospitals. Since this recommendation, only a handful of US hospitals have instituted partial or full implementation of CPOE.2 Some of the factors limiting the widespread implementation of CPOE are internal institutional resistance, considerable financial costs, and technology limitations.3–6
CPOE implementation is much more than an information technology change. Rather, CPOE involves a major health care delivery paradigm shift in administrative and clinical policies and functions. Administratively, as noted by a leading CPOE expert group, "successful CPOE is not a technology implementation but a redesign of a complex clinical process, integrating the technology at key points to enhance and optimize ordering decisions."5 In other words, a paperless trail that makes sense from inventory management to frontline notification of critical drug orders must be designed and checked. Clinically, the introduction of CPOE requires a new level of integration among all aspects of the hospital. In addition, CPOE algorithms can be designed to move clinicians from expert opinion-based to evidence-based clinical decision-making through computer-supported knowledge management.5,6 For example, antibiotic choices can be tailored through the CPOE environment. While ordering antibiotics, clinicians can view the appropriate treatment guidelines and references online during the ordering process, which can assist them in making better therapeutic decisions.
Hallmarks of CPOE implementation include a high level of leadership, widespread employee commitment to the project, availability of resources, access to technology, comprehensive training, and communication.3,5–8 This review presents the pertinent features of CPOE and illustrates the importance of these factors by examining the implementation process at a tertiary pediatric hospital. The case study depicts the changes during the prelive, live, and postlive time periods (Fig 1) and outline the CPOE implementation strategy and the management of organizational change at the Children’s Hospital of Pittsburgh (CHP).
CPOE BACKGROUND
CPOE Design
In response to the IOM report, a consortium of Fortune 500 companies formed the Leapfrog Group, with the goal of promoting greater safety standards in hospitals. The IOM and the newly formed Leapfrog Group identified systems such as CPOE as one of the important technologies for reducing adverse events and promoting standardization in health care institutions.2,9 In pediatric settings, the CPOE effects are very relevant, as reported in a 2001 pediatric study that focused on adverse drug events (ADEs). Kaushal et al10 reported that, among nearly 11000 medication orders, there were 5.7% medication errors, 1.1% potential ADEs, and 0.24% definite ADEs. They also determined that a significant increase in potential ADEs occurred in NICUs and, more importantly, the majority of potential ADEs occurred at the ordering stage; ADEs included incorrect dosing, antibiotics, and intravenous medications. Kaushal et al10 concluded that a CPOE system could have prevented 93% of the potential ADEs. Many other reports offered compelling evidence for the potential benefits of CPOE in both adult and pediatric settings.
Fundamentally, CPOE is a system in which physicians enter orders directly into a computer interface instead of writing them out on paper. The actual utility of CPOE is in the capacity for specific edits or alerts applied automatically by the system. The algorithms use current specific patient information (eg, weight or allergies) to prevent possible errors.2–6 In some instances, the drug dose is calculated automatically on the basis of the current patient weight. The system is designed to augment physicians’ decision-making process by providing edits and relevant information at the time of order entry.2,5,11
The introduction of a CPOE system requires a shift in everyday practices of all members of a health care team. Change is daunting for many physicians, nurses, and health administrators, which is why a clear understanding of CPOE and its benefits, possible drawbacks, and obstacles is necessary. An institution must overcome these barriers for a successful transition. Organizational change is a cyclical process that requires perpetual evaluation. CPOE implementation is no different, and hospitals should anticipate this requirement and plan for periodic previews and reviews of system upgrades (Fig 2). Figure 2, adapted from the software life cycle, provides a model for the "life cycle" of organizational change. First the leaders analyze the needs and goals of the organization. Then the leadership team assesses the risks and benefits and identifies the necessary resources for a successful change. Next, the design team details the project specifications and crafts a coordinated plan. The plan is executed and the results are validated and compared with stated objectives. Subsequently, the leadership team evaluates the overall project impact, reassesses organizational needs, and starts the process again.
Many issues affect the deployment of CPOE throughout a health care institution. User satisfaction is a major determinant in the success of the new system. Several studies have analyzed these factors and determined that most user satisfaction is correlated strongly with end-user efficiency, and satisfaction ultimately affects the long-term success of CPOE.12 Therefore, identifying barriers to user satisfaction in general and points of contention in a particular institution is an important exercise in preparation for CPOE. Barriers are linked frequently to poor preconceptions held by health care providers regarding the computer order system.13 Common fears of CPOE held by physicians include the following: (1) CPOE will "deintellectualize" care with protocols; (2) CPOE will promote "cookbook" medicine; (3) CPOE will increase time at computers at the expense of patient interactions; (4) edits (alerts) will interrupt workflow and thought flow; and (5) cultural traditions and practice routines will be altered.3,6,11,13–16 To combat such negative notions and obstacles to implementation, early and consistent clinician involvement is critical in the development and maintenance of CPOE.5,13
CPOE Efficiency
Most CPOE benefits include process improvement, cost-conscious decision-making, clinical decision support, and efficiency.3,5,13,17–19 Time efficiency incorporates all of these factors and is a high priority in hospitals.20 In fact, often user satisfaction is correlated strongly with the efficiency aspects of CPOE systems, rather than the particular technologic innovations or decision-support features of a particular system.11,12,21 For instance, clinicians immediately recognize the value of having laboratory reports and current medications on one computer screen as they are making decisions, but the fact that it takes a tremendous amount of software manipulation to make this happen does not matter to them. Multiple studies indicate that CPOE decreases the overall time required to process orders, compared with traditional, paper-based methods. Consequently, physicians have more time to spend on unique aspects of a particular case and student or resident education.3,18,22 Furthermore, CPOE allows the flexibility for orders to be entered from essentially any location in the hospital, which again promotes work efficiency and unaltered workflow.6 One impressive example of the time-saving possibilities of CPOE systems came from a study involving information retrieval. The average time required for a computerized antibiotics management program to retrieve the same information as an infectious disease specialist was 3.5 seconds, compared with 14 minutes for the specialist.22 Also, with standardization of orders, calls between physicians and the pharmacy have been shown to be reduced drastically, which again saves time and money and prevents errors.23 These time-saving measures all address the fears of deintellectualization and excess time spent at a computer, at the loss of direct patient care.
CPOE Effects on Workflow
Studies claim that CPOE interrupts workflow and alters practices, including multitasking, decision support, and error prevention. An important underlying theme emphasizes the intrinsic limitations of human knowledge, which substantiate the use of clinical decision-support programs and the standardization of ordering algorithms that augment existing expert knowledge.3,24–26 In reality, CPOE systems are meant to enhance clinicians’ decision-making and not to control or to enforce decisions.14,24,27,28
As many studies suggest, the CPOE threat of "cookbook" medicine is unfounded and CPOE scarcely alters established practice traditions. The positive aspect is that clinicians, armed with more information, can spend more time on problem-solving and direct patient care. With less reliance on rote memorization, which can lead to sensory or factual overload, clinicians can provide more error prevention and consistent prescribing patterns. In a study involving an antibiotics management program, the number of drug allergy alerts was reduced from 405 to 87 and the number of adverse events decreased from 28 to 4 over time.22 In a previous study performed by the same investigators, a survey indicated that physicians applied knowledge from CPOE to similar patients, which suggests that the clinicians "learned" from the CPOE system.28 Early in the CHP experience, we demonstrated similar decreases in serious ADEs.29 These studies provide additional evidence for many of the positive outcomes of CPOE. Furthermore, study results offer cogent evidence for constructing project goals for future CPOE implementation.
Technology Integration
Although there are many fears regarding CPOE, a smooth transition can be achieved with proper orientation and implementation strategies. Although a primary consideration in the development of CPOE is the technologic requirements of the system, clearly human considerations and benefits, such as improved workflow efficiency, quality care, error prevention, and enhanced clinical decision-making, are crucial to the adoption of CPOE. Institutional leaders and managers must consider the following questions. (1) Is the CPOE software compatible with current computer systems and resources (2) What is the complexity of the system (3) Is the introduction of the system feasible in the current environment (4) Should we implement a packaged product or develop a "home-grown" system6,19,30 For instance, a hospital system may have an established laboratory information management system that is the industry standard, and the leaders must consider the costs of "scrapping" the laboratory system and building another laboratory information system unique to the CPOE system. Similar considerations arise with accounting, radiology, and pharmacy systems that may be in place already. The needs and requirements of the organization determine the answers to these questions, which should be reflected in the operational strategy for the project.31
CHP IMPLEMENTATION ISSUES AND STRATEGIES
Development
In 2002, the CHP launched an effort to implement a hospital-wide introduction of CPOE. The CHP system went "live" in October 2002, but the preparation process started 1 year before the first computerized order was placed. Although the intense preparation seemed exhaustive, it was critical to the success of the CPOE phase-in at CHP. CPOE, like many other global organizational changes, can be uncomfortable and onerous at times, but the transformation is justified by the eventual outcome of improving patient safety and health care quality.
In the introductory phases of CPOE implementation at CHP, an information technology firm introduced system components, past system integration issues, and, most importantly, the promise of a customized CPOE system to meet the needs of a pediatric hospital. The entire process started >1 year before the actual live implementation, through organization of stakeholders, leadership workshops, committee formation, employee education, and readiness assessments. From the outset, involvement in the project included a wide range of hospital personnel, from top executives to frontline providers. Vertical integration within the hospital project team set the stage for the successful hospital-wide implementation.
The tenets of the CPOE implementation strategy included evaluating environments, reducing risks, and continuing maintenance. When practice environments are evaluated, identification of potential trouble areas is useful. For example, the CHP respiratory department already used a computer system for data management. Therefore, the new CPOE environment requires integration between the old department laboratory and administrative data system and the CPOE system. Another factor to consider in the analysis of the current setting is the context in which orders are placed. For instance, in the PICUs, the volume of orders, including a substantial number of verbal orders, posed possible barriers to a smooth CPOE transition. Therefore, an upgraded policy regarding verbal orders and communication with ICU stakeholders before the implementation of CPOE were crucial to the success of the new system. With careful analyses, new software and technology supporting system application risks can be reduced.31 Lastly, software maintenance is incredibly important in the overall project success. Common software maintenance factors include corrective, adaptive, and perfective changes.31 Corrective changes refer to the identification and elimination of computer system errors; adaptive responses involve the accommodation of the computer system to organizational changes; and perfective maneuvers involve the ability to modify the computer program over time, with improved efficiency in the system.31 All of these factors are important in constructing and maintaining a CPOE design and implementation strategy.
Leadership Orientation
The first integrated gathering at CHP was a leadership workshop conducted by the hospital leadership and the CPOE firm. The goal of the workshop was to provide background on information technology integration, to discuss common problems related to information technology changes, and to allow participants to organize their thoughts regarding the goals of CPOE implementation. The workshop message centered on the correlation between implementation success and change management. The facilitators cautioned leaders not to focus only on technology-centered changes. The 3 goals of this session were for participants (1) to articulate goals and objectives that this project would enable; (2) to identify challenges or lessons learned from recent enterprise-wide information technology changes; and (3) to identify design activities required for transformation. Major stakeholders used this exercise to focus the agenda and to form a committee structure. This type of meeting was ceremonial in some ways, but it served as an identifiable starting point for the multidisciplinary team.
Project Governance
Committee assignments included policy development, workflow changes, specific end-user requirements, and guidelines for rule creation and training (Fig 3 and Table 1). One key committee, the physicians’ advisory committee, was formed 1 year before the live date and continued after implementation. The committee consisted of clinicians (largely physicians but also pharmacists) and technical support staff members. The committee’s overall strategy was aimed at facilitating physician transition and providing vigilance regarding system-wide medical issues. The physician-led committee catered to physician workforce needs and customized the ordering software for pediatric hospital requirements (eg, weight-based dosing). Recommendations from the physicians’ advisory group were implemented and monitored by a physician leader, with the assistance of the technology support staff, on a daily basis. Recommendations included downtime protocols, error reporting, software updates, and pharmacy protocols. Physician access to system design changes on an on-going basis is critical to the continued success of the CPOE system.
Rules Management
A major function of the physicians’ advisory committee is rules management. A rule is a computer algorithm designed by the programmers to alert or to remind clinicians about a particular clinical parameter. A synchronous order entry rule fires in response to a triggering event, ie, the order. For example, when an order is placed for an aminoglycoside (the trigger), a rule can fire that alerts the user to the patient’s elevated creatinine level. Another synchronous rule can fire to recommend best practices and to remind the user to order a serum creatinine assay if there is no laboratory result from the previous 48 hours. Similarly, a rule can fire that alerts the user to order a peak or trough blood level assay at the time of the medication order. An asynchronous order entry rule is triggered in response to an event that occurs after the order has been written and executed. For the patient described above who is receiving an aminoglycoside, the receipt of an elevated creatinine blood level (the trigger here) would fire an asynchronous rule that notifies a caregiver that an abnormal blood level has been reported. The clinician is thus alerted to modify the existing order because of the new occurrence of the elevated creatinine level. There can be many different kinds of events that cause a rule to fire. In addition to when an order is placed, triggers can occur when a patient is admitted, transferred, or discharged. Some of our rules include reminders about standard medication times, pregnancy indicators when a radiograph is being ordered, and a rule that requires documentation of weight and allergies before any medication order can be written. The goal of rules management is to create safety reminders that protect patients from the most deleterious outcomes while not overwhelming end-users with insignificant warnings. Good rules fire occasionally, under appropriate circumstances, and when there is the greatest potential for impact on patient treatment. The physician advisory committee must anticipate the behavior modifications associated with implementing rule sets and create support tactics that support end-users. One tactic used by CHP is to test rules against the current ordering environment and to demonstrate the impact of the new rules. Finally, regular maintenance of rules is an important task for the physicians’ advisory committee.
Another committee central to planning the CPOE integration was the process redesign committee. The overall goal was to incorporate CPOE into the everyday hospital inpatient work environment. The process redesign committee examined inpatient unit processes and workflow issues. It also identified a number of activities that would be changed by CPOE, as well as some that would be eliminated. Ultimately, many of the changes identified were related to nursing care; therefore, the impact of CPOE on nursing workflow would be substantial. For example, the CPOE system was designed to link nursing notations with orders, so that items such as vital signs (eg, patient weight) and other care notes could be observed by all care providers through the same CPOE patient screen. Such integration reduced the amount of paperwork but also altered the way the staff functioned. Therefore, the committee not only needed to pinpoint changes in function attributable to CPOE but also coordinated education activities about workflow-related changes for the nursing staff.
Readiness Assessment
After several months of committee work, a group was formed to create, to disperse (7 months before the live date), and to analyze a readiness assessment survey. The objective was to determine the committee effectiveness and problem areas. Four hundred five employees responded to the CPOE survey, generating trends of opinion regarding personal preference, job impact, effectiveness of training, change imperatives, and perceived rewards or recognition attached to implementation. A second survey, titled stakeholder analysis, was given to a smaller control group (n = 46) in various departments (eg, pharmacists, residents, respiratory therapists, health unit coordinators, and nurses). Recognizing stakeholder concerns is key to preventing many obstacles to transition. The survey revealed some insights into the change process but, surprisingly, most respondents were satisfied with the preparation; therefore, the work continued as planned previously. In summary, the evaluation/reevaluation process through surveys is key to understanding and gaining participation from critical participants in the organization.
Advertisement
Near the end of the prelive period, an extensive network of committees was formed explicitly for the purpose of supporting and easing the anxiety of change. Figure 4 shows a detailed transition map, facilitated by the integration of all committees. The next important phase in the introduction of CPOE was advertising the go-live event and the critical changes that would occur. The committee work was relayed to the general hospital through newsletters, e-mails, and posters. General employee meetings were held once per week for 1 month before the live date. These meetings reviewed CPOE implementation status, departmental readiness, go-live strategy, and organizational downtime procedures. The open forum allowed important questions to be answered, so that all hospital employees were informed and knowledgeable regarding CPOE.
GO-LIVE PHASE (OCTOBER 2002)
Probably the most important and fundamental activity necessary for a smooth transition to CPOE is staff CPOE training. Training systems must be widely accessible and encompass a comprehensive approach to system functioning. In other words, training programs should match the end-user profile and introduce end-users to real but hypothetical problems. Poor training may lead to a lack of system understanding, which can result in frustration, poor acceptance, and a lack of full utilization.2,5,6,16,32 The CPOE training team, led by the education department, was formed in the prelive period and continued into the live and postlive periods. Training strategies were designed for optimal effectiveness with minimal work interference. The team set a lofty goal of educating 3000 end-users before the live date. Formal instruction was initiated 3 months before the live date, when the go-live system was essentially operationally complete. Early training proved to be a useful support tactic to quench change anxiety. Users had ample time to practice using the system and to gain comfort before the stressful live period.
One key feature of the training strategy was to select, to train, and to acknowledge a cadre of super-users in the hospital. A super-user is an individual who is expected to know more about the CPOE system and to assist regular end-users with problems on the floors. Training was intended to provide familiarity and guidelines for use of the system. The pretraining regimen included hands-on, in-house sessions, after which a training manual was distributed. The manual included a user guide, a quick reference book, and a CD-ROM for take-home training in order entry and viewing. The user guide was created by the hospital and software vendor specifically for the CHP CPOE system. The very extensive guide provided an overview of the system, common troubleshooting solutions, CD-ROM installation procedures, and detailed sections regarding specific uses of the system. Chapters included patient access lists, multi-patient task lists, Kardex, documenting vital signs, allergies, results viewing, physician in-box, batch charge entry, and a charge viewer. Each section was accompanied by pictures of pertinent viewing screens and instructions on use. The guide also allowed for interaction with the CD-ROM through viewing scenarios and assessments. Furthermore, every individual was given an identification number and password, to ensure continued, flexible, monitored training. During the live period, hands-on, in-house training was available 24 hours per day, 7 days per week. By the time go-live was reached, 2261 clinical staff members, including 493 physicians, had been trained successfully; there were 223 super-users.
Extensive training in the prelive and live periods is a beneficial method for reducing anxiety. To continue with this support in the postlive period, attention to necessary modifications and prompt responses to complaints are vital. The leader of the physicians’ advisory committee continued to be an important clinical liaison during the postlive period. Everyday concerns regarding CPOE were directed to this physician, who was able to facilitate modifications if necessary. This provided a tangible point of action for employees and aided in supporting change.
HOW CPOE HAS CHANGED "THE SYSTEM"
Successful CPOE design and implementation require strong executive and clinical leadership, with a focused but integrated committee structure. The role of executive and cabinet council leadership must be pervasive throughout the change process but should not overshadow and diminish the contributions of frontline personnel. The primary role of leadership is to articulate and to promote strategic change, to commit human and financial resources, to critique global decisions, and to ensure budget and timeline objectives. All committees should report their findings and disseminate the results to the group. Finally, problems should be categorized according to the degree of risk posed to the overall project timeline, to focus resources in solving the problem. Through the integration of all parties, the barriers to both development and diffusion of CPOE are reduced dramatically.
Analysis of the process of CPOE implementation at CHP provides a model for other institutions interested in CPOE. The described barriers to CPOE implementation were overcome, and the initial benefits of CPOE are coming to fruition. CPOE introduction has created improved clinical integration and increased efficiency. Most importantly, however, safety indicators suggest that the most significant types of medical errors are decreasing.29 Figure 5 provides an example of the effects of CPOE on the clinical processes of CHP. The lessons learned from the entire CPOE implementation process are many; however, a few stand out as general guidelines for other institutions considering CPOE implementation. Table 2 provides summaries of the lessons learned, as well as points to avoid when undertaking CPOE implementation.
CONCLUSIONS
The beneficial hallmarks of CPOE include increased work efficiency and, most importantly, improved patient safety. However, realization of these benefits is the product of shifts in hospital culture and practices. Careful management of the systematic changes accompanying introduction of CPOE is crucial to successful CPOE implementation. This management includes integration of, and cooperation between, various hospital departments. CPOE is an important innovation that requires total organizational commitment and sustained support. Consequently, our patients appreciate the improved safety and care that CPOE supports.
FOOTNOTES
Accepted May 25, 2005.
No conflict of interest declared.
REFERENCES
Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Healthcare System. Washington, DC: National Academy Press; 1999
Sittig DF, Stead WW. Computer-based physician order entry: the state of the art. J Am Med Inform Assoc. 1994;1 :108 –123
Metzger J, Turisco F. Computerized Physician Order Entry: A Look at the Vendor Marketplace and Getting Started. New York, NY: First Consulting Group; 2001:1–36
Drazen E, Kilbridge P, Metzger J, Turisco F. A Primer of Physician Order Entry. Oakland, CA: First Consulting Group; 2000:1–43
Doolan DF, Bates DW. Computerized physician order entry systems in hospitals: mandates and incentives. Health Aff (Milwood). 2002;21 :180 –187
Ahmad A, Teater P, Bentley TD, et al. Key attributes of a successful physician order entry system implementation in a multi-hospital environment. J Am Med Inform Assoc. 2002;9 :16 –24
Dick RS, Steen EB, Detmer DE, eds. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: National Academy Press; 1997
Soumerai SB, Avorn J. Crossing the Quality Chasm: A New Health Care System for the 21st Century. Washington, DC: National Academy Press; 2001
Kaushal R, Bates DW, Landrigan C. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285 :2114 –2120
Lee F, Teater J, Spurr CD, Bates DW. Implementation of physician order entry: user satisfaction and self-reported usage patterns. J Am Med Inform Assoc. 1996;3 :42 –55
Murff HJ, Kannry J. Physician satisfaction with two order entry systems. J Am Med Inform Assoc. 2001;8 :499 –509
Massaro T. Introducing physician order entry at a major academic center, II: impact on medical education. Acad Med. 1993;68 :25 –30
Pestotnik SL, Classen DC, Evans RS, Burke JP. Implementing antibiotic practice guidelines through computer-assisted decision support: clinical and financial outcomes. Ann Intern Med. 1996;124 :884 –890
Chisholm CD, Collison EK, Nelson DR, Cordell WH. Emergency department workplace interruptions: are emergency physicians "interrupt-driven" and "multitasking" Acad Emerg Med. 2000;7 :1239 –1243
Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003;326 :314 –318
Chisholm CD, Dornfeld AM, Nelson DR, Cordell WH. Work interrupted: a comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38 :146 –151
Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians’ time utilization in ambulatory care internal medicine practices. J Am Med Inform Assoc. 2001;8 :361 –371
Benson T. Why general practitioners use computers and hospital doctors do not, part 2: scalability. BMJ. 2002;325 :1090 –1093
Massaro T. Introducing physician order entry at a major academic medical center, I: impact of organizational culture and behavior. Acad Med. 1993;68 :20 –25
Tierney WM, Overhage JM, McDonald CJ, Wolinksy FD. Medical students’ and housestaff’s opinions of computerized order-writing. Acad Med. 1994;69 :386 –389
Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other anti-infective agents. N Engl J Med. 1998;338 :232 –238
Ogura H, Sagara E, Iwata M, et al. Online support functions of prescription audit in an integrated hospital information system. Med Inform (Lond). 1988;13 :161 –169
McDonald C. Protocol-based computer-reminders, the quality of care and the non-perfectability of man. N Engl J Med. 1976;295 :1351 –1355
McDonald C. Physician response to computer reminders. JAMA. 1980;244 :1579 –1581
Greenes R. Knowledge management as an aid to medical decision making and education: the Explorer-1 system. Medinfo. 1986;88 :895 –899
Kawahara NE, Jordan FM. Influencing prescribing behavior by adapting computerized order-entry pathways. Am J Hosp Pharm. 1989;46 :1798 –1801
Evans R, Classen DC, Pestotnik SL, Lundsgaarde HP, Burke JP. Improving empiric antibiotic selection using computer decision support. Arch Intern Med. 1994;154 :878 –884
Upperman JS, Staley P, Friend K, et al. The impact of hospitalwide computerized physician order entry on medical errors in a pediatric hospital. J Pediatr Surg. 2005;40 :57 –59
Dick RS, Steen EB, Detmer DE, eds. The Computer-Based Patient Record: An Essential Technology for Health Care. Washington, DC: National Academy Press; 1997
Orthner HF, Blum BI. Implementing health care information systems. In: Orthner HF, Blum BI, eds. Implementing Health Care Information Systems. New York, NY: Springer-Verlag; 1989:3–21
Barnett GO. The application of computer-based medical-record systems in ambulatory practice. In: Orthner HF, Blum BI, eds. Implementing Health Care Information Systems. New York, NY: Springer-Verlag; 1989:85–99(Jeffrey S. Upperman, MD, )