Barriers and Motivators for Making Error Reports from Family Medicine Offices: A Report from the American Academy of Family Physic
http://www.100md.com
《家庭医学美国版医学期刊》
Department of Family Medicine, University of Cincinnati, Cincinnati, OH (NE)
American Academy of Family Physicians, Kansas City, KS (DG, EB)
Department of Family Medicine, University of Chicago, Chicago, IL (JH)
Abstract
Context: Reporting of medical errors is a widely recognized mechanism for initiating patient safety improvement, yet we know little about the feasibility of error reporting in physician offices, where the majority of medical care in the United States is rendered.
Objective: To identify barriers and motivators for error reporting by family physicians and their office staff based on the experiences of those participating in a testing process error reporting study.
Design: Qualitative focus group study, analyzed using the editing method.
Setting: Eight volunteer practices of the American Academy of Family Physicians National Research Network.
Participants: 139 physicians, nurse practitioners, physician assistants, nurses, and staff who took part in 18 focus groups.
Instrument: Interview questions asked about making reports, what prevents more reports from being made, and decisions about when to make reports.
Results: Four factors were seen as central to making error reports: the burden of effort to report, clarity regarding the information requested in an error report, the perceived benefit to the reporter, and properties of the error (eg, severity, responsibility). The most commonly mentioned barriers were related to the high burden of effort to report and lack of clarity regarding the requested information. The most commonly mentioned motivator was perceived benefit.
Conclusion: Successful error reporting systems for physicians’ offices will need to have low reporting burden, have great clarity regarding the information requested, provide direct benefit through feedback useful to reporters, and take into account error severity and personal responsibility.
The reporting of medical errors or events is an essential activity for improving patient safety, not just in hospitals but in ambulatory care settings as well.1–3 Error reports "channel attention, shape interpretations and serve as springboards for action."2 Error reporting is one of several mechanisms for identifying areas needing improvement.1,4 A strength of error reporting is that it occurs on the front lines of care and, therefore, has the potential to increase mindfulness of safety issues as they occur in real time.4 A weakness is that most errors are not reported.1,5 Many reasons have been proposed for this underreporting, including underrecognition,6,7 confusion about definitions,7–10 fear of blame and punishment,9–12 concerns about anonymity and confidentiality,9,11,13 and the amount of time and effort required to report.9,11,12,14 In addition, a belief that reporting will make no difference has also been cited as a reason for underreporting.10,11,14 Less is known about what encourages health care personnel to make reports.9,11,14
With the passage of the Patient Safety and Quality Improvement Act of 2005,15 it is likely that error and event reporting will spread from hospitals, where it is common, to ambulatory care settings, where it rarely occurs and is not part of routine work. As with most patient safety research, the majority of work on error reporting has been performed in a hospital setting.6,7,9,10,12,14,16 There is, however, a small body of literature about error reporting from primary care practices.11,17 For example, Beasley and Karsh explored what 14 family physicians and office staff would like in an error reporting system.11,17
To better delineate the factors for successfully reporting errors in an ambulatory care setting, we chose to draw from the experiences of family physicians and their office staff who were participating in an errors reporting study. We held focus groups at the participating offices to discuss their experiences with making error reports, specifically looking for barriers to and motivators for reporting errors.
Methods
Setting
This study took place in 8 selected volunteer family physician offices: 4 private practices and 4 family medicine residency clinics. All offices had at least one physician-member of the American Academy of Family Physicians (AAFP) National Research Network. This study received approval from the University of Missouri-Kansas City Institutional Review Board (IRB) and from individual site IRBs as required. Physicians and staff members at all sites signed and returned consent forms.
Error Reporting
This focus group study occurred within the context of a larger study designed to collect and describe testing process errors. We elected to study testing process errors because tests are ordered during 39% of primary care outpatient encounters,18 and testing errors are a serious safety threat to patients.19 The testing process encompasses a series of decisions and actions that occur from the time a test is ordered to the time the appropriate follow-up action is taken with the patient.19 Testing process errors account for 14% to 47% of reported errors in office practice.20–22
At each of the 8 practices, physicians, nurse practitioners (NPs), physician assistants (PAs), office staff, and nurses submitted anonymous reports of errors observed in the course of clinical care related to the laboratory, radiology, and diagnostic testing processes. Errors were defined as "anything that happened in your own practice related to the testing process that should not have happened, that was not anticipated, and that makes you say, ‘That should not happen in my practice, and I don’t want it to happen again’." Reports were made anonymously either over a secure Web site or by filling out and mailing a paper form. Practices and individuals could determine how they wished to file reports. Reporting occurred for 8 months.
Focus Groups
We chose focus groups to achieve a wide breadth of responses around issues related to error reporting and the effects of error reporting on office systems.23,24 Focus groups use the "dynamic and interactive exchange among participants to produce multiple stories and diverse experiences,"25 and we desired to fully explore the error reporting experience by all participants. We visited every practice approximately 6 months after they began reporting. All practice staff were invited to participate, and the groups were held before or after office hours or during lunch. Lead study physicians and site research coordinators (usually a nurse or office manager) at each practice arranged the time and place for their focus groups, as well as extended the invitations to their staff and physicians, with advice and input from the research investigators. Depending on the size of the practice, between 1 and 3 groups were held at each office. In the larger practices, physicians, residents, NPs, and PAs were interviewed separately from the nurses and staff. However, this was not possible at the smaller practices, where groups were mixed.
Data Collection
An initial interview guide was developed from the literature,20,26–28 reviewed by the research team and revised before being used. Minor modifications were made after the first focus group to improve clarity and understanding. Questions asked about making error reports included: What is the time commitment involved in error reporting Did you feel adequately trained to make reports Could you be making more reports What prevents you from making more reports How do you feel about reporting an error What made you decide to report some errors (or types of errors) and not others We also included in the analysis spontaneous comments made throughout the focus group interviews that related to barriers and motivators for reporting.
All focus groups were led by an experienced qualitative researcher and focus group moderator (NCE) or the research associate (DG), who was trained and observed by NCE. All practices except one allowed audiotaping; however, technical problems at one other practice meant that for two practices, extensive notes were taken in lieu of audiotaping. Audiotapes were transcribed and compared with the original recording for accuracy. All names and identifying information were removed from the transcripts. Focus group participants also provided demographic information.
Data Analysis
Data were analyzed using the "editing method,"29,30 using NVivo 2.0 software (QSR International, Victoria, Australia). In this method, while acknowledging our preconceptions about error reporting (which all researchers hold), we sorted the interview data into meaningful segments that relate to the purpose of the study. We then placed these segments into coding categories derived from the data using NVivo 2.0 to store, display, and assist with organizing the categories.31 We explicitly checked the categories against each other and the original data, searching for patterns and themes. Each transcript was read and coded by NCE. For improved validity, an additional analyst (DG) read and coded one third of the transcripts. During coding, categories were added or modified as needed as we drew on the original transcripts for meaningful segments of text. The analysts then met and compared coding, making changes as needed to achieve consensus. NCE then re-coded all the transcripts a second time. Coding categories, illustrative quotes, and initial theme formation were then reviewed and discussed by all members of the study team through meetings, phone conferences, and E-mail discussions. From this analysis, themes related to perceived barriers and motivators for making error reports were reported, and a model was developed of important factors in an ambulatory error reporting system.
Results
The demographics of the office sites and the focus group participants are shown in Table 1. Nearly all the nurses and staff were women, whereas 64% of the physicians, NPs, and PAs were men. The offices were located in 7 states and included 2 rural locations. Staff and physicians at these offices submitted 597 usable reports. The vast majority (519) were submitted by computer, but all practices submitted reports both by paper and computer. Practices submitted an average of 75 reports (range, 38 to 170). The time required by participants to complete an error reporting form was 2 to 5 minutes. Findings from these error reports will be reported elsewhere.
Eighteen focus groups were convened. Two practices had 1 combined group each, 2 offices had 2 groups (separate for clinicians and staff), and 4 practices had 3 groups (separate for clinicians and staff). Within the broad categories of barriers and motivators, most issues discussed by the participants could be assigned to one of four factors: the burden of effort to make a report, the perceived benefit from making a report, the clarity about what to report, and the properties of the error itself, such as severity (Tables 2 and 3).
Barriers to Reporting
The most frequently reported barriers were those related to burden of effort, specifically a lack of time to make the report and forgetfulness (Table 2). Participants noted that, "Sure it’s only 5 minutes of your time, but who’s got 5 minutes" or "It’s just one more added thing when we’re already pushed to the limit." Participants also noted that they both forgot about making reports at all ("To be honest, I pretty much have forgotten about it") or they would plan on making a report later, but, "I think I’ll wait until I get a few minutes and then I’ll do that and then at the end of the day you’ve already forgot the incident." Some participants mentioned that information required of the reporting tool (like patient demographics) was not always readily available. In addition, local computer problems and access made making online reports difficult for some participants.
Confusions about what to report and who should report were also noted. These included lack of clarity about what constituted an error, the specific information requested in the report, and whether error identification and reporting applied to those in the back office. For example, "Working in medical records ... None of it applies to what I do, but it does apply to others." Repetitive errors that seem to happen again and again, as well as rare errors, were unlikely to be reported. Comments related to the frequent errors noted the "fatigue" factor leading to "detracting from wanting to report it." A rare error wasn’t reported because, "I remember thinking, I have never seen this before and I don’t think it’ll ever happen again so I didn’t report it."
Another barrier to reporting was related to the severity of the error, where less serious errors were less likely to be reported. For example, this participant noted that, "I think for me, the level of it (the error) was just not that big a deal. I haven’t reported any—but that doesn’t mean I haven’t seen them." Feeling personally responsible for an error was a barrier for some participants. For example, "I never thought of reporting something that I did," and "If it’s something I goofed on it’s a lot easier to say, I’m not sure that was an error."
Motivators for Making Reports
There was less consensus regarding motivators for reporting, and most motivating categories were mentioned by only two or three practices. Whereas many motivators were the inverse of the barriers, other categories were purely motivators; their absence wasn’t noted to be a significant barrier (Table 3). The most commonly mentioned reason for reporting was receiving some perceived benefit. This was expressed as, "If it makes it so that they’ll learn from the process then it’s worth it," and, "If we can make it known and get it corrected it frees up a lot of time for all the other stuff that we have to do." Although this error reporting study did not provide site-based feedback to the practices because of IRB restrictions, participants still hoped, "that as they (the investigators) collate this information and they feed it back to us, I can trend (error types)."
Receiving some emotional benefit by having an outlet to vent was also noted: "There is a little emotional quality to it for me, if it gave me more angst I’m more likely to report it." Interestingly, although some participants noted feeling personally responsible for an error deterred them from making a report, others felt that personal responsibility was a motivator to report the error. For example, "I’m actually more likely to report the ones that I make. I feel guilty about them." The promise of anonymity was also seen as a motivating factor in making reports.
Motivating factors related to previously mentioned barriers included easing the burden of effort to report; some participants wanted to shift the burden to a dedicated office reporter: "It would be easier to make reports if it would be a data entry system where actually one person made all the reports and information was just given to that person." The severity of the error was also a motivating factor, for example, "I guess where I felt that harm was more would determine whether I reported it or not."
Discussion
In the hospital setting, "incident reports" have been part of the culture for many years, but in the office-based practice setting, making such reports is rare.32 The Patient Safety and Quality Improvement Act of 2005 would support the type of error reporting done by our study participants, and their opinions and thoughts may be useful as future patient safety organizations develop their error reporting systems. The decision to report an error is a complex one, and from our study, 4 factors are considered most often by participants making error reports: the burden of effort to make the report, the perceived benefit from making the report, the clarity about what to report, and the properties of the specific error, such as severity or who is responsible for the error.
Previous researchers have proposed models for error reporting systems. Uribe and colleagues propose a model of intersecting circles of control (organizational and individual) with factors placed within these circles in a hierarchal manner based on their ability to be controlled or modified.14 Karsh and colleagues propose a multilevel systems model of technology design and implementation that has three levels (organization, system, and individual), and within each level the technology is fitted to the user, the task, the organization, and the environment.11 Uribe’s model was based on input from physicians and nurses at an academic medical center, whereas Karsh’s was based on input from 14 family physicians and their office staff with unknown experiences in error reporting. In Figure 1, we present an alternative model for an error reporting decision making and support system in a family medicine office, based on the practical application of our participants’ responses and previously reported theoretical models.11,14 We describe questions that people make when deciding to report an error and the organizational and system components that support error reporting by the 4 factors our study found to be of most importance to our participants (burden of effort, perceived benefit, clarity of request, and error properties).1,14,33 We believe that this model serves as a concrete and usable foundation for developing an error reporting system in family medicine.
The importance of reporting burden was made clear by our participants. Finding the time and remembering to report in the midst of a busy, stressful practice setting is difficult. This finding is not surprising because most research has found that ease of use and time are important factors in choosing to report errors.9,11,13,14 In this study, reports took an average of 2 to 5 minutes to complete. However, this estimate does not account for the break in work flow, time to access the reporting tool (paper or computer), and time to collect the information required for the report. This latter point was noted by participants in several groups who said that some information requested, such as demographic information about the patient, often necessitated having the chart present. In addition, requested information may not even be available at the time an error is discovered. For example, although patient outcomes in a hospital setting may be known in hours or even minutes, the outcomes from an error in family medicine may not be evident for days or weeks. This difference between the theoretical time needed to complete a report and the real time needed in the office setting, as well as the availability of required information for the error report, became clear only when those actually making such reports were questioned. Some of our participants felt the burden could be reduced by having one person make all the error reports, but this entails internal reporting to a peer or supervisor, which has its own set of problems.16 Other possibilities for future consideration include reporting systems that allow data to be entered at several points in time as information and outcomes become known and are still fresh in the reporter’s memory.
Participants are willing to overcome burdens to reporting when they feel the benefits are high. Although our study did not provide direct practice-level feedback, a desired feature of reporting systems in other studies,9,11,17,34 our participants noted they were learning about errors just by making the reports.4 For some, there was also an emotional benefit to reporting, giving participants a tangible method for dealing with the "angst," "frustration," and "guilt" that might accompany making or discovering errors. The role of emotion in decision making has become clearer in recent years and must be considered when designing an error reporting system.35 For a reporting system to maintain itself, however, ongoing benefits, including emotional support, regular feedback, and perceived quality improvement, will need to be an integral part of a system.11,14 Individuals who report to patient safety organizations created as a result of the Patient Safety and Quality Improvement Act of 2005 will have protection from discovery for confidential error reports. As a result, it may be feasible to integrate an educational and supportive feedback loop into the reporting system and subsequently to improve quality as an outcome of reporting. This will be a major perceived benefit that was missing from our study.
Error reporters also need clarity about what is wanted and needed in a report and who should make reports. Despite our best efforts at training, there was still confusion about what actually encompassed a testing process error. Individuals in some groups felt that their type of work (eg, clerical) was not included or that certain types of events didn’t really justify being reported, including those very rare and very common events. Clear definitions, with examples, need to be part of a reporting system. Many studies have found that the severity of the error (and resultant adverse event) is related to decisions to make reports.9–11,13,14,16 This, in turn, is related to that very definition of what constitutes an error8,36 and harm.37 Designers of systems also need to be aware of the apparent paradox surrounding the reporting of errors for which participants feel a personal sense of responsibility. When individuals feel they are responsible for an error, conflicting stories emerge about how this translates into an error report. Some find it cathartic to report, whereas others find it perhaps easier to justify or soft-sell their action as a face-saving response. Reporting systems need to be diligent about definitions, explanations, and training. Knowing that individuals are more likely to report those errors they perceive as serious, further research is needed among potential reporters to assist in developing a usable definition.8 The assurance and clarification of anonymity and confidentiality are also important issues, because other studies have found that these factors are cited in decision making about error reporting.5,11,14
This study has several limitations. Participants made error reports part of a research study with guaranteed anonymity. Factors relating to other types of error reporting systems might not have been mentioned with this frame of reference. At least one member of each practice was also committed to practice-based research. This altruistic benefit of moving the discipline’s understanding forward was seen by some as a benefit of this error reporting study but might not apply to other situations. As mentioned, our IRB required study protocol, which restricted feedback to the practices, may have blunted the true impact that feedback has on the reporting process. The focus groups allowed us to collect a wide range and breadth of responses, but we cannot quantify which are most important or which might apply to specific subgroups. The nonhomogenous nature of some groups (physicians, staff, and nurses in one group) may have stifled some conversation but was necessary to include small practices. We did separate physicians from staff and nurses whenever possible. Our findings, however, can be useful for future research and interventions to help define qualities of an error reporting system. Patient input was not included in the study, either as error reporters or focus group participants. They will have unique and important contributions to safety and will need to be included in future studies.
Staff and physicians at busy family practice offices across America were willing to identify and submit error reports about testing process errors for 8 months. Their insights into the process confirm that error reporting does constitute another burden on their time but that when they perceive benefits to themselves, their practices, and their patients, they are willing to take the time to submit reports. However, time commitment and confusions about what information and what type of error should be reported are barriers to the usefulness and quality of the reports and must be directly addressed when developing error reporting systems.
Notes
This article was externally peer-reviewed.
Funding: Supported by Agency for Healthcare Research and Quality Grant R21 HS13554-01.
Conflict of interest: none declared.
References
Kaplan H, Barach P. Incident reporting: science or protoscience Ten years later. Qual Saf Health Care 2002; 11: 144–5.
Sutcliffe KM. Defining and classifying medical error: lessons for learning. Qual Saf Health Care 2004; 13: 8–9.
West D, Westfall JM, Araya-Guerra R, et al. Using reported primary care errors to develop and implement patient safety interventions: a report from the ASIPS Collaborative. In: Advances in patient safety: from research to implementation. Rockville (MD): Agency for Healthcare Research and Quality; 2005 Publication no. 050021.
Elder NC, Graham D, Hickner J. Can a national primary care error-reporting system make a difference in medical practice Available from: http://www.npsf.org/html/Focus.html Focus on Patient Safety 2004; 7: 4–5.
Johnson CW. How will we get the data and what will we do with it then Issues in the reporting of adverse healthcare events. Qual Saf Health Care 2003; 12 Suppl 2: ii64–7.
Hobgood C, Xie J, Weiner B, Hooker J. Error identification, disclosure, and reporting: practice patterns of three emergency medicine provider types. Acad Emerg Med 2004; 11: 196–9.
Tamuz M, Thomas EJ, Franchois KE. Defining and classifying medical error: lessons for patient safety reporting systems. Qual Saf Health Care 2004; 13: 13–20.
Dovey SM, Phillips RL. What should we report to medical error reporting systems Qual Saf Health Care 2004; 13: 322–3.
Jeffe DB, Dunagan WC, Garbutt J, et al. Using focus groups to understand physicians’ and nurses’ perspectives on error reporting in hospitals. Jt Comm J Qual Saf 2004; 30: 471–9.
Taylor JA, Brownstein D, Christakis DA, et al. Use of incident reports by physicians and nurses to document medical errors in pediatric patients. Pediatrics 2004; 114: 729–35.
Karsh BT, Escoto KH, Beasley JW, Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon 2006; 37: 283–95.
Wakefield DS, Wakefield BJ, Uden-Holman T, Blegen MA. Perceived barriers in reporting medication administration errors. Best Pract Benchmarking Healthc 1996; 1: 191–7.
Coyle YM, Mercer SQ, Murphy-Cullen CL, Schneider GW, Hynan LS. Effectiveness of a graduate medical education program for improving medical event reporting attitude and behavior. Qual Saf Health Care 2005; 14: 383–8.
Uribe CL, Schweikhart SB, Pathak DS, Dow M, Marsh GB. Perceived barriers to medical-error reporting: an exploratory investigation. J Healthc Manag 2002; 47: 263–79.
Carney J. Patient Safety and Quality Improvement Act of 2005. Bricker and Eckler Health Care Client Bulletin 2005; 05–06: 1–4.
Lawton R, Parker D. Barriers to incident reporting in a healthcare system. Qual Saf Health Care 2002; 11: 15–8.
Beasley JW, Escoto KH, Karsh BT. Design elements for a primary care medical error reporting system. WMJ 2004; 103: 56–9.
Cherry D, Burt C, Woodwell D. National Ambulatory Medical Care Survey, 2001 Summary, Advance Data from Vital and Health Statistics. Hyattsville (MD): National Center for Health Statistics; 2003.
Hickner JM, Fernald DH, Harris DM, Poon EG, Elder NC, Mold JW. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf 2005; 31: 81–9.
Dovey SM, Meyers DS, Phillips RL, Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care 2002; 11: 233–8.
Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: A report from the ASIPS Collaborative. Annals of Fam Med 2004; 2: 327–332.
Phillips R, Dovey S, Graham D, Elder N, Hickner J. Learning from different lenses: Reports of medical errors in primary care by clinicians, staff and patients. J Patient Safety 2006; 2(3): 140–146.
Morgan DL. Successful focus groups: advancing the state of the art. Newbury Park (CA): Sage Press; 1993.
Fern E. Advanced focus group research. Thousand Oaks (CA): Sage Press; 2001.
Brown J. The use of focus groups in clinical research. In: Crabtree B, Miller W, editors. Doing Qualitative Research. 2nd Ed. Thousand Oaks (CA): Sage Press; 1999. pp. 109–26.
Boohaker EA, Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med 1996; 156: 327–31.
Mold JW, Cacy DS, Dalbir DK. Management of laboratory test results in family practice. An OKPRN study. Oklahoma Physicans Resource/Research Network. J Fam Pract 2000; 49: 709–15.
Nutting PA, Main DS, Fischer PM, Stull TM, Pontious M, Seifert M, Jr, et al. Toward optimal laboratory use. Problems in laboratory testing in primary care. JAMA 1996; 275: 635–9.
Miller W, Crabtree B. Clinical research. A multi-method typology and qualitative roadmap. In: Crabtree B, Miller W, editors. Doing qualitative research. 2nd ed. Thousand Oaks (CA): Sage Press; 1999. pp. 3–30.
Miller W, Crabtree B. Qualitative analysis: how to begin making sense. Fam Prac Res J 1994; 14: 289–97.
Gibbs G. Qualitative data analysis. Explorations with NVivo. Philadelphia: Open University Press; 2002.
Fischer G, Fetters MD, Munro AP, Goldman EB. Adverse events in primary care identified from a risk-management database. J Fam Prac 1997; 45: 40–6.
Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care 2004; 13: 388–94.
Battles JB, Kaplan HS, Van der Schaaf TW, Shea CE. The attributes of medical event-reporting systems: experience with a prototype medical event-reporting system for transfusion medicine. Arch Pathol Lab Med 1998; 122: 231–8.
Zinn J. Risk, affect and emotion. In: Forum: Qualitative Social Research; 2006. Available from: www.qualitative-research.net/fqs-texts/1-06/06-1-29-E.htm.
Woods D, Cook R. Mistaking error. In: Youngberg B, Harlie M, editors. The patient safety handbook. Jones and Bartlett Publishers, Inc., Sudbury (MA); 2003. pp. 95–109.
Elder NC, Vonder Meulen MB, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med 2004; 2: 125–9.(Nancy C. Elder, MD, MSPH, Deborah Graham)
American Academy of Family Physicians, Kansas City, KS (DG, EB)
Department of Family Medicine, University of Chicago, Chicago, IL (JH)
Abstract
Context: Reporting of medical errors is a widely recognized mechanism for initiating patient safety improvement, yet we know little about the feasibility of error reporting in physician offices, where the majority of medical care in the United States is rendered.
Objective: To identify barriers and motivators for error reporting by family physicians and their office staff based on the experiences of those participating in a testing process error reporting study.
Design: Qualitative focus group study, analyzed using the editing method.
Setting: Eight volunteer practices of the American Academy of Family Physicians National Research Network.
Participants: 139 physicians, nurse practitioners, physician assistants, nurses, and staff who took part in 18 focus groups.
Instrument: Interview questions asked about making reports, what prevents more reports from being made, and decisions about when to make reports.
Results: Four factors were seen as central to making error reports: the burden of effort to report, clarity regarding the information requested in an error report, the perceived benefit to the reporter, and properties of the error (eg, severity, responsibility). The most commonly mentioned barriers were related to the high burden of effort to report and lack of clarity regarding the requested information. The most commonly mentioned motivator was perceived benefit.
Conclusion: Successful error reporting systems for physicians’ offices will need to have low reporting burden, have great clarity regarding the information requested, provide direct benefit through feedback useful to reporters, and take into account error severity and personal responsibility.
The reporting of medical errors or events is an essential activity for improving patient safety, not just in hospitals but in ambulatory care settings as well.1–3 Error reports "channel attention, shape interpretations and serve as springboards for action."2 Error reporting is one of several mechanisms for identifying areas needing improvement.1,4 A strength of error reporting is that it occurs on the front lines of care and, therefore, has the potential to increase mindfulness of safety issues as they occur in real time.4 A weakness is that most errors are not reported.1,5 Many reasons have been proposed for this underreporting, including underrecognition,6,7 confusion about definitions,7–10 fear of blame and punishment,9–12 concerns about anonymity and confidentiality,9,11,13 and the amount of time and effort required to report.9,11,12,14 In addition, a belief that reporting will make no difference has also been cited as a reason for underreporting.10,11,14 Less is known about what encourages health care personnel to make reports.9,11,14
With the passage of the Patient Safety and Quality Improvement Act of 2005,15 it is likely that error and event reporting will spread from hospitals, where it is common, to ambulatory care settings, where it rarely occurs and is not part of routine work. As with most patient safety research, the majority of work on error reporting has been performed in a hospital setting.6,7,9,10,12,14,16 There is, however, a small body of literature about error reporting from primary care practices.11,17 For example, Beasley and Karsh explored what 14 family physicians and office staff would like in an error reporting system.11,17
To better delineate the factors for successfully reporting errors in an ambulatory care setting, we chose to draw from the experiences of family physicians and their office staff who were participating in an errors reporting study. We held focus groups at the participating offices to discuss their experiences with making error reports, specifically looking for barriers to and motivators for reporting errors.
Methods
Setting
This study took place in 8 selected volunteer family physician offices: 4 private practices and 4 family medicine residency clinics. All offices had at least one physician-member of the American Academy of Family Physicians (AAFP) National Research Network. This study received approval from the University of Missouri-Kansas City Institutional Review Board (IRB) and from individual site IRBs as required. Physicians and staff members at all sites signed and returned consent forms.
Error Reporting
This focus group study occurred within the context of a larger study designed to collect and describe testing process errors. We elected to study testing process errors because tests are ordered during 39% of primary care outpatient encounters,18 and testing errors are a serious safety threat to patients.19 The testing process encompasses a series of decisions and actions that occur from the time a test is ordered to the time the appropriate follow-up action is taken with the patient.19 Testing process errors account for 14% to 47% of reported errors in office practice.20–22
At each of the 8 practices, physicians, nurse practitioners (NPs), physician assistants (PAs), office staff, and nurses submitted anonymous reports of errors observed in the course of clinical care related to the laboratory, radiology, and diagnostic testing processes. Errors were defined as "anything that happened in your own practice related to the testing process that should not have happened, that was not anticipated, and that makes you say, ‘That should not happen in my practice, and I don’t want it to happen again’." Reports were made anonymously either over a secure Web site or by filling out and mailing a paper form. Practices and individuals could determine how they wished to file reports. Reporting occurred for 8 months.
Focus Groups
We chose focus groups to achieve a wide breadth of responses around issues related to error reporting and the effects of error reporting on office systems.23,24 Focus groups use the "dynamic and interactive exchange among participants to produce multiple stories and diverse experiences,"25 and we desired to fully explore the error reporting experience by all participants. We visited every practice approximately 6 months after they began reporting. All practice staff were invited to participate, and the groups were held before or after office hours or during lunch. Lead study physicians and site research coordinators (usually a nurse or office manager) at each practice arranged the time and place for their focus groups, as well as extended the invitations to their staff and physicians, with advice and input from the research investigators. Depending on the size of the practice, between 1 and 3 groups were held at each office. In the larger practices, physicians, residents, NPs, and PAs were interviewed separately from the nurses and staff. However, this was not possible at the smaller practices, where groups were mixed.
Data Collection
An initial interview guide was developed from the literature,20,26–28 reviewed by the research team and revised before being used. Minor modifications were made after the first focus group to improve clarity and understanding. Questions asked about making error reports included: What is the time commitment involved in error reporting Did you feel adequately trained to make reports Could you be making more reports What prevents you from making more reports How do you feel about reporting an error What made you decide to report some errors (or types of errors) and not others We also included in the analysis spontaneous comments made throughout the focus group interviews that related to barriers and motivators for reporting.
All focus groups were led by an experienced qualitative researcher and focus group moderator (NCE) or the research associate (DG), who was trained and observed by NCE. All practices except one allowed audiotaping; however, technical problems at one other practice meant that for two practices, extensive notes were taken in lieu of audiotaping. Audiotapes were transcribed and compared with the original recording for accuracy. All names and identifying information were removed from the transcripts. Focus group participants also provided demographic information.
Data Analysis
Data were analyzed using the "editing method,"29,30 using NVivo 2.0 software (QSR International, Victoria, Australia). In this method, while acknowledging our preconceptions about error reporting (which all researchers hold), we sorted the interview data into meaningful segments that relate to the purpose of the study. We then placed these segments into coding categories derived from the data using NVivo 2.0 to store, display, and assist with organizing the categories.31 We explicitly checked the categories against each other and the original data, searching for patterns and themes. Each transcript was read and coded by NCE. For improved validity, an additional analyst (DG) read and coded one third of the transcripts. During coding, categories were added or modified as needed as we drew on the original transcripts for meaningful segments of text. The analysts then met and compared coding, making changes as needed to achieve consensus. NCE then re-coded all the transcripts a second time. Coding categories, illustrative quotes, and initial theme formation were then reviewed and discussed by all members of the study team through meetings, phone conferences, and E-mail discussions. From this analysis, themes related to perceived barriers and motivators for making error reports were reported, and a model was developed of important factors in an ambulatory error reporting system.
Results
The demographics of the office sites and the focus group participants are shown in Table 1. Nearly all the nurses and staff were women, whereas 64% of the physicians, NPs, and PAs were men. The offices were located in 7 states and included 2 rural locations. Staff and physicians at these offices submitted 597 usable reports. The vast majority (519) were submitted by computer, but all practices submitted reports both by paper and computer. Practices submitted an average of 75 reports (range, 38 to 170). The time required by participants to complete an error reporting form was 2 to 5 minutes. Findings from these error reports will be reported elsewhere.
Eighteen focus groups were convened. Two practices had 1 combined group each, 2 offices had 2 groups (separate for clinicians and staff), and 4 practices had 3 groups (separate for clinicians and staff). Within the broad categories of barriers and motivators, most issues discussed by the participants could be assigned to one of four factors: the burden of effort to make a report, the perceived benefit from making a report, the clarity about what to report, and the properties of the error itself, such as severity (Tables 2 and 3).
Barriers to Reporting
The most frequently reported barriers were those related to burden of effort, specifically a lack of time to make the report and forgetfulness (Table 2). Participants noted that, "Sure it’s only 5 minutes of your time, but who’s got 5 minutes" or "It’s just one more added thing when we’re already pushed to the limit." Participants also noted that they both forgot about making reports at all ("To be honest, I pretty much have forgotten about it") or they would plan on making a report later, but, "I think I’ll wait until I get a few minutes and then I’ll do that and then at the end of the day you’ve already forgot the incident." Some participants mentioned that information required of the reporting tool (like patient demographics) was not always readily available. In addition, local computer problems and access made making online reports difficult for some participants.
Confusions about what to report and who should report were also noted. These included lack of clarity about what constituted an error, the specific information requested in the report, and whether error identification and reporting applied to those in the back office. For example, "Working in medical records ... None of it applies to what I do, but it does apply to others." Repetitive errors that seem to happen again and again, as well as rare errors, were unlikely to be reported. Comments related to the frequent errors noted the "fatigue" factor leading to "detracting from wanting to report it." A rare error wasn’t reported because, "I remember thinking, I have never seen this before and I don’t think it’ll ever happen again so I didn’t report it."
Another barrier to reporting was related to the severity of the error, where less serious errors were less likely to be reported. For example, this participant noted that, "I think for me, the level of it (the error) was just not that big a deal. I haven’t reported any—but that doesn’t mean I haven’t seen them." Feeling personally responsible for an error was a barrier for some participants. For example, "I never thought of reporting something that I did," and "If it’s something I goofed on it’s a lot easier to say, I’m not sure that was an error."
Motivators for Making Reports
There was less consensus regarding motivators for reporting, and most motivating categories were mentioned by only two or three practices. Whereas many motivators were the inverse of the barriers, other categories were purely motivators; their absence wasn’t noted to be a significant barrier (Table 3). The most commonly mentioned reason for reporting was receiving some perceived benefit. This was expressed as, "If it makes it so that they’ll learn from the process then it’s worth it," and, "If we can make it known and get it corrected it frees up a lot of time for all the other stuff that we have to do." Although this error reporting study did not provide site-based feedback to the practices because of IRB restrictions, participants still hoped, "that as they (the investigators) collate this information and they feed it back to us, I can trend (error types)."
Receiving some emotional benefit by having an outlet to vent was also noted: "There is a little emotional quality to it for me, if it gave me more angst I’m more likely to report it." Interestingly, although some participants noted feeling personally responsible for an error deterred them from making a report, others felt that personal responsibility was a motivator to report the error. For example, "I’m actually more likely to report the ones that I make. I feel guilty about them." The promise of anonymity was also seen as a motivating factor in making reports.
Motivating factors related to previously mentioned barriers included easing the burden of effort to report; some participants wanted to shift the burden to a dedicated office reporter: "It would be easier to make reports if it would be a data entry system where actually one person made all the reports and information was just given to that person." The severity of the error was also a motivating factor, for example, "I guess where I felt that harm was more would determine whether I reported it or not."
Discussion
In the hospital setting, "incident reports" have been part of the culture for many years, but in the office-based practice setting, making such reports is rare.32 The Patient Safety and Quality Improvement Act of 2005 would support the type of error reporting done by our study participants, and their opinions and thoughts may be useful as future patient safety organizations develop their error reporting systems. The decision to report an error is a complex one, and from our study, 4 factors are considered most often by participants making error reports: the burden of effort to make the report, the perceived benefit from making the report, the clarity about what to report, and the properties of the specific error, such as severity or who is responsible for the error.
Previous researchers have proposed models for error reporting systems. Uribe and colleagues propose a model of intersecting circles of control (organizational and individual) with factors placed within these circles in a hierarchal manner based on their ability to be controlled or modified.14 Karsh and colleagues propose a multilevel systems model of technology design and implementation that has three levels (organization, system, and individual), and within each level the technology is fitted to the user, the task, the organization, and the environment.11 Uribe’s model was based on input from physicians and nurses at an academic medical center, whereas Karsh’s was based on input from 14 family physicians and their office staff with unknown experiences in error reporting. In Figure 1, we present an alternative model for an error reporting decision making and support system in a family medicine office, based on the practical application of our participants’ responses and previously reported theoretical models.11,14 We describe questions that people make when deciding to report an error and the organizational and system components that support error reporting by the 4 factors our study found to be of most importance to our participants (burden of effort, perceived benefit, clarity of request, and error properties).1,14,33 We believe that this model serves as a concrete and usable foundation for developing an error reporting system in family medicine.
The importance of reporting burden was made clear by our participants. Finding the time and remembering to report in the midst of a busy, stressful practice setting is difficult. This finding is not surprising because most research has found that ease of use and time are important factors in choosing to report errors.9,11,13,14 In this study, reports took an average of 2 to 5 minutes to complete. However, this estimate does not account for the break in work flow, time to access the reporting tool (paper or computer), and time to collect the information required for the report. This latter point was noted by participants in several groups who said that some information requested, such as demographic information about the patient, often necessitated having the chart present. In addition, requested information may not even be available at the time an error is discovered. For example, although patient outcomes in a hospital setting may be known in hours or even minutes, the outcomes from an error in family medicine may not be evident for days or weeks. This difference between the theoretical time needed to complete a report and the real time needed in the office setting, as well as the availability of required information for the error report, became clear only when those actually making such reports were questioned. Some of our participants felt the burden could be reduced by having one person make all the error reports, but this entails internal reporting to a peer or supervisor, which has its own set of problems.16 Other possibilities for future consideration include reporting systems that allow data to be entered at several points in time as information and outcomes become known and are still fresh in the reporter’s memory.
Participants are willing to overcome burdens to reporting when they feel the benefits are high. Although our study did not provide direct practice-level feedback, a desired feature of reporting systems in other studies,9,11,17,34 our participants noted they were learning about errors just by making the reports.4 For some, there was also an emotional benefit to reporting, giving participants a tangible method for dealing with the "angst," "frustration," and "guilt" that might accompany making or discovering errors. The role of emotion in decision making has become clearer in recent years and must be considered when designing an error reporting system.35 For a reporting system to maintain itself, however, ongoing benefits, including emotional support, regular feedback, and perceived quality improvement, will need to be an integral part of a system.11,14 Individuals who report to patient safety organizations created as a result of the Patient Safety and Quality Improvement Act of 2005 will have protection from discovery for confidential error reports. As a result, it may be feasible to integrate an educational and supportive feedback loop into the reporting system and subsequently to improve quality as an outcome of reporting. This will be a major perceived benefit that was missing from our study.
Error reporters also need clarity about what is wanted and needed in a report and who should make reports. Despite our best efforts at training, there was still confusion about what actually encompassed a testing process error. Individuals in some groups felt that their type of work (eg, clerical) was not included or that certain types of events didn’t really justify being reported, including those very rare and very common events. Clear definitions, with examples, need to be part of a reporting system. Many studies have found that the severity of the error (and resultant adverse event) is related to decisions to make reports.9–11,13,14,16 This, in turn, is related to that very definition of what constitutes an error8,36 and harm.37 Designers of systems also need to be aware of the apparent paradox surrounding the reporting of errors for which participants feel a personal sense of responsibility. When individuals feel they are responsible for an error, conflicting stories emerge about how this translates into an error report. Some find it cathartic to report, whereas others find it perhaps easier to justify or soft-sell their action as a face-saving response. Reporting systems need to be diligent about definitions, explanations, and training. Knowing that individuals are more likely to report those errors they perceive as serious, further research is needed among potential reporters to assist in developing a usable definition.8 The assurance and clarification of anonymity and confidentiality are also important issues, because other studies have found that these factors are cited in decision making about error reporting.5,11,14
This study has several limitations. Participants made error reports part of a research study with guaranteed anonymity. Factors relating to other types of error reporting systems might not have been mentioned with this frame of reference. At least one member of each practice was also committed to practice-based research. This altruistic benefit of moving the discipline’s understanding forward was seen by some as a benefit of this error reporting study but might not apply to other situations. As mentioned, our IRB required study protocol, which restricted feedback to the practices, may have blunted the true impact that feedback has on the reporting process. The focus groups allowed us to collect a wide range and breadth of responses, but we cannot quantify which are most important or which might apply to specific subgroups. The nonhomogenous nature of some groups (physicians, staff, and nurses in one group) may have stifled some conversation but was necessary to include small practices. We did separate physicians from staff and nurses whenever possible. Our findings, however, can be useful for future research and interventions to help define qualities of an error reporting system. Patient input was not included in the study, either as error reporters or focus group participants. They will have unique and important contributions to safety and will need to be included in future studies.
Staff and physicians at busy family practice offices across America were willing to identify and submit error reports about testing process errors for 8 months. Their insights into the process confirm that error reporting does constitute another burden on their time but that when they perceive benefits to themselves, their practices, and their patients, they are willing to take the time to submit reports. However, time commitment and confusions about what information and what type of error should be reported are barriers to the usefulness and quality of the reports and must be directly addressed when developing error reporting systems.
Notes
This article was externally peer-reviewed.
Funding: Supported by Agency for Healthcare Research and Quality Grant R21 HS13554-01.
Conflict of interest: none declared.
References
Kaplan H, Barach P. Incident reporting: science or protoscience Ten years later. Qual Saf Health Care 2002; 11: 144–5.
Sutcliffe KM. Defining and classifying medical error: lessons for learning. Qual Saf Health Care 2004; 13: 8–9.
West D, Westfall JM, Araya-Guerra R, et al. Using reported primary care errors to develop and implement patient safety interventions: a report from the ASIPS Collaborative. In: Advances in patient safety: from research to implementation. Rockville (MD): Agency for Healthcare Research and Quality; 2005 Publication no. 050021.
Elder NC, Graham D, Hickner J. Can a national primary care error-reporting system make a difference in medical practice Available from: http://www.npsf.org/html/Focus.html Focus on Patient Safety 2004; 7: 4–5.
Johnson CW. How will we get the data and what will we do with it then Issues in the reporting of adverse healthcare events. Qual Saf Health Care 2003; 12 Suppl 2: ii64–7.
Hobgood C, Xie J, Weiner B, Hooker J. Error identification, disclosure, and reporting: practice patterns of three emergency medicine provider types. Acad Emerg Med 2004; 11: 196–9.
Tamuz M, Thomas EJ, Franchois KE. Defining and classifying medical error: lessons for patient safety reporting systems. Qual Saf Health Care 2004; 13: 13–20.
Dovey SM, Phillips RL. What should we report to medical error reporting systems Qual Saf Health Care 2004; 13: 322–3.
Jeffe DB, Dunagan WC, Garbutt J, et al. Using focus groups to understand physicians’ and nurses’ perspectives on error reporting in hospitals. Jt Comm J Qual Saf 2004; 30: 471–9.
Taylor JA, Brownstein D, Christakis DA, et al. Use of incident reports by physicians and nurses to document medical errors in pediatric patients. Pediatrics 2004; 114: 729–35.
Karsh BT, Escoto KH, Beasley JW, Holden RJ. Toward a theoretical approach to medical error reporting system research and design. Appl Ergon 2006; 37: 283–95.
Wakefield DS, Wakefield BJ, Uden-Holman T, Blegen MA. Perceived barriers in reporting medication administration errors. Best Pract Benchmarking Healthc 1996; 1: 191–7.
Coyle YM, Mercer SQ, Murphy-Cullen CL, Schneider GW, Hynan LS. Effectiveness of a graduate medical education program for improving medical event reporting attitude and behavior. Qual Saf Health Care 2005; 14: 383–8.
Uribe CL, Schweikhart SB, Pathak DS, Dow M, Marsh GB. Perceived barriers to medical-error reporting: an exploratory investigation. J Healthc Manag 2002; 47: 263–79.
Carney J. Patient Safety and Quality Improvement Act of 2005. Bricker and Eckler Health Care Client Bulletin 2005; 05–06: 1–4.
Lawton R, Parker D. Barriers to incident reporting in a healthcare system. Qual Saf Health Care 2002; 11: 15–8.
Beasley JW, Escoto KH, Karsh BT. Design elements for a primary care medical error reporting system. WMJ 2004; 103: 56–9.
Cherry D, Burt C, Woodwell D. National Ambulatory Medical Care Survey, 2001 Summary, Advance Data from Vital and Health Statistics. Hyattsville (MD): National Center for Health Statistics; 2003.
Hickner JM, Fernald DH, Harris DM, Poon EG, Elder NC, Mold JW. Issues and initiatives in the testing process in primary care physician offices. Jt Comm J Qual Patient Saf 2005; 31: 81–9.
Dovey SM, Meyers DS, Phillips RL, Jr, et al. A preliminary taxonomy of medical errors in family practice. Qual Saf Health Care 2002; 11: 233–8.
Fernald DH, Pace WD, Harris DM, West DR, Main DS, Westfall JM. Event reporting to a primary care patient safety reporting system: A report from the ASIPS Collaborative. Annals of Fam Med 2004; 2: 327–332.
Phillips R, Dovey S, Graham D, Elder N, Hickner J. Learning from different lenses: Reports of medical errors in primary care by clinicians, staff and patients. J Patient Safety 2006; 2(3): 140–146.
Morgan DL. Successful focus groups: advancing the state of the art. Newbury Park (CA): Sage Press; 1993.
Fern E. Advanced focus group research. Thousand Oaks (CA): Sage Press; 2001.
Brown J. The use of focus groups in clinical research. In: Crabtree B, Miller W, editors. Doing Qualitative Research. 2nd Ed. Thousand Oaks (CA): Sage Press; 1999. pp. 109–26.
Boohaker EA, Ward RE, Uman JE, McCarthy BD. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med 1996; 156: 327–31.
Mold JW, Cacy DS, Dalbir DK. Management of laboratory test results in family practice. An OKPRN study. Oklahoma Physicans Resource/Research Network. J Fam Pract 2000; 49: 709–15.
Nutting PA, Main DS, Fischer PM, Stull TM, Pontious M, Seifert M, Jr, et al. Toward optimal laboratory use. Problems in laboratory testing in primary care. JAMA 1996; 275: 635–9.
Miller W, Crabtree B. Clinical research. A multi-method typology and qualitative roadmap. In: Crabtree B, Miller W, editors. Doing qualitative research. 2nd ed. Thousand Oaks (CA): Sage Press; 1999. pp. 3–30.
Miller W, Crabtree B. Qualitative analysis: how to begin making sense. Fam Prac Res J 1994; 14: 289–97.
Gibbs G. Qualitative data analysis. Explorations with NVivo. Philadelphia: Open University Press; 2002.
Fischer G, Fetters MD, Munro AP, Goldman EB. Adverse events in primary care identified from a risk-management database. J Fam Prac 1997; 45: 40–6.
Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care 2004; 13: 388–94.
Battles JB, Kaplan HS, Van der Schaaf TW, Shea CE. The attributes of medical event-reporting systems: experience with a prototype medical event-reporting system for transfusion medicine. Arch Pathol Lab Med 1998; 122: 231–8.
Zinn J. Risk, affect and emotion. In: Forum: Qualitative Social Research; 2006. Available from: www.qualitative-research.net/fqs-texts/1-06/06-1-29-E.htm.
Woods D, Cook R. Mistaking error. In: Youngberg B, Harlie M, editors. The patient safety handbook. Jones and Bartlett Publishers, Inc., Sudbury (MA); 2003. pp. 95–109.
Elder NC, Vonder Meulen MB, Cassedy A. The identification of medical errors by family physicians during outpatient visits. Ann Fam Med 2004; 2: 125–9.(Nancy C. Elder, MD, MSPH, Deborah Graham)