Clinical Impact Associated with Corrected Results in Clinical Microbiology Testing
http://www.100md.com
微生物临床杂志 2005年第5期
Department of Laboratory Medicine
Department of Internal Medicine, University of Washington School of Medicine, Seattle, Washington
ABSTRACT
We developed a strategy to determine the clinical impact associated with errors in clinical microbiology testing. Over a 9-month period, we used a sequential three-stage method to prospectively evaluate 480 consecutive corrected microbiology laboratory reports. The three stages were physician review of the corrected report, medical record review, and interview with the clinician(s) taking care of the patient. Of the 480 corrected reports, 301 (62.7%) were ruled out for significant clinical impact by physician review and an additional 25 cases (5.2%) were ruled out for clinical impact by medical record review. This left 154 cases (32.1%) that required clinician interview to determine clinical impact. The clinician interview revealed that 32 (6.7%) of the corrected reports were associated with adverse clinical impact. Of these 32 cases, 19 (59.4%) involved delayed therapy, 8 (25.0%) involved unnecessary therapy, 8 (25.0%) were associated with inappropriate therapy, and 4 (12.5%) were associated with an increased level of care. The laboratory was entirely responsible for the error in 28 (87.5%) of the 32 cases and partially responsible in the other 4 cases (12.5%). Twenty-six (81.3%) of the 32 cases involved potentially preventable analytic errors that were due to lack of knowledge (cognitive error). In summary, we used evaluation of corrected reports to identify laboratory errors with adverse clinical impact, and most of the errors were amenable to laboratory-based interventions. Our method has the potential to be implemented in other laboratory settings to identify and characterize errors that impact patient safety.
INTRODUCTION
In 2000, the Institute of Medicine published the report, To Err is Human (5), which stated that adverse events are not uncommon and result in an estimated 44,000 to 98,000 hospital deaths per year. This publication has generated widespread interest in strategies for detecting and reducing adverse events related to medical errors. There are a number of published studies on clinical laboratory errors, and these focus primarily on the rate of laboratory errors and the classification of the errors (for a review, see reference 2). However, there is little data on laboratory-related adverse events, defined as physical insults or injuries due to laboratory error and not due to the patient's underlying condition.
There are two obstacles to studying laboratory-related adverse events, which must be overcome to carry out quality improvement projects aimed at reducing these events. The first obstacle is that most laboratories do not collect patient outcome data related to laboratory errors. These data are needed to understand the clinical consequences of laboratory error and to prioritize error reduction measures. The second is the practical challenge of detecting the relatively few cases of laboratory-related adverse events among the large number of tests performed. This obstacle is largely due to the lack of effective, rapid screening methods that laboratory personnel can apply to potential cases.
Sources of potential cases of laboratory-related adverse events include internal laboratory incident reports, risk management incident reports, physician complaints and other forms of physician collaboration, and a variety of daily information system reports. Daily information system reports can be automatically generated and highlight specific areas of suboptimal laboratory practice. Examples of information system reports include turnaround time reports which highlight test results associated with poor turnaround times and the daily list of corrected laboratory reports. A corrected laboratory report consists of an erroneous test result and the correction that was required to that result after the result's initial release. Each of these sources of cases has advantages and disadvantages for detecting laboratory-related adverse events. Although no single source can capture all adverse events, any of the sources can be used for quality improvement projects intended to improve patient safety. Some sources of cases also generate an unmanageably high number of cases and therefore require effective screening methods to significantly enrich for cases that cause adverse events.
In this report we describe a three-stage strategy that was applied prospectively to corrected laboratory reports to identify and characterize the clinical impact associated with problems in clinical microbiology testing. The results indicate that the strategy is feasible to implement and reproducible and detects clinically significant, preventable problems in clinical microbiology testing that are amenable to interventions.
MATERIALS AND METHODS
Participating institutions. The study was approved by the institutional review board of the host institutions. Two medical centers participated in the study, and each has a full-service microbiology laboratory that performs bacterial, mycobacterial, fungal, and parasitic testing. Virology testing was performed at a different laboratory and was not included in this study. Annually, one of the laboratories receives approximately 150,000 specimens and the other receives approximately 100,000 specimens. Each laboratory receives about 65% of the specimens from inpatients and the remainder from outpatients.
Corrected reports. In this prospective study, corrected reports were used as the source of cases of potential laboratory-related adverse events. In the laboratory information system (LIS) used by the participating laboratories, corrected reports were generated in the manual entry system of the LIS by pulling up the incorrect report by accession number, using a special prefix typed in before the accession number. The corrected report was then entered. Each corrected report in the LIS consisted of a description of the result as originally reported with its associated date and time and then the correction to that result with its associated date and time. The LIS had a database search function that allowed flagging of corrected reports, and a list of these corrected reports was automatically generated each morning. The list of corrected reports generated on Monday to Friday was reviewed the same day the list was generated. The reports created on Saturday and Sunday were reviewed on Monday. Routine updates of microbiology results prior to finalization were not considered corrections and were excluded from the study. Examples of routine updates include blood culture daily updates or specific final organism identification that was consistent with the preliminary result.
Evaluating the corrected reports, stage 1. We used a sequential, three-stage method to determine the presence or absence of clinical impact (Fig. 1). The first stage consisted of a rapid, daily review of corrected laboratory reports by a physician (A.P.L. or J.S.) specializing in infectious disease (ID) to identify those cases representing potentially clinically significant errors. This stage ruled out most of the corrected reports for further investigation since most were trivial modifications of unlikely clinical significance. General inclusion criteria for significant corrections included (i) removal or addition of organisms from Gram stain or culture results on a specimen from a sterile site, or pathogenic organisms that are not part of the normal flora on a specimen from a nonsterile site, and (ii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that could cause inappropriate antimicrobial therapy. Corrections of errors with low potential clinical impact were excluded, including (i) trivial changes in a report that do not affect the content of the laboratory result, such as minor typographical corrections; (ii) addition or removal of a nonpathogenic organism in the report that is an obvious contaminant from the site or strictly part of the normal flora; and (iii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that would not normally cause inappropriate antibiotic coverage and treatment.
Evaluating the corrected reports, stage 2. The second stage of evaluation of the corrected reports, which was applied only to reports that were scored as positive for potential clinical impact in stage one, consisted of medical record review (performed by S.Y., a pathology resident) to look for documentation of adverse events attributable to the laboratory error (Fig. 1). An adverse clinical impact attributable to laboratory error was considered present when one or more of the following occurred that could not be accounted for by the patient's underlying condition: (i) unnecessary or inappropriate drug therapy, where unnecessary therapy was defined as administration of antibiotics based on the incorrect report, when no antibiotic was necessary (inappropriate therapy was an indication by the clinician that the initial therapy based on the incorrect report was suboptimal and there was an improvement in antibiotic therapy based on the corrected report); (ii) delayed treatment, defined as an indication by the clinician that therapy would have been initiated hours to days earlier if the clinician had received the correct report instead of the incorrect report; (iii) unnecessary invasive test or procedure; (iv) transient morbidity (lasting <1 week) or prolonged morbidity (lasting >1 week), where morbidity is defined as fever, dyspnea, pain, or any other forms of physical discomfort or functional impairment; (v) increased level of care including emergency room visit, admission of outpatient to the hospital, transfer from regular ward to the intensive care unit, surgery or additional procedures, consultation by specialists, isolation precautions, and prolonged hospital stay; and (vi) death (since medical records can be incomplete and adverse events often are minimized or unreported [8, 14], absence of such documentation did not exclude a case from further investigation). The medical record review primarily served to exclude cases from further investigation when adverse events were clearly not possible. For examples, cases were excluded if the patient had died of an unrelated cause before the erroneous test result was reported or when available medical records clearly indicated that appropriate treatment had been implemented despite the initial error.
Evaluating the corrected reports, stage 3. Cases involving errors that were potentially clinically significant and where clinical impact could not be excluded by record review were selected for the third stage of the method, which was a detailed investigation to determine the patient outcome (Fig. 1). This investigation was performed by one of us (S.Y.) and involved additional review of medical records when necessary and a prompt (within 48 to 72 h) phone interview of at least one clinician directly involved in the care of the affected patient. Specifically, clinicians were asked to assess the presence or absence of the adverse clinical impacts listed previously. Occasionally, the clinicians were contacted again 1 to 2 weeks later to assess the longer-term clinical outcome. This was done in cases when the clinic impact could not be completely assessed at the time of the interview. For example, in a few cases, additional time and data were needed (e.g., laboratory tests, radiologic exams, follow-up physical exams) by the clinician to accurately assess the clinical impact of the error.
When an undesirable clinical impact related to a laboratory error was confirmed, the responsible laboratory was notified of the clinical consequences of the initial error, and the specific cause and nature of the error were further classified using a slightly modified version of a recently published classification scheme (1). The purpose of this classification scheme is to help choose interventions that will reduce the likelihood of the future occurrence of the error. Briefly, the cases were classified by specific clinical impact, the responsibility for the error (laboratory, nonlaboratory, or both), phase of testing (preanalytic, analytic, or postanalytic), and preventability. To facilitate the choice of effective intervention strategies, preventable errors were further categorized based on a cognitive psychology model as a cognitive error (also known as a mistake), which occurs due to lack of knowledge or poor judgment or a noncognitive error (also known as a slip), which occurs due to interruptions in a relatively automatic process (11, 12).
Intra- and interrater reproducibility. An ID physician (A.P.L.) performed the stage 1 evaluation of the corrected reports for about 90% of the corrected reports for this study. For the remainder of the reports, stage 1 evaluation was performed by a second ID physician (J.S.). Both physicians were blinded to the identity of the patient and the laboratory originating the report. To assess the specificity of the stage 1 review of the corrected report by the ID physician, 30 cases that were scored as negative after stage 1 were investigated for patient outcome using interviews of the involved care providers. To assess interrater reliability for stage 1 evaluation, 73 (15.2%) of the 480 corrected reports were chosen randomly and evaluated by both ID physicians. To assess intrarater reliability for the stage 1 evaluation, one of the physicians reevaluated 100 randomly chosen cases at a later date along with new cases, and the second physician reevaluated 94 cases. Inter- and intrarater reliability was measured using the Cohen kappa statistic (6).
RESULTS
Overview. The study covered a 9-month period encompassing parts of 2003 and 2004. During this time, approximately 164,000 specimens were submitted for testing to the two laboratories and there were 480 reports that required correction after finalization. Thus, approximately, 0.3% of the specimens submitted were associated with a corrected report. Table 1 summarizes the results of the three-stage strategy that was applied to the 480 corrected reports to detect cases involving adverse clinical impact. Table 2 gives six examples of cases which illustrate the overall method.
As shown in Table 1, 301 (62.7%) of the 480 cases could be ruled out for significant clinical impact by rapid review of each corrected report by an ID physician. An additional 25 cases (5.2%) could be ruled out for clinical impact after medical record review. This left 154 cases (32.1%) that required a clinician interview to determine clinical impact. The clinician interview revealed that, overall, 32 (6.7%) of the corrected reports were associated with an adverse clinical impact. Thus, approximately, 0.02% of the specimens submitted during the study period were associated with both a corrected report and an adverse impact.
Of the 30 cases that were scored by either reviewer as negative for clinical impact after stage 1 evaluation, a clinician interview showed that none resulted in a clinical impact. This confirmed that stage 1 evaluation was highly specific. In contrast, 32 (17.9%) of the 179 cases that were scored as positive after stage 1 led to an undesirable clinical impact. The difference in clinical impact between cases that were scored as negative for potential clinical impact in stage 1 and those that were scored as positive was statistically significant (P = 0.01, chi-square test).
Inter- and intrarater reliability. There was moderate interrater reliability regarding the results of stage 1 evaluation of the corrected reports by the two reviewers ( = 0.50) and substantial intrarater reliability ( = 0.79 and 0.83, respectively, for the two reviewers).
Classification of laboratory-related adverse events. (i) Summary. Table 3 summarizes the classification of the 32 cases in which there was an adverse clinical impact. The cases were classified according to the specific impact, responsibility for the error, the phase of laboratory testing involved, preventability, and error type if the error was preventable. Most errors were associated with delayed, inappropriate, or unnecessary therapy. The majority of errors were preventable, cognitive, analytic errors that occurred inside the laboratory, and thus were likely to be amenable to laboratory-based interventions, such as increased education or supervision.
(ii) Clinical impact. Of the 32 cases of laboratory-related adverse events, the most common adverse impact was delay in appropriate therapy, (range, 0.5 to 7 days; average, 1.7 days), and occurred in 19 (59.4%) cases. Inappropriate or unnecessary therapy was also common, and each was involved in eight (25.0%) cases. In four (12.5%) cases, the laboratory error led to an increased level of care, in the forms of requiring infectious disease consultations, transfer to the intensive care unit, and, in one case, surgery to evacuate empyema. In five (15.6%) cases, the care providers felt that the error clearly caused additional transient morbidity of the patient (prolonged fever, sepsis, dyspnea, persistent diarrhea, prolonged hospital stay) that could not be entirely accounted for by the patient's underlying condition.
(iii) Responsibility. Of the 32 laboratory-related adverse events, the laboratory bore all responsibility for the error in 28 (87.5%) of the cases and at least some responsibility in four (12.5%) of the cases. In four (12.5%) cases, nonlaboratory factors also played a role; for example, in one case a physician failed to act appropriately even after being notified of the correct result.
(iv) Phase of testing. Thirty-one (96.9%) of the 32 cases of adverse events involved an analytic error. None (0%) of 32 cases involved preanalytic errors, and one (3.1%) case was due to a postanalytic error, which was a data entry error. Of the 31 cases of analytic errors, 20 (64.5%) involved problems with Gram stain interpretation, 6 (19.4%) involved colony morphology evaluation, 7 (22.6%) related to biochemical testing, 1 (3.1%) was a misinterpretation of a trichrome-stained slide for ovum and parasite examination, and 1 (3.1%) involved in vitro antibiotic susceptibility testing. In 6 (19.4%) of the 31 analytic errors, there was failure to follow an established policy or procedure for the appropriate workup of specimens.
(v) Preventability and error type. Of the 32 cases with adverse clinical impact, 4 (12.5%) cases were considered unpreventable due to highly atypical Gram stain patterns that resulted in the initial misinterpretation by experienced technologists, 26 (81.3%) cases were preventable cognitive errors due to lack of knowledge, and 2 (6.3%) were noncognitive errors. One of the noncognitive errors was the postanalytic data entry error, and the second case was a slide of a body fluid that was inadvertently read before it was stained, causing yeast forms to be missed.
DISCUSSION
To our knowledge, this is the first study to describe using corrected laboratory reports as the basis for identifying and characterizing adverse events related to problems in microbiology testing. The method is reproducible and shows promise to be applicable throughout the clinical laboratory. The results of the study show that evaluation of corrected laboratory reports can reveal a significant number of preventable laboratory-related adverse events.
The use of corrected reports as a source of adverse events has a number of advantages. First, the list of corrected reports can be generated automatically by the laboratory information system, and therefore dependence on voluntary reporting of errors is avoided. In addition, the list of corrected reports can reflect events occurring within the last 24 h, allowing timely investigation. In contrast, incident reports, which are commonly used to detect laboratory errors (1), require voluntary reporting and usually have significant delays which limit the completeness and accuracy of investigation (15).
Corrected reports have a number of limitations. First, they miss many potential cases of laboratory-related adverse events. For example, delays in laboratory testing are a common source of potential adverse events and most delays do not generate a corrected report since the results are usually correct. Another problem with corrected reports is they are unlikely to randomly sample laboratory-related adverse events. Errors occurring in the laboratory are more likely to be identified and corrected by laboratory staff than errors occurring outside the laboratory, such as mislabeling and suboptimal specimens. In this study, the laboratory was entirely or partially responsible for all 32 cases of adverse events. In addition, the corrected-report method probably is biased toward detecting analytic errors rather than pre- or postanalytic errors. The method also tends to reveal problems with Gram stains, since the results of this screening test are usually reexamined and amended when subsequent culture results appear discrepant. For these reasons, the results obtained should not be extrapolated to estimate the overall rate or type of errors in the laboratory. Complete detection and characterization of laboratory-related adverse events would require the addition of other sources of cases, for example, incident reports, physician complaints, turnaround time reports, etc.
Our results suggest that the first two stages of evaluating the corrected reports, which are relatively rapid, produce a manageable number of cases for the more detailed, time-consuming investigation which takes place in the third stage. Thirty-two percent (Table 1) of all corrected reports were screened positive after the second stage of investigation, meaning that clinical impact had not been ruled out. This is one or two cases per day that would receive the detailed investigation.
In addition, the results indicate that the first stage of the method separates cases with clinical impact from those with no clinical impact. Thus, 32 (17.9%) of the 179 cases that were screened as positive after stage 1 had an adverse clinical impact revealed by a clinician interview. In contrast, when 30 cases screened as negative in stage 1 were investigated in a similar fashion, none had a clinical impact (P = 0.01, chi-square test).
A variety of clinical impacts were attributed to the laboratory errors, most commonly, delayed, inappropriate, or unnecessary therapy (Table 3). In most cases it was difficult to attribute additional clinical morbidity entirely to the laboratory error because of the complex medical conditions of many of the patients. However, increased morbidity due to laboratory error was likely to be present in a significant fraction of these cases. We were able to definitively confirm five cases of increased morbidity, including prolonged sepsis, diarrhea, fever, or dyspnea.
The evaluation of the clinical impact of the laboratory error was not always straightforward. First, there was the difficulty in deciding what the appropriate management would be without the error. For example, one clinician may start antibiotic therapy when new organisms are identified on the corrected urine culture report and state that the therapy was delayed by the error; a second clinician may consider this overtreatment of bacterial colonization. The second physician would not change patient management in this scenario and would consider the error to have no clinical impact. A second methodological problem was that it was not always possible to separate the impact of the laboratory error from the effects of the patient's underlying condition. For example, in one case concerning a patient with an abscess, a laboratory error delayed the appropriate therapy and exacerbated the condition. However, it was not possible to accurately determine the laboratory error's contribution to the patient's morbidity.
Our approach to the problem of obtaining accurate patient outcomes was to defer to the assessment of the physician caring for the patient, as this physician had the most complete information on the patient and was most likely to arrive at the appropriate clinical assessment. This physician was also in the best position to separate the impact of the laboratory error from the effects of the patient's underlying condition. However, in our experience, many clinicians still found this difficult to do and tended to be conservative in their assessment of the impact of the laboratory error. Not surprisingly, it was usually much easier for the clinicians to identify relatively objective impacts such as delayed therapy than impacts such as increased morbidity that require subjective assessment. Therefore, our results likely underestimate some of the clinical consequences of the laboratory errors.
Most of the errors occurred in the analytic phase of testing. In contrast, previous studies of laboratory specialties such as chemistry (7), molecular genetic testing (4), or the stat laboratory (10), as well as previous studies of laboratories in primary care (9) locations or the laboratory as a whole (1, 3, 13) described errors that were most frequently preanalytic. To a certain extent, this difference may reflect the bias of the different methods and sources of cases used. However, the difference may also reflect unique problems in clinical microbiology testing. In clinical microbiology testing, there is heavy reliance on manual work and subjective interpretation, whereas laboratory analyses in sections such as chemistry tend to be largely automated and objective. This point is highlighted by the high number of cases associated with errors in Gram stain interpretation. Gram stains are a manual, subjective, and largely preliminary screening test. The accuracy is affected by many factors, including sampling variability, technical problems, and the expertise of the reader.
Most of the 32 cases of adverse events involved preventable, cognitive errors under the laboratory's authority (Table 3). These cognitive errors are likely to respond to measures aimed at increasing the education, experience, and supervision of laboratory personnel (1, 12). Several such intervention strategies have already been implemented by the laboratories participating in the study.
In summary, the strategy of evaluating selected corrected laboratory reports is effective in identifying preventable adverse events associated with problems in microbiology testing. The errors identified by this method were largely analytic errors of the preventable cognitive type that occurred inside the microbiology laboratory. The errors are likely amenable to laboratory intervention strategies aimed at increasing education and supervision of the laboratory personnel. The methods described here show promise to be applicable throughout the clinical laboratory. Logical directions for future work include determining the portability of the method to other institutions and laboratory divisions (e.g., clinical chemistry, hematology) and modifications to the method to improve its utility. Ideally, the methods would be used in combination with other sources of adverse events, such as incident reports, other management reports (e.g., turnaround time reports), physician complaints, and other forms of collaboration with caregivers.
ACKNOWLEDGMENTS
We thank Ferric Fang, Brandi Limbago, Michael Haas, Yolanda Houze, and John Quick for sharing their professional expertise regarding this work.
Present address: Kaiser-Permanente, Vallejo Medical Center, 975 Sereno Drive, Vallejo, CA 94589.
REFERENCES
Astion, M. L., K. G. Shojania, T. R. Hamill, S. Kim, and V. L. Ng. 2003. Classifying laboratory incident reports to identify problems that jeopardize patient safety. Am. J. Clin. Pathol. 120:18-26.
Bonini, P., M. Plebani, F. Ceriotti, and F. Rubboli. 2002. Errors in laboratory medicine. Clin. Chem. 48:691-698.
Goldschmidt, H. M. J., and R. W. Lent. 1995. Gross errors and work flow analysis in the clinical laboratory. Klin. Biochem. Metab. 3:131-140.
Hofgartner, W. T., and J. F. Tait. 1999. Frequency of problems during clinical molecular-genetic testing. Am. J. Clin. Pathol. 112:14-21.
Kohn, L., J. Corrigan, and M. Donaldson. 2000. To err is human: building a safer health system. National Academic Press, Washington, D.C.
Kundel, H. L., and M. Polansky. 2003. Measurement of observer agreement. Radiology 228:303-308.
Lapworth, R., and T. K. Teal. 1994. Laboratory blunders revisited. Ann. Clin. Biochem. 31(Pt. 1):78-84.
Luck, J., J. W. Peabody, T. R. Dresselhaus, M. Lee, and P. Glassman. 2000. How well does chart abstraction measure quality A prospective comparison of standardized patients with the medical record. Am. J. Med. 108:642-649.
Nutting, P. A., D. S. Main, P. M. Fischer, T. M. Stull, M. Pontious, M. Seifert, Jr., D. J. Boone, and S. Holcomb. 1996. Toward optimal laboratory use. Problems in laboratory testing in primary care. JAMA 275:635-639.
Plebani, M., and P. Carraro. 1997. Mistakes in a stat laboratory: types and frequency. Clin. Chem. 43:1348-1351.
Reason, J. T. 1990. Human error. Cambridge University Press, New York, N.Y.
Shojania, K. G., H. Wald, and R. Gross. 2002. Understanding medical error and improving patient safety in the inpatient setting. Med. Clin. N. Am. 86:847-867.
Stahl, M., E. D. Lund, and I. Brandslund. 1998. Reasons for a laboratory's inability to report results for requested analytical tests. Clin. Chem. 44:2195-2197.
Thomas, E. J., and L. A. Petersen. 2003. Measuring errors and adverse events in health care. J. Gen. Intern. Med. 18:61-67.
Wald, H., and K. G. Shojania. 2001. Incident reporting, p. 41-50. In K. G. Shojania, B. W. Duncan, and K. M. McDonald (ed.), Making health care safer: a critical analysis of patient safety practices. Agency for Healthcare Research and Quality, Rockville, Md.(Shan Yuan, Michael L. Ast)
Department of Internal Medicine, University of Washington School of Medicine, Seattle, Washington
ABSTRACT
We developed a strategy to determine the clinical impact associated with errors in clinical microbiology testing. Over a 9-month period, we used a sequential three-stage method to prospectively evaluate 480 consecutive corrected microbiology laboratory reports. The three stages were physician review of the corrected report, medical record review, and interview with the clinician(s) taking care of the patient. Of the 480 corrected reports, 301 (62.7%) were ruled out for significant clinical impact by physician review and an additional 25 cases (5.2%) were ruled out for clinical impact by medical record review. This left 154 cases (32.1%) that required clinician interview to determine clinical impact. The clinician interview revealed that 32 (6.7%) of the corrected reports were associated with adverse clinical impact. Of these 32 cases, 19 (59.4%) involved delayed therapy, 8 (25.0%) involved unnecessary therapy, 8 (25.0%) were associated with inappropriate therapy, and 4 (12.5%) were associated with an increased level of care. The laboratory was entirely responsible for the error in 28 (87.5%) of the 32 cases and partially responsible in the other 4 cases (12.5%). Twenty-six (81.3%) of the 32 cases involved potentially preventable analytic errors that were due to lack of knowledge (cognitive error). In summary, we used evaluation of corrected reports to identify laboratory errors with adverse clinical impact, and most of the errors were amenable to laboratory-based interventions. Our method has the potential to be implemented in other laboratory settings to identify and characterize errors that impact patient safety.
INTRODUCTION
In 2000, the Institute of Medicine published the report, To Err is Human (5), which stated that adverse events are not uncommon and result in an estimated 44,000 to 98,000 hospital deaths per year. This publication has generated widespread interest in strategies for detecting and reducing adverse events related to medical errors. There are a number of published studies on clinical laboratory errors, and these focus primarily on the rate of laboratory errors and the classification of the errors (for a review, see reference 2). However, there is little data on laboratory-related adverse events, defined as physical insults or injuries due to laboratory error and not due to the patient's underlying condition.
There are two obstacles to studying laboratory-related adverse events, which must be overcome to carry out quality improvement projects aimed at reducing these events. The first obstacle is that most laboratories do not collect patient outcome data related to laboratory errors. These data are needed to understand the clinical consequences of laboratory error and to prioritize error reduction measures. The second is the practical challenge of detecting the relatively few cases of laboratory-related adverse events among the large number of tests performed. This obstacle is largely due to the lack of effective, rapid screening methods that laboratory personnel can apply to potential cases.
Sources of potential cases of laboratory-related adverse events include internal laboratory incident reports, risk management incident reports, physician complaints and other forms of physician collaboration, and a variety of daily information system reports. Daily information system reports can be automatically generated and highlight specific areas of suboptimal laboratory practice. Examples of information system reports include turnaround time reports which highlight test results associated with poor turnaround times and the daily list of corrected laboratory reports. A corrected laboratory report consists of an erroneous test result and the correction that was required to that result after the result's initial release. Each of these sources of cases has advantages and disadvantages for detecting laboratory-related adverse events. Although no single source can capture all adverse events, any of the sources can be used for quality improvement projects intended to improve patient safety. Some sources of cases also generate an unmanageably high number of cases and therefore require effective screening methods to significantly enrich for cases that cause adverse events.
In this report we describe a three-stage strategy that was applied prospectively to corrected laboratory reports to identify and characterize the clinical impact associated with problems in clinical microbiology testing. The results indicate that the strategy is feasible to implement and reproducible and detects clinically significant, preventable problems in clinical microbiology testing that are amenable to interventions.
MATERIALS AND METHODS
Participating institutions. The study was approved by the institutional review board of the host institutions. Two medical centers participated in the study, and each has a full-service microbiology laboratory that performs bacterial, mycobacterial, fungal, and parasitic testing. Virology testing was performed at a different laboratory and was not included in this study. Annually, one of the laboratories receives approximately 150,000 specimens and the other receives approximately 100,000 specimens. Each laboratory receives about 65% of the specimens from inpatients and the remainder from outpatients.
Corrected reports. In this prospective study, corrected reports were used as the source of cases of potential laboratory-related adverse events. In the laboratory information system (LIS) used by the participating laboratories, corrected reports were generated in the manual entry system of the LIS by pulling up the incorrect report by accession number, using a special prefix typed in before the accession number. The corrected report was then entered. Each corrected report in the LIS consisted of a description of the result as originally reported with its associated date and time and then the correction to that result with its associated date and time. The LIS had a database search function that allowed flagging of corrected reports, and a list of these corrected reports was automatically generated each morning. The list of corrected reports generated on Monday to Friday was reviewed the same day the list was generated. The reports created on Saturday and Sunday were reviewed on Monday. Routine updates of microbiology results prior to finalization were not considered corrections and were excluded from the study. Examples of routine updates include blood culture daily updates or specific final organism identification that was consistent with the preliminary result.
Evaluating the corrected reports, stage 1. We used a sequential, three-stage method to determine the presence or absence of clinical impact (Fig. 1). The first stage consisted of a rapid, daily review of corrected laboratory reports by a physician (A.P.L. or J.S.) specializing in infectious disease (ID) to identify those cases representing potentially clinically significant errors. This stage ruled out most of the corrected reports for further investigation since most were trivial modifications of unlikely clinical significance. General inclusion criteria for significant corrections included (i) removal or addition of organisms from Gram stain or culture results on a specimen from a sterile site, or pathogenic organisms that are not part of the normal flora on a specimen from a nonsterile site, and (ii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that could cause inappropriate antimicrobial therapy. Corrections of errors with low potential clinical impact were excluded, including (i) trivial changes in a report that do not affect the content of the laboratory result, such as minor typographical corrections; (ii) addition or removal of a nonpathogenic organism in the report that is an obvious contaminant from the site or strictly part of the normal flora; and (iii) misidentification of organisms by Gram stain, culture gross morphology, biochemical testing, or molecular testing that would not normally cause inappropriate antibiotic coverage and treatment.
Evaluating the corrected reports, stage 2. The second stage of evaluation of the corrected reports, which was applied only to reports that were scored as positive for potential clinical impact in stage one, consisted of medical record review (performed by S.Y., a pathology resident) to look for documentation of adverse events attributable to the laboratory error (Fig. 1). An adverse clinical impact attributable to laboratory error was considered present when one or more of the following occurred that could not be accounted for by the patient's underlying condition: (i) unnecessary or inappropriate drug therapy, where unnecessary therapy was defined as administration of antibiotics based on the incorrect report, when no antibiotic was necessary (inappropriate therapy was an indication by the clinician that the initial therapy based on the incorrect report was suboptimal and there was an improvement in antibiotic therapy based on the corrected report); (ii) delayed treatment, defined as an indication by the clinician that therapy would have been initiated hours to days earlier if the clinician had received the correct report instead of the incorrect report; (iii) unnecessary invasive test or procedure; (iv) transient morbidity (lasting <1 week) or prolonged morbidity (lasting >1 week), where morbidity is defined as fever, dyspnea, pain, or any other forms of physical discomfort or functional impairment; (v) increased level of care including emergency room visit, admission of outpatient to the hospital, transfer from regular ward to the intensive care unit, surgery or additional procedures, consultation by specialists, isolation precautions, and prolonged hospital stay; and (vi) death (since medical records can be incomplete and adverse events often are minimized or unreported [8, 14], absence of such documentation did not exclude a case from further investigation). The medical record review primarily served to exclude cases from further investigation when adverse events were clearly not possible. For examples, cases were excluded if the patient had died of an unrelated cause before the erroneous test result was reported or when available medical records clearly indicated that appropriate treatment had been implemented despite the initial error.
Evaluating the corrected reports, stage 3. Cases involving errors that were potentially clinically significant and where clinical impact could not be excluded by record review were selected for the third stage of the method, which was a detailed investigation to determine the patient outcome (Fig. 1). This investigation was performed by one of us (S.Y.) and involved additional review of medical records when necessary and a prompt (within 48 to 72 h) phone interview of at least one clinician directly involved in the care of the affected patient. Specifically, clinicians were asked to assess the presence or absence of the adverse clinical impacts listed previously. Occasionally, the clinicians were contacted again 1 to 2 weeks later to assess the longer-term clinical outcome. This was done in cases when the clinic impact could not be completely assessed at the time of the interview. For example, in a few cases, additional time and data were needed (e.g., laboratory tests, radiologic exams, follow-up physical exams) by the clinician to accurately assess the clinical impact of the error.
When an undesirable clinical impact related to a laboratory error was confirmed, the responsible laboratory was notified of the clinical consequences of the initial error, and the specific cause and nature of the error were further classified using a slightly modified version of a recently published classification scheme (1). The purpose of this classification scheme is to help choose interventions that will reduce the likelihood of the future occurrence of the error. Briefly, the cases were classified by specific clinical impact, the responsibility for the error (laboratory, nonlaboratory, or both), phase of testing (preanalytic, analytic, or postanalytic), and preventability. To facilitate the choice of effective intervention strategies, preventable errors were further categorized based on a cognitive psychology model as a cognitive error (also known as a mistake), which occurs due to lack of knowledge or poor judgment or a noncognitive error (also known as a slip), which occurs due to interruptions in a relatively automatic process (11, 12).
Intra- and interrater reproducibility. An ID physician (A.P.L.) performed the stage 1 evaluation of the corrected reports for about 90% of the corrected reports for this study. For the remainder of the reports, stage 1 evaluation was performed by a second ID physician (J.S.). Both physicians were blinded to the identity of the patient and the laboratory originating the report. To assess the specificity of the stage 1 review of the corrected report by the ID physician, 30 cases that were scored as negative after stage 1 were investigated for patient outcome using interviews of the involved care providers. To assess interrater reliability for stage 1 evaluation, 73 (15.2%) of the 480 corrected reports were chosen randomly and evaluated by both ID physicians. To assess intrarater reliability for the stage 1 evaluation, one of the physicians reevaluated 100 randomly chosen cases at a later date along with new cases, and the second physician reevaluated 94 cases. Inter- and intrarater reliability was measured using the Cohen kappa statistic (6).
RESULTS
Overview. The study covered a 9-month period encompassing parts of 2003 and 2004. During this time, approximately 164,000 specimens were submitted for testing to the two laboratories and there were 480 reports that required correction after finalization. Thus, approximately, 0.3% of the specimens submitted were associated with a corrected report. Table 1 summarizes the results of the three-stage strategy that was applied to the 480 corrected reports to detect cases involving adverse clinical impact. Table 2 gives six examples of cases which illustrate the overall method.
As shown in Table 1, 301 (62.7%) of the 480 cases could be ruled out for significant clinical impact by rapid review of each corrected report by an ID physician. An additional 25 cases (5.2%) could be ruled out for clinical impact after medical record review. This left 154 cases (32.1%) that required a clinician interview to determine clinical impact. The clinician interview revealed that, overall, 32 (6.7%) of the corrected reports were associated with an adverse clinical impact. Thus, approximately, 0.02% of the specimens submitted during the study period were associated with both a corrected report and an adverse impact.
Of the 30 cases that were scored by either reviewer as negative for clinical impact after stage 1 evaluation, a clinician interview showed that none resulted in a clinical impact. This confirmed that stage 1 evaluation was highly specific. In contrast, 32 (17.9%) of the 179 cases that were scored as positive after stage 1 led to an undesirable clinical impact. The difference in clinical impact between cases that were scored as negative for potential clinical impact in stage 1 and those that were scored as positive was statistically significant (P = 0.01, chi-square test).
Inter- and intrarater reliability. There was moderate interrater reliability regarding the results of stage 1 evaluation of the corrected reports by the two reviewers ( = 0.50) and substantial intrarater reliability ( = 0.79 and 0.83, respectively, for the two reviewers).
Classification of laboratory-related adverse events. (i) Summary. Table 3 summarizes the classification of the 32 cases in which there was an adverse clinical impact. The cases were classified according to the specific impact, responsibility for the error, the phase of laboratory testing involved, preventability, and error type if the error was preventable. Most errors were associated with delayed, inappropriate, or unnecessary therapy. The majority of errors were preventable, cognitive, analytic errors that occurred inside the laboratory, and thus were likely to be amenable to laboratory-based interventions, such as increased education or supervision.
(ii) Clinical impact. Of the 32 cases of laboratory-related adverse events, the most common adverse impact was delay in appropriate therapy, (range, 0.5 to 7 days; average, 1.7 days), and occurred in 19 (59.4%) cases. Inappropriate or unnecessary therapy was also common, and each was involved in eight (25.0%) cases. In four (12.5%) cases, the laboratory error led to an increased level of care, in the forms of requiring infectious disease consultations, transfer to the intensive care unit, and, in one case, surgery to evacuate empyema. In five (15.6%) cases, the care providers felt that the error clearly caused additional transient morbidity of the patient (prolonged fever, sepsis, dyspnea, persistent diarrhea, prolonged hospital stay) that could not be entirely accounted for by the patient's underlying condition.
(iii) Responsibility. Of the 32 laboratory-related adverse events, the laboratory bore all responsibility for the error in 28 (87.5%) of the cases and at least some responsibility in four (12.5%) of the cases. In four (12.5%) cases, nonlaboratory factors also played a role; for example, in one case a physician failed to act appropriately even after being notified of the correct result.
(iv) Phase of testing. Thirty-one (96.9%) of the 32 cases of adverse events involved an analytic error. None (0%) of 32 cases involved preanalytic errors, and one (3.1%) case was due to a postanalytic error, which was a data entry error. Of the 31 cases of analytic errors, 20 (64.5%) involved problems with Gram stain interpretation, 6 (19.4%) involved colony morphology evaluation, 7 (22.6%) related to biochemical testing, 1 (3.1%) was a misinterpretation of a trichrome-stained slide for ovum and parasite examination, and 1 (3.1%) involved in vitro antibiotic susceptibility testing. In 6 (19.4%) of the 31 analytic errors, there was failure to follow an established policy or procedure for the appropriate workup of specimens.
(v) Preventability and error type. Of the 32 cases with adverse clinical impact, 4 (12.5%) cases were considered unpreventable due to highly atypical Gram stain patterns that resulted in the initial misinterpretation by experienced technologists, 26 (81.3%) cases were preventable cognitive errors due to lack of knowledge, and 2 (6.3%) were noncognitive errors. One of the noncognitive errors was the postanalytic data entry error, and the second case was a slide of a body fluid that was inadvertently read before it was stained, causing yeast forms to be missed.
DISCUSSION
To our knowledge, this is the first study to describe using corrected laboratory reports as the basis for identifying and characterizing adverse events related to problems in microbiology testing. The method is reproducible and shows promise to be applicable throughout the clinical laboratory. The results of the study show that evaluation of corrected laboratory reports can reveal a significant number of preventable laboratory-related adverse events.
The use of corrected reports as a source of adverse events has a number of advantages. First, the list of corrected reports can be generated automatically by the laboratory information system, and therefore dependence on voluntary reporting of errors is avoided. In addition, the list of corrected reports can reflect events occurring within the last 24 h, allowing timely investigation. In contrast, incident reports, which are commonly used to detect laboratory errors (1), require voluntary reporting and usually have significant delays which limit the completeness and accuracy of investigation (15).
Corrected reports have a number of limitations. First, they miss many potential cases of laboratory-related adverse events. For example, delays in laboratory testing are a common source of potential adverse events and most delays do not generate a corrected report since the results are usually correct. Another problem with corrected reports is they are unlikely to randomly sample laboratory-related adverse events. Errors occurring in the laboratory are more likely to be identified and corrected by laboratory staff than errors occurring outside the laboratory, such as mislabeling and suboptimal specimens. In this study, the laboratory was entirely or partially responsible for all 32 cases of adverse events. In addition, the corrected-report method probably is biased toward detecting analytic errors rather than pre- or postanalytic errors. The method also tends to reveal problems with Gram stains, since the results of this screening test are usually reexamined and amended when subsequent culture results appear discrepant. For these reasons, the results obtained should not be extrapolated to estimate the overall rate or type of errors in the laboratory. Complete detection and characterization of laboratory-related adverse events would require the addition of other sources of cases, for example, incident reports, physician complaints, turnaround time reports, etc.
Our results suggest that the first two stages of evaluating the corrected reports, which are relatively rapid, produce a manageable number of cases for the more detailed, time-consuming investigation which takes place in the third stage. Thirty-two percent (Table 1) of all corrected reports were screened positive after the second stage of investigation, meaning that clinical impact had not been ruled out. This is one or two cases per day that would receive the detailed investigation.
In addition, the results indicate that the first stage of the method separates cases with clinical impact from those with no clinical impact. Thus, 32 (17.9%) of the 179 cases that were screened as positive after stage 1 had an adverse clinical impact revealed by a clinician interview. In contrast, when 30 cases screened as negative in stage 1 were investigated in a similar fashion, none had a clinical impact (P = 0.01, chi-square test).
A variety of clinical impacts were attributed to the laboratory errors, most commonly, delayed, inappropriate, or unnecessary therapy (Table 3). In most cases it was difficult to attribute additional clinical morbidity entirely to the laboratory error because of the complex medical conditions of many of the patients. However, increased morbidity due to laboratory error was likely to be present in a significant fraction of these cases. We were able to definitively confirm five cases of increased morbidity, including prolonged sepsis, diarrhea, fever, or dyspnea.
The evaluation of the clinical impact of the laboratory error was not always straightforward. First, there was the difficulty in deciding what the appropriate management would be without the error. For example, one clinician may start antibiotic therapy when new organisms are identified on the corrected urine culture report and state that the therapy was delayed by the error; a second clinician may consider this overtreatment of bacterial colonization. The second physician would not change patient management in this scenario and would consider the error to have no clinical impact. A second methodological problem was that it was not always possible to separate the impact of the laboratory error from the effects of the patient's underlying condition. For example, in one case concerning a patient with an abscess, a laboratory error delayed the appropriate therapy and exacerbated the condition. However, it was not possible to accurately determine the laboratory error's contribution to the patient's morbidity.
Our approach to the problem of obtaining accurate patient outcomes was to defer to the assessment of the physician caring for the patient, as this physician had the most complete information on the patient and was most likely to arrive at the appropriate clinical assessment. This physician was also in the best position to separate the impact of the laboratory error from the effects of the patient's underlying condition. However, in our experience, many clinicians still found this difficult to do and tended to be conservative in their assessment of the impact of the laboratory error. Not surprisingly, it was usually much easier for the clinicians to identify relatively objective impacts such as delayed therapy than impacts such as increased morbidity that require subjective assessment. Therefore, our results likely underestimate some of the clinical consequences of the laboratory errors.
Most of the errors occurred in the analytic phase of testing. In contrast, previous studies of laboratory specialties such as chemistry (7), molecular genetic testing (4), or the stat laboratory (10), as well as previous studies of laboratories in primary care (9) locations or the laboratory as a whole (1, 3, 13) described errors that were most frequently preanalytic. To a certain extent, this difference may reflect the bias of the different methods and sources of cases used. However, the difference may also reflect unique problems in clinical microbiology testing. In clinical microbiology testing, there is heavy reliance on manual work and subjective interpretation, whereas laboratory analyses in sections such as chemistry tend to be largely automated and objective. This point is highlighted by the high number of cases associated with errors in Gram stain interpretation. Gram stains are a manual, subjective, and largely preliminary screening test. The accuracy is affected by many factors, including sampling variability, technical problems, and the expertise of the reader.
Most of the 32 cases of adverse events involved preventable, cognitive errors under the laboratory's authority (Table 3). These cognitive errors are likely to respond to measures aimed at increasing the education, experience, and supervision of laboratory personnel (1, 12). Several such intervention strategies have already been implemented by the laboratories participating in the study.
In summary, the strategy of evaluating selected corrected laboratory reports is effective in identifying preventable adverse events associated with problems in microbiology testing. The errors identified by this method were largely analytic errors of the preventable cognitive type that occurred inside the microbiology laboratory. The errors are likely amenable to laboratory intervention strategies aimed at increasing education and supervision of the laboratory personnel. The methods described here show promise to be applicable throughout the clinical laboratory. Logical directions for future work include determining the portability of the method to other institutions and laboratory divisions (e.g., clinical chemistry, hematology) and modifications to the method to improve its utility. Ideally, the methods would be used in combination with other sources of adverse events, such as incident reports, other management reports (e.g., turnaround time reports), physician complaints, and other forms of collaboration with caregivers.
ACKNOWLEDGMENTS
We thank Ferric Fang, Brandi Limbago, Michael Haas, Yolanda Houze, and John Quick for sharing their professional expertise regarding this work.
Present address: Kaiser-Permanente, Vallejo Medical Center, 975 Sereno Drive, Vallejo, CA 94589.
REFERENCES
Astion, M. L., K. G. Shojania, T. R. Hamill, S. Kim, and V. L. Ng. 2003. Classifying laboratory incident reports to identify problems that jeopardize patient safety. Am. J. Clin. Pathol. 120:18-26.
Bonini, P., M. Plebani, F. Ceriotti, and F. Rubboli. 2002. Errors in laboratory medicine. Clin. Chem. 48:691-698.
Goldschmidt, H. M. J., and R. W. Lent. 1995. Gross errors and work flow analysis in the clinical laboratory. Klin. Biochem. Metab. 3:131-140.
Hofgartner, W. T., and J. F. Tait. 1999. Frequency of problems during clinical molecular-genetic testing. Am. J. Clin. Pathol. 112:14-21.
Kohn, L., J. Corrigan, and M. Donaldson. 2000. To err is human: building a safer health system. National Academic Press, Washington, D.C.
Kundel, H. L., and M. Polansky. 2003. Measurement of observer agreement. Radiology 228:303-308.
Lapworth, R., and T. K. Teal. 1994. Laboratory blunders revisited. Ann. Clin. Biochem. 31(Pt. 1):78-84.
Luck, J., J. W. Peabody, T. R. Dresselhaus, M. Lee, and P. Glassman. 2000. How well does chart abstraction measure quality A prospective comparison of standardized patients with the medical record. Am. J. Med. 108:642-649.
Nutting, P. A., D. S. Main, P. M. Fischer, T. M. Stull, M. Pontious, M. Seifert, Jr., D. J. Boone, and S. Holcomb. 1996. Toward optimal laboratory use. Problems in laboratory testing in primary care. JAMA 275:635-639.
Plebani, M., and P. Carraro. 1997. Mistakes in a stat laboratory: types and frequency. Clin. Chem. 43:1348-1351.
Reason, J. T. 1990. Human error. Cambridge University Press, New York, N.Y.
Shojania, K. G., H. Wald, and R. Gross. 2002. Understanding medical error and improving patient safety in the inpatient setting. Med. Clin. N. Am. 86:847-867.
Stahl, M., E. D. Lund, and I. Brandslund. 1998. Reasons for a laboratory's inability to report results for requested analytical tests. Clin. Chem. 44:2195-2197.
Thomas, E. J., and L. A. Petersen. 2003. Measuring errors and adverse events in health care. J. Gen. Intern. Med. 18:61-67.
Wald, H., and K. G. Shojania. 2001. Incident reporting, p. 41-50. In K. G. Shojania, B. W. Duncan, and K. M. McDonald (ed.), Making health care safer: a critical analysis of patient safety practices. Agency for Healthcare Research and Quality, Rockville, Md.(Shan Yuan, Michael L. Ast)