Effectiveness of Medical Resident Education in Mechanical Ventilation
Department of Medicine, University of North Carolina at Chapel Hill School of Medicine, Chapel Hill, North Carolina; Department of Medicine, Vanderbilt University School of Medicine, Nashville, Tennessee; Department of Medicine, Duke University School of Medicine, Durham, North Carolina; Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland; Department of Medicine, University of California at San Francisco School of Medicine, San Francisco, California; Department of Medicine, University of Colorado School of Medicine, Denver, Colorado; Department of Medicine, New York University School of Medicine, New York, New York; Upper Chesapeake Health System, Bel Air, Maryland; and Department of Medicine, University of Chicago School of Medicine, Chicago, Illinois*6')1d?, 百拇医药
ABSTRACT*6')1d?, 百拇医药
TOP*6')1d?, 百拇医药
ABSTRACT*6')1d?, 百拇医药
INTRODUCTION*6')1d?, 百拇医药
METHODS*6')1d?, 百拇医药
RESULTS*6')1d?, 百拇医药
DISCUSSION*6')1d?, 百拇医药
REFERENCES*6')1d?, 百拇医药
Specific methods of mechanical ventilation management reduce mortality and lower health care costs. However, in the face of a predicted deficit of intensivists, it is unclear whether residency programs are training internists to provide effective care for patients who require mechanical ventilation. To evaluate these educational outcomes, we administered a validated 19-item case-based test and survey to resident physicians at 31 diverse U.S. internal medicine residency programs nationwide. Of 347 senior residents, 259 (75%) responded. The mean test score was 74% correct (SD, 14%; range, 37 to 100%). Important items representing evidence-based standards of critical care answered incorrectly were as follows: use of appropriate tidal volume in the acute respiratory distress syndrome (48% incorrect), identifying a patient ready for a weaning trial (38% incorrect), and recognizing indication for noninvasive ventilation (27% incorrect). Most accurately identified pneumothorax (86% correct) and increased intrathoracic positive end-expiratory pressure (93% correct). Better scores were associated with "closed" versus "open" intensive care unit organization (76 versus 71% correct, p = 0.001), resident perception of greater versus lesser ventilator knowledge (79 versus 71% correct, p = 0.001), and graduation from a U.S. versus international medical school (75 versus 69% correct, p = 0.033). Although overall training satisfaction correlated strongly with program use of learning objectives (r = 0.89, p < 0.0001), only 46% reported being satisfied with their mechanical ventilation training. We conclude that senior residents may not be gaining essential evidence-based knowledge needed to provide effective care for patients who require mechanical ventilation. Residency programs should emphasize evidence-based learning objectives to guide mechanical ventilation instruction.
Key Words: clinical competence • education, medical • educational measurement • internship and residency • mechanical ventilation9z\6), http://www.100md.com
INTRODUCTION9z\6), http://www.100md.com
TOP9z\6), http://www.100md.com
ABSTRACT9z\6), http://www.100md.com
INTRODUCTION9z\6), http://www.100md.com
METHODS9z\6), http://www.100md.com
RESULTS9z\6), http://www.100md.com
DISCUSSION9z\6), http://www.100md.com
REFERENCES9z\6), http://www.100md.com
Randomized, controlled clinical trials have demonstrated that specific methods of mechanical ventilator management can improve mortality (1, 2), decrease the duration of mechanical ventilation and intensive care unit (ICU) stay (2, 3), and reduce costs and ICU complications (3) for the nearly 1.5 million U.S. patients who require mechanical ventilation each year (4). Critically ill patients with acute respiratory failure should be treated by evidence-based clinical practice guidelines as the standard of care (5–7).9z\6), http://www.100md.com
Intensivists care for only 36.8% of critically ill patients in the United States (8), whereas general internists, surgeons, and anesthesiologists manage the rest. National surveys have reported that 59% of general internists use mechanical ventilators in practice and that 67% of patients admitted to ICUs remain on the service of the primary physician (9). Of particular interest in the care of the critically ill is the growing numbers of hospitalists, most of whom are general internists who provide care for ICU patients (10). Overall, physicians trained in internal medicine direct 63% of U.S. ICUs, and only half of ICU medical directors and intensivists are certified in critical care medicine (8, 11).
In the coming decade, the United States may experience a significant unmet demand for physicians who are able to provide care to critically ill patients, primarily because of the aging of the population (8). Greater numbers of older patients are being treated in intensive care units than ever before (12), with those over age 65 years incurring about 60% of all ICU days (8). The management of patients requiring mechanical ventilation is a particular concern, as the incidence of acute respiratory failure requiring mechanical ventilation increases nearly 10-fold between the ages of 55 and 85 years (13). Because even more generalists will likely be needed to care for these patients in the future, it is incumbent on residency training programs to teach medical residents important elements of the management of persons requiring mechanical ventilation, including evidence-based standards of care.o96]li(, http://www.100md.com
It is unknown how successful residency programs have been in educating trainees in the management of mechanically ventilated patients. Therefore, we created a specific written mechanical ventilation test and questionnaire with the goals of (1) measuring the knowledge thought necessary by a panel of experts for graduating internal medicine residents to provide effective care for ventilated patients, (2) describing the perceptions of residents and residency program directors of the adequacy of this knowledge, and (3) assessing the characteristics of residents and residency programs that are associated with greater mechanical ventilator knowledge and satisfaction with training. We also surveyed residency program directors to understand how programs provide mechanical ventilation education and evaluate the outcomes of this instruction.
METHODS%?{e#%, 百拇医药
TOP%?{e#%, 百拇医药
ABSTRACT%?{e#%, 百拇医药
INTRODUCTION%?{e#%, 百拇医药
METHODS%?{e#%, 百拇医药
RESULTS%?{e#%, 百拇医药
DISCUSSION%?{e#%, 百拇医药
REFERENCES%?{e#%, 百拇医药
Item Development%?{e#%, 百拇医药
The overall goal was to create a short but inclusive test that would measure essential and evidence-based knowledge necessary for a senior medical resident to provide effective care for a mechanically ventilated patient. Eight board-certified medical intensivists with extensive experience in practicing and teaching critical care medicine in an academic setting, as well as in publishing on mechanical ventilation and medical education topics, were involved in test writing. This group aimed to compile a case-based, multiple choice examination written in the style of the American Board of Internal Medicine certification test. The group intended the test to address five specific areas of knowledge relevant to mechanical ventilation: (1) cardiopulmonary physiology, (2) indications for mechanical ventilation, (3) diagnosis and management of mechanical ventilator complications, (4) use of different ventilator modes and settings, and (5) interpretation of data from mechanical ventilators. The test consisted of four case vignettes featuring patients with commonly encountered potential indications for mechanical ventilation such as chronic obstructive pulmonary disease, asthma, acute respiratory distress syndrome (ARDS), and pneumonia. Multiple choice questions with four single-response options followed each case presentation.
Item Reductionv|ax'/@, http://www.100md.com
Twenty-five questions were initially generated and circulated by e-mail among the expert group members for comment. Four questions were dropped that related to issues not adequately supported by the medical literature, and two others were dropped because they were felt to be of lesser importance to a nonintensivist. All but five items remaining were changed on the basis of comments by group members, compiled by the primary author, and recirculated among group members three times before a pilot test consisting of 19 items was agreed on unanimously. A questionnaire was appended to the test to gather demographic information about the participants and to allow residents to report their perceptions about their training in mechanical ventilation as well as satisfaction with training.v|ax'/@, http://www.100md.com
Sample Selectionv|ax'/@, http://www.100md.com
We aimed to recruit about 30 to 40 residency programs for this study, seeking to include programs diverse in size, academic affiliation, and regional location. Our goal was to understand the outcomes of education in mechanical ventilation at a full range of programs.
Between December 2000 and January 2001, we sent e-mail messages to the 291 internal medicine training programs with functional e-mail addresses on the Accreditation Council of Graduate Medical Education's Web site (14). These messages described the study protocol and requested the participation of each program in the study. A total of 137 programs responded, of which 82 (28%) agreed to participate in the study. Two months later, we polled program directors again to assess expected compliance with the study protocol and feasibility for the program. At this point 12 failed to respond, 24 either could not guarantee full participation in the study protocol or expected a significant number of residents to be off site, and 8 declined to participate. We then chose a sample of 26 programs from the 36 remaining to assemble as diverse a group of programs as possible yet equally distributed in terms of geographic region, small versus large numbers of residents in the program, urban versus suburban or rural setting, and university versus community hospital affiliation.
Validity Assessment of the Questionnaire10jnrh, 百拇医药
Before testing a larger sample of residents, we sought to determine whether our questionnaire had evidence of validity (15), that is, if it truly measured mechanical ventilator knowledge of senior residents. The involvement of mechanical ventilation experts in the test development fulfilled assumptions for content validity, or the degree of appropriateness of test items included.10jnrh, 百拇医药
To evaluate more rigorous aspects of validity, the initial test was administered by residency or fellowship program directors at 5 university medical centers randomly selected from the 36 that agreed to participate. A total of 132 participants returned tests, including 103 internal medicine residents of all three levels of training, 19 pulmonary and critical care medicine fellows, and 10 attending physicians trained in pulmonary and critical care medicine. The anonymous tests took about 45 minutes to finish and were completed as a proctored group in most cases. The mean percentage of correct answers on the pilot test ranged from 67% (SD, 17%) for postgraduate first-year residents to 95% (SD, 6%) for attending physicians.
We evaluated the criterion validity of the questionnaire by testing for an empirical association between test scores and duration of training. Using one-way analysis of variance testing, we found that test scores increased significantly overall with year of training (p < 0.0001) from interns to senior residents. Last, we aimed to demonstrate evidence of construct validity, the theoretical assessment of validity based on the strength of relationships between test scores and other measured variables (15). We hypothesized that test scores would have high correlation with year of training (r > 0.6) but low correlations (r < 0.15) with testing site. Because our test was designed to assess knowledge needed by a senior resident to manage mechanically ventilated patients, we also hypothesized that test scores would increase significantly between each year of training, but not between fellows and attending physicians. These relationships behaved as expected, as we found Spearman correlation coefficients of 0.57 (p < 0.0001) and 0.03 (p = 0.49) between test scores and duration of training and test site, respectively. Using two-sample t tests, we also showed that there were significant differences in scores between each year of training (all p < 0.001), although there were no significant differences between the scores of fellows and attending physicians (p = 0.35). We made no further changes to the test (see resident test in online supplement).
Final Questionnaire Administrationd}t., 百拇医药
A total of 26 residency programs with 347 eligible senior residents not involved in the preliminary test participated in the final testing phase between April and July 2001. We tested only senior residents as close to the end of their training as possible, because we wished to measure the success of training programs in providing adequate education in mechanical ventilation over the course of a resident's entire 3 years of training. Program directors or chief medical residents administered the anonymous questionnaires, with no time limit, to senior residents at a single group gathering whenever possible. Most tests were proctored by a program director or chief resident, who then returned them by mail to the main study site. Program directors completed questionnaires before the residents' anonymous answer sheets were collected.d}t., 百拇医药
Variablesd}t., 百拇医药
The main outcome variable was percentage of correct answers on the mechanical ventilator test. Independent variables measured included characteristics of participants (satisfaction with training, perceived barriers to learning, and experiences in the ICU), residency programs (methods of resident evaluation, ICU organization, type of educational program in mechanical ventilation), and the perceptions of both residents and their program directors about the adequacy of trainee knowledge. Response choices for items addressing training perceptions included "strongly agree," "agree," "disagree," and "strongly disagree," although some included binary ("yes" and "no") responses. ICU organization was characterized as "open" if patients were monitored primarily by resident teams with both ICU and ward duties and "closed" if teams provided care solely for ICU patients.d}t., 百拇医药
Statistical Analysisd}t., 百拇医药
Univariate descriptive statistics were calculated for participants. We also used two-sample t tests or one-way analysis of variance tests to compare and report test scores by participant and program-level characteristics. Testing also was done using Wilcoxon rank sum and Kruskal–Wallis tests, although the results were similar to those obtained by parametric methods. Pearson(Christopher E. Cox, Shannon S. Carson, E. Wesley Ely, Joseph A. Govert, Joanne M. Garrett, Roy G. Br)
ABSTRACT*6')1d?, 百拇医药
TOP*6')1d?, 百拇医药
ABSTRACT*6')1d?, 百拇医药
INTRODUCTION*6')1d?, 百拇医药
METHODS*6')1d?, 百拇医药
RESULTS*6')1d?, 百拇医药
DISCUSSION*6')1d?, 百拇医药
REFERENCES*6')1d?, 百拇医药
Specific methods of mechanical ventilation management reduce mortality and lower health care costs. However, in the face of a predicted deficit of intensivists, it is unclear whether residency programs are training internists to provide effective care for patients who require mechanical ventilation. To evaluate these educational outcomes, we administered a validated 19-item case-based test and survey to resident physicians at 31 diverse U.S. internal medicine residency programs nationwide. Of 347 senior residents, 259 (75%) responded. The mean test score was 74% correct (SD, 14%; range, 37 to 100%). Important items representing evidence-based standards of critical care answered incorrectly were as follows: use of appropriate tidal volume in the acute respiratory distress syndrome (48% incorrect), identifying a patient ready for a weaning trial (38% incorrect), and recognizing indication for noninvasive ventilation (27% incorrect). Most accurately identified pneumothorax (86% correct) and increased intrathoracic positive end-expiratory pressure (93% correct). Better scores were associated with "closed" versus "open" intensive care unit organization (76 versus 71% correct, p = 0.001), resident perception of greater versus lesser ventilator knowledge (79 versus 71% correct, p = 0.001), and graduation from a U.S. versus international medical school (75 versus 69% correct, p = 0.033). Although overall training satisfaction correlated strongly with program use of learning objectives (r = 0.89, p < 0.0001), only 46% reported being satisfied with their mechanical ventilation training. We conclude that senior residents may not be gaining essential evidence-based knowledge needed to provide effective care for patients who require mechanical ventilation. Residency programs should emphasize evidence-based learning objectives to guide mechanical ventilation instruction.
Key Words: clinical competence • education, medical • educational measurement • internship and residency • mechanical ventilation9z\6), http://www.100md.com
INTRODUCTION9z\6), http://www.100md.com
TOP9z\6), http://www.100md.com
ABSTRACT9z\6), http://www.100md.com
INTRODUCTION9z\6), http://www.100md.com
METHODS9z\6), http://www.100md.com
RESULTS9z\6), http://www.100md.com
DISCUSSION9z\6), http://www.100md.com
REFERENCES9z\6), http://www.100md.com
Randomized, controlled clinical trials have demonstrated that specific methods of mechanical ventilator management can improve mortality (1, 2), decrease the duration of mechanical ventilation and intensive care unit (ICU) stay (2, 3), and reduce costs and ICU complications (3) for the nearly 1.5 million U.S. patients who require mechanical ventilation each year (4). Critically ill patients with acute respiratory failure should be treated by evidence-based clinical practice guidelines as the standard of care (5–7).9z\6), http://www.100md.com
Intensivists care for only 36.8% of critically ill patients in the United States (8), whereas general internists, surgeons, and anesthesiologists manage the rest. National surveys have reported that 59% of general internists use mechanical ventilators in practice and that 67% of patients admitted to ICUs remain on the service of the primary physician (9). Of particular interest in the care of the critically ill is the growing numbers of hospitalists, most of whom are general internists who provide care for ICU patients (10). Overall, physicians trained in internal medicine direct 63% of U.S. ICUs, and only half of ICU medical directors and intensivists are certified in critical care medicine (8, 11).
In the coming decade, the United States may experience a significant unmet demand for physicians who are able to provide care to critically ill patients, primarily because of the aging of the population (8). Greater numbers of older patients are being treated in intensive care units than ever before (12), with those over age 65 years incurring about 60% of all ICU days (8). The management of patients requiring mechanical ventilation is a particular concern, as the incidence of acute respiratory failure requiring mechanical ventilation increases nearly 10-fold between the ages of 55 and 85 years (13). Because even more generalists will likely be needed to care for these patients in the future, it is incumbent on residency training programs to teach medical residents important elements of the management of persons requiring mechanical ventilation, including evidence-based standards of care.o96]li(, http://www.100md.com
It is unknown how successful residency programs have been in educating trainees in the management of mechanically ventilated patients. Therefore, we created a specific written mechanical ventilation test and questionnaire with the goals of (1) measuring the knowledge thought necessary by a panel of experts for graduating internal medicine residents to provide effective care for ventilated patients, (2) describing the perceptions of residents and residency program directors of the adequacy of this knowledge, and (3) assessing the characteristics of residents and residency programs that are associated with greater mechanical ventilator knowledge and satisfaction with training. We also surveyed residency program directors to understand how programs provide mechanical ventilation education and evaluate the outcomes of this instruction.
METHODS%?{e#%, 百拇医药
TOP%?{e#%, 百拇医药
ABSTRACT%?{e#%, 百拇医药
INTRODUCTION%?{e#%, 百拇医药
METHODS%?{e#%, 百拇医药
RESULTS%?{e#%, 百拇医药
DISCUSSION%?{e#%, 百拇医药
REFERENCES%?{e#%, 百拇医药
Item Development%?{e#%, 百拇医药
The overall goal was to create a short but inclusive test that would measure essential and evidence-based knowledge necessary for a senior medical resident to provide effective care for a mechanically ventilated patient. Eight board-certified medical intensivists with extensive experience in practicing and teaching critical care medicine in an academic setting, as well as in publishing on mechanical ventilation and medical education topics, were involved in test writing. This group aimed to compile a case-based, multiple choice examination written in the style of the American Board of Internal Medicine certification test. The group intended the test to address five specific areas of knowledge relevant to mechanical ventilation: (1) cardiopulmonary physiology, (2) indications for mechanical ventilation, (3) diagnosis and management of mechanical ventilator complications, (4) use of different ventilator modes and settings, and (5) interpretation of data from mechanical ventilators. The test consisted of four case vignettes featuring patients with commonly encountered potential indications for mechanical ventilation such as chronic obstructive pulmonary disease, asthma, acute respiratory distress syndrome (ARDS), and pneumonia. Multiple choice questions with four single-response options followed each case presentation.
Item Reductionv|ax'/@, http://www.100md.com
Twenty-five questions were initially generated and circulated by e-mail among the expert group members for comment. Four questions were dropped that related to issues not adequately supported by the medical literature, and two others were dropped because they were felt to be of lesser importance to a nonintensivist. All but five items remaining were changed on the basis of comments by group members, compiled by the primary author, and recirculated among group members three times before a pilot test consisting of 19 items was agreed on unanimously. A questionnaire was appended to the test to gather demographic information about the participants and to allow residents to report their perceptions about their training in mechanical ventilation as well as satisfaction with training.v|ax'/@, http://www.100md.com
Sample Selectionv|ax'/@, http://www.100md.com
We aimed to recruit about 30 to 40 residency programs for this study, seeking to include programs diverse in size, academic affiliation, and regional location. Our goal was to understand the outcomes of education in mechanical ventilation at a full range of programs.
Between December 2000 and January 2001, we sent e-mail messages to the 291 internal medicine training programs with functional e-mail addresses on the Accreditation Council of Graduate Medical Education's Web site (14). These messages described the study protocol and requested the participation of each program in the study. A total of 137 programs responded, of which 82 (28%) agreed to participate in the study. Two months later, we polled program directors again to assess expected compliance with the study protocol and feasibility for the program. At this point 12 failed to respond, 24 either could not guarantee full participation in the study protocol or expected a significant number of residents to be off site, and 8 declined to participate. We then chose a sample of 26 programs from the 36 remaining to assemble as diverse a group of programs as possible yet equally distributed in terms of geographic region, small versus large numbers of residents in the program, urban versus suburban or rural setting, and university versus community hospital affiliation.
Validity Assessment of the Questionnaire10jnrh, 百拇医药
Before testing a larger sample of residents, we sought to determine whether our questionnaire had evidence of validity (15), that is, if it truly measured mechanical ventilator knowledge of senior residents. The involvement of mechanical ventilation experts in the test development fulfilled assumptions for content validity, or the degree of appropriateness of test items included.10jnrh, 百拇医药
To evaluate more rigorous aspects of validity, the initial test was administered by residency or fellowship program directors at 5 university medical centers randomly selected from the 36 that agreed to participate. A total of 132 participants returned tests, including 103 internal medicine residents of all three levels of training, 19 pulmonary and critical care medicine fellows, and 10 attending physicians trained in pulmonary and critical care medicine. The anonymous tests took about 45 minutes to finish and were completed as a proctored group in most cases. The mean percentage of correct answers on the pilot test ranged from 67% (SD, 17%) for postgraduate first-year residents to 95% (SD, 6%) for attending physicians.
We evaluated the criterion validity of the questionnaire by testing for an empirical association between test scores and duration of training. Using one-way analysis of variance testing, we found that test scores increased significantly overall with year of training (p < 0.0001) from interns to senior residents. Last, we aimed to demonstrate evidence of construct validity, the theoretical assessment of validity based on the strength of relationships between test scores and other measured variables (15). We hypothesized that test scores would have high correlation with year of training (r > 0.6) but low correlations (r < 0.15) with testing site. Because our test was designed to assess knowledge needed by a senior resident to manage mechanically ventilated patients, we also hypothesized that test scores would increase significantly between each year of training, but not between fellows and attending physicians. These relationships behaved as expected, as we found Spearman correlation coefficients of 0.57 (p < 0.0001) and 0.03 (p = 0.49) between test scores and duration of training and test site, respectively. Using two-sample t tests, we also showed that there were significant differences in scores between each year of training (all p < 0.001), although there were no significant differences between the scores of fellows and attending physicians (p = 0.35). We made no further changes to the test (see resident test in online supplement).
Final Questionnaire Administrationd}t., 百拇医药
A total of 26 residency programs with 347 eligible senior residents not involved in the preliminary test participated in the final testing phase between April and July 2001. We tested only senior residents as close to the end of their training as possible, because we wished to measure the success of training programs in providing adequate education in mechanical ventilation over the course of a resident's entire 3 years of training. Program directors or chief medical residents administered the anonymous questionnaires, with no time limit, to senior residents at a single group gathering whenever possible. Most tests were proctored by a program director or chief resident, who then returned them by mail to the main study site. Program directors completed questionnaires before the residents' anonymous answer sheets were collected.d}t., 百拇医药
Variablesd}t., 百拇医药
The main outcome variable was percentage of correct answers on the mechanical ventilator test. Independent variables measured included characteristics of participants (satisfaction with training, perceived barriers to learning, and experiences in the ICU), residency programs (methods of resident evaluation, ICU organization, type of educational program in mechanical ventilation), and the perceptions of both residents and their program directors about the adequacy of trainee knowledge. Response choices for items addressing training perceptions included "strongly agree," "agree," "disagree," and "strongly disagree," although some included binary ("yes" and "no") responses. ICU organization was characterized as "open" if patients were monitored primarily by resident teams with both ICU and ward duties and "closed" if teams provided care solely for ICU patients.d}t., 百拇医药
Statistical Analysisd}t., 百拇医药
Univariate descriptive statistics were calculated for participants. We also used two-sample t tests or one-way analysis of variance tests to compare and report test scores by participant and program-level characteristics. Testing also was done using Wilcoxon rank sum and Kruskal–Wallis tests, although the results were similar to those obtained by parametric methods. Pearson(Christopher E. Cox, Shannon S. Carson, E. Wesley Ely, Joseph A. Govert, Joanne M. Garrett, Roy G. Br)