The Step 2 Clinical-Skills Examination
http://www.100md.com
《新英格兰医药杂志》
Editor's note: This is the first in a series of occasional articles on the education of physicians.
From their introduction in the late 1800s, medical licensing examinations in the United States included tests of clinical skills. The most recent modification of the bedside examination came in 1961, when instead of having decentralized examiners observe candidates, an examiner from the National Board of Medical Examiners (NBME) heard candidates as they took a history and watched them perform a physical examination. Three years later, however, the board analyzed the results of that examination and determined that it failed to produce the degree of statistical reliability that had been anticipated; thus, the examination was discontinued.1 The hope was to replace the unreliable examination with something better in the future, but problems persisted in subsequent iterations of the examination. As a result, American medical students have not been tested on their clinical skills as part of a licensing examination since 1964. Now all that is about to change.
The U.S. Medical Licensing Examination (USMLE) is cosponsored by the NBME and the Federation of State Medical Boards (FSMB), the association of state boards that govern the practice of medicine (see Figure). This three-step examination is required for medical licensure in the United States. In June 2004, a clinical-skills examination will be added to Step 2 of the USMLE, and beginning with the medical school graduating class of 2005, students will be required to pass it in order to be licensed. The examination will consist of a one-day test in which students will examine 12 standardized patients — laypeople who have been trained to simulate patients (by giving a medical history and, in some cases, even mimicking physical findings) and to evaluate students during the clinical encounter. The test will have three component scores: an integrated clinical-encounter score consisting of an assessment of skills in gathering data, including history taking and physical examination, and writing a note about the patient; an assessment of communication and other interpersonal skills; and an assessment of proficiency in spoken English. Students must pass all three components in order to pass the examination. The score will be reported as pass or fail. Physicians with the D.O. degree may also achieve initial licensure through the Comprehensive Osteopathic Medical Licensing Examination of the National Board of Osteopathic Medical Examiners. A clinical-skills component is scheduled to be added to this examination program by 2005.
Figure. A Time Line of Medical Licensing Examinations in the United States.
The NBME certifying examinations were available to students and graduates of accredited U.S. schools conferring the M.D. degree from 1916 through 1994. Nearly all state licensing authorities accepted passage of these examinations in lieu of other examination requirements, and about 80 percent of U.S. medical graduates were licensed on the basis of these tests. From the late 1950s until 1992, graduates of medical schools outside the United States and Canada were required to pass qualifying examinations that were administered by the Educational Commission for Foreign Medical Graduates (ECFMG) before taking the state-sponsored licensing examination. Beginning in 1968, all doctors who were eligible for licensure in a given state could meet the examination requirements by taking either the examinations developed by the individual state or the Federation Licensing Examination (FLEX). Between 1992 and 1994, the USMLE replaced all these separate examinations.
According to Dr. Donald Melnick, president of the NBME, there are strong reasons why such a clinical-skills component of the licensing examination is necessary. Physicians need clinical skills to be effective, and licensing examinations are expected to ensure that they have the competencies they require for practice. Those who license physicians have a duty to protect the public and to demonstrate the requisite quality of medicine in the United States. Moreover, the testing method that is being implemented (the examination of standardized patients) has been shown to be a reliable, valid, and feasible means of assessing fundamental clinical skills. More than 200 studies have assessed the reliability and validity of the techniques used in this examination, and the NBME has administered prototype examinations to thousands of students. Research by the NBME and other groups has shown that clinical-skills examinations measure relevant skill sets that are different from those measured by the multiple-choice questions included in the current Steps 1, 2, and 3 of the licensing examination.
Despite these arguments, the plan to reintroduce clinical-skills testing has engendered considerable controversy. The American Medical Student Association (AMSA) has opposed the test since 1999. In 2002, the American Medical Association (AMA) passed a resolution to oppose the examination "by any means, including possible legal action." The Council of Deans of the Association of American Medical Colleges (AAMC) has also expressed reservations about the cost of the test. AMSA and the Medical Student Section of the AMA, as well as the AMA as a whole, have put forth several arguments in opposition to the examination. One of the most important has to do with the expense involved. The fee for the examination will be $975, which does not include the travel and lodging costs for students, most of whom will have to travel in order to reach one of the five testing sites (located in Atlanta, Chicago, Houston, Los Angeles, and Philadelphia). This cost is a new addition to the large debt load carried by 80 percent of medical students — a debt that currently averages about $104,000 and is increasing steeply.
A second critical argument concerns logistic considerations. In order to allow for adequate remediation in the fourth year of medical school if deficiencies are identified, the NBME expects that most students will take this examination at the end of the third year or early in the fourth year. These are the critical months for rotations that lead to career choices and the selection of a residency program — an inopportune time to create further logistic constraints for students who are trying to schedule sought-after rotations that may influence their success in being matched with a desired program.
Furthermore, the student groups have argued that most medical schools are already evaluating clinical competencies by means of in-house assessments; they propose having the Liaison Committee on Medical Education, which is sponsored by the AAMC and the AMA, oversee the assessments that are already in place. But this committee, as the accrediting authority for medical education programs leading to the M.D. degree, is not involved in the licensure of physicians or the assessment of individual students beyond providing the accreditation of the medical school. Licensure rests with the state medical boards, which require passage of the USMLE.
The AMA, for its part, takes issue with the lack of proven validity of the test in terms of its ability to identify students who might have subsequent difficulty in the practice of medicine. There is, however, limited evidence that any examination, including the existing components of the USMLE, predicts long-term outcomes. Data from Canada show that scores on the Canadian licensure examination, which includes both cognitive- and clinical-skills components, do correlate with good clinical care.2 My colleagues and I have shown that medical students who engage in unprofessional behavior in medical school are more than twice as likely as other medical students to be disciplined by a state medical board when they are licensed physicians.3 But there is no precedent for requiring proof that a test predicts the future performance of a student, and the public has little patience for such an intellectual debate. According to a Harris poll commissioned by the FSMB, patients believe that doctors should be tested on their clinical skills.4 Those who administer the USMLE can also monitor performance outcomes prospectively.
The clinical-skills examination is scheduled to begin in June 2004. The AAMC, which supports the examination, has pledged to work with the NBME and the FSMB to minimize the financial burden placed on students. After discussions among the NBME, the AMSA, and the AMA, some, but not all, of the opposition has been lifted. This opposition is not binding, however, since the NBME and the FSMB, as independent agencies, can proceed with the clinical-skills examination despite opposition by the AMA.
The NBME has explained that adding several more testing sites would increase the cost of the examination and decrease testing reliability, because some testing facilities would be underused. However, the NBME anticipates that the patterns of utilization of the testing sites within the first year or two will demonstrate a need for a sixth testing location in the Northeast. Medical schools are also encouraged to include the cost of the examination in their financial-aid calculations in order to offset the expense for students.
Nonetheless, the AMSA continues to oppose the examination. In November 2003, in an effort to address the remaining concerns, the AMA sent a letter to the deans of medical schools encouraging them to ensure that there is an appropriate emphasis on clinical-skills training in the curriculum and to develop mechanisms for assisting students in meeting the costs of the examination. The letter also discouraged schools from making passage of the examination a requirement for graduation for at least five years after it is implemented.
In spite of the controversy, many of us who are involved in medical education believe that implementing the clinical-skills examination is the right thing to do. The success of the examination will depend on the ability of medical schools and students to overcome the logistic barriers, and the road will surely be bumpy. But the reality is that licensure represents a public trust in the physician's skills. In this era of increased professional accountability, how can the licensure process not include proof of clinical skills? The implementation of the Step 2 clinical-skills examination makes explicit to medical schools the importance of these skills. It is likely to drive curricular reform toward better instruction and assessment of clinical skills, and that may be the test's most enduring benefit. After all, we all study for what's on the test.
Source Information
From the Office of the Dean, Student Affairs, and the Department of Medicine, School of Medicine, University of California, San Francisco.
References
Hubbard JP, Levit EJ. The National Board of Medical Examiners: the first seventy years: a continuing commitment to excellence. Philadelphia: National Board of Medical Examiners, 1985.
Tamblyn R, Abrahamowicz M, Dauphinee WD, et al. Association between licensure examination scores and practice in primary care. JAMA 2002;288:3019-3026.
Papadakis MA, Hodgson CA, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med 2004;79:244-249.
United States medical licensing examination: results from Harris Interactive poll. (Accessed March 17, 2004, at http://www.usmle.org/news/cse/newsrelease2503.htm.)(Maxine A. Papadakis, M.D.)
From their introduction in the late 1800s, medical licensing examinations in the United States included tests of clinical skills. The most recent modification of the bedside examination came in 1961, when instead of having decentralized examiners observe candidates, an examiner from the National Board of Medical Examiners (NBME) heard candidates as they took a history and watched them perform a physical examination. Three years later, however, the board analyzed the results of that examination and determined that it failed to produce the degree of statistical reliability that had been anticipated; thus, the examination was discontinued.1 The hope was to replace the unreliable examination with something better in the future, but problems persisted in subsequent iterations of the examination. As a result, American medical students have not been tested on their clinical skills as part of a licensing examination since 1964. Now all that is about to change.
The U.S. Medical Licensing Examination (USMLE) is cosponsored by the NBME and the Federation of State Medical Boards (FSMB), the association of state boards that govern the practice of medicine (see Figure). This three-step examination is required for medical licensure in the United States. In June 2004, a clinical-skills examination will be added to Step 2 of the USMLE, and beginning with the medical school graduating class of 2005, students will be required to pass it in order to be licensed. The examination will consist of a one-day test in which students will examine 12 standardized patients — laypeople who have been trained to simulate patients (by giving a medical history and, in some cases, even mimicking physical findings) and to evaluate students during the clinical encounter. The test will have three component scores: an integrated clinical-encounter score consisting of an assessment of skills in gathering data, including history taking and physical examination, and writing a note about the patient; an assessment of communication and other interpersonal skills; and an assessment of proficiency in spoken English. Students must pass all three components in order to pass the examination. The score will be reported as pass or fail. Physicians with the D.O. degree may also achieve initial licensure through the Comprehensive Osteopathic Medical Licensing Examination of the National Board of Osteopathic Medical Examiners. A clinical-skills component is scheduled to be added to this examination program by 2005.
Figure. A Time Line of Medical Licensing Examinations in the United States.
The NBME certifying examinations were available to students and graduates of accredited U.S. schools conferring the M.D. degree from 1916 through 1994. Nearly all state licensing authorities accepted passage of these examinations in lieu of other examination requirements, and about 80 percent of U.S. medical graduates were licensed on the basis of these tests. From the late 1950s until 1992, graduates of medical schools outside the United States and Canada were required to pass qualifying examinations that were administered by the Educational Commission for Foreign Medical Graduates (ECFMG) before taking the state-sponsored licensing examination. Beginning in 1968, all doctors who were eligible for licensure in a given state could meet the examination requirements by taking either the examinations developed by the individual state or the Federation Licensing Examination (FLEX). Between 1992 and 1994, the USMLE replaced all these separate examinations.
According to Dr. Donald Melnick, president of the NBME, there are strong reasons why such a clinical-skills component of the licensing examination is necessary. Physicians need clinical skills to be effective, and licensing examinations are expected to ensure that they have the competencies they require for practice. Those who license physicians have a duty to protect the public and to demonstrate the requisite quality of medicine in the United States. Moreover, the testing method that is being implemented (the examination of standardized patients) has been shown to be a reliable, valid, and feasible means of assessing fundamental clinical skills. More than 200 studies have assessed the reliability and validity of the techniques used in this examination, and the NBME has administered prototype examinations to thousands of students. Research by the NBME and other groups has shown that clinical-skills examinations measure relevant skill sets that are different from those measured by the multiple-choice questions included in the current Steps 1, 2, and 3 of the licensing examination.
Despite these arguments, the plan to reintroduce clinical-skills testing has engendered considerable controversy. The American Medical Student Association (AMSA) has opposed the test since 1999. In 2002, the American Medical Association (AMA) passed a resolution to oppose the examination "by any means, including possible legal action." The Council of Deans of the Association of American Medical Colleges (AAMC) has also expressed reservations about the cost of the test. AMSA and the Medical Student Section of the AMA, as well as the AMA as a whole, have put forth several arguments in opposition to the examination. One of the most important has to do with the expense involved. The fee for the examination will be $975, which does not include the travel and lodging costs for students, most of whom will have to travel in order to reach one of the five testing sites (located in Atlanta, Chicago, Houston, Los Angeles, and Philadelphia). This cost is a new addition to the large debt load carried by 80 percent of medical students — a debt that currently averages about $104,000 and is increasing steeply.
A second critical argument concerns logistic considerations. In order to allow for adequate remediation in the fourth year of medical school if deficiencies are identified, the NBME expects that most students will take this examination at the end of the third year or early in the fourth year. These are the critical months for rotations that lead to career choices and the selection of a residency program — an inopportune time to create further logistic constraints for students who are trying to schedule sought-after rotations that may influence their success in being matched with a desired program.
Furthermore, the student groups have argued that most medical schools are already evaluating clinical competencies by means of in-house assessments; they propose having the Liaison Committee on Medical Education, which is sponsored by the AAMC and the AMA, oversee the assessments that are already in place. But this committee, as the accrediting authority for medical education programs leading to the M.D. degree, is not involved in the licensure of physicians or the assessment of individual students beyond providing the accreditation of the medical school. Licensure rests with the state medical boards, which require passage of the USMLE.
The AMA, for its part, takes issue with the lack of proven validity of the test in terms of its ability to identify students who might have subsequent difficulty in the practice of medicine. There is, however, limited evidence that any examination, including the existing components of the USMLE, predicts long-term outcomes. Data from Canada show that scores on the Canadian licensure examination, which includes both cognitive- and clinical-skills components, do correlate with good clinical care.2 My colleagues and I have shown that medical students who engage in unprofessional behavior in medical school are more than twice as likely as other medical students to be disciplined by a state medical board when they are licensed physicians.3 But there is no precedent for requiring proof that a test predicts the future performance of a student, and the public has little patience for such an intellectual debate. According to a Harris poll commissioned by the FSMB, patients believe that doctors should be tested on their clinical skills.4 Those who administer the USMLE can also monitor performance outcomes prospectively.
The clinical-skills examination is scheduled to begin in June 2004. The AAMC, which supports the examination, has pledged to work with the NBME and the FSMB to minimize the financial burden placed on students. After discussions among the NBME, the AMSA, and the AMA, some, but not all, of the opposition has been lifted. This opposition is not binding, however, since the NBME and the FSMB, as independent agencies, can proceed with the clinical-skills examination despite opposition by the AMA.
The NBME has explained that adding several more testing sites would increase the cost of the examination and decrease testing reliability, because some testing facilities would be underused. However, the NBME anticipates that the patterns of utilization of the testing sites within the first year or two will demonstrate a need for a sixth testing location in the Northeast. Medical schools are also encouraged to include the cost of the examination in their financial-aid calculations in order to offset the expense for students.
Nonetheless, the AMSA continues to oppose the examination. In November 2003, in an effort to address the remaining concerns, the AMA sent a letter to the deans of medical schools encouraging them to ensure that there is an appropriate emphasis on clinical-skills training in the curriculum and to develop mechanisms for assisting students in meeting the costs of the examination. The letter also discouraged schools from making passage of the examination a requirement for graduation for at least five years after it is implemented.
In spite of the controversy, many of us who are involved in medical education believe that implementing the clinical-skills examination is the right thing to do. The success of the examination will depend on the ability of medical schools and students to overcome the logistic barriers, and the road will surely be bumpy. But the reality is that licensure represents a public trust in the physician's skills. In this era of increased professional accountability, how can the licensure process not include proof of clinical skills? The implementation of the Step 2 clinical-skills examination makes explicit to medical schools the importance of these skills. It is likely to drive curricular reform toward better instruction and assessment of clinical skills, and that may be the test's most enduring benefit. After all, we all study for what's on the test.
Source Information
From the Office of the Dean, Student Affairs, and the Department of Medicine, School of Medicine, University of California, San Francisco.
References
Hubbard JP, Levit EJ. The National Board of Medical Examiners: the first seventy years: a continuing commitment to excellence. Philadelphia: National Board of Medical Examiners, 1985.
Tamblyn R, Abrahamowicz M, Dauphinee WD, et al. Association between licensure examination scores and practice in primary care. JAMA 2002;288:3019-3026.
Papadakis MA, Hodgson CA, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med 2004;79:244-249.
United States medical licensing examination: results from Harris Interactive poll. (Accessed March 17, 2004, at http://www.usmle.org/news/cse/newsrelease2503.htm.)(Maxine A. Papadakis, M.D.)