18
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
USMLE program in some capacity. These individuals represent 58 different medical and osteopathic
licensing boards throughout the United States.
USMLE Policies
The USMLE recommends that state medical boards require the dates of passing Step 1, Step 2, and
Step 3 to occur within a seven-year period. The program, however, recommends that state medical
boards consider additional time for individuals completing a dual degree program (MD/PhD;
DO/PhD). Additionally, the USMLE program imposes a limit of no more than six attempts to pass
each of the Step or Step Components. Additional attempts are allowed only at the written request of
a state medical board.
Most state medical boards utilizing the USMLE impose both time and attempt limits on the USMLE
as part of their requirements for obtaining an initial medical license. Currently, 40 out of 53 medical
boards impose some limit on the number of attempts at the USMLE; 45 out of 53 medical boards
impose a time limitation for the completion of the USMLE sequence. For a complete listing, please
visit:
www.fsmb.org/licensure/usmle-step-3/state_specific
.
Specific requirements for taking and retaking USMLE are provided in the FAQs on the USMLE
website
at:
www.usmle.org/frequently-asked-questions/
.
For information on exceptions to USMLE policy, contact the FSMB or visit the USMLE website
at
www.usmle.org/bulletin/eligibility/
.
19
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
USMLE Data and Research
Aggregate Performance Data
The USMLE program publishes aggregate performance data for all Steps since the program’s
inception. These data include examinee volume and passing percentages categorized by first-taker and
repeater examinees; US/Canadian and international students/graduates; allopathic and osteopathic
examinees. These performance data are available at the USMLE website
at
www.usmle.org/performance-data/
.
Passing rates and examinee counts for 2015-2016 are provided for each Step in this report’s Appendix.
Research Agenda
Each year, the USMLE Composite Committee reviews and endorses a research agenda for the
program. The committee endorsed the following research themes and/or topics for the program for
2017-2018: enhancements to the USMLE; relating scores and pass/fail outcomes to external measures;
determining strategies for providing meaningful performance feedback to examinees and stakeholders;
and USMLE security procedures.
2016 Publications
Below is a list of program-related publications by USMLE staff in 2015. A more extensive listing
(2009-2016) is available on the USMLE website at
http://usmle.org/data-research/
.
Clauser J, Hambleton R, Baldwin P. The effect of rating unfamiliar items on Angoff passing scores.
Educational and Psychological Measurement. 2016; Oct 10. [Epub ahead of print]
Margolis MJ, Mee JM, Clauser BE, Winward M, Clauser JC. Effect of content knowledge on
Angoff-style standard setting judgments.
Educational Measurement: Issues and Practice. 2016;35(1):29-37.
Cuddy MM, Winward ML, Johnston MM, Lipner RS, Clauser BE. Evaluating validity evidence for
USMLE Step 2 Clinical Skills data gathering and data interpretation scores: Does performance
predict history-taking and physical examination ratings for first-year internal medicine residents?
Academic Medicine. 2016;91:133-139.
Prober CG, Kolars JC, First LR, Melnick DE. A plea to reassess the role of United States Medical
Licensing Examination Step 1 scores in residency selection.
Academic Medicine. 2016;91:12-15.
Raymond MR, Ling Y, Grabovsky I. Investigating the performance of second language medical
students on lengthy clinical vignettes.
Evaluation & the Health Professions. 2016; Oct 19. [Epub ahead
of print]
Katsufrakis PJ, Uhler TA, Jones LD. The residency application process: pursuing improved
outcomes through better understanding of the issues.
Academic Medicine. 2016;91:1483-1487.
20
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Standard Setting
USMLE General Procedures for Standard Setting
The USMLE system for setting standards is established by the USMLE Composite Committee, which
includes representatives of the ECFMG, FSMB, NBME and the public. The system specifies the
kinds of data to be gathered and how the data are to be gathered, the frequency of reviewing the
standards and adjusting them, and assigns the judgment task to the Management Committee. The
Management Committee, jointly appointed by the FSMB and NBME, must use the procedures
defined by the Composite Committee, but is free to set the standard and revise the standard as it
deems necessary. The decision of the Management Committee is final; no superior governing
committee is authorized to alter its decision. The Management Committee includes those with
educational, licensing, and clinical practice perspectives, as well as a representative from the public.
Current policy requires that the Management Committee review the effectiveness of Step standards at
least annually. A comprehensive review and possible adjustment of the standard must be undertaken
approximately every four years. In addition, when there are any major changes to the design or format
of the Step examination, the Management Committee is asked to establish new passing requirements
for the redesigned components. USMLE believes that there must be an opportunity for review and
adjustment of standards in order to reflect the realities of change in the content of medicine, the nature
of the test, the characteristics of examinees, and the expectations of stakeholders. Such review of the
standard is essential to assure that the judgment inherent in defining the standard reflects current
conditions, not those that were pertinent in the past.
Mandated Data Sources Informing the Judgment Process
USMLE policy mandates the use of four categories of data in making judgments about standards.
These are:
•
Content-referenced judgments of experts. Content experts provide their opinions, based
upon review of content and examinee performance, on the appropriate requirements for
passing the examination.
•
Survey of stakeholders. Expectations of stakeholders for the percent of examinees, to whom
the stakeholder is exposed, that should pass the examination.
•
Cohort performance trends. Trends in examinee performance over a long period of time and
the effect of repeated attempts at the examinations on the failure rate in a defined cohort of
examinees.
•
Confidence intervals in the region of the cut-score. Estimates of numbers of misclassified
examinees based on historical distributions of examinee performance and the measurement
error in the scale area under consideration for the cut-score.
Setting the Standard
The Management Committee meets to consider the collected data. As part of this process the
committee reviews all of the data collection processes and considers the combined data as part of the
decision-making process. Typically, the question posed of the committee is whether the externally
collected data, performance trends, and score reliability data suggest that the current standards need
to be changed. The committee can allow the standards to remain the same or can vote to make a
change. If the latter occurs then the committee identifies the new performance requirements.
21
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
USMLE policy requires that standards be implemented on the first day of the month following the
decision of the Management Committee. Information regarding the timing of the standard setting
process and its outcomes is posted on the USMLE website.
22
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Resources
Websites:
Multiple avenues for obtaining additional information on the USMLE exist:
•
USMLE website (
www.usmle.org
) provides the most current information on the program.
•
FSMB website (
www.fsmb.org
) contains information specific to USMLE Step 3.
•
NBME website (
www.nbme.org
) contains information specific to registering for USMLE
Steps 1, 2CK and 2CK for students and graduates of U.S. and Canadian medical schools.
•
ECFMG website at (
www.ecfmg.org)
provides information on ECFMG certification and
registering for USMLE Steps 1, 2CK and 2CK for students and graduates of international
medical schools seeking information.
Written materials:
•
USMLE Bulletin of Information – provides USMLE policies and procedures and can be accessed
from the main page of the USMLE website (
www.usmle.org
).
•
NBME
Examiner – the official newsletter of the NBME & provides additional information on
USMLE; the current
and archived issues can be found under the Publications tab
at
www.nbme.org
.
•
Journal of Medical Regulation (previously the Journal of Medical Licensure and Discipline) – published
by the FSMB, the
Journal occasionally provides informational articles summarizing major
aspects of the USMLE program
. Topics covered include Step 2 Clinical Skills, the
development of multiple-choice questions for test content, research, and processes for
maintaining program security. The following articles are available
at
http://jmr.fsmb.org/archives/
or upon request from the FSMB:
o
“Implementing Strategic Changes to the USMLE.”
Journal of Medical Regulation. Vol.
100, No. 3, 2014.
o
“An Assessment of USMLE Examinees Found to Have Engaged in Irregular
Behavior, 1992-2006.”
Journal of Medical Regulation. Vol. 95, No. 4, 2010.
o
“Developing Content for the United States Medical Licensing Examination.”
Journal
of Medical Licensure and Discipline. Vol. 95, No. 2, 2009.
o
“Maintaining the Integrity of the Unites States Medical Licensing Examination.”
Journal of Medical Licensure and Discipline. Vol. 92, No. 3, 2006.
o
“The Introduction of Clinical Skills Assessment into the United States Medical
Licensing Examination (USMLE): A Description of the USMLE Step 2 Clinical Skills
(CS).”
Journal of Medical Licensure and Discipline. Vol. 91, No. 3, 2005.
o
“The United States Licensing Examination.”
The Journal of Medical Licensure and
Discipline. Vol. 91, No. 1, 2005.
23
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Key contacts:
The following individuals are key contacts for state medical boards on matters involving the USMLE.
David Johnson, MA
Michael Barone, MD
Federation of State Medical Boards
National Board of Medical Examiners
Sr. Vice President, Assessment Services
Vice President, Licensure Programs
400 Fuller Wiser Road
3750 Market Street
Euless, Texas 76039
Philadelphia, PA 19104-3190
817-868-4081;
djohnson@fsmb.org
215-590-9743;
mbarone@nbme.org
Amy Buono
Office of the USMLE Secretariat
3750 Market Street
Philadelphia, PA 19104-3190
215-590-9877;
abuono@nbme.org
24
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
APPENDIX
The data tables below are extracted from the performance data provided on the USMLE website
at
http://www.usmle.org/performance-data/
. Similar data are available for all years of the USMLE
program.
Table 1
2016 STEP 1 ADMINISTRATIONS *
Number Tested and Percent Passing
# Tested
# Passing
Examinees from US/Canadian Schools that Grant:
MD Degree
21,122
94%
1st Takers
20,122
96%
Repeaters**
1,000
64%
DO Degree
3,454
93%
1st Takers
3,398
94%
Repeaters**
56
75%
Total US/Canadian
24,576
94%
Examinees from Non-US/Canadian Schools
1st Takers
15,031
78%
Repeaters**
2,575
39%
Total non-US/Canadian
17,606
72%
Notes for Table 1
* Represents data for examinees tested in 2016 and reported through February 1, 2017.
** The # tested listed for repeaters represent examinations given, not the number of examinees for
the specified time period.
25
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Table 2
2015-2016 STEP 2 CLINICAL KNOWLEDGE (CK) ADMINISTRATIONS *
Number Tested and Percent Passing
# Tested
# Passing
Examinees from US/Canadian Schools that Grant:
MD Degree
21,515
96%
1st Takers
20,535
97%
Repeaters**
980
71%
DO Degree
2,272
94%
1st Takers
2,228
94%
Repeaters**
44
84%
Total US/Canadian
23,787
95%
Examinees from Non-US/Canadian Schools
1st Takers
12,720
80%
Repeaters**
2,738
53%
Total non-US/Canadian
15,440
75%
Notes for Table 2
* Data for Step 2 CK are provided for examinees tested during the period of July 1, 2015 to June 30,
2016.
** The # tested listed for repeaters represent examinations given, not the number of examinees for
the specified time period.
26
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Table 3
2015-2016 STEP 2 CLINICAL SKILLS (CS) ADMINISTRATIONS *
Number Tested and Percent Passing
# Tested
# Passing
Examinees from US/Canadian Schools that Grant:
MD Degree
20,662
97%
1st Takers
19,906
97%
Repeaters**
716
85%
DO Degree
46
91%
1st Takers
46
91%
Repeaters**
0
NA
Total US/Canadian
20,668
97%
Examinees from Non-US/Canadian Schools
1st Takers
12,051
82%
Repeaters**
2,300
71%
Total non-US/Canadian
14,351
81%
Notes for Table 3
* * Data for Step 2 CS are provided for examinees tested during the period of July 1, 2015 to June
30, 2016.
** The # tested listed for repeaters represent examinations given, not the number of examinees for
the specified time period.
27
Annual Report on USMLE to Medical Licensing Authorities in the U.S. | 2017
Table 4
2016 STEP 3 ADMINISTRATIONS *
Number Tested and Percent Passing
# Tested
# Passing
Examinees from US/Canadian Schools that Grant:
MD Degree
19,574
96%
1st Takers
18,997
97%
Repeaters**
597
70%
DO Degree
21
85%
1st Takers
20
95%
Repeaters**
1
§
Total US/Canadian
19,595
96%
Examinees from Non-US/Canadian Schools
1st Takers
8,804
86%
Repeaters**
1,355
53%
Total non-US/Canadian
10,159
81%
Notes for Table 4
* The table represents data for examinees tested in 2016 with scores reported by February 1, 2017.
** The # tested listed for repeaters represent examinations given, not the number of examinees for
the specified time period.
§
USMLE does not report percent for cohort populations of five or fewer examinations
Document Outline - Cover
- Table of Contents
- Executive Summary
- Introduction and program overview
- USMLE program news 2015-2017
- USMLE strategic communication outreach
- USMLE enhancements
- Medical licensing authorities and the USMLE
- USMLE data and research
- Standard setting
- Resources
- Appendix
Dostları ilə paylaş: |