Predicting USMLE failure
Is poor performance on NBME clinical subject examinations associated with a failing score on the USMLE step 3 examination?
Dong T et al. Academic Medicine 2014;89(5):762-766.
Reviewed by Jeanine Ronan
What was the study question?
The investigators evaluated whether or not poor performance on NBME Clinical Subject Examinations (shelf examinations) was associated with subsequent failure on the USMLE Step 3 Examination.
How was the study done?
Shelf examination performance for each core clerkship was reviewed from five years of graduating classes from the Uniformed Services University. Poor performance was defined as scoring less than one standard deviation below the national mean on any examination. Students were divided into three groups: students with no episodes of poor performance, students who had poor performance on only one clerkship and students with poor performance on two or more clerkships. Using contingency table analyses and regression modeling, they determined if group status was associated with failure on USMLE Step 3.
What were the results?
During the study period, 23 students failed Step 3 on the first attempt. Students with poor performance on one shelf exam had a 14.23 fold higher odds of failing Step 3, Students with poor performance on two or more shelf examinations had 33.41 fold higher odds when compared to those without any episodes of poor performance. The main limitation of this study was that it was conducted with data from only one institution.
What are the implications?
If students at risk of failure of Step 3 can be identified earlier due to shelf examination performance, medical schools can recommend appropriate remediation programs. In addition, student self-awareness about the risk of potential failure can be very motivating to encourage further study.
Editor's Note: A recent discussion on the COMSEP listserv centered on the pros and cons of using the shelf examination in the pediatric clerkship. Here is one more reason those who favor the shelf exam have to justify its use (JG).