Is it worth the effort?
What aspects of Letters of Recommendation Predict Performance in Medical School? Findings from One Institution DeZee KJ et al. Academic Medicine 2014;89:1408-1415.
Is it worth the effort?
Reviewed by Srividya Naganathan
What was the study question?
Do letters of recommendation (LOR's) from medical school applications predict medical students' performance in school?
How was the study done?
This was a retrospective cohort study of three consecutive years of graduating students at the Uniformed Services University of Health Sciences at Bethesda, Maryland. The 2 groups of students studied were: the top graduates elected to the Alpha Omega Alpha (AOA) Honor medical society and the "bottom of the class" (graduates who had the lowest grade point average (GPA)). Seventy six different characteristics were analyzed by 2 blinded investigators from the first 3 LOR's for each student.
What were the results?
A total of 437 LOR's were analyzed and divided into 4 categories: general characteristics, author characteristics, student characteristics and comparative rankings. The general characteristics were statistically similar in the 2 groups. The only difference observed in the author characteristics was that the AOA students' LOR's were more likely written by a supervisor/employer compared to the bottom of the class group (36% vs 22%, P=0.001). In the student characteristics, the difference seen was the author description of how well they knew the student, with AOA students classified as "very well" known as compared to bottom of the class students (41% vs 22%, P=0.003). Among the comparative rankings, AOA students were more likely to be labeled as "best compared to peers" (41% vs 17%, P=0.01) in contrast to bottom of class students who were more likely to have nonpositive comments (13% vs 6%, P=0.005).
What are the implications of these findings?
LOR's have been traditionally used in the application process for undergraduate and graduate medical training. It is questionable if they reliably predict educational outcomes. This study showed that LOR's have limited value in the admission process as very few characteristics had statistical significance.
Editor's note: Admission committees at both the UG and PG level spend a great deal of time reading LOR. And, we, as faculty members, spend a great deal of time writing such letters for prospective residents. This study confirms what many of us have suspected for a while - these letters may not be that informative when selecting students (SLB).