Lynne . Smith,,Torrance,California,Cameron . Escovedo,,Los Angeles,CA,Deborah . Lehman,,Los Angeles,CA,Anju . Relan,,Los Angeles,CA,LuAnn . Wilkerson,,Los Angeles,CA,Lee Todd . Miller,,Los Angeles,CA
Background: The potential for self-assessments tools to improve self-directed learning and metacognition is well recognized.
Objective: To evaluate the effects of an optional practice exam on performance on a CLIPP-based written final examination.
Methods: From 2011-2013, third year medical students, on pediatric clerkship, created multiple-choice questions based on 32 required CLIPP cases. Questions were reviewed and edited by two faculty members and a senior medical student for internal consistency, content validity, and effective test writing practices, resulting in a 281-question item bank. From this, an online, multiple-choice self-assessment was developed that medical students could voluntarily access prior to the end of rotation CLIPP exam. Students’ use and completion of the practice exam was recorded. A 100-question CLIPP exam was administered, in accordance with the MedU-CLIPP protocol, during subsequent pediatric block rotations. To eliminate potential performance gains arising from the act of writing practice questions, we analyzed final examination scores of medical students from 2013-2014- a group that did not contribute to the creation of the online question bank, in order to determine the practice effect.
Results: Data from 194 third year medical students were analyzed. 75 (38.7%) students completed the full practice test, with 119 (61.3%) choosing to take partial or no practice test. Mean practice test score for users was 75.73%, S.D.=8.00, N=75. CLIPP exam mean score for practice test users was 81.36% (S.D.= 8.15), and for the partial or no practice test group, 79.39% (S.D.= 7.85). An independent-samples t-test performed to detect differences between users, and partial or non-users of the practice test showed no significant difference on the CLIPP exam scores favoring student-generated questions (95% CI, -.343 to 4.29,), t(153.04) = 1.67, p=.098).
Discussion: Our preliminary analysis of data shows that final exam performance was not affected by the utilization of a student-generated practice exam. Further analysis is warranted to reveal the effectiveness of a combination of creating questions and having practice questions available for test preparation. Differential volumes of practice may also be effective in predicting CLIPP final exam success, which we intend to investigate as a follow up study.