Council on Medical Student Education in Pediatrics


Search This Site

Journal Club

Changes on the medical education horizon

Conceptual and practical challenges in the assessment of physician competence. Whitehead CR et al. Medical Teacher 2015; 27:245-251.

Reviewed by Gary Beck

What was the purpose?
This article discusses a transitional step to expand the arsenal of assessment methods used for competency-based medical education.

What was reported?
This paper addresses the tension between meeting accreditation standards and developing meaningful assessment tools of competency-based medical education. The authors describe how competency frameworks should be considered. For example, the ACGME competencies of Professionalism and Interpersonal Skills and Communication have similar frameworks, but there are differences in content and structure. The authors suggest that as contexts change, competency frameworks need to adjust to reflect the circumstances. Competency frameworks help provide clear yet abstract models of real world constructs that multiple-choice exams cannot measure.

What were the recommendations?
The article emphasizes that educators need to rethink appropriate methods of assessing competency frameworks, focusing on two methodologies to accomplish this: ethnography and realist evaluation. Ethnographic assessment involves gathering information about social interactions through observation, discussions, and analysis of written documents. Realist evaluation looks for patterns of behavior in different contexts by gathering qualitative and quantitative data to develop a detailed understanding of outcomes in those different contexts. For each of these methods, rigorous qualitative analysis would need to be undertaken; quantitative analysis could be used with realist evaluations as well. These were just two examples, but the authors stated there are other methods that could likely be used.

What are the implications of these findings?
Assessment as a driver of education needs to be reconsidered. With the growing emphasis on the Core Entrustable Professional Activities for Entering Residency, use of traditional methods of assessment need to be reconsidered. What kind of multiple choice questions informs about a student’s professional behavior? Are artificial experiences inherent in OSCEs sufficient to inform educators about professionalism? These are issues that face clerkship directors and medical schools as the focus on competency-based medical student education expands.

Editor’s note: This may be an area where COMSEP could play a leadership role and develop innovative assessment methods that could be used - and studied - at each of our institutions (SLB).

Return to Journal Club