Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96. Reviewed by Bill Varade, MD; University of Rochester, NY
Epstein RM. Assessment in medical education. N Engl J Med 2007;356:387-96.
Reviewed by Bill Varade, MD; University of Rochester, NY
What is the problem and what is known about it so far?
Clerkship directors and others charged with assessing the competency of trainees and other professionals face challenges in developing and utilizing appropriate, reliable, and valid assessment tools. Part of the problem lies in defining what is being measured – a particular area of competence – since it is often context-specific (setting, disease specifics, etc.), content-specific (information gathering and clinical reasoning), and developmental (novices tend to use rule-based formulas while experts can make rapid, context-based judgments in ambiguous situations). Importantly, in any given individual, development of competence will usually vary across different contents and contexts reflecting the particulars of training, exposure, personal interests, etc.
Some competencies are more easily measured than others. Medical facts tend to be more easily evaluated though may not accurately predict clinical competence. Accurately gauging clinical reasoning ability usually requires more sophisticated and varied assessments while evaluation of competencies such as professionalism and the ability to work effectively in teams becomes still more problematic.
The purpose of assessment varies as well, being formative or summative and being used for the personal development of the learner, for selection for advanced training, or for certification purposes. Not all assessment tools can be used interchangeably for all these goals and no one tool is appropriate for all situations.
What did the researchers find?
This is a review article, not a trial. In it, the author first provides a concise summary of some of the theoretical foundations of assessment as well as the potential benefits and perils of assessment. A broad survey of the various assessment tools available to medical educators is presented. Importantly, the appropriate context for using the tools as well as the pros and cons for each method are discussed. Future directions and challenges in assessment are addressed. These include assessments of quality of care and patient safety, teamwork, and professionalism. The need to use multiple types of assessments longitudinally to gain an accurate, comprehensive picture of an individual is stressed. However, the proper selection and weighting of these individual tools to provide this global picture remains a problem to be resolved. The need for standardization of assessments within and between institutions to allow meaningful comparisons of learners needs to be balanced with the development of specific clerkship and institutional evaluation tools that reflect their unique curriculum and culture. Also, the problems of avoiding the unintended effects of assessment as well as the special situation of evaluating clinical expertise are discussed.
What were the limitations of the study?
The article presumes some basic familiarity with the assessment tools discussed. The format does not permit an extensive review of each item. Further discussion of the costs (both in terms of dollars and manpower required) would be helpful for those considering using a particular tool. However, a comprehensive reference list is provided to allow further investigation of specifics of the individual techniques.
What are the implications of the study?
The article provides an informative compilation of assessment tools available to medical educators. The table outlining the domains assessed by the individual tools, their suggested use, limitations and strengths as well as the references will be especially useful for the clerkship director looking for new means to assess students' competencies or to round out currently used assessment tools.
Editorial Comment: It is hard to know whether to be happy that educational articles now appear in the NEJM or sad that the content of this article has been found in most Clerkship Guides and Curricula (including COMSEP) for more than a decade. Nonetheless, this is a well written review of the subject. (Bill Raskza).