|
Epstein RM. Assessment in medical education. N
Engl J Med 2007;356:387-96.
Reviewed by Bill Varade, MD; University of
Rochester, NY
What is the problem and what is known about it so far?
Clerkship directors and others charged with assessing the competency
of trainees and other professionals face challenges in developing
and utilizing appropriate, reliable, and valid assessment tools.
Part of the problem lies in defining what is being measured –
a particular area of competence – since it is often context-specific
(setting, disease specifics, etc.), content-specific (information
gathering and clinical reasoning), and developmental (novices tend
to use rule-based formulas while experts can make rapid, context-based
judgments in ambiguous situations). Importantly, in any given individual,
development of competence will usually vary across different contents
and contexts reflecting the particulars of training, exposure, personal
interests, etc.
Some competencies are more easily measured than others. Medical
facts tend to be more easily evaluated though may not accurately
predict clinical competence. Accurately gauging clinical reasoning
ability usually requires more sophisticated and varied assessments
while evaluation of competencies such as professionalism and the
ability to work effectively in teams becomes still more problematic.
The purpose of assessment varies as well, being formative or
summative and being used for the personal development of the learner,
for selection for advanced training, or for certification purposes.
Not all assessment tools can be used interchangeably for all these
goals and no one tool is appropriate for all situations.
What did the researchers find?
This is a review article, not a trial. In it, the author first
provides a concise summary of some of the theoretical foundations
of assessment as well as the potential benefits and perils of assessment.
A broad survey of the various assessment tools available to medical
educators is presented. Importantly, the appropriate context for
using the tools as well as the pros and cons for each method are
discussed. Future directions and challenges in assessment are addressed.
These include assessments of quality of care and patient safety,
teamwork, and professionalism. The need to use multiple types of
assessments longitudinally to gain an accurate, comprehensive picture
of an individual is stressed. However, the proper selection and
weighting of these individual tools to provide this global picture
remains a problem to be resolved. The need for standardization of
assessments within and between institutions to allow meaningful
comparisons of learners needs to be balanced with the development
of specific clerkship and institutional evaluation tools that reflect
their unique curriculum and culture. Also, the problems of avoiding
the unintended effects of assessment as well as the special situation
of evaluating clinical expertise are discussed.
What were the limitations of the study?
The article presumes some basic familiarity with the assessment
tools discussed. The format does not permit an extensive review
of each item. Further discussion of the costs (both in terms of
dollars and manpower required) would be helpful for those considering
using a particular tool. However, a comprehensive reference list
is provided to allow further investigation of specifics of the individual
techniques.
What are the implications of the study?
The article provides an informative compilation of assessment
tools available to medical educators. The table outlining the domains
assessed by the individual tools, their suggested use, limitations
and strengths as well as the references will be especially useful
for the clerkship director looking for new means to assess students'
competencies or to round out currently used assessment tools.
Editorial Comment: It is hard to know whether to be happy
that educational articles now appear in the NEJM or sad that the
content of this article has been found in most Clerkship Guides
and Curricula (including COMSEP) for more than a decade. Nonetheless,
this is a well written review of the subject. (Bill Raskza).
| 
