CLINICAL REASONING ASSESSMENT: A WORKSHOP FOR DEVELOPING SCRIPT CONCORDANCE ASSESSMENT ITEMS.
Rebecca A Maldonado,University of Colorado School of Medicine,Aurora,CO; Debra Herrmann,George Washington University,,; Meghan D. Treitz,Childrens Hospital Colorado,,

Rationale:

Clinical reasoning skills are a key feature of the practice of medicine, but are often difficult to assess in learners.  A newer assessment modality of script concordance testing (SCT) has been developed and validated in several studies, and is being used in medical schools outside the US, during the clerkship years and residency training. The SCT is based upon the theory that clinical reasoning develops as pattern recognition (scripts) when learners are presented with clinical cases.  The scripts are used to apply key features of the patient’s clinical presentation to confirm or rule out hypotheses in the differential diagnosis. The learners’ decisions to increase or decrease the probability of a specific diagnosis in the differential actually being the correct diagnosis can be quantified, and compared to the decision-making process of a panel of experienced clinicians. This comparison provides a measure of the learner’s clinical reasoning ability.



Objectives:
  1. Describe the theory of script concordance testing (SCT) as a measure of clinical reasoning.
  2. Understand the reliability of script concordance testing.
  3. Recognize the parameters for developing an expert clinician panel
  4. Construct SCT items for clinical reasoning assessment.
  5. Conduct analytic tests to assess the quality of SCT items


Methods and Content:

In this workshop, the background of script concordance testing, its reliability, and the development of comparison expert panel item responses will be presented.  This is an ideal workshop for clerkship directors, as participants will learn an effective approach to assess learner’s clinical reasoning during the clerkship experience. The 2-hour workshop will consist of interactive group sessions and skill building exercises to develop and validate SCT assessment items

 

 

5 min

.

 

Activity:

Introductions

Objectives

Agenda for workshop

Presenter:

Maldonado

10 min

Objective: 1

 

Activity:

Brief didactic on script concordance theory

Presenter:

Herrmann

5 min

Objective: 4

 

 

Activity:

Interactive large group session on the construction of SCT items

Presenter:

Maldonado

30 min

Objective: 4

 

 

Activity:

Participants will develop case vignettes with 3-4 SCT items each

Facilitator(s):

All

5 min

Objective: 3

 

 

Activity:

Brief didactic on constructing an expert panel for validating SCT items

Presenter:

Herrmann

20 min

Objective: 5

 

 

Activity:

Participants will serve as expert panel members utilizing a set of SCT items

Facilitator(s):

All

10 min

 

Activity:

Large group debrief on expert panel experience.

 

15 min

Objective: 2 & 5

 

 

 

 

Activity:

Large group interactive session on SCT reliability literature and instructions for conducting analytic tests on the expert panel responses in order to determine reliability of specific SCT items

Presenter:

Treitz

10 min

 

Activity:

Q & A session

Facilitators:

All