And you thought writing a MCQ was hard! How to construct and implement script concordance tests: insights from a systematic review. Dory V et al. Medical Education 2012;46:552-563.
Reviewed by Christy Peterson
What was the study's purpose?
To provide educators with guidelines on how to design and implement a script concordance test (SCT).
What is a script concordance test?
It is a way to assess clinical reasoning in a test situation. It was devised by Bernard Charlin et al about 15 years ago. A SCT presents students with a short case such as "A two year old boy presents with an acute onset of respiratory distress. He is previously well, there is no family history of respiratory conditions, his growth is normal." Then, the student ranks how likely or not different diagnoses are (such as foreign body aspiration, epiglotitis, etc) based on extra information provided (such as no immunizations, presence of fever, etc).
How was the study done?
This is a systemic review. Researchers searched lots of sources and found 82 relevant references, most of which focused on justifying that SCTs are effective. About one quarter of the references explored how best to design and implement an SCT.
What were the results?
Tests with 75 questions (3 - 4 questions per case) should produce reliable scores. A team of 10-20 clinicians are required to build a test with good validity.
What are the implications of these findings?
SCTs could be used at the clerkship level and have been shown to be valid, reliable and discriminatory. Much more research, though, is required before SCTs should be used for high-stakes summative assessments.
Editor's note: Wow… a panel of 10-20 people to write a SCT! And we thought writing multiple choice questions were time consuming. Maybe a group of COMSEP members would like to tackle a SCT that all could use! If you want to learn more about this form of assessment, look for articles by Bernard Charlin (from the Universite de Montreal in Canada) who dreamt up this idea and has been writing about it ever since! (SLB)