Journal Club

Cook, D. (2005). The Research We Still Are Not Doing: An Agenda for the Study of Computer-Based Learning. Acad Med, 80(6), 541-548.

Reviewed by Norman Berman, Dartmouth Medical School

Abstract: In 1994 Friedman published a paper, about computer-assisted instruction (CAI) research titled "The Research We Should Be Doing."¹ In this brief but seminal paper Friedman explained some of the difficulties inherent in performing media-comparative studies of CAI programs. He then went on to propose a CAI research agenda in which studies "might explicitly contrast different approaches to the design of computer-based instruction." Unfortunately, although this paper is often referenced, the majority of the published CAI literature today is comprised of media-comparative studies, which compare CAI to other teaching methods.

The current paper by Cook is meant to readdress the issue that was not solved by Friedman's paper. Cook explores confounding in media-comparative research and, using concepts promoted by Geoff Norman, he discusses the importance of a tightly controlled setting and that the various factors that contribute to a result are systematically varied based on a theory of causation. Savvy readers may recognize this as the scientific method!

Cook then proposes a framework for future studies that views all instructional design, in computer-based learning or other methods, as having 4 levels - medium, configuration, instructional method and presentation. Medium, as suggested by the term media-comparative studies, is the level where computer-assisted instruction is compared to other media such as books or lectures, a level at which meaningful comparison is not possible. Configuration refers to the differences within a given media format, such as CD-ROM vs. web, or synchronous vs. asynchronous learning. Instructional method refers to the teaching techniques that support a learning process, such as case-based or problem-based learning or the use of simulators. Finally, presentation refers to the elements of the medium that enhance the intervention, such as multimedia, use of hyperlinks and the use of interactive questioning. Cook suggests that to avoid confounding, research is best done within rather than between levels.

Finally, Cook discusses some possible themes and outcomes for research including adaptation to individual difference, just-in-time learning, simulation and integration. Importantly, there is recognition of the value of rigorous qualitative studies in CAI research.

Review: Cook effectively reiterates the message of Friedman's classic paper, and then goes on to propose a valuable new framework for future research on CAI. This paper should be required reading for anyone interested in pursuing research related to computer-assisted instruction. As COMSEP moves forward in CAI research, this article will be very helpful in focusing our efforts appropriately.

(Norm provides a great review of an area that is becoming increasingly more important in all areas of education. As some of you know I have been getting my Masters of Health Professional Education (MHPE) on-line and it has been quite eye opening about what type of a student I am on-line and with CAI. Have you ever participated as a learner with CAI? If not you should give it a try sometime from the learner's perspective. Have you tried to think about ways to research computer assisted instruction (CAI)? Would data about these different elements of CAI influence your use of this material? - Robin