The Evaluation of Computer-Assisted Instruction
A great deal of research has been conducted to evaluate the effectiveness of computer tutorials and other forms of what is commonly described as computer assisted instruction (CAI). Most teachers do not follow the research reports that appear in research journals; nevertheless, they should pay some attention to the general conclusions of educational researchers because these findings should eventually be translated into changes in classroom practice.
Summary of Research on CAI. Because of this huge volume of information, it is common to rely on reviews rather than the results of individual studies in evaluating the effectiveness of CAI (Bayraktar, 2001–2002; Christmann, Badgett, & Lucking, 1997; Fletcher-Flinn & Gravatt, 1995; Tamim, Bernard, Borokhovski, Abrami & Schmid, 2011; Waxman, Lin, & Michko, 2003). This approach, called meta-analysis, combines the findings from many studies in an effort to come to a general conclusion and also to identify variables that may account for deviations from this general conclusion. It does appear that interest in general comparisons between CAI and “traditional” instruction is declining for reasons we will describe after a brief summary of this more traditional research. Tutorials and drill activities have been studied most extensively. These studies have found that technology seems to offer a moderate advantage over traditional instruction, with tutorials somewhat more effective than drill activities, particularly for younger and less able students (Lepper & Gurtner, 1989). The effect is labeled “moderate” because 66 percent of students taught using CAI performed better than the average for a group taught more traditionally.
One of the more recent reviews compared more current CAI research with older studies and suggests that the advantage of CAI may be increasing. However, the general benefits are still described as moderate (Fletcher-Flinn & Gravatt, 1995). A review of the research evaluating simulations reached a similar conclusion (Thomas & Hooper, 1991). With the exception of the work done with the Jasper Woodbury interactive video materials, research evaluating exploratory learning environments is much less extensive.
Even the moderate advantage for CAI instruction has been questioned, however. Critics point out that studies producing no advantage for CAI are less likely to be published, that many studies involving CAI do not control study time to make certain that students receiving CAI do not work longer, and that computers in many situations are so novel that students may respond to them more positively because of the uniqueness of learning with technology. For example, a meta-analysis determined that short-term CAI experiences produce a larger advantage over traditional instruction than comparisons involving instructional programs of longer duration, and the author interpreted this trend as supporting the novelty or motivational explanation (Bayraktar, 2001–2002). Others accept the findings that CAI may offer an advantage but argue that CAI materials are often just more carefully developed and that there is no intrinsic advantage in the actual method of instruction (Clark, 1985; Fletcher-Flinn & Gravatt, 1995). Our argument that researchers seem to have moved beyond comparisons of CAI and traditional instruction is very much related to this argument. The questions of interest now seem to be more specific. For example, the review of interactive multimodal variables (Moreno & Mayer, 2007) considers what combinations of media types and what forms of interactivity are most productive. While this may frustrate those who assumed simple and straightforward recommendations would be forthcoming, the acknowledgement that teaching and learning are complex seems no different when associated with questions regarding technology than with other educational topics.