Chapter 13: Designing and conducting summative evaluations

“Summative evaluation is the process of collecting data and information to make decisions about whether the instruction actually works as intended in the performance context; further, it is used to determine whether progress is being made in ameliorating the performance problems that prompted the instructional design and development effort. The main purpose in summative evaluation is to determine whether given instruction meets expectations.” (p. 343)

“Interest in summative evaluation has shifted from comparisons of innovations and statements of posttest performance to demonstrations of learner performance in the performance context where the skills were intended for use. Are the skills used by the learner in the workplace, and do they work? To answer these questions, there are two areas summative evaluation studies typically examine: the qualities of the instruction and the impact of the instruction on the organization. Instructional quality questions are answered using expert judgment reviews of the instructional materials and procedures. Organizational impact questions are answered using studies of skill transfer into the worksite after instruction is completed.” (p. 344)

“Summative evaluations are conducted to make decisions about whether to maintain, adopt, or adapt instruction. The primary evaluator in a summative evaluation is rarely the designer or developer of the instruction; the evaluator is frequently unfamiliar with the materials, the organization requesting the evaluation, or the setting in which the materials are evaluated. Such evaluators are referred to as external evaluators; these evaluators are preferred for summative evaluations because they have no personal investment in the instruction and are likely to be more objective about the strengths and weaknesses of the instruction.” (p. 364)

“Instructional designers make excellent summative evaluators because of their understanding of the instructional design process, the characteristics of well-designed instruction, and the criteria for evaluating instruction. These skills provide them with the expertise for designing and conducting the expert judgment as well as the impact analysis phases of the summative evaluation.” (p. 364)