Chapter 9: RDE

Brief Overview

This chapter, by explaining Campbell & Stanley's (1963) threats to internal validity, shows students why they should question studies that claim to have established cause-effect relationships. The chapter shows the problems with both
  1. pretest-posttest designs and
  2. two-group designs that do not employ random assignment.
Thus, the stage is set for the topic of the next chapter: the simple experiment.

Visually Enhanced Summary

We begin by discussing a simple approach to establishing that a treatment causes an effect:

Then, we show how this simple approach fails to work in the two-group case because of selection. Figure 9-1

( pages 258-259 make the point that arbitrary assignment does not make groups equal.

We then show how matching, which seems such a sensible idea, fails to make the groups equal. We explain that matching does not work because researchers cannot match on all possible participant variables and matching on pretest scores does not work because pretest scores are not

  1. perfectly measuring the pretest variable. Because of random measurement error, you can have groups that were matched on pretest scores that are not matched on the pretest variable. In such a case, the matched groups may regress toward different means on the posttest
  2. measuring the only participant variable that determines posttest scores. If groups are similar on the pretest variable, but differ in other respects, the groups may score differently on the posttest. For example, the groups may naturally grow apart (a selection by maturation interaction).
    Figure 9-2 and Table 9-1 (both on page 266) combine to summarize the essence of our critique of nonrandomized, two-group designs. The figure below shows that neither unmatched nor matched group designs have internal validity.

We then turn to showing how the pretest-posttest design fails to control the threats to internal validity. As you can see from Figure 9.5 (p. 273), we divide the problems of the pretest-posttest design into two types:

  1. Factors that do not change participants, but do change average scores on the posttest.
  2. Nontreatment factors that change both participants' and their scores:
Table 9.3 (p. 274) provides students with a list of questions to ask of a "before-after" study. The chapter concludes by
  1. Discussing how ruling out the eight threats to internal validity would enable us to make cause-effect statements, thereby getting students ready for Chapter 14 (Single-n designs and Quasi-Experiments)
  2. Discussing common conflicts between internal and external validity concerns and why psychologists, when forced to choose, tend to make design decisions that boost internal validity at the expense of external validity
  3. Suggesting that the simplest way to establish internal validity is to use the simple experiment, the topic of the next chapter.

Back to Chapter 9 Menu

Back to Research Design Explained Main Menu