Brief Summary of Chapter 9
To establish that a treatment causes an effect, you must:
- introduce a treatment
- observe a change in behavior
- make sure that nothing else besides the treatment could be responsible for the change in behavior
It is very hard to say that the treatment is the only thing that caused the
change in behavior. As Campbell and Stanley have pointed out, there are eight
things other than the treatment that might cause a change in behavior.
If you try a before-after design, you have to worry that the change in behavior could be due to:
- History
- Maturation
- Testing
- Instrumentation
- Regression
- Mortality
Note that history, maturation, and testing could produce real changes in your participants, whereas instrumentation, regression, and mortality could result in your "after" scores differing from your
"before" scores, even though your participants didn't change at all.
(See Table 9-3 or Figure 9-5) for a review.
If you compare a no-treatment group with a treatment group, the question is: Do
the groups differ now because one group got the treatment--or did the two groups
differ to start with? In other words, with a two-group design, you have to worry
about selection. As Table 9-1 illustrates, it is very hard to eliminate the selection threat. Basically, the problem is that you can't get two identical groups of participants.
Some people naively think that matching will give you identical groups, but those people are failing to realize that:
- You aren't matching on every characteristic so the groups, even though they score the same on your matching task, may naturally grow apart by the time you retest them.
- You are matching on scores rather than on actual characteristics. Scores contain random error. So, the question is, "Are you groups' pretest scores similar because the groups really are similar or are their pretest scores similar because you selected scores that are loaded with random error?" If random error created the illusion of the groups being similar for the pretest, it probably won't recreate that illusion at the end of the study. In other words, the difference between the scores at the end of the study may not be due to the treatment, but to regression.
Back to Chapter 9 Menu