Brief Summary of Chapter 14

To establish that a treatment causes an effect, you must

1. introduce a treatment
2. observe a change in behavior
3. make sure that nothing else besides the treatment could be responsible for the change in behavior

Researchers using randomized experiments rely on statistical tests to argue that it is unlikely that anything other than treatment caused the change in behavior. Single-n researchers try to prevent factors that could change the behavior from varying. Studying participants under highly controlled laboratory conditions, for example, reduces the risk of the change being due to history. Maturation, however, still might be a problem. Therefore, single-n researchers prefer A (no treatment)-B (treatment)-A (no-treatment) designs to A (no-treatment)-B (treatment) designs.

Quasi-experimenters can't keeping all of the critical factors constant. Therefore, they must rely on their wits to rule out Campbell and Stanley's eight threats to internal validity. For example, if they used a before-after design, they would have to worry that the change in behavior could be due to:

• History
• Maturation
• Testing
• Instrumentation
• Regression
• Mortality

Fortunately, by adding several more times of measurement to a before-after design, quasi-experimenters can better estimate the effects of these threats. Thus, if there are only modest changes in behavior in the weeks before the treatment is administered, but a large change in behavior right after the treatment is administered, then the quasi-experimenter would be relatively confident that the change was due to the treatment. However, it is still possible that the difference was not due to the treatment. A change in the environment (history), rather than the treatment, could be responsible for the sudden change. For example, suppose you are trying to change people's attitudes toward government regulation of food. Then, the week you administer the treatment, there is an outbreak of food poisoning. The food poisoning outbreak may have more effect than your manipulation. To minimize these problems, you might use a reversal time-series design. In that case, you would get baseline data, introduce the treatment, collect data, withdraw the treatment, and collect data. Your hope would be that when you withdrew the treatment, the behavior would revert back to what it was before you introduced the treatment.

Instead of using a time-series design, you might compare a no-treatment group with a treatment group. A big question with this design is: Do the groups differ now because one group got the treatment--or did the two groups differ to start with? In other words, with a two-group design, you have to worry about selection. It is very hard to eliminate the selection threat. Basically, the problem is that you can't get two identical groups of participants.

Some people naively think that matching will give you identical groups, but those people are failing to realize that:

• You aren't matching on every characteristic so the groups, even though they score the same on your matching task, may naturally grow apart by the time you retest them.
• You are matching on scores rather than on actual characteristics. Scores contain random error. So, the question is: Are you groups' pretest scores similar because the groups really are similar or are their pretest scores similar because you selected scores that are loaded with random error? If random error created the illusion of the groups being similar for the pretest, it probably won't recreate that illusion at the end of the study. In other words, the difference between the scores at the end of the study may not be due to the treatment, but to regression.

Back to Chapter 14 Menu