Chapter 5 Summary

Brief Overview

This chapter deals with four major topics
  1. Errors in measurement, especially the distinction between random error and bias. This article provides examples that may help students understand the difference.
  2. Reliability and steps one can take to determine whether random error is leaking into the measure, where it is leaking in, and how to stop the leaks.
  3. Evaluating/making a case for a measure's construct validaty.
  4. Choosing/devising a valid treatment manipulation.

More Detailed Summary

The chapter opens by stressing the value of 
  1. operational definitions for allowing psychological science to be and
  2. understanding operational definitions for
That is,

Next, we discuss error in measurement. We stress that

  • bias is different than random error. The referee example we use seems to help students make this distinction.
  • observers introduce both random error and bias.
  • there are ways to reduce observer errors (see Table 5-1, p. 153), including one of the most popular in all of psychology - virtually eliminating the observer.
  • standardizing procedures can reduce both random error and bias
  • blind procedures can reduce bias (The example of the professor grading exams seems to help students understand the value of standardization, blind techniques, and reducing the role of the observer), but do not necessarily reduce random error.
  • psychological tests often use many items in an attempt to balance out the random error
  • participants may bias by trying to make a good impression (social desirability bias) and by obeying demand characteristics.
  • there are ways (see Table 5-2, p. 157) of reducing the impact of these subject biases

    The concept map below summarizes first part of the section on measurement errors (the part in which we distinguish between random error and bias).

    The concept map below summarizes the next part (in which we discuss how to reduce observer errors).

    The concept map below summarizes the key points of the last part (in which we try to reduce measurement errors due to the participant).

    The next part of the chapter deals with reliability. The "key points box" (Box 5-3, p. 174) and the flow chart ( Figure 5-6, p. 175) summarize the main points in this section.

    Next, we move from reliability to validity. Figure 5-7 (page 184) summarizes the main ideas quite well.

    One interesting way to talk about the value of validity is to discuss some "witch tests," such as the one that if a person floats, rather than drowns, they are a witch. The basic logic of this test is lampooned in the movie "Monty Python and the Holy Grail" (the 4 min scene that starts 17:25 into the movie. It is also available on YouTube here). For a shorter and more serious critique, you can use Virginia Governor Tim Kaine's statement (quoted on p. 21 of the July 24, 2006 Newsweek) about why he decided to pardon Grace Sherwood ( In 1706, Sherwood had been convicted of being a witch) : "With 300 years of hindsight, we all certainly can agree that trial by water is an injustice."

    We end the chapter by providing tips for manipulating variables. As Table 5-5 (p. 185) illustrates, most of the principles behind measuring variables also can be applied to manipulating variables. Table 5-6 (p. 191) reviews the rest of the chapter by naming the three basic kinds of manipulations (instructional, environmental, stooges) and listing the pros and cons of each type.

    Back to Chapter 5 Menu

    Back to Research Design Explained Home Page