Chapter 6 begins by showing students that picking a measure is not simply a matter of picking a measure with the highest validity correlation for at least two reasons.
- To get valid measurements, the fit between a measure's weaknesses and the particular research design's strengths is often more important than the measure's reported validity.
- Researchers always have concerns beyond validity, such as:
- Sensitivity: Being able to detect even small differences between groups
- Scales of Measurement: Being able to make specific statements about differences between groups
- Ethics: Complying with APA's ethical code
- Practical: Concerns such as using a measure that you can afford and that your research sponsors will accept as valid (the political value of having a measure with face validity).
We then point out that sensitivity is important because
At one level, having a sensitive measure is simple. All you need to do is make sure
- Researchers cannot accept the null hypothesis
- Treatment effects are often small because
- ethical considerations often restrict the intensity of the treatment.
- practical concerns often restrict the duration of the treatment.
- the treatment is, at best, only one of many determinants of the participants' behavior.
- Small effects are important.
- your measure produces a variety of scores,
- and produces those different scores for the right¬ rather than the wrong ¬reasons.
One "wrong " reason for getting a variety of scores would be random error. To reduce the chances that the variations in scores are not due to random error, we want a reliable measure. (The rat example on page 124 helps most students see how increased reliability can pay off in increased sensitivity.)
The right reason for getting a variety of scores would be that the measure is measuring what it is supposed to measure. Therefore, to maximize the chances that the variations in scores are for the right reasons, we should maximize the validity of our measure. Novices often fail to maximize the validity of their measures because they haven't precisely defined what it is they are trying to measure.
To get a measure that can produce a wide variety of scores:
Once you have a measure that could possibly produce a wide variety of scores, you have a chance of making it actually produce a wide variety of scores. To maximize your chances of producing a measure that will produce a wide variety of scores, pilot test your measure.
- avoid "all-or-none" measures such as "alive or dead," "happy or sad," etc.
- add scale points to your measure by asking "how much" rather than "whether," and by adding questions/observations.
Put another way:
Scales of measurement
We start by differentiating between the different scales of measurement. Box 6-1 ("Numbers and the toll ticket") make two concepts quite clear to most students:
We then show how different research questions require different scales of measurement. If you want to review the scale of measurement needed to answer different types of research questions, you may want to display Table 6-2.
- the differences between the different scales of measurement, and
- why psychological measurements are rarely ratio scale
Having established that answering certain research questions requires having a certain scale of measurement, we discuss which measures are commonly assumed to produce which kind of data. In reviewing this material, you may want to display Table 6-3.
Thus, you may want to structure your discussion of scales of measurement as follows:
In conclusion, the following concept map summarizes the chapter's main points.
Back to Chapter 6 Menu
Back to Research Design Explained Home Page