Mottos/Mantras/Maxims/Take-Home Lessons From Chapter 6

1. Like any design decision, choosing a measure for your study involves making tradeoffs. You need to make the right tradeoffs for your particular study.
2. So, when "shopping" for a measure, don't just ask: "Which is the most valid measure?"  Instead, ask "Which measure will be most likely to get me an accurate answer to my research question?" Asking that question, involves asking at least five other questions:
   1. Which measure would best avoid the main threat to my study's construct validity? For example, if participant bias is a serious threat, you would probably not use a self-report measure,
      but if observer bias or random error was a more serious threat, you might want a self-report measure.
   2. Will the measure be sensitive enough to detect the difference I expect to see? If you are going to study hundreds of participants and your treatment is expected to have a large effect, do not worry about sensitivity. If, on the other hand, you are studying only a few dozen participants and your effect may be small, sensitivity will be very important.
   3. "Will the measure allow me to make the comparisons I need to make?" If your hypothesis is just that two conditions will be different, any valid measure will do. If, however, you want to make more specific comparisons (e.g., does one condition have more of the construct than another? how much more of the construct does one condition have than another? how many times more of the construct does one condition have than the other?), you will need a measure that provides the level of measurement that allows you to make that comparison.
   4. "Can I afford it?" For example, using some psychological tests is very expensive, using some measures will require expensive equipment, and using observational measures may require you to hire and train observers.
   5. "Is it ethical?"
3. Failing to  have a sensitive measure can cause  you to fail to find a difference (i.e., to get null results, which are inconclusive).
4. All other things being equal, the purer the measure, the more sensitive and accurate it will be. The purest and most valid measures tend to be reliable (not affected by random error) and as direct as possible (the less a measure is affected by factors other than the construct, the more valid and sensitive it can be).
5. A sensitive measure should provide a variety of scores. To see whether your participants are likely to get a variety of scores, pretest your measure. If participants aren't getting a variety of scores, consider adding questions, adding scale points, and adjusting the difficulty of the questions.
6. A sensitive measure is sensitive to differences between conditions; a sensitive and valid measure is sensitive to differences--in terms of what you want to measure-- between conditions.
7. All valid measurement involves assigning participants who differ on a characteristic different scores on that characteristic. But measurements differ in terms of the comparisons they allow you to make.
8. If you simply want to compare participants or groups to see whether participants differ on the characteristic you want to measure, even using the lowest level of measurement -- nominal, in which the numbers are substitutes for  category names and higher scores don't reflect more of a quality-- will do.
9. If you need to determine whether participants/groups have more of a quality than another, you need a measure in which higher scores reflect that participants have more of the measured characteristic. That is, you need at least   ordinal measurement in which you could use participants' scores to order participants from those having the least of the measured quality to those having the most.
10. If  you need to determine whether the difference between two conditions is more than than the difference between two other conditions ( (e.g., you need to know that the difference in terms of the construct you are measuring between a person scoring a 7 and a person scoring a 5 is greater than the difference between a person scoring a 3 and a person scoring a 2), you need an (equal) interval level of measurement. Equal interval scales allow you to to make  comparisons involving amounts (i.e.,  "how much more" statements).
11. If you need to say how many times more of a quality a participant or group has, you need ratio scale measurement.
12. It is hard to get ratio scale level of measurement in psychology; most of our measures would be considered interval, at best.
13. Face validity is not scientific validity. Indeed face validity may increase participant bias and thus harm construct validity. However, face validity may affect how non-scientists regard your research.