Lecture 6.2: Issues other than validity that should be considered if you are to select the best measure for your study

I. Sensitivity: Will the measure be able to detect the differences you need to detect?

A. Why sensitivity is important:

1. Often, the effects of a single variable on behavior are small. Yet, small effects can be important (e.g., if television violence increased aggression by 1% that would be a small, but important relationship).

2. A fact about the null hypothesis that makes sensitivity important:

B. Examples of other science's trying to increase the sensitivity of their measuring systems--from telescopes to microscopes

C. Achieving the necessary level of sensitivity:

1. Look for high validity

2. Look for high reliability

3. Allow scores to vary:

a. Avoid behaviors that are resistant to change (established habits, important commitments)

b. Avoid measures that produce a limited range of scores (health: 1=alive, 2=dead)

c. Ask "how much do you..." rather than just "Do you..."

d. Add scale points to a rating scale

4. Pilot test your measure to be sure that scores actually do vary

1. Scores should vary--

2. But for the right reasons

II. Will the measure allow you to make the kinds of comparisons you need to make?

A. What is measurement? Assigning numbers to observations

B. The different scales of measurement: All numbers are not alike

1. Nominal numbers: Swapping numbers for names. Different numbers representing different states, types, kinds

a. Bigger numbers don't mean more of a quality

b. Numbers are so arbitrary they can be reversed

(e.g. 1 = male, 2 = female or 1 = female, 2 = male)

2. Ordinal numbers: When bigger means more

a. Number have a meaningful order (Ranked data)

3. Interval scale numbers: Knowing how much more

a. Numbers can be added (Although controversial, scores from rating scales are often added up to get an average rating)

b. Making ratios between interval numbers doesn't make sense (a person who circles a "5" on a 1-5 scale does not feel 5 times as positively about the issue as someone who circles a "1")

4. Ratio scales: Zeroing in on perfection

a. May only occur with magnitude estimation

b. Can add ratio numbers and can make ratio comparisons

C. Why our numbers do not always measure up:

1. Often, we don't even try to measure the absolute zero

2. Usually, we don't have perfect validity

D. Which level of measurement do you need?

1. When you need ratio scale data: Rarely

2. When you need at least interval scale data

a. When you need to know whether one condition changed more than another

b. When mapping functional relationships

3. When ordinal data is sufficient:

You only need to know whether one group did better than another

4. When you only need nominal scale data:

You only need to know whether your groups differ

E. Conclusions about scales of measurement

III. Ethical and practical considerations:

A. Preventing subject biases versus using an unobtrusive measure (Validity vs. Ethics)

B. Recording actual behavior versus using quick and easy self-report measure (Validity vs. Practical issues)

C. Trading sensitivity for face validity