1. What questions would you have of a researcher who said that the no-treatment and treatment groups were identical before the start of the study?
1. How could the researcher know that the groups were matched on every relevant variable?
2. What might happen if the researcher only matched on the outcome variable?
3. The researcher matches groups based on scores. However, given that scores are affected by random error, would two groups that were measured as the same actually be the same?
4. See Figure 9-2 on page 347.
5. See Table 9.1 on page 354.
2. In all of the following cases, the researcher wants to make cause-effect statements. What threats to internal validity is the researcher apparently overlooking?
a. Employees are interviewed on job satisfaction. Bosses undergo a three week training program. When employees are re-interviewed a second time, dissatisfaction seems to be even higher. Therefore, the researcher concludes that the training program caused further employee dissatisfaction.
History—other events besides the training program have happened in the past three weeks. For example, layoffs or salary cuts could have occurred.
Instrumentation—The interviewer may have developed more rapport and been more direct in the second interview. Thus, the second time around, the measure was not the same and not administered the same way.
b. After completing a voluntary workshop on improving the company's image, workers are surveyed. Worker who attended the workshop are now more committed than those in the "no-treatment" group who did not make the workshop. Researcher's conclusion: The workshop made workers more committed.
Obvious selection problem—even before the workshop, volunteers were probably more committed than non-volunteers.
c. After a 6-month training program, employee productivity improves. Conclusion: Training program caused increased productivity.
Maturation: New workers might have naturally improved their skills
over that period.
History: Other events (a new incentive system, a better supervisor, better technology) that happened over the last six months could be responsible for the rise in productivity.
Regression: Would be a likely problem if training was instituted because productivity was at an all time low.
Mortality: Poorer workers may have left the company.
d. Morale is at an all-time low. As a result, the company hires a "humor consultant." A month later, workers are surveyed and morale has improved. Conclusion: The consultant improved morale.
Regression is the most likely suspect.
Also likely are
Mortality (unhappy people leaving)
History (management making other changes)
e. Two groups of workers are matched on commitment to the company. One group is asked to attend a two-week workshop on improving the company’s image, the other is the no-treatment group. Workers who complete the workshop are more committed than those in the “no-treatment” group. Researcher’s conclusion: The workshop made workers more committed.
Selection (not all workers who are asked will go) and mortality (people dropping out) are prime suspects.
3. A hypnotist claims that hypnosis can cause increases in strength. To "prove" this claim, the hypnotist has participants see how many times they can squeeze a hand-grip in two minutes. Then, he hypnotizes them and has them practice for two weeks. At the end of two weeks, they can squeeze the hand-grips together many more times than they could at the beginning. Other than hypnosis, what could have caused this effect?
Hints: See Table 9.2. Also, note that practice might make better (use the correct term for this effect) and that hand-grips might get easier to press once they are "broken in" (again, use the correct term for such an effect).
4. How could a quack psychologist or doctor take advantage of regression toward the mean to make it appear that certain phony treatments actually worked?
If the quack takes people who are feeling unusually bad, those people will tend to improve on their own. That is, they will naturally rebound to their normal levels of health or happiness and the quack can take the credit.
Why should a baseball team’s general manager consider regression toward the mean when considering a trade for a player who made the All-Star team last season?
Obviously, things went well for the player that year: no injuries, no really bad breaks. Next year, the player may not be so lucky.
5. How could a participant's score on an ability test change even though the person's actual ability had not?
Hints: To answer this question, ask yourself how your score on a test might change if you took the same test several times. As you might suspect, the questions you would ask yourself are a subset of the questions listed in Table 9.2 ("Questions to ask when examining a pretest-posttest study"). However, given that you are interested in how scores might change when the underlying ability did not change, you would not ask the questions that asked whether a real change was due to a nontreatment factor. Instead, you would focus on the questions that ask whether the scores--rather than the participants--may have changed? with scores changing rather than participants changing. Finally, note that because you are comparing one person's pretest score to that same person's posttest score, you would not have to worry about mortality.
6. Suppose a memory researcher administers a memory test to a group of residents at a nursing home. He finds a group of grade school students that score the same as the older patients on the memory pretest. He then administers an experimental memory drug to the older patients. A year later, he gives both groups a posttest.
a. If the researcher finds that the older patients now have a worse memory than the grade-school students, what can the researcher conclude? Why?
Nothing—the results could be due to a selection by maturation interaction due to the school children's memories improving and the older patients' memories naturally staying the same or declining slightly.
b. If the researcher finds that the older patients now have a better memory than the grade
school students, what can the researcher conclude? Why?
The researcher might have an easier time concluding that the drug improves memory because the difference is opposite of what would be expected on the basis of selection by maturation interactions. However, history effects are still possible (if other interventions are going on at the nursing home) and regression might be possible (if the children selected had unusually high scores for their grade level).
7. Suppose there is a correlation between the use of night lights in an infant’s room an increased incidence of near-sightedness later. What might account for this relationship?
Hint: See the second and third paragraphs on page 332 (they start with "Second,.." and "Third,...) respectively. The points they make about a sugar-hyperactivity link can be applied to the nightlights-vision link. If you need more help, go back to Box 7.2 on p. 229.
8. What is the difference between
a. testing and instrumentation?
The difference between testing and instrumentation is that in testing participants may remember things from the previous test and therefore score higher, whereas in instrumentation, the actual measuring instrument changes or the way it is administered changes.
b. history and maturation?
History refers to changes due to events in the participant's environment (other than the treatment).
Maturation refers to changes due to biological changes in the participant.
9. Suppose a researcher reports that a certain argument strategy has an effect, but only on those participants who hold extreme attitudes. Why might the researcher be mistaken about the effects of the persuasive strategy?
Hint: See Table 9.2 on page 356.
10. What is the difference between internal and external validity?
Internal validity refers to whether you can make the statement that, in a given study, with these participants, the treatment caused an effect.
External validity refers to whether you can generalize what you discovered in a particular study to other people, situations, and times.