1.
What questions would you have of a researcher who said that the no-treatment and treatment groups
were identical before the start of the
study?
Hints:
1. How could the researcher know that the groups were matched on every relevant variable?
2. What might happen if the researcher only matched on the outcome variable?
3. The researcher matches groups based on scores. However, given that scores are affected by random error, would two groups that were measured as the same actually be the same?
4. See Figure 9-2 on page 347.
5. See Table 9.1 on page 354.
2.
In all of the following cases, the researcher wants to make cause-effect
statements. What threats to internal validity is the
researcher apparently overlooking?
a. Employees are interviewed on job satisfaction. Bosses undergo a three week training program. When employees are re-interviewed a second time, dissatisfaction seems to be even higher. Therefore, the researcher concludes that the training program caused further employee dissatisfaction.
History—other events besides the
training program have happened in the past three weeks. For
example, layoffs or salary cuts could have occurred.
Instrumentation—The
interviewer may have developed more rapport and been more direct in the second
interview. Thus, the second time around, the measure
was not the same and not administered the same way.
b.
After completing a voluntary workshop on improving the company's image, workers
are surveyed. Worker who attended the workshop are now
more committed than those in the
"no-treatment" group who did not make the workshop.
Researcher's conclusion: The workshop made workers more committed.
Obvious selection problem—even
before the workshop, volunteers were probably more committed than
non-volunteers.
c.
After a 6-month training program, employee productivity improves. Conclusion: Training program caused increased
productivity.
Maturation: New workers might
have naturally improved their skills
over that period.
History: Other events (a new
incentive system, a better supervisor, better technology) that happened over
the last six months could be responsible for the rise in productivity.
Regression: Would be a likely
problem if training was instituted because productivity was at an all time low.
Mortality: Poorer workers may
have left the company.
d.
Morale is at an all-time low. As a result, the company
hires a "humor consultant." A month later,
workers are surveyed and morale has improved. Conclusion:
The consultant improved morale.
Regression is the most likely
suspect.
Also likely are
Mortality (unhappy people
leaving)
History (management making other changes)
e. Two groups of workers are
matched on commitment to the company. One group is
asked to attend a two-week workshop on improving the company’s image, the other
is the no-treatment group. Workers who complete the
workshop are more committed than those in the “no-treatment” group. Researcher’s conclusion: The workshop made workers more
committed.
Selection
(not all workers who are asked will go) and mortality (people dropping out) are
prime suspects.
3.
A hypnotist claims that hypnosis can cause increases in strength. To "prove" this claim, the hypnotist has
participants see how many times they can squeeze a hand-grip in two minutes. Then, he hypnotizes them and has them practice for two
weeks. At the end of two weeks, they can squeeze the
hand-grips together many more times than they could at the beginning. Other than hypnosis, what could have caused this effect?
Hints: See Table 9.2. Also, note that practice might make better (use the correct term for this effect) and that hand-grips might get easier to press once they are "broken in" (again, use the correct term for such an effect).
4.
How could a quack psychologist or doctor take advantage of regression toward
the mean to make it appear that certain phony treatments actually worked?
If
the quack takes people who are feeling unusually bad, those people will tend to
improve on their own. That is, they will naturally
rebound to their normal levels of health or happiness and the quack can take
the credit.
Why should a baseball
team’s general manager consider regression toward the mean when considering a
trade for a player who made the All-Star team last season?
Obviously,
things went well for the player that year: no injuries, no really bad breaks. Next year, the player may not be so lucky.
5. How could a participant's score on an ability test change even though the person's actual ability had not?
Hints: To answer this question, ask yourself how your score on a test might change if you took the same test several times. As you might suspect, the questions you would ask yourself are a subset of the questions listed in Table 9.2 ("Questions to ask when examining a pretest-posttest study"). However, given that you are interested in how scores might change when the underlying ability did not change, you would not ask the questions that asked whether a real change was due to a nontreatment factor. Instead, you would focus on the questions that ask whether the scores--rather than the participants--may have changed? with scores changing rather than participants changing. Finally, note that because you are comparing one person's pretest score to that same person's posttest score, you would not have to worry about mortality.
6.
Suppose a memory researcher administers a memory test to a group of residents
at a nursing home. He finds a group of grade school
students that score the same as the older patients on the memory pretest. He
then administers an experimental memory drug to the older patients. A year later, he gives both groups a posttest.
a. If the researcher finds that the older patients now have a worse
memory than the grade-school students, what can the researcher conclude? Why?
Nothing—the
results could be due to a selection by maturation interaction due to the school
children's memories improving and the older patients' memories naturally
staying the same or declining slightly.
b. If the researcher finds that the
older patients now have a better memory than the grade
school
students, what can the researcher conclude? Why?
The
researcher might have an easier time concluding that the drug improves memory
because the difference is opposite of what would be expected on the basis of
selection by maturation interactions. However, history
effects are still possible (if other interventions are going on at the nursing
home) and regression might be possible (if the children selected had unusually
high scores for their grade level).
7. Suppose there is a correlation between the use of night
lights in an infant’s room an increased incidence of near-sightedness later. What might account for this relationship?
Hint: See the second and third paragraphs on page 332 (they start with "Second,.." and "Third,...) respectively. The points they make about a sugar-hyperactivity link can be applied to the nightlights-vision link. If you need more help, go back to Box 7.2 on p. 229.
8.
What is the difference between
a. testing and instrumentation?
The
difference between testing and instrumentation is that in testing participants
may remember things from the previous test and therefore score higher, whereas
in instrumentation, the actual measuring instrument changes or the way it is
administered changes.
b. history and
maturation?
History refers to changes due to
events in the participant's environment (other than the treatment).
Maturation refers to changes due
to biological changes in the participant.
9.
Suppose a researcher reports that a certain argument strategy has an effect,
but only on those participants who hold extreme attitudes. Why
might the researcher be mistaken about the effects of the persuasive strategy?
Hint: See Table 9.2 on page 356.
10.
What is the difference between internal and external validity?
Internal
validity
refers to whether you can make the statement that, in a given study, with these
participants, the treatment caused an effect.
External
validity
refers to whether you can
generalize what you discovered in a particular study to
other people, situations, and times.