1.
A professor has a class of 40 students. Half of the
students chose to take a test after every chapter (chapter test condition)
outside of class. The other half of the students chose
to take in-class, "unit tests." Unit tests
covered four chapters. The professor finds no
statistically significant differences between the groups on their scores on a
comprehensive final exam. The professor then concludes
that type of testing does not affect performance.
a.
Is this an experiment?
Hint: What is a key feature of an experiment? If you
are not sure, see the last heading on page 375. Does the professor's study have
that feature?
b.
Is the professor's conclusion reasonable? Why or why
not?
Hints:
1. If this study is not an experiment, can a cause-effect conclusion be drawn?
2. What conclusions can be drawn from null (nonsignificant) results? If you are not sure, see page 379.
2. Participants are randomly assigned to meditation or no meditation
condition. The meditation group meditates three times
a week. The meditation group reports being significantly more
relaxed than the no meditation group.
a.
Why might the results of this experiment be less clear-cut than they may first
appear?
There
is a construct validity problem. The meditation group
may feel more relaxed because of a placebo effect or they may simply report
being more relaxed because they think that is what the experimenter wants them
to say. In addition, it may be that the tense people
dropped out of the experimental group because they were unable or unwilling to
keep to the schedule of meditating three times a week.
b.
How would you improve this experiment?
The
experiment could be improved by improving the control group. For
example, the control group might be assigned to keep to a schedule where they
would listen to classical music three times a week. Alternatively,
they might be asked to keep to a schedule where they would have “quiet time”
three times a week.
3.
Theresa fails to find a significant difference between her control group and
her experimental group t(10)=2.11,
not significant.
a.
Given that her results are not significant, what–if anything– would you advise
her to conclude?
Hint: Are null results inconclusive? If you aren't sure, see page 379.
b.
What would you advise her to do? (Hint: You know that
her t test, based on 10 degrees of
freedom, was not significant. What does the fact that
she has 10 degrees of freedom tell you?)
Hints: How many participants did she have? (To
answer this question, see the first full paragraph on p. 401.) Is this number of
participants consistent with point 5 in Table 10.3? If you are still not sure
about what to do, see the first paragraph on page 387.
4.
A training program significantly improves worker performance.
What should you know before advising a company to invest in such a
training program?
You
should know how big the difference was. A statistically significant difference may not be big
enough to be worth paying for.
5.
Jerry's control group is the football team, the experimental group is the
baseball team. He assigned the groups to condition
using random assignment. Is there a problem with
Jerry's experiment? If so, what is it? Why is it a problem?
Hints: Is there independent random assignment? If you are still not sure what the problem is, read the first paragraph on page 373.
6.
Students were randomly assigned to two different strategies of studying for an
exam. One group used visual imagery, the other group
was told to study their normal way.
The
visual imagery group scored a 88% on the test as compared to a 76% for the
control group. This difference was not significant.
a.
What, if anything, can the experimenter conclude?
Nothing—null
results are inconclusive.
b.
If the difference had been significant, what would you have concluded? What changes in the study would have made it easier to be
sure of your conclusions?
Imagery
seems to improve recall. We would be more confident of
our conclusions if they hadn't used an “empty control group.”
Ideally, the control group would have gotten some placebo-type treatment
(a lecture on the importance of studying).
c.
"To be sure that they are studying the way they should, why don't you have the imagery people
form one study group and have the control group form another study group." Is this good advice? Why or
why not?
This is bad advice because that would mean violating
independence.
d.
"Just get a random sample of students who typically use imagery and
compare them to a sample of students who don't use imagery. That
will do the same thing as random assignment" Is this good advice? Why or why not?
This is bad advice. Random
sampling is very different from random assignment. People
who typically use imagery may differ from people who don't typically use
imagery in a wide variety of ways. They are probably
more visual thinkers and may do better in art, architecture, geometry, and
chemistry than people who do not typically use imagery.
7.
Bob and Judy are doing basically the same study. However,
Bob has decided to put his risk of a Type 1 error at .05 whereas Judy has put
her risk of a Type 1 error at .01. That is, Bob is
willing to take a 5 in 100 risk that the results declared significant are due
to chance, whereas Judy is only willing to take a 1/100 risk of "significant results" being due to
chance.
a.
If Judy has 22 participants in her study, what t value would he need to get significant results?
Hints: See the p = .01 column in Table 1 on page 680--and remember df will be number of participants - 2. Also, don't forget that, to be statistically significant, obtained t's absolute value should be greater than the tabled value.
b.
If Bob has 22 participants in his study, what t value would he need to get significant results?
Hints: See the p = .05 column in Table 1 on page 680--and remember df will be number of participants - 2. Also, don't forget that, to be statistically significant, obtained t's absolute value should be greater than the tabled value.
c.
Who is more likely to make a Type 1 error? Why?
Hints: If the null hypothesis is true, who is taking a larger risk of making a Type 1 error? This is as easy as determining whether .05 is greater than .01.
d.
Who is more likely to make a Type II error? Why?
Hints: Is there a tradeoff between risking a Type I error and risking a Type II error? Is the person who is cautious about not falling for a false alarm at risk of overlooking a real effect?
8.
Gerald's dependent measure is the order in which people turned in their exam
(1st, 2nd, 3rd, etc.). Can Gerald use a t test on this
data? Why or why not? What
would you advise Gerald to do in future studies?
Gerald
should not use a t test because he
has ordinal data. Because
he has ordinal data, computing means for the control group and the experimental
group (a first step in doing a t
test) would be misleading. Next time, Gerald should
record at what time people turned in their exam. Then,
Gerald would have data that were at least interval.
9.
Are the results of Experiment A or Experiment B more likely to be significant? Why?
|
EXPERIMENT A |
EXPERIMENT B |
||
|
CONTROL
GROUP |
EXPERIMENTAL
GROUP |
CONTROL
GROUP |
EXPERIMENTAL
GROUP |
|
3 |
4 |
0 |
0 |
|
4 |
5 |
4 |
5 |
|
5 |
6 |
8 |
10 |
Hints: How do the results for the two experiments differ? (It is not in terms of means). If you need more help, see re-read pages 398-399. If you need even more help, put the data from each experiment into an online t test calculator, such as this one. Which experiment produces the lowest p value (the lower the p value, the more likely the results are to be significant)? Given that the means are the same in both experiments, why does one experiment have a lower p value? Aside: Neither experiment produced statistically significant results--mostly because neither experiment studied enough participants.
10.
Are the results of experiment A or
experiment B more likely to be significant? Why?
|
EXPERIMENT
A |
EXPERIMENT
B |
|||
|
CONTROL GROUP |
EXPERIMENTAL GROUP |
CONTROL GROUP |
EXPERIMENTAL GROUP |
|
|
3 |
4 |
3 |
4 |
|
|
4 |
5 |
4 |
5 |
|
|
5 |
6 |
5 |
6 |
|
|
|
|
3 |
4 |
|
|
|
|
4 |
5 |
|
|
|
|
5 |
6 |
|
|
|
|
3 |
4 |
|
|
|
|
4 |
5 |
|
|
|
|
5 |
6 |
|
Experiment
B because it is based on more participants. Having
more participants allows random error more opportunities to balance out. Consequently, with more participants, a moderate
difference between the groups is less likely to be due to chance alone. When we do the calculations, we find that for Experiment A
t(4) = 1.225, which is not significant, and
that for Experiment B, t (16) = 2.449, which is significant.