THE SIMPLE EXPERIMENT
I. An active method: The experimenter makes things happen.
II. Starts with causal hypotheses
A. Experimental hypothesis: Treatment has an effect.
B. Null hypothesis: Treatment has no effect.
One either rejects the null hypothesis or fails to reject it: The null hypothesis is never supported. The inability to find a difference doesn't mean that a difference doesn't exist. This fact affects what hypotheses researchers can test (they can't prove that something has no effect or that two manipulations have identical effects) and how researchers interpret null results.
III. Have to administer the Independent variable (IV):
Treatment variable, varied independently of participant's wishes. In simple experiment, there are two levels of IV (often low and high).
IV. Dependent variable (DV): Response that you are measuring (often some kind of score). Response depends on participant and you hope it depends on treatment level.
V. Memory tricks for keeping IV and DV straight.
A. Participants get IV before DV, so IV can be thought of as 1V (first variable) or you could keep the order straight by saying to yourself "I must ID (identify) the variables.
B. To identify (ID) which is which, remember that the experimental hypothesis can always be restated as:
Independent Variable will cause a change in the scores on the Dependent Variable.
VI. Experimental group: Gets higher level of treatment than control (comparison) group.
A. Why we need -- the "better than what?" question.
B. Why groups aren't really groups:
1. Prevent group from influencing individual participants' responses (mob versus individual behavior)
2. Want to randomize not only individual differences, but also differences between testing sessions (time of day, violations of standardization, etc.)
VII. How are participants assigned to "group" (condition)?
A. Not by experimenter's whim
B. Not by participant's whim
C. By random assignment!!!!!!!!
1. What random assignment is
2. What random assignment isn't
a. It isn't random selection (random assignment to--not random sampling from)
b. It isn't arbitrary assignment
VIII. Why do people fail to randomly assign participants to a group when it can be done so easily and when it would be ethical to do so?
A. People fail to execute it correctly because:
1. Ignorance of how to use random numbers table
2. Intuitive belief that they know what random is and so they disbelieve random numbers table if results don't look random
(Ex: The following three patterns of coin flips are all equally likely:HHHHHH vs. HTHTHT vs. HTTHTH)
B. People fail to understand value of random assignment. Random assignment better than arbitrary means because random assignment allows you to use statistics to establish internal validity.
IX. Are there any problems with random assignment?
Yes, random assignment, like all methods, fails to equalize groups, but
A. Groups will be relatively equal if large sample sizes are used and observations are independent.
B. Statistics can be used to account for chance differences between groups. If groups differ by more than what statistics would estimate chance would do, results are statistically significant.
Statistically significant results are not always:
3. In the direction you expected (treatment may make people worse)
X. Can anything go wrong with statistics?
Yes, the statistical significance decision can err in one of two ways:
A. Type 1 errors: " False positives." Can specify how vulnerable you'll be, but can't eliminate the possibility of making a Type 1 error.
1. Implications of using a .05 level
2. Implications of using a .01 level
B. Type 2 error: "False negatives." Can reduce the likelihood of Type 2 error (and therefore increase power) by:
1. Reducing random error
a. Using reliable measures
b. Standardizing procedures
c. Carefully coding data
d. Using a homogenous group of participants
2. Give random error a better chance of balancing out by using a large number of participants.
3. Make the difference between groups so big that it would be hard for random error to hide it by using a strong manipulation of the IV.
XI. Conclusions about the simple experiment:
A. Easy to maximize chances that study will be internally valid
B. However, steps taken to improve power, may hurt external validity
1. Controlled situation improves power, may hurt ability to generalize to less controlled situations
2. Homogeneous participants improves power, may hurt ability to generalize to other types of participants
3. Strong manipulation of IV may improve power, but hurt ability to generalize to more naturally occurring levels of IV.
C. Yet, the simple experiment and external validity are by no means incompatible:
1. Simple experiments can be done anywhere: Control is not needed because of random assignment.
2 You can combine random selection and random assignment to get a study that is internally and externally valid.
3. Some experiments capture the essential features of real life situations (much like an aviation engineer's wind tunnel)
D. The simple experiment and construct validity
a. Experimenter bias: Rosenthal
b. Subject bias
2. Single and double blind techniques can help.
3. Empty control group hurts construct validity.
To Lecture 10.2: Statistics
Back to Chapter 10 Main Menu