Statistical significance

Statistical significance of a correlational coefficient only indicates that the relationship isn't due entirely to sampling error. Specifically, stress that
  1. As Table 7-5 (p. 197) shows, in a correlational design, statistical significance has nothing to do with internal validity. The only reason statistically significant results in experiments indicate that the treatment caused an effect was because the design controlled for third variable factors and established that the treatment preceded the effect.
  2. The meaning of statistical significance does not differ depending on whether the significant results come from an ANOVA, a t-test between means, or a t-test to determine if the correlation coefficient is significantly greater than zero.
  3. In a correlational design, statistical significance does not necessarily imply external validity. If the data can be viewed as a random sample of a certain population, then a significant relationship means that the relationship would hold in that population.
  4. As Handout 7.3 shows, null results must be interpreted very cautiously. A demonstration to help students realize the problem of accepting null results is described in
Harcum, E. R. (1989). Demonstrating that an ability does not exist. Teaching of
Psychology, 16, 85-86.

Back to Chapter 7 Main Menu