Demonstration research and meta-analysis in parapsychology

From 1945 to 1983, 60 studies by 17 investigators were located that investigated correlations between each subject's extraversion score and ESP score (Honorton, Ferrari, & Bern, 1990). A diagram of the correlations showed an approximate bell shape; when weighted by degrees of freedom, the weighted mean r was .14. For free-response studies, there was a significant correlation between extraversion and ESP scores (r = .20, z = 4.46); this effect was homogeneous across investigators and extraversion scales. For forced-choice studies, there was a significant correlation between ESP and extraversion only for those studies that reported the ESP results to the subjects before extraversion was measured. Some data exist that suggest the extraversion scores could have been temporarily inflated as a result of the positive feedback subjects had been given on their ESP scoring. However, an experiment that set out to test the hypothesis that subjects' knowledge of their ESP scores would bias their responses to the personality test yielded chance results (Krishna & Rao, 1991).

In a related meta-analysis of 16 studies, scores on the Defense Mechanism Test were found to correlate with high ESP-scoring (Haraldsson, 1992). A meta-analysis of several studies utilizing hypnotic induction found higher ESP scoring with hypnosis than in nonhypnosis conditions (Schechter, 1984). A meta-analysis of telepathy-clairvoyance dream studies with free-response targets at one laboratory yielded a joint p value of .000002 for outside judges' evaluations and .002 for subjects' evaluations (Child, 1985).

The effect sizes yielded by these various meta-analyses ranged from a low of .0003 for the RNG research through .01 for dice PK, .09 for precognition, .09 for extraversion, and .29 for the ganzfeld studies. However, these effect sizes cannot be interpreted fairly without attention being given to their significance levels and the concomitant sample sizes. Effect sizes with small numbers of subjects are unstable, and a thorough evaluation of demonstration research data must include a consideration of probability measures.

Conclusion

Many critics are unimpressed by the use of meta-analysis because the experiments included in these analyses represent various degrees of quality and sophistication. The consistency shown by them, it is alleged, may be due to a replication of errors rather than of psi. Slavin (1984), commenting on meta-analysis in general, points out that "meta-analysis is no better than the studies that went into it". Other critics suspect that meta-analysis portrays research results as typically homogeneous, ignoring variables that may have been crucial to a complete understanding of the phenomena under study (Raudenbush, 1991).

Nonetheless, issues of quality control have been considered in these analyses. Most of the cited studies have given differential weights to each study on the basis of experimental quality; except for the dice meta-analysis, this procedure has yielded no significant relationships between effect size and quality. Furthermore, the meta-analyses typically are used by parapsychologists (e.g., Broughton, 1991) to point out that a general level of replication has been achieved, but not that a phenomenon has been conclusively demonstrated. It is possible that the effects observed in these meta-analyses can be explained by something other than psi, for example, shortcomings in the understanding of randomness and independence. Nevertheless, there is an anomaly here that needs an explanation. If psi does not exist, there is little to be lost by further research because it might uncover an alternative explanation for the anomaly. If psi does exist, it is important to conduct other types of investigations so that its underlying processes can be more completely described and understood. This is a critical component of the PA's agenda for the future. Whatever the outcome may be, it cannot help but add to the sum of knowledge about humanity and the human condition.