Demonstration research and meta-analysis in parapsychology

While meta-analysis may be a step toward an "objective" summarization of the state of the art in a given scientific area, it is not necessarily a shortcut to arrive at scientific "truth." Fine (1990) has pointed out: "At its very best, meta-analysis cannot determine what is true, it only can attempt to measure what is". In other words, meta-analysis can be considered "a glorified literature review" (Adler, 1990, p. 4). However, meta-analysis not only pools data from many studies; it trims the experimental pool where necessary, removing extremely flawed studies and calculating the number of studies that could have remained unreported. Meta-analysis may show that testing for statistical significance can actually obscure underlying regularities and processes in individual studies. Further, meta-analysis has the potential of requiring major changes in the way psychologists view the general research process.

Meta-Analyses in Ganzfeld Psi Research

The use of meta-analyses has helped address one of the central questions in parapsychology: Has psi research demonstrated any sort of anomalous effect? Many recent experiments in parapsychology tend to use more complex target material than the cards and dice used in earlier investigations, partially to maintain subjects' interest and partially because they are thought to more closely resemble spontaneous psi occurrences. These experiments fall under the general heading of "free-response" experiments because subjects are asked to give a verbal or written description of the target, rather than being forced to make a choice from a small set of discrete possibilities. Various types of target materials have been used, including pictures, videotape segments, and small objects.

A typical experiment begins by assembling a large pool of potential targets, often in packets of four. Similarity of targets within a packet is kept to a minimum to make evaluation easier. After the subject is sequestered in an isolated room, a packet is selected randomly from the pool, and the target is selected at random from the packet. The subject is asked to provide a description of what he or she thinks is the target. After the subject's description has been recorded and secured against possible later alteration, a judge (who may or may not be the subject) receives a copy of the subject's description and duplicates of the four possible targets that were in the packet (one of which was the actual target). The judge blindly ranks the four choices from most to least likely to have been the target, or simply chooses the one that most closely matches the subject's description (Utts, 1991).

If ranks are used, the statistical analysis proceeds by summing the ranks given to the actual targets over a series of trials and comparing the sum to what would be expected by chance. If the selection method is used, a "direct hit" occurs if the correct target is chosen. The number of direct hits over a series of trials is compared to the number expected from binomial expansion. In a correctly designed experiment with four potential targets, the probability of a direct hit by chance is .25 on each trial in situations where the trials are independent of each other (Utts, 1991).