Updating the ganzfeld database: A victim of its own success? - Statistical Data Included

**********

The term psi denotes anomalous processes of information transfer such as telepathy and other forms of extrasensory perception that are currently unexplained in terms of known physical or biological mechanisms. The question of whether psi actually exists continues to be controversial. In 1994, Bem and Honorton summarized meta-analyses of approximately 50 studies from 10 separate laboratories that appeared to provide replicable evidence for psi using an experimental protocol known as the ganzfeld procedure.

In most studies using the ganzfeld procedure, two participants--a "sender" and a "receiver"--are sequestered in separate, acoustically isolated rooms. For approximately 30 min, the sender concentrates on a randomly selected stimulus target, for example, an art print, a photograph, or a brief videotaped sequence. During the same period, the receiver is immersed in a mild form of perceptual isolation called the ganzfeld (total field) while providing a continuous verbal report of his or her ongoing thoughts, feelings, and images. At the completion of the ganzfeld period, the receiver is shown several stimuli (usually four) and, without knowing which stimulus was the target, is asked to rate the degree to which each matches the thoughts, feelings, and images experienced during the ganzfeld period. If the receiver assigns the highest rating to the target stimulus, it is scored as a hit. Thus, if the experiment uses judging sets containing four stimuli (the target and three decoys or control stimuli), the hit rate expected by chance is 25%. (1)

In their article, Bem and Honorton (1994) reported a hit rate of 35% (p < [10.sup.-9]) for 28 ganzfeld studies conducted between 1974 and 1981 and a hit rate of 32% (p = .0008) for 10 computer-controlled ("autoganzfeld") studies conducted between 1983 and 1989 that had been specifically designed to eliminate methodological flaws identified in some of the earlier studies.

More recently, Milton and Wiseman (1999) published a follow-up meta-analysis of 30 additional ganzfeld studies that had been conducted from 1987 through 1997. They concluded that these studies did not yield an overall significant effect, thereby calling into question the replicability of the ganzfeld procedure (see Storm & Ertel, 2001, for a critique of that meta-analysis). Milton subsequently organized and initiated an Internet debate of the ganzfeld research, a debate that was edited for publication by Schmeidler and Edge (1999). In her own contribution to that debate, Milton (1999) noted that when replications published after the Milton--Wiseman cutoff date are added to the database, the accumulated studies do, in fact, achieve statistical significance. Even so, however, the mean effect size of these more recent studies is still significantly smaller than those reported by Bem and Honorton for the two earlier databases.

The z scores of the studies in the Milton--Wiseman database are significantly heterogeneous, and one of the observations made during the online debate was that several studies contributing negative z scores to the analysis had used procedures that deviated markedly from the standard ganzfeld protocol. Such a development is neither bad nor unexpected. Many psi researchers believe that the reliability of the basic procedure is sufficiently well established to warrant using it as a tool for the further exploration of psi. Thus, rather than continuing to conduct exact replications, they have been modifying the procedure and extending it into unknown territory. Not unexpectedly, such deviations from exact replication are at increased risk for failure. For example, rather than using visual stimuli, Willin (1996a, 1996b) modified the ganzfeld procedure to test whether senders could communicate musical targets to receivers. They could not. When such studies are thrown into an undifferentiated meta-analysis, the overa ll effect size is thereby reduced and, perversely, the ganzfeld procedure becomes a victim of its own success.

In the present study, we sought to test this explanation for the apparent decline in ganzfeld effect sizes. Three independent raters unfamiliar with the recent ganzfeld studies and uninformed as to the studies' Outcomes rated the degree to which each of the recent studies deviated from the standard ganzfeld protocol. The database was then reexamined to test the hypothesis that effect sizes are positively correlated with the degree to which the experimental procedures adhere to the standard protocol.

METHOD

Studies Included in the Analysis

In addition to the 30 studies analyzed by Milton and Wiseman (1999), an additional 10 studies were located by examining the six major publication outlets for parapsychological research. Many of these studies had been completed but not yet published prior to the cutoff date set by Milton and Wiseman for their meta-analysis. Following Milton and Wiseman, we treated separate experimental series within a given report separately but not experimental conditions within a given series. Two studies in the Milton--Wiseman sample that were originally reported in the Parapsychological Association's Proceedings of Presented Papers were replaced by their published reports in archival journals. These substitutions did not affect the statistical outcomes reported by Milton and Wiseman for these studies. Table 1 lists all 40 studies, with the 10 new studies identified by asterisks.