Tests of Extra-Sensory Perception

THE PROCEEDINGS OF THE AMERICAN SOCIETY FOR PSYCHICAL RESEARCH ~ Tests of Extra-Sensory Perception Among A.S.P.R. Members BY DR. GARDNER MURPHY AND ERNEST TAVES Columbia University During the academic year 1937-38 the present writers were carrying on a study of extra-sensory perception among normal persons, whose task it was to try to “per- ceive’ various concealed materials. Sometimes the material was a few feet away; sometimes at distances ranging up to 3,000 miles. Much of this work has already been pub- lished in the Journal of Parapsychology.! The Executive Secretary of the A.S.P.R. participated in a number of these experiments, visiting our laboratory at Columbia University, and also making a large number of long- distance calls on material in our laboratory. Owing largely to her interest, several other members of the A.S.P.R. began to take part in the work during the spring and summer of 1938. Their work, with one type of material, up to November 1938 is included in the publication men- tioned in Footnote 1. The main bulk of the data to be Teported here, however, is new, consisting of the results of work done at the New York offices of the A.S.P.R. between November 1938 and March 1940. The method of organizing a research group at the A.S.P.R. offices was as follows: The Executive Secretary Teported to the President the willingness of the present writers to conduct experiments at the Society offices under Carefully controlled experimental conditions. The President atranged a meeting at which the senior author made a brief address on some current ESP problems and the junior — ™ Murphy, G. and Taves, E., Covariance methods in the comparison of extra- Sensory tasks, J. Parapsychol., 1938, vol. 3, no. 1, 38-78.

2] Proceedings of the A.S.P.R. author assisted in the demonstration of a number of cur- rent methods. Fifteen volunteers expressed a readiness to come once or twice a week in the late afternoon to take part in telepathy and clairvoyance tests under the present writers’ direction. The meetings immediately began, the usual times being Monday at 4:30 p.m. and Friday at 4:30 p.m. There were usually five or six subjects present at the Friday meetings, and eight or nine at the Monday meetings, some persons at times attending both. Subjects were free to participate at any time, and at no time was any subject discouraged from participating in the experiments. Neither was any person subjected to undue pressure to participate. The subjects were, that is, entirely free to participate or not at any time. The general background of our problem was this: Prior to 1938 large-scale investigations of the capacity of ordinary normal persons to read or guess material not accessible through the ordinary channels of sense had, as far as we can ascertain, never given a clearly positive result. The results of J. E. Coover? at Stanford University, making use of one hundred subjects, each of whom called playing- cards a thousand times, are admittedly ambiguous; the results of S. G. Soal* in the analysis of material broadcast | in England appeared completely negative; the 1924 radio — test made by the present senior author did not lend itself — to statistical treatment, while the statistical treatment of the Zenith Radio data obtained during the winter of 1937-8 — is notoriously difficult to interpret.‘ As far as we have ascertained, clear-cut, positive results in clairvoyance, under — good experimental conditions, have regularly been reported — not from an ordinary, normal, unselected population, but from specially gifted persons. This is true, for example, 2Coover, J. E., Experiments in psychical research at Stanford University, Stanford University Press, 1917, xxiv + 641. 8 Soal, S. G., Experiments in supernormal perception at a distance, Proceed- ings of the S.P.R., 1931-32, vol. 40, 165. 4 Goodfellow, L. D., A psychological interpretation of the results of the Zenith radio experiment in telepathy, J. Exper. Psychol., 1938, vol. 23, 601-632. — See also, Stuart, C. E., A review of recent criticisms of ESP research, II, — J. Parapsychol., 1939, vol. 3, no. 2, 194-205. Tests of Extra-Sensory Perception [3 of the classical British, French, Dutch, and German work, and, as Dr. Rhine has pointed out over and over again, is true even in the exceptionally successful Duke University experiments.’ To the present writers, then, there seemed to be plenty of room for one more effort to ascertain whetffer extra-chance results are obtainable from a group of subjects distinguishable from the general public only in terms of their interest, not in terms of previous demonstra- tions of high scoring ability. It is, of course, to be granted immediately that members of a society for psychical research may, in a sense, be a select group, selected because of some actual gift in them- selves or in their relatives. The point is, however, a hazard- ous one. There is nothing in our contact with the group, or in accounts of their experiences, to suggest that they differ clearly from an ordinary sample of the population, except that their educational level and range of intellectual interest are above average, while at the same time their membership in such an organization doubtless means also a less skeptical attitude towards the existence of such phenomena than is ordinarily found in comparable groups of intellectuals.° Procedure One of the great needs in psychical research for a half century has been the development of some simple method of experimentation which any competent investigator could follow, so that the results from many investigators working with many subjects could be compared. If one man with one method gets positive results in the study of telepathy, clair- voyance, precognition, and other allied phenomena, and another man is unable to get the same results, the data are —_——. 5 Rhine, J. B., Extra-sensory perception, Boston S.P.R., 1934. ®° Dr. W.F. Prince, in a report on Human Experiences, Bulletin of the B.S.P.R., 1931, found a large number of eminent persons, most of them college graduates, who reported psychic experiences, but these did not constitute by any means a majority of the group to whom the questionnaire was sent. In an unpublished questionnaire among Columbia graduate and undergraduate groups, the senior author found about 60% willing to respond, and of these about two-thirds reported positively on having had one or more “psychic experiences” (without any commitment as to the explanation of the experience).

4] Proceedings of the A.S.P.R. likely to seem ambiguous to other investigators and to most readers. In any border-line field a repeatable method is important in order that basic incontestable facts can be independently ascertained. One feature of such a repeatable method would be the selection of a type of material which could be used with many different groups of subjects. Another important criterion would be that the method be usable with the ordinary run of normal persons, not merely with special sensitives. A positive finding in one laboratory based on the use of a specially selected group of sensitives might be quite valid in itself, but since the very question at issue is likely to be the genuineness of the result, the method should be one which can give clear results with any ordinary normal population, so that other normal populations can be compared with it. When the opportunity presented itself to carry out experi- mental studies of extra-sensory perception at the New York offices of the A.S.P.R., the present writers undertook to — start from the very beginning in formulating a method which would meet the stated requirements. This meant, in q the first place, working with a group of subjects who did not claim to be and who, in fact, were not exceptional persons or “gifted sensitives.”” It meant, secondly, that a very large variety of simple tests of telepathy and clair- voyance should be devised, in the conviction that capacities — of the normal person might well appear in relation to some ~ test methods which would be altogether absent with other — test methods. Our present report is a description of our quest for such a method, rather than a final report on any spectacular — results. It is felt that the data obtained are of value and importance only in so far as they clarify, both for us and ~ for other investigators, the general problem of methods of — testing for extra-sensory capacities in ordinary normal — persons. If we move one short step in the direction of devising a universally valid and suitable method which will J Tests of Extra-Sensory Perception [5 in subsequent years lead to more general agreement among investigators as to what kind of results can be obtained with a uniform method, our present report will have served its purpose. Now as regards our general procedure. It had been G. M.’s practice in earlier years to carry out with each new subject a series of crude preliminary experiments in order to awaken the subject’s interest and to gauge roughly the likelihood of finding anything worthy of sustained research. Such preliminary experiments, analogous to those reported by Upton Sinclair’ and by Réné Warcollier®, are seldom capable of giving results which can be clearly appraised in terms of chance coincidence. Sinclair’s experiments, for example, though brilliant in results, can only be evaluated strictly if the total number of impressions correctly trans- mitted can be compared with a definite number indicating the successes which might conceivably be attributed to the operation of chance alone. As pointed out by E. T. in the March number of the Journal of the A.S.P.R.°, a statistical method for the treatment of chaotic and casual material of this sort is not only unavailable but inconceivable. In practice, crude preliminary experiments may actually get the subjects into bad working habits and make strict work more difficult to achieve at a later time. We determined, therefore, from the beginning to use material which could be evaluated strictly in terms of the theory of probability. This is not at all a difficult thing to do, nor, which is important, does it necessitate that the material be entirely without interest to the subjects. A second requirement was that the material must be categorically out of the range of the known senses. This was achieved by the following procedure: The subjects in Most cases sat in the south room (room A in Figure 1) of the A.S.P.R. offices on the 9th floor at 40 East 34th Street, ; Sinclair, U., Mental radio, Pasadena, 1930. Warcollier, R., Experiments in telepathy, Boston S.P.R., 1938. 9 eae : i ey ay a a in psychical research, Journal of the A.S.P.R., 1940,

6] Proceedings of the A.S.P.R. New York City, either gathered around a central table or ~ in other chairs nearby, as shown in the floor plan. On some — days, when the number of subjects was small, it was found more convenient to use the smaller room E. There is no indication in the results that this variable was of any importance. In all tests of pure clairvoyance in which no human mind — was or had been occupied with the nature and order of the stimulus-material, the material to be called or guessed was concealed in a cardboard box or other opaque container. This material consisted in most cases of decks of cards of — various sorts which had been elaborately shuffled and cut, © face down, by a person other than the experimenter. Thus, for example, E. T. would shuffle a deck of cards and would 7 place the deck in a cardboard box, and the box would be — given to G. M. as he started down-town to the A.S.P.R. — office, or vice versa. In some experiments, in which the same ~ deck of cards was used several times at one session, the experimenter retired to room B, C, or D, shutting the door, where he shuffled the cards thoroughly, cut the deck while © not looking at it, placed the deck in the box, and took the — box into the experimental room. As the empirical controls of shuffling in a later section of this report indicate, all _ persons who shuffled the cards achieved a random order of — cards in the deck, i.e., the shuffling was adequate no matter — who performed the operation.” 4 10In addition to the tests of clairvoyance in which the actual arrangement of a the material to be guessed was known to no one, there were two other procedures: (1) The “pure telepathy” (PT) procedure in which the ex- ia perimenter acted as agent or asked another member of the group to act as _ agent, choosing the material and concentrating upon it, but having no ' physical reference point or object on which to concentrate. (2) There were _ also some instances of the procedure which Rhine calls GESP (general — extra-sensory perception), in which the agent gazes at the material to be transmitted. In this case correct perception of the material might con- ceivably be due either to the telepathic factor (i.e., contact with the agent’s mind), or to the clairvoyant factor (i.e., contact with the stimulus materi: itself). In general, most telepathic and GESP material, depending as it does upon voluntary selection or some very inadequate method of selecting what is to be sent, lends itself only very poorly to statistical appraisal. A small amount of work has been done with GESP in which the material to” be transmitted was chosen by a “pure chance” method (involving the use of dice and similar material). These data will be reported in due time; they are not yet sufficient to warrant any conclusions. ‘L “olg SHUIO “Y'd'S'V FHL 40 NVIg x007,7 Tests of Extra-Sensory Perception ( > j§PH & © & D — a — 2 eo] ae aa OE | A RR NS) [7

8] Proceedings of the A.S.P.R. Sometimes both experimenters attended the meeting, but as a rule only one was present. As the tables show, E. T. gathered well over half of the data. On two occasions Dr. Gustave Gilbert, who was familiar with our methods, took | charge of the meeting when neither of us was free at the appointed hour. A comparison between results obtained by the different experimenters will be made later. Also, Dr. Gilbert at times assisted G. M. and shuffled the cards. As just noted, however, the identity of the shuffler does not appear to be an important variable. The subjects, then, seated themselves informally at the tables in the experimental room, and followed the instruc- tions given them by the experimenter, namely, to “guess” card orders in various kinds of decks. Figure 2 indicates the way in which the subjects proceeded. The group is shown seated around a table, recording their guesses, or “calls.” It is important here, we believe, to stress the infor- mality of the laboratory situation. Without relaxing actual experimental control, the experimenters attempted to make the subjects feel completely at ease; all subjects were extremely cooperative, and the resulting atmosphere was conducive to the collection of considerable quantities of data under rigid, yet informal, conditions. It is now necessary to define in detail the special materials used in the clairvoyance tests, bearing always in mind that these were varied somewhat from week to week in accord- ance with the judgment of the experimenter. A large amount of “50-50” material was made up; that is to say, material in which each guess should have, on a purely statistical basis, an even chance of being correct. Here we used red and black cards, red and yellow cards, yellow and black, black and white, red and white, and, less frequently, numerous other combinations. We used photo- graphs of men and of women, and we used several pairs of symbols from the Rhine set, e.g., permitting a choice be- ee pn oe From Lert to Richt ARE Mrs. Hines, Miss WELLMAN, Miss Mount, A.S.P.R. Susyecrs SEATED AT TABLE. Ficure 2.

Tests of Extra-Sensory Perception [9 tween circles and crosses, or squares and wavy lines. In addition to this 50-50 material, we used playing-cards, where the odds are 1 in 4 of getting the suit correct, 1 in 13 of getting the numerical value correct, and 1 in 52 of guessing the card exactly. We also made a great deal of use of “Rook” cards, the deck consisting of fifty-six cards with 4 suits (printed in different colors), and the cards being numbered simply from 1 to 14 in each suit. The subjects were usually required to call the top 14 and the bottom 14 cards of this deck, although occasionally the entire deck or the top 28 cards were called. Most of the Rook data are from calling either the top and bottom 14 cards, or from calling just the top 14 cards, and in this paper we are reporting only on such calls; we are obtaining more data on Rook calls when the subjects are required to call either 28 or 56 cards in a row, but these experiments are not yet complete. We also made considerable use of what we call the “5-dot” task, an outgrowth of a method used with interest- ing results at Columbia. Upon 5 completely blank cards, plain white stock with no mark on either side, large black dots were marked in ink. These 5 were then shuffled into a deck of 45 identical blank cards, bearing no markings of any sort. After a thorough shuffling, and cutting, face down, the 50 cards were placed in a cardboard box and the subjects were required to indicate on which of the 50 cards : there was a black dot, being allowed only 5 guesses in which to locate the marked cards—guessing, for example, that cards 2, 18, 35, 40 and 49, counting down from the top, were the marked ones. The chance expectation here is one correct guess in every two complete executions of the task. (The chance that an individual call is right is one in ten, and this is multiplied by 5, since there are 5 guesses. In the course of two executions of the task, 10 guesses are made as to the location of a card bearing a dot, and, by chance alone, one of these ten guesses may be expected to be correct. )

10] Proceedings of the A.S.P.R. The subjects were provided with record sheets, with a carbon paper and duplicate attached, so that all their “calls” — were recorded in duplicate. Before the experimenter opened — the box of cards, each subject handed in one copy of his © calls to him. This item seems worthy of some emphasis here © in view of current criticisms of published ESP research. It is important to guard against errors in recording, either by subjects or by the experimenter. The routine was, therefore, for the experimenter to record the target order — as he opened the box, and before he looked at anyone’s | calls. The method of scoring for official purposes, when the data were assembled at Columbia, was to juxtapose the i target order and the call order as written down by the subjects on record sheets. For unofficial purposes of inform- ing the subjects as to their progress, it was common to read | off to them the actual order, allowing them to check their own scores after each task, but the carbon of the call order had, of course, previously been handed in, permitting a — completely independent checking against the official target — order. Results We are now ready for the results. We shall here give and discuss the significant findings. Complete data will be f found in the Appendices. Regarding the general problem of method which we raised in the introduction, it should be made clear first of all that, under the conditions described, all of the 50-50 tasks except the red-white must be regarded as giving © results so completely comparable with chance expectation as to justify no discussion whatever. In most of the 50-50 tasks this group of normal subjects working with this group of experimenters obtained nothing of parapsycho- logical significance. As an example, Table I shows the number of guesses regarding faces of men and of women in photographs, the number of successes, and the number of successes expected if chance alone were operating. With- Tests of Extra-Sensory Perception [11 TasBLE I. Sets of photographs of men and of women, shuffled and placed in boxes, and called in “down-through” order. Numbertot ‘calls, ual cei ol aya ie a en 1600 Number On Correce (oalle ne ul vi luslventsshnsenaes 807 FEXPOCTER DY NOMATCCUipesereys eset estloei eels uouciee acetates 800 Deviation trom wchance Wiican sine erdsadeesis +7 out statistical discussion, it is sufficient to report here that these scores are well within the limits of chance expecta- tion. The same is true for the other 50-50 tasks; there is no evidence for the operation of other than chance factors. The complete data for all 50-50 tasks are given in Ap- pendix I. Of the tasks with odds other than 50-50, none gave significant results except the Rook cards. The data from the Rook and the red-white cards, how- ever, are in a different category. These results are very surprising, and appear to warrant the view that the general problem of method raised at the beginning has been some- what advanced towards the point of ultimate solution. On February 20, 1940, when all of the data in hand at that time were brought together, it became evident that the correct calls on the top 14 and bottom 14 Rook cards in the deck of 56 were considerably in excess of chance expec- tation. Table II presents the relevant Rook data up to this date; the letter P (probability) indicates the likelihood of getting such a result by chance alone (“random sampling”), ie. there are only four chances in 1000 of getting such high Rook scores through the operation of nothing more than chance. The Rook scores, then, suggested that we were making Progress towards a method. But the data in hand by February 20th pointed to another result just as striking, Namely, a very consistent tendency for our subjects to go below chance on the red-white decks. The complete batch

12] Proceedings of the A.S.P.R Tasie II. Data on Rook calls to February 20, 1940. IN MEAICI ORE CAMS) 2 cy hecaeeatere eluent Mal ah oo iad 14,826 Gonreaiieae yl utMnlnee tet eC sc tonal wl 1,143 ACMenICer eR DEGEALION ees cn ney MLN al a 1,059 Demanen trom) chance U0 onc wa Ui, +84 nr nt ran Sag UU MU A dee i SU Mau .004 Taste III. Data on red-white calls to February 20, 1940. IMMDED NOL Calista cc tieniie WEL) Auman adic) a 2,650 Marrectiical tae it, va AU MMM ANY oe MADE esa Nica INES 1,230 AGHATICEERDECtBHON se clculil Meret emer Maat hy Hyeey is 1,325 Deviation roma chance liu lie —95 | et Lara Ooi HARPS aay LUA MOA E EAH OPED AN ave re) .0002 of data collected up to this time, including all tasks, would — be extremely unlikely to give any task as high as the Rook, — or any task as low as the red-white through the operation — of chance factors. As seen in Table III, the red-white calls _ on February 20, 1940, numbered 2650, and the successes — numbered 1230 as against the 1325 expected by chance. The P value here is .0002, which means that there are only © 2 chances in ten thousand that a chance (random) sample — would give such low scores on the red-white deck. (For evaluating the obtained deviations from chance 7 expectation the binomial statistics are used. The P value is determined by computing in the case of the Rook data, for — example, the moments of the distribution (1/14 + 13/14)* and substituting in the Charlier expression of the gen- — eralized normal curve. In this connection see Huntingtons’ paper, A rating table for card-matching experiments, in the ' Journal of Parapsychology, 1937, vol. 1, no. 4, 198.) With these data in hand, it was immediately determined to gather two types of additional information: (1) Both — E. T. and G. M. must gather considerably more material _ with the Rook and red-white decks with the same group of — subjects to ascertain whether the positive trend for Rook — and the negative trend for red-white would continue. (2) Tests of Extra-Sensory Perception [13 Completely independent investigators, preferably those hav- ing no convictions on the question of extra-sensory per- ception, should be asked to collaborate, working entirely independently and with fresh groups of subjects. Accordingly, a considerable amount of additional Rook and red-white data were gathered at the A.S.P.R. meetings, and the research carried forward through March 31, 1940, at which time the method was finally terminated so far as the present report is concerned. Concurrently, G. M. wrote to professional psychologists of his acquaintance teaching at universities and colleges in various parts of the United States, instructing them in the preparation of the Rook and red-white cards, and asking them to get their classes to call them by the down-through (DT) method. The results from these classroom experiments will be reported when available. The additional data from the A.S.P.R., however, have clearly continued to support the results already indicated, and in the following section of this report the complete data, up through March 31, 1940, will be presented. Analysis of data collected through March 31, 1940 I. Rook. Approximately 3,000 additional Rook calls (first and last 14 cards in the deck) were collected between February 20 and March 31. This additional sample of data also revealed the positive trend which was previously in evidence, and the entire number of calls and successes is indicated in Table IV. This table represents the entire Rook data for the method described, and the P value lies beyond the ordinary limits of chance variation. Taste IV. Complete Rook data, top and bottom decks of 14. Naniber of Callenvacca dieu i leah afl un Naa Jed) 17,612 Worrect (callanpagerian h Mise bn Oe hi a alana 1,361 ATTN emcee CERN ica aes das atsrssscdapassaecds vase aee 1,258 Deviation from chance .......... i ehsddsedvceatisstuneees +103 IBRD CSTRRIN WARN Wie NEC Clits UN TEU NG HARON eae PLS I 0015

ta Proceedings of the AS p R. Table IV Premmber of quests re vant Rook data, butiil sh nett it a positive dwn ae ronnswered. We know: that we havé f sufficient ma avn ii chance expectation, a deviation Wide ac has ti y to be designated sig. nificant, but cae subjects th to what kind of performance on the part of en mii f i deviation, is to be attributed, - Does the dé”™ froin relat TOM MARY scores just a little over chance, 0 ely few ®Xtremely high scores? Do the subje“? Fore CCsistently a trj Mostly at the chance ley ; a they score es, or are both ah el, with occasiona extreme sco!” if OMens resent? am a t of considera : Pp f ese questions af© ble intere it i St, and in order to it 1s necessary to tabula,’ ( , answer then! : at ’ es of jhe various scores, e the obtained fre. Frequency z 3 7 p 3. DISTRIBUTION oF R Fro" 9m Stoney —_ Top 14 Tests of Extra-Sensory Perception [15 Chance expectation for the deck of 14 Rook cards is iit but this does not mean that all scores above 1 are definitely extra-chance. Rather, chance will produce a distribution of all the scores from 0 to 14 (although no scores as high as 14 are expected by chance unless the number of trials is astronomically large), the average of which will be 1. It is possible, then, to compute the expected chance distribution, which may then be compared with the obtained distribution of scores by means of standard statistical techniques. This process yields the additional information we desire. In Figures 3, 4, and 5 are shown the expected and obtained distributions of the various Rook scores. From these curves and from the data presented in Table V it is apparent that ms Ise —_._.. Obtained ; if & 15 Q + y So \ 25 o ial 4 2 a J + Ss 6 by Score Ficurs 4. Duisrrrpution or Rook Scores — Borrom 14

16] Proceedings of the A.S.P.R Obtesined ah en PR te Expected 2e0 3 Frequenc » } } Score Ficure 5. Distrrsurton or Rook Scores — Top 14 anp Bortom 14 CoMBINED 7 the deviation from chance expectation shown in Table TV is attributable both to a deficiency of scores of 0 and a surplus of scores of 4, 5, and 6 (6 being the highest score we have ever obtained on a deck of 14 Rook cards). Taste V, Comparison of expected and obtained distributions of Rook scores—top 14 and bottom 14. Obtained Expected Score Frequency Frequency Difference 0 404 446 42 1 493 480 13 2 247 240 /, 3 89 74 15 4 20 16 4 5 3 v4 1 6-14 Z 1 1 a Tests of Extra-Sensory Perception [17 Scores of 1 and 2 are near the expected frequencies, scores of 0 are less frequent then expected by chance, and scores of 4, 5, and 6 are in excess of chance expectation. In interpreting Table V it is important to note that the actual size of the deviation from chance must be considered with respect to the magnitude of the score. The deviation of 4, for example, for the score of 4 is considerably more significant than the deviation of 13 for the score of 1. Deviations of the obtained frequencies from expectation are considerably more significant where the score is high than where the score is only slightly deviant from chance. Next in the analysis of the Rook data comes the question whether the differences between the obtained frequencies and the frequencies expected by chance represent a sig- nificant deviation, or whether these differences are of such small magnitude that they might well be obtained in a procedure involving nothing more than random sampling. The Chi-square procedure provides such a test of sig- nificance, and obtained values are presented in Table VI. Taste VI. Chi-square and P for the various Rook distributions. Distribution Chi-square Ya Top 14 9.105 .06 Bottom 14 2205 70 Both 9.447 05 The P values of .06 and .05 in Table VI are not small enough to be termed significant, but they are suggestive. What we have so far, then, is the knowledge that the total deviation from chance of the Rook scores is significant, and that the departure from expectation occurs not in one place in the distribution of scores, but is more or less spread out over the entire distribution, so that the obtained distribution still does represent a pretty fair “fit” to the expected dis- tribution. Stated differently, this means that our significant deviation from chance was not obtained in spectacular Spurts of high scoring, but was obtained more or less con-

18] Proceedings of the A.S.P.R. Tests of Extra-Sensory Perception [19 sistently during the research. This is, perhaps, an indica- ST UT OT EEE EEL UE Taste VII. Breakdown of Rook data by individual subjects. tion that normal people have generally available a small quantity of some psychic factor. This is speculative, but it Sibieet Ne a indicates the next step in the analysis, which is to break a jus 0 CM 31 _7 down the scores in terms of individual subjects to determine EB 2 1 EM 2 0 whether the positive deviation is from all subjects more or MB 4 1 GM 18 3 less equally, or principally from a few “gifted” subjects. OB 2 0 HM 100 abe This analysis was performed, and the results are shown in VB 4 2 MM 118 : 09 Table VII. Forty-two subjects participated in the Rook CD a Mi ie ; mn experiments ; some subjects called the deck but once, whereas ae 6 _2 Jo 1 A one subject called the deck 222 times. Of the 42 subjects, BF 12 2 AP 20 bind 14 are below chance expectation, 23 are above, and 5 sub- HF 4 10 hi 47 2 jects show no deviation whatever. The table shows that RF 30 8 MP 46 ai over a third of the total deviation came from one subject, SF 80 4 re ' ey J. G., whereas considerably less than a tenth of the total as 2 IR 54 8 number of calls came from this subject. JG 900 37 90007 MR 2 pe The average score of the entire group is 1.081 correct AH 8 ae ot) 4 i calls per deck of 14 cards, chance expectation being 1.000. aoe ved Ng CV 2 mn It is of interest to know whether the entire group, exclud- MH 19 1 AW 222 13 ing the exceptional case of J. G., scored significantly above EK 2 0 FW 1 2 chance. The average score for the remainder of the group, Total deviation 103 then, is 1.056, and if J. G.’s scores are reduced to this level, Total N 1258 statistical analysis reveals that there are 92 chances in 100 that the “true” difference between expected and obtained scores is greater than zero, and is positive. The result is suggestive, but since it still leaves 8 chances in 100 that the difference is not greater than zero, no conclu- sion that the group as a whole scored significantly above chance is indicated. Rather, the writers are of the opin- ion that there is a certain amount of evidence that the entire group did manifest some small ability to score above chance; without the results of J. G.’s calls the data are merely suggestive, but the entire mass of data is quite striking. *N is the number of decks of 14 called, ** D is the deviation from chance expectation. deck was a correct guess most likely. In other words, when guessing the order of cards in a deck, is spatial posi- — tion in the deck a factor of importance? Rhine” had pre- viously reported interesting results with respect to this question, and we wondered whether our data contained any such result. For this particular treatment of the data we used only the calls from J. G., since here, apparently, a genuine extra-chance factor was producing high scores, and here would be the most promising field in ‘which to Search for any such effect. J. G.’s correct guesses were therefore tabulated according to position in the deck; the Tesults are shown in Table VIII and in Figure 6. —_—_—_—_— 1 Ibid, p. 124 ff, The analysis of the Rook data was almost complete at this point, but there was one other question in which we were interested — the question at which points in the

20] Proceedings of the A.S.P.R EN) 3 s = Chance ratio +S q-6 7-8 9-1 (2-19 Position in deck - Ratio of successes Fa expectation Ficure 6. GrapH SHow1nc Distripution or J. G.’s Rook Successes Taste VIII. Tabulation of J. G.’s correct calls at each position in the deck of 14 Rook cards. Position in Total correct deck calls 1 De SAUTER ML MEA ae Ca TALE an IN 10 PONE TBR RIOMIRUERELN WSR Acer ag UL TENT 3 SRI ALSTOM RRC Marne Wh Mt BA CSRS 5 “cAI AGIA SOLER IURR Ise WADURS Ne MBI UR OR ALIN HOA ALG RE 15 SS PRNVABUU ALIEN UME Hirst ore pal Cg OOM OSL ae MU 10 ja SUM Dt peat to a LR Se UTADA 15 VAD NOMED SDS MURA AIG SETUP UC aS Wee CDA RY 6 ‘CURL DE ASE SATS UA ela BONN ARIE RIGO Up 17 USAR IR URIOT SI CROSUID RE MS a SSCA SOTTO 6 OIA ey Se LOWERS APD ODS ES RA nas a HSL 9 ih D ate Gea penal ea Nabe WANA MeL th) Su 9 EAU tO Dace) AUR ae eMn Pea Alt AUER ORR 12 LES TEAR e he Eee MEN REAR ate EEL Pk OME OT Ws BALE vf PURI al toll & Jae) WU ERE cy a 3 Tests of Extra-Sensory Perception [21 The table reveals a general tendency for successes to cluster toward the first third of the deck, and when the values are averages and plotted as in Figure 6 the relation- ship becomes apparent. This “inverted U” curve is one of the types of curves reported previously by Rhine, although most of the Duke University subjects characteristically pro- duced a regular U curve, opposite to that shown in Figure 6. 2. Red-White. As shown in Table III, the red-white data up to February 20, 1940 were distinctly extra-chance, correct calls being so far below expectation that there was no question as to the reliability of the trend. This, of course, was an interesting result, but to us it had little value unless it could be repeated. Consequently we decided to collect more red-white data under similar conditions, to see whether the P value would remain significant or would increase until it became of little value. Accordingly, over 4,000 additional red-white calls were collected, so that the data were almost trebled. These additional calls also revealed the negative trend present in the original red-white data. . In collecting these additional data, decks of both 25 and 50 cards were used, the trend being equally negative in each case; to expedite statistical analysis, the decks of 50 are considered as two decks of 25. The complete data are shown in Table IX. Taste IX. Complete red-white data as of March 31, 1940. Number tof calley ea Cea Sa en 6,975 COPTECE CaIES Me Ue Uo UN Gis Mt Als UO ou 3,343 Chance expectation tiie lait late ain ii set 3,487.5 Deviation) from chance Wu le —144.5 1 SAUCE MBIAU Ee OONCU AI SRST MENUDO UH) Ae .000,276 It is apparent that the entire red-white data are, by this analysis, beyond the limits of ordinary chance variation— the negative deviation is statistically significant, indicating that in the present research when subjects were asked

Freg uency a 22] Proceedings of the A.S.P.R. to guess red-white cards, they scored below expectation to such an extent that chance is excluded as an explanation. The next step in the red-white analysis was to plot the frequency distributions of obtained and expected scores. These curves are shown in Figure 7. From visual inspec- tion it is plain that the obtained curve is not simply skewed to the left, as might be expected, but is moved bodily to the lower end of the score scale. For almost every score value below the chance expectation of 12.5 the obtained scores exceed the expected, whereas for every score value above the expected mean the expected scores exceed the obtained, <= Expected a= Obfained Poy = =o 8 B® & ov © » 45698629 © 4 42 13 4 15 (6 17 18 19 20 24 Score—-Deck of 25 red-white cards Ficure 7. Oxsrarnep AND Expectep FREQUENCY DISTRIBUTIONS OF SCORES on Deck or Rep-WaHiITE Carps Tests of Extra-Sensory Perception [23 as shown clearly in Table X. This is an interesting result, indicating a linear departure from chance expectation. The obtained negative deviation is not attributable to a large number of extremely low scores, nor to a spectacular defi- ciency of extremely high scores—it is rather that some constant factor is apparently at work on all scores, high or low, making each score just a little less than it should be, resulting in the bodily shift of the entire distribution in a negative direction. Since the departure from the expected curve is still a good fit for the obtained curve as the Chi- square and P values in Table XI indicate. TasLtE X. Table showing comparison of expected and obtained distributions of red-white scores—deck of 25. Obtained Expected Score Frequency Frequency Difference 19-25 1 2 —l 18 1 4 —3 17 8 9 —l 16 10 17 —7 15 ae 27 —5 14 32 of, —5 13 41 43 —2 12 48 43 5 11 41 37 4 10 33 27 6 9 18 17 1 8 15 9 6 7 7 2 3 6 1 Z —l 5-0 1 1 0 Taste XI. Values obtained in Chi-square analysis of red-white scores. CHioacgema de 1h ikas sites dicstsak ookaseyy bulstaversoasestenees ameunabins 16.34 Di lssciaiesaanes ses RyeaueadcHarb eae thnsuetuatsouah vatouaciusgs seats 13 ReaD ee PSUR Neto UATA MET PRVTR EIN PORG OIA BS NVC 22

24] Proceedings of the A.S.P.R. In the case of the-red-white data, as in the case of the Rook data, the total number of hits, or correct calls, was tabulated by individual subjects, with the results shown in Table XII. Here the group effect is considerably more marked than in the case of the Rook data. Twenty-four subjects took part in the red-white experiments, some call- ing the deck but twice, one subject calling the deck 59 times. Of the 24 subjects, 17 are below expectation, only 6 are above expectation, and 1 subject shows no deviation. As in the case of the Rook data, here also is one exceptional subject, this time A. W., whose red-white scores up to April 1, 1940 were so low that the chances are 250 to 1 against an explanation in terms of chance alone. Excluding for the moment the data from A. W., the remainder of the group scored so low that the chances are about 170 to 1 against a chance explanation. When the entire data are considered together, however, the negative deviation is so large as to be expected by chance only once in 4,000 such ~ periods of experimentation. A negative result of this kind, from a purely statistical standpoint, is precisely as significant as an equally positive — deviation would be. One must, in a sense, know what the cards are in order to miss them enough to produce a sig- _ nificantly negative deviation. Such a negative result is 7 rather striking, and demands an attempt at an explanation. The present writers have tentatively adopted a hypothesis, but further research is necessary before much can be said | about it. It would appear, at least, that there may be some- j thing about the red-white situation which arouses nega- tivism. This may be because of obscure affective factors— _ the symbolic values of red and of white are deeply ingrained — in our culture, at least, and this general affective tone of — the material used may have something to do with the type — of result obtained. Beyond that, however, the authors are — not at present willing to proceed. Tests of Extra-Sensory Perception [25 TaBLE XII. Breakdown of red-white data by individual subjects. Subject N Correct calls Deviation iP MB Z Bo —3 239 AB 12 133 —17 028 BB ) 30 —7.5 053 ED 4 46 -4 .242 KD 6 79 0 .500 RF 12 165 15 042 SF 36 430 —15 .166 GG 3 40 2:5 363 JG 8 92 tat 145 GH 30 368 —7 318 KH 6 74 —l 468 DH 9 109 —3.5 345 MH 9 106 —6.5 Wa EK va 28 3 239 HM 9 102 —10.5 .092 MM 15 197 95 176 CM 9 99 —13.5 .036 LP 14 179 4 359 AP 12 136 —l4 053 MP 4 45 —5 184 IR 11 139 We) 452 ET 3 35 —2.5 322 AX 3 31 —6.5 082 AW 59 662 —50.5 .004 Empirical Controls The results just reported, however interesting, would nevertheless mean little unless proper empirical controls were made. It is necessary, that is, to determine the degree to which the shuffling of the various decks of cards was adequate—to determine whether the materials used were truly random, or were in some systematic way deviant from random sampling. If in the red-white decks, for example, there was a consistent tendency for the red cards to be found at the top of the deck, and a subject similarly tended to guess more red cards at first and more white cards later, a spurious positive result would be obtained.

26] Proceedings of the A.S.P.R. Tests of Extra-Sensory Perception [27 chance only once in the course of 4,000 such experiments. Second, when Rook cards are guessed 14 at a time, the scores are high to a degree to be expected by chance but once in the course of 670 such experiments. It is possible that the results are partly due to the personalities of the experimenters, or to fatigue, excitement, health, diet of the subjects, and other such multitudinous factors, but to the writers it seems more likely that the attitude of the subject toward the task is the most important single variable. It cannot be too strongly emphasized that we have striven chiefly to develop a method, and that until our results have been independently confirmed they cannot be incorporated into the general body of science. They may have been due in part to tangible or intangible aspects of the procedure, such as the type of material used, the manner of preparing it, or the attitudes consciously or unconsciously entertained by the experimenters or by the subjects. We believe that the method can be successfully repeated, but nothing is less characteristic of science than premature declarations of success. A few words about plans for subsequent experimentation seem in order. Our experience has been that relatively few subjects are willing to give the time to carry out rigidly controlled experiments permitting statistical evaluation over a long period of time. A given type of material has to be used a long time before one has a clear-cut result. Never- theless, two forward steps are possible. Perhaps a few members of the Society, reading this report, will be suf- ficiently interested to join and work with the group. Second, there is now enough material in hand from some of these Procedures to justify starting with other types of objects to be guessed, involving other colors and pictures and various materials with more direct and unambiguous feeling tone or emotional value. It is true that such material quickly loses its special appeal, but the recent study of Pratt and Woodruff” shows that a change of method every Paw Pratt, J. G. and Woodruff, J. L., Size of stimulus symbols in extra- Sensory perception, J. Parapsychol., 1939, vol. 3, no. 2, 121-158. While it is true that in the subject’s calls such deviations — from random sampling are present, results will yet be genuine provided that the target, or deck of cards to be called, represents a random sample. To make this deter- — mination, then, either of two methods may be used: (1) _ Each target deck may be checked against the succeeding — target deck, the correspondences being noted. A consistent — extra-chance tendency in the decks will then produce extra- chance frequency distributions. (2) The frequencies of each ~ kind of material used at each point through the deck may _ be tabulated. Such an analysis would reveal any tendency for the fifth card in the red-white deck to be red more than — it should be by chance, or any similar tendency. Either of — these two controls is probably adequate for all practical purposes; in any case, negative results with both types of — control should satisfy most readers as to the adequacy of shuffling. Both such controls were made, the results of — which are shown in Appendices III, IV, V, and VI. The obtained P values indicate that the shuffling, for all — experimenters, in both the case of the Rook and red-white q data, was adequate. Summary We should like now to recapitulate the main trends of — the work done in the two winters’ investigation at the — A.S.P.R. offices. Most of the work appears to confirm the — conclusion voiced by Coover that ordinary subjects, work- — ing in an ordinary mental state, with statistically measur- — able material, do not average significantly above chance — expectation. Some of the results, however, do indicate a process which © the statistician would not permit us to treat as chance effects — (nor, in view of the procedure, as effects due to sensory 4 cues, imperfect shuffling, etc.). First, the scores on the simple choice of red or white cards are paradoxically low, ql not merely low enough to arouse interest, but so extremely ~ low that such a result would be expected to appear by —

28] Proceedings of the A.S.P.R. few trials, provided only that the statistical treatment _ remains uniform, is even advantageous. It is of the utmost importance that the current work of Pratt and Woodruff _ just mentioned be repeated somewhere outside of the Duke University laboratory. Here high scores are apparently obtained from normal persons by a rigid method of control. Third, the ingenious method of Whately Carington,® — informally reported from Cambridge, England, could well be adapted to our group performances and urgently calls for repetition. These are among the things to be done during the coming season. 13 Carington, W., Experiments on the paranormal cognition of drawings, to appear in J. Parapsychol., vol. 4, no. 1 Appendices A few words as to the most convenient method of reading the tables may be in order. The column at the left in Appen- dix I designates the kind of material used in a given set of experiments. The capital N in the second column indicates the number of individual tests given by a given method. There were, for example, 2794 playing-cards upon which guesses were made. The next column, “hits expected,” tells the number of right guesses to be expected on the basis of chance alone. For example, the likelihood of getting the color right when one is limited to ordinary guessing is 50-50, so that half of 2794, or 1397 should be correct. The next column, “hits obtained,” tells the actual number successfully guessed, which, in this case, deviates from chance expecta- tion by only 3, as the next column shows. The next column, “\/npq” is a standard measure to tell within what range the scores may be expected to vary; ” is the number of calls, AppenDIx I. Summary of all 50-50 card data. Material N Hits Exp. Hits Obt. Dev. Wnpq D/Vnpq P* Playing Cards.. 2794 1397 1400 +3 2643 0.11 .46 Circle-Blank .... 3275 1637.5 1652 +14.5 28.61 0.51 31 Square-Waves.. 775 387.5 381 —6.5 13.91 0.47 .32 Yellow-Blue ... 270 135 121 —14 8.2 1.71. .04 Yellow-Black .. 1085 542.5 527 —15.5 16.47 0.94 .17 Circle-Cross ... 8010 4005 4084 +79 44.75 1.77 .04 Masc.-Fem. .... 1600 800 807 +7 20.00 0.35 .36 Black-White .. 2275 1137.5 1124 —13.5 23.85 0.57 .28 Red-Yellow ..... 625 SHI Pre pends) 0,0 MMe aie Best bass 100016 Red-White ...... 6975, 3487.5 3343 —1445 41.7 3.46 .0003 SROTAL sons 27684 13842 13739 —103 832 124 .10 * The values in this column are not completely accurate, being based simply on D/Vnpq. Actually, as indicated on page 12, the statistical procedure for 4 precise evaluation is considerably more complex. These P values, however, are sufficiently good approximations to indicate whether significant results are being obtained. When a critical case is found, the more precise analysis Is needed. ;

30] Proceedings of the A.S.P.R. Tests of Extra-Sensory Perception [31 Appenp1x II. Results obtained checking each red-white deck against / AppENpIx III. Red-white target control, showing frequencies of red the succeeding deck. cards at each position in the deck for each experi- RT EAR TTR iN 1350 piieaenb ETN eros 1 URS OM RuO UENOST RON UAE rena HOMLOTEO OE TO REE 675 ra ch sud EET OOO ANT NEE OS 667 Position in deck on i, Ai LD [coca tsi Stas le ARE ONS EUS IURN IDG! SN UROUD SCALIA aI id OUR —8s ; puncc nu raa hn : ' ; SPU intawtesnesansceatinapesnelarssespaieneaiautubsasvel wide Me alusauns lua byeeync 34 RN Ue ne 24 6 0 a Ae cies eounthaatusenasel a lasles 17 6 3 p is the probability that a given call is correct, and q is — Bi PAULO RN DA 21 6 3 the probability that it is incorrect. This measure, the stand-_ Ce ars 19 6 1 ard deviation, will include about 68% of all the scores which — Aaa ashe EEN 17 9 2 would appear in a large chance series. The procedure here AAPM eateaO ie a! Oe 24 4 3 is exactly as if we said, “If these are Japanese, I should ~ Pe MANOA TPS LETTE 21 5 4 expect them to be about 5 feet 5 inches” but if we know TO 21 4 2 that about 68% of all adult Japanese men are between 5 © | SIMI EF SHAY ARO ROY 28 8 2 feet 3 inches and 5 feet 7 inches in height, the 2 inches, © VD icasoss ion VORA ale 23 7 2 measured in either direction from 5 feet 5 inches is the o ‘ Be I Msn LS cz AINE 22 5 2 or standard deviation, the measure that we are here using. _ TA ssossonsssaotecinnonsncatoons 22 7 0 The reader, without puzzling about statistical detail, can LAC MAN eT EAH 22 5 1 see that the further a deviation is from the expected num- ~ 16 eA 16 6 1 ber, the more standard deviations or units it is from the ~ UF ia iolaatelaeaale ie. 17 3: 2 expected number, and the more surprising it is. A value pF Sapurdret en OPE ALOR 19 6 4 of more than 30 above chance arises by chance alone only ~ 1D stash deantaueaiabtiinetiaed 21 9 1 once in about 700 cases. If the reader glances down this 2 schecseoivieceans opeeelinar ene 21 8 1 next-to-the-last column he will see that with all of our © BL ecenedeuedeal vollathodttabps 19 5 3 materials except the red-white the data are within the 1 snail vabailndbacupecaitcs 14 4 1 category of ordinary chance expectation. The results with yx TU vebtes ona SBN 16 5 k the circle-cross are suggestive, and with the yellow-blue BA sssestoensdeictpbeetipatiens 21 6 1 almost the same, but no statistician would accept such 25) aconssabntsacipuveesiniodices 15 6 2 values as conclusive, and certainly no psychical researcher © should do so either. The P value, of course, reveals nothing whatever of the Cause of a phenomenon, but it permits a concise statement as to whether a result is reasonably attributable to chance or not, provided one has a large amount of material per- Mitting uniform treatment, and provided one makes full t use of the necessary statistical safeguards. The column which means the most, and which fortunately ~ can be explained more easily, is the last column, the one — captioned P. The P value is simply the likelihood that a — random chance collection of material would give such a deviation from the expected mean as is actually obtained. ©

32] Proceedings of the A.S.P.R. Tests of Extra-Sensory Perception [33 ApprENDIX IV. Distribution of 5-Dot Scores. » AppenpDIx VI. Rook control, ET’s shuffling. Score fo te D ey ” Position in Rook Deck (ORBEA es 148 166 —18 324 1.952 0 0 12S an Ghe tone oe Or TO Me Ponds 14 | Total UN ey tT RAN t 106 92 +14 196 1.849 1 106 i, i2 9 16 9 10 9 i2 i2 12 19 10 12 10 12| 10 2 es SU it ic coal isi A i 19] 17/14 00096. 7. Bae 15,9) 42/6 a2 eto age CARN lb: PARLE 1212 Ser i RA oe) ASL AB RS a) a 281 280 4.937 163 RR EC ee VOR eg a Ln ne AR CRU AN FOAM AM ree a |10| 5 7 9 11 10 14 9 12 10 12 16 13 8 9] 145 CY EC Aa an ey aC CMS ia be inn Cae aN Fo | 14 | 178 Nig 281 PB 15. 1016 1 ae AS 10 808 Se = 692 0 ONES AG RAE SA Ve TR SM AM Me te 2 To be within chance expectation, the values in the GM i 5 ; U i rp i Hf a va i 4 in He iy hi hie ii column should range between 12 and 30, in the ET column, © 3 A] 907 ARO) BN Ae aR IO, Bs a2 a a . . vu between 1 and 9. Inspection shows that all the obtained — 0 a a en ae ae a ee eh) Re eH SU values in these two columns lie within the expected ranges. - : 13 wt M, ri 8 ‘ iA ; q " i a L 5p In the GG column the amount of data is so small that no | Total | 154 154 154 154 154 154 154 154 154 154 154 154 154 154 [2156 single position in the deck could reveal a significant devia- — V2¥2 — V2N—1 = 1.77. As in Appendix V, the obtained value indi- tion, ie. values of O and 4, which are the maximum and — cates that the shuffling of the Rook cards was adequate. minimum values possible of achievement, are within a 3o j range. The table indicates that the shuffling of red-white — cards was adequate for each experimenter. APPENDIX V. Rook control, GM’s shuffling. Position in Rook Deck A a eS Gale CSE GM aR OU MON Ch PET Site | Total 14 A eV A eA aT a A SAN A oH 13 TAS Ao OARIN ANODE NY NI SY UMM HDC OC AERC AW Binns pau | 59 12 BURA Ne OA OO: Dei imate hae td! | aN Meg Rea SETA 02K NB NS AHN NA GT A 2 CU Pe, Roi Wa | 62 G& | 10 SLAIN LAR AN SRS UUM FAS cage AL DO HG ees TR Ps A | 55 ng 9 GNSS VGA ec ARORD: AGIAN Dans RUE: atc er RCE NY eg UT | 62 2 Rae REM As Ee Mee a a an UR On AS MI OE AM A Ue a Se ae ARS eran cr OUD RMR aI JAE PARR SRE RC UP I Ry LU iD gh ve S) PTT SIN ps MUN AN a Seis Re” SHE SHE: LS Sh la | 70 DE TRIE GU DCE Simp Ei AG ASME iS TABS A Aine age a 3 SSUES REIMAN LT Ua SPINS SHU REUSE RN URC EL | 58 f 2 te AN AMR RS BoaPRes TRUM: Ne oY” aT UMTS RR a Pay dp | 66 1 SHAR CANA UCAS eSARn on EAM ctu Okc AMES at uns ete deus HES Total | 61 61 61 61 61 61 61 61 61 61 61 61 61 61 |854 . —————<— a V2x2 — V2N—1 = 0.38. This value would have to exceed 2.00 mate- rially before any extra-chance factor would be indicated. The obtained value i of 0.38 indicates that GM’s shuffling was adequate. a a el ee