Designs for Tests with Free Response Material

Reprinted from the Journal of the American Society for Psychical Research Volume LVI— October 1962 — Number 4 Designs for Tests with Free Response Material W. G, Rot A Study from the Psychical Research Foundation, Inc. INTRODUCTION The tests in this outline have been developed for research with “mediums” and other seemingly gifted subjects who sometimes are averse to conventional parapsychological methods, such as card-guessing trials. The designs have been adapted from the accustomed practices of these persons and are likely to be accept- able at least to those who are interested in scientific research. Since the methods allow for the free expression of alleged psychical impressions, they permit the exploration of phenomena which cannot otherwise be approached, such as those which bear on the problem of post-mortem survival. It is recommended that a three-step test design be adopted. First, the investigator should satisfy himself in the course of informal observations that the subject appears to have psychical ability. Second, one of the designs in this outline should be selected according to the type of phenomena produced by the subject. An inventive experimenter will be able to extend the basic designs to other situations and types of test if the need arises. For instance, the methods described in Sections I and II can be used for studies of “psychic” or “spiritual” diagnosis and cure. Blood, sputum, or other samples from patients (or their names) can be used in the place of token-objects. A short pilot experiment should then be carried out with the selected design. If this is successful, a con- firmatory series should be planned and conducted. It is essential in this third phase that the investigator decide on the length and type of test beforehand and closely follow his plan. In the conduct of each type of test described in this paper, one experiment consists of four or more tests which are evaluated together (as shown in Section IV). For convenience, such a series is referred to as a “set.” In pilot experiments (the second step) one set suffices, though more may be required to reach a judgment regarding the subject’s abilities. As a safe rule for the third step, or confirmatory work, about fifty individual tests should be carried out before a satisfactory statistical appraisal can be made. If each set consists of four tests, twelve sets are required; if there are five tests to a set, ten sets are needed, and so on. The number of tests in each set should remain the same in a series of sets.

Reprinted from the Fournal of the American Society for Psychical Research Volume LVII — January 1963 — Number 1 Errata Mr, Walter A. Carrithers, Jr., writes that Footnote 6 on page 133 of his article, ““Madame Blavatsy: ‘One of the World’s Great Jokers’ ” which appeared in the July, 1962, number of the Journal should conclude, ‘“‘Nov., 1884, p. 71.” as In “Designs for Tests with Free srt 4% Material” by W. G. Roll in the October, 1962, number of the Journal, the annotation mark for “correct and particularly significant” ((V)) was left out by the printer in the following places: Page 187, Sample 1, line 10: “V) for correct and particularly significant” Page 187, Sample 2, lines 5-6: “‘a man’s voice (+/) spoke about insulin-shock or -shot ((V)). The person’s hand feela numb (()).. . ” Page 188, 2nd main paragraph, lines 3-5: sae. he can count as 2 or more, as he decides, only he must decide on the value he assigns it before the tests. I generally count (/) as 2.” Page 189, bottom line of Section II: ‘No. 2 would rate a () if true” Page 189, line 2 of Record Sheet: “Write (V) for correct and particularly significant” Page 193, line 3 from bottom: “With (VY) counted as 2, the score is 7.” PRINTED IN BRUGES, BELGIUM, BY THE ST CATHERINE PRESS, LTD

185 Designs for Tests with Free Response Material Though the number of tests in each set must be the same, the subjects and cooperators may be different. A set consisting of four to six tests generally can be completed in about one hour. 1. Toxen-Opject Tests In token-object tests at asap known as “psychometry’’), the subject, by touching or looking at an object, diy can obtain information which refers to events associated with it or with the life of some person who has been in contact with it. In each experimental set, one subject takes part, and there should be four or more cooperators. Each submits a token to the experi- menter, this object belonging cither to himself or to deceased or living relatives or associates. The cooperators also consent to annotate the statements produced by the subject. It is important that the subject should not be able to infer anything from his sensory contact with the object which could contribute to the score. To this end, the objects must be cither concealed (say in boxes or envelopes of similar shape and weight) or, if not concealed, must all be of the same kind. For instance, hair tokens from persons of the same sex and age groups can be used. Although the subject may infer from a perfume scent that a lock of hair belongs to a woman (or from a grey streak, to an elderly person), this information will not help towards a higher score since all cooperators in the test are likely to accept the related statements as applying to them. It is also im: t that the experimenter or others present during the test should not know which token-object Kadir, — oe any one _ —_ 7 might unwitti i the subject’s responses. ir tokens co ncebeig af aapablyr a ptionillbie g-agtit gg Magoo arrange to have the subject work behind a screen, or in some other way insure that he (the experimenter) does not see the hair; and he must instruct the subject not to refer to its color, or the eye and skin color of the person to whom it belongs. All tokens should be in similar containers, whether or not they are to be taken out during the test. Hair tokens should be tied i i container should have an token, The ter and others t at the test must not Ean Ace Bee cindkne ue only. ane win i ihe imenter has an assistant. If not, the experimenter may request the cooperators each to select a numbered container, out of sight of the experimenter, to place the token in it, and to write down on the code sheet which mark the container of his token has. Journal of the American Society for Psychical Research 186 If concealed tokens of varying weights are used, these are the experimenter in the lower halves of a set of similar OXes; boxes are weighed and wads of cotton or similar material are placed in them in different quantities to equalize the weights and prevent rattling. One or more of the cooperators are then called in and asked to place the numbered tops on the boxes and to make a record on code sheet of which box contains which object. This is done out of sight of the experimenter, and the code sheet is not consulted by him until after the experiment. The tokens can be given to the subject by the experimenter bject at a time) at the same session or they can be spaced over several. The subject's statements are recorded by hand or recording machine. At the end of a set, the experimenter edits the statements. For instance, if locks of hair are used, any reference to the hair, eye, or skin color is deleted, since this would tell the cooperators which statements were intended for their tokens. (If the experimen- ter knows the toe and was himself present during the tests, the = ould not have made any such references, as noted earlier.) Repetitions are also deleted. Parentheses for the annotations of the cooperators are then inserted after each item of information. No firm rules can be given for this. The experimenter must be guided by his own judgment as to what constitutes oer items of information. inally, copies of the statements from all the tests in a set are for each cooperator and sent to him for annotation, with i ions as to the time of the tests. The name of the cooperator is written on top of the first page of his copy of the statements, and @ space reserved for the date of annotating the transcript. The cooperators are also given instructions for annotating the records. A sample of this is included here (see Sample 1) and will explain the annotation marks. A sample of a record sheet from a set con- Sisting of four tests also is included (see Sample 2). Remember that there would be four copies of these statements, one to each cooper- ator. This sample has the annotation marks of the cooperator to om it was sent and also the identifying marks and scores (in —) filled in by the experimenter, as explained below, after the sheet was returned to him. i g ° Sample 1 Instructions to Cooperators in Token-Object Tests Here is a copy of the statements produced by the subjects the sessions making up this experimental series. As a rule, the statements appearing together are intended to apply to the

xd tw hm 187 Designs for Tests with Free Response Material same token. However, it is possible that some statements and whole To may refer to another, so you are asked to examine y cach statement, determining whether or not it is applicable to the circumstances of the n whose token you supplied, even if the statement occurs in the midst of sentences which do not apply. The pairs of parentheses following the statements of the sensitive = to = used for your annotations. The following marks should used: VV for correct and particularly significant ¥ for correct X for incorrect ? for doubtful or unverifiable Each annotation should refer to the information contained in the words immediately preceding the parentheses in question. _Please do not add or remove any parentheses, but feel free to give your comments on the statements and the way they are subdivided on an accompanying sheet. Try to read all the statements as if were meant for you, j them correct, incorrect, etc., accordingly. P make an entry in every pair of parentheses. At the end, ng cpl gael ead pair of parentheses been annotated. When finished, please write the date and time of annotating the statements in the space reserved. Your responses to the statements will be kept confidential. Sample 2 Statements from Four Token-Object Tests (with annotation marks and scores added) Cooperator: Mr. Peter Gimble: D. Time of Annotations: March 30, 1961, 5 p.m. Time of Experiment: March 9, 1961, 3:30 p.m. A man’s voice (4/) spoke about insulin-shock or -shot (+/). The person’s hand feels numb (+/), it draws (+/), and hurts (X). The feeling goes up to here (subject indicates area just below elbow) (X). Numbness starts slow (X) and builds up (X). It’s in both hands (X). I associate it with a man (4/) because of the man’s voice. Time of Experiment: March 16, 1961, 5 p.m. Melancholy (4/), sad (+/) feeling. A recent death or tragedy or —S very sad (7), An almost dark X). Al feeling o griet (X). ‘ of loss (X). iheeter a eee Journal of the American Society for Psychical Research 188 away (X). Sample belongs to a woman (X), a female personal- of Experiment: March 16, 1961, 8: 30 p. m. An impression of a royal blue castle (X) and a black lacquer table (X). A woman (X) with dark but not black hair (X). Some- i between the throat and waist (4/). of iment: March 27, 1961, 7 p.m. Person has a peeling (X), sunburnt (X) arm. Not a bad burn There is a pain in the jaws (X). Person has been to a dentist Gx}, The walls in the dentist's 0 ce are turquoise (X), a pale shade of turquoise (X). A feeling of wanting to go to sleep (+/), on the verge of blacking out (X). When the records have been returned by the cooperators, the i adds up the score for each test. He disregards the the X’s and ?’s, and counts a */ as 1. A 4/ he can count as 2 or more, as he decides, only he must decide on the value he assigns it before the tests. I generally count 4/ as 2. The score of each test is written in the bottom left-hand margin after the statements on each copy. Then and only then, the imenter determines which token-object was used in which test by consulting the code sheet. The identifying mark (A, B, C, or D, in the present sample) on the container of the token-object of cooperator is copied after his name on the top of the first page of the copy of the statements sent to him. The identifying marks are Ssaiber on the top left-hand margin of the statements from each test in a set (see Sample 2), The hope is, of course, that cach cooperator will find in the test many correct statements devoted to his object, but few in the others. Il. “Brrer-Reapinc” Tests This method is intended for tests with a subject working with @ group of cooperators present together. It provides a scientific framework for a common “‘séance room” situation. The following is an adaptation of so-called “‘billet-reading,” but other token-objects than written sheets of pee may be used as long as these are identical in order to insure their owners cannot be normally discovered either by the subject, the experi- menter, the cooperators, or others present during the test. One session is devoted to an experimental set of four or more tests. One subject and four or more cooperators participate. The Material consists of a tape recorder (hand-written notes may be used instead), identical blank sheets of paper, opaque envelopes, and record sheets which consist of series of columns with spaces

189 Designs for Tests with Free Response Material numbered 1-100, and room for the participant’s annotations (see Sample 3). If there is any question as to the opaqueness, the sheets GEA tir dacloap’ ich anes aluminum foil, or a similar material before being placed in the envelopes. Each of the participants writes his name, the name of a deceased person, and a question or whatever it be, on a sheet of paper. The experimenter collects these, goes into another room and inserts each sheet into an envelope, out of sight of the subject and the cooperators. It is important that he must not himself have any way of knowing which sheet is in which envelope. He should therefore use a screen or similar device when carrying out this task. If the imenter has an assistant, this task can be delegated to him. assistant does not enter the experimental room or communi- cate with the subject or participants till after the tests.) The Cee ees Sent miteontiedy 0G WEN they lettete of the phabet. oe aavladianen cattaremmeam inte: te subj accusto to, experiment can in. The tigics teoobdet- fs-toxand-on: (ot-tlie-nese-titer 4v sendy) and the experimenter gives the subject the first envelope. The subject (or the experimenter) reads the code mark on the envelope into the tape for later identification. The subject then produces his state- ments pertaining to the envelope. The experimenter must always be in a position to watch that the envelopes are not being tampered with. This procedure is repeated for cach envelope, after which they are returned to the experimenter. It is then time for the cooperators to make their annotations and a record sheet (see Sample 3) is given to each. It is explained that the tape recorder will ly be turned on for a play-back Be ee ee ee : b ¢ | ; F 4 : Sample 3 RECORD SHEET : Peter Gimble. Time of Test: March 9, 1961 ; 3:30 p.m. Cooperator: Write +/ for correct and particularly . v for correct, X for incorrect, and ? for doubtful or unverifiable. Draw a line Journal of the American Society for Psychical Research 190 beneath the last number of each test. If there are more than 100 jtems to be annotated, use another record sheet, changing 1 to ‘101, 2 to 102, etc. The experimenter decides how to separate the material into items for annotation, as in other token tests (see Section I),

191 Designs for Tests with Free Response Material Once the hie — Sa the play-back and annotation can begin. experimenter stops the tape (or ceases reading from his notes) after each statement, ‘oy give Sccekares time for their annotations. At the end of the statements ini to the first envelope, they are told to draw a line beneath the las annotated number on their records, and to write the identifying letter of the next envelope by the following number, and so on to the end. The record sheets are then collected by the experimenter after he has made sure that they are filled in correctly. The experiment has now been completed and it remains only to open the envelopes to determine who is associated with which. The letter on the envelope of each cooperator should be copied onto his record sheet. It will probably not be convenient to Rca ee poms pe at ~~ rp of the test. However, the Pp pants get a idea of the results if are told which envelope and identification mark is theirs, and if he tape is then played back again (or the notes again read aloud), The items scored correct by each cooperator for each envelope are then added ven, experimenter, using the spaces at the bottom of the record 8 III. ESP-Projection Tests In LESP-projection (“traveling clairvoyance” or “out-of-the- body” experience), the subject has the experience of being in a place different from where his body is and of observing events there which, later it is found, occurred. Sometimes the objects allegedly observed include the subject's body; and occasion: y it is claimed that the subject was seen by others at the place where he had the impression of projecting himself. Reported cases of ESP-projection generally are associated with changed states of consciousness brought about by drugs, sleep, ga ee (Se dissociation. However, tests in this guide are designed for investigating subjects who already possess psychical abilities of the types in question; the conditions which ee about such capacities are not considered. experimental design described here is equally suitable for other free response ESP tests. An experiment (a set) consists of four or more tests. In each, the subject is instructed to project himself to a different location or to the same location at different times, and to report what he observes to the experimenter, who keeps a record. One or more cooperators who can observe or are familiar with the target locations assist in the experiment. They are provided with copies of the subject’s statements at the end of the experiment and requested to annotate these, without knowing Journal of the American Society for Psychical Research 192 which statements are meant to apply to which circumstances. The cooperators either receive a full transcript of the statements (similar to the one inserted in Section I) or the statements are played back by a tape recorder, or read aloud and annotated on a record sheet as explained in Section II. Occasionally a subject repeats part of the experimenter’s instructions in such a way that the cooperators might know or be able to guess where the subject attempted to D edo himself, if were to see this part of the transcript. Such statements must be deleted or edited by the experimenter before making out the copies for the cooperators. For instance, assume that the subject has been instructed to visit a room in a hotel and he remarks that “the lobby is crowded with people.” The cooperator who is associated with the hotel will at once know that the record where this sentence appears relates to the hotel. For this reason it has to be deleted. If another target location is a cabin on a ship and the subject says that “the bunk is under the porthole,” this, too, will tell the cooperators which location the subject is aiming for, and the statement must be deleted (or edited to “the bed is under the window’’). It is not always easy to decide whether all such clues have been removed. It is best therefore to equalize the locations. For instance, if it is desirable to use a hotel room, all target locations in the experiment should be hotel rooms, or all should be living rooms, or all bedrooms, etc. In a convenient adaptation of this procedure, there is only one cooperator and the results can be assessed immediately, The subject is asked to project himself to a certain room where he will observe the cooperator, and to rt back on his activities. The imental set is divided into four or more tests of, say, ten minutes each. At the end of a set, the statements are copied for annotation, or, more simply, read aloud to the cooperator who can annotate them by using the kind of record sheets shown in Sample 3. The cooperator uses one record sheet for each test period. If @ set consists of four tests, all the items in all tests are annotated according to the accuracy with which they describe the cooperator’s Getivities in exch period, For instance, if during the first period he sharpened a pencil while smoking a cigarette, he annotates on the first record the subject’s statements for each period according to whether they seem to refer to smoking or pencil sharpening; on the next record sheet he evaluates the items according to his activities during the second period, and so on. He must keep a record of his activities during each period as a guide for his anno- tations. Before the annotation can begin, it is important that the experi- menter remove any clues from the records that can inform the

193 Designs for Tests with Free Response Material cooperator which statements were meant for which periods. For instance, if the subject says, “He is no longer sitting down,” the cooperator, when he later sees or hears this statement, will infer that it was not said during the first period, and the annotation and assessment may be biased as a result. For the same reason, the order of the tests must be randomized before being presented to the cooperator. IV. CaLcuLaTinGc THE RESULTS The best way at present to determine the statistical significance of tests with verbal material is the Pratt-Birge method. follow- nia oy Oa has been adapted from an article by J. G. Pratt. en the experimenter has completed adding the scores for each of the tests of a set, he enters them in a table, as in the following example where four cooperators took part (see Sample 4). He sums up the numbers in the rows and columns and in the “target diagonal.” He is then ready for the analysis, a standard statistical procedure, easy for anyone to complete (a table with squares and square roots will, however, be a help). Sample 4 Cooperator’s Annotations Smith Gimble (D) The spaces in Sample 4 are made up of the number of items checked as correct by the cooperators. For instance, Gimble found 5 correct items in the statements the subject produced when this person held Gimble’s token or projected himself to Gimble’s location; (at the time of annotating these statements, Gimble did not know that they were intended for him). Of these, two were “correct and particularly significant’ (see samples of statements and record sheet in Sections I and II), With 4/ counted as 2, the score is 7, He also found 2 correct items in Smith’s record, and | each in Olson's and Peters’. T "Appraising Verbel Test Materiel in Perepeychology,” Journal of Pere- peychology, Vol 12, December, 1948, pp. 236-256 J ¢ Journal of the American Society for Psychical Research 194 The total number of items is 35. Of these, 18 were hits (the sum of the items on the diagonal), The mean number of hits expected by chance is one fourth of the number of items, that is 35/4 or 8.75. In this experiment the deviation from chance is 18 less 8.75, that is, 9.25. Some additional figures are needed to complete the assessment: (1) fda keprelbecl ants | Samael Sar 1225. 2) The sum of the squares of the individual spaces (S) = 143. 3) The sum of the squares of the separate row totals (R) = 315. 4) a ee er er Sonn (C) = 1. te) The number of tests (t) = 4. (6) The square of the number of tests (T) = 16. ; 2 ae a Variance = a | In the present experiment this works out at: 1225 + (16 x 143) — (4 x 315) — (4 x 341) _ 889 _ 9.6 1s + 6 =i rine If the experimental series consists of 12 sets (48 tests), the 12 individual variances are simply added together, as are the 12 deviations. Let us say that the total deviation in the 12 sets is 36 and the total variance is 225. The critical ratio, which indicates the statistical significance of the result, is obtained by dividing the total deviation the standard deviation. The standard deviation is easily calculated by taking the square root of the total variance (V/225). The critical ratio (CR) thus works out at 36/15 = 2.40. Earlier in this paper I suggested that the experimenter should complete one or more exploratory sets before engaging on the full series poncocap for a statistical assessment. I have not considered the question of how large the deviation in a single set should be in order to justify conclusive testing. An average deviation of 3 in the 12 sets (assuming a total variance of would yield a critical ratio of 2.40. In other words, if the exploratory set described above had given a somewhat smaller deviation than the one shown (9.25), er testing would still have been indicated. If there are more items or more cooperators in an exploratory set, the deviation has to be proportionately larger to be suggestive. A critical ratio of 2.33, which has odds of 100 to one, is usually considered statistically significant. If the experimenter is p for negative scores as well as positive ones (that is, if he that the subject may avoid giving correct impressions), a critical ratio of 2.58 has to be reached to obtain the 100 to | level.

195 Designs for Tests with Free Response Material CONCLUDING REMARKS This paper is part of an attempt to encourage research with so-called “‘sensitives,” “mediums,” and other gifted subjects whose abilities have been neglected in recent years, partially because they often operate in religious and occult areas, with which the para- psychologist is anxious not to appear to be associated, and partially because there have been no suitable methods of assessment. This field of research is important not only because the potential subjects often appear to possess remarkable and recurrent abilities in paranormal cognition but also because these powers may throw light on problems which have not yet been closely studied, such as the question of post-mortem existence. A guide for tests with run of the mill subjects would be incom- plete without a section on the psychological conditions favorable to success. The investigations outlined here, however, are primarily intended for persons who already are known, or suspected to possess, psi abilities. The main purpose of the tests is to allow for scientific assessment without radically changing the favorable conditions under which the results are produced. This is not to say that anybody can get scores with these proce- dures, even from capable percipients. The subject’s knowledge that he is being observed, the personality of the investigator, and other factors may hamper results. For this reason a three-stage approach is recommended. This offers an opportunity, first of all, for the investigator to study the subject under the percipient’s normal working conditions and to ascertain whether results can be forth- coming while he is being observed. At the second stage, the appropriate test design is applied with such adjustments as seem advisable. If the subject continues to produce under these con- ditions, a confirmatory series is initiated. If the experiments at the second stage of the research are unsuccessful, but there were strong indications of psi in the first stage, further search for a fruitful method may be undertaken. There is wide scope for improving and adding to the designs. Psychical Research Foundation, Inc. Box 6116 College Station Durham, North Carolina PRINTED IN BRUGES, BELGIUM, BY THE ST CATHERINE PRESS, LTD.