The Effect of Experimenter Differences and Subjects' Belief Level upon ESP Scores

Subjects’ Belief L perpen fe naire and by two self-rating scales, The hypothesis that i= shies Cee ESP was con significant bi ‘The hypothesis (Van de Castle's) s ypothesis roc significant. A multiple in Pa nearly reached fy arose of the other, Exploratory measures of the sub ESP were also This experiment was cd : Foundation, Inc. Dr. is now Direc : for Psychical Research and Mr. Dean is Res features of the investigatt Robert L. Van de Castle. We The Effect of Experimenter Differences 159 perception (6). This emphasis in normal perception led para- psychologists to explore the same factors in extrasensory perception, Dr. Schmeidler, who was a student of one of the “new look” psychologists, Dr. Gardner Murphy, has done the lion's share of the work concerning conative factors in ESP. She found that subjects’ belief that ESP is either ible or impossible strongly affects their ESP scoring level (5). Her results, which show remark- able consistency throughout 20 years of research, indicate that believers (sheep) score higher in ESP tests than disbelievers (goats), and that disbelievers exhibit an avoidance pattern; that is, they score below oe expectation. The disbelievers’ results are in line with so-called “negative ception” findings in general ychology. In Schmeidler’s work, this differential scoring apardce ts belief categories applied only to well-adjusted subjects (as judged by the Munroe Inspection Technique), while the scoring of malad- justed subjects was less predictable (4). The ESP scores of persons with a strong interest in theoretical problems (as tested by the Allport-Vernon Study of Values) were more predictable than those of _ having little interest in theory (5). e essential features of Schmeidler’s experiments enjoyed a number of replications by other researchers (1, 2, 5, 7). Generally concurring results emerged in seven out of nine experiments (Table 1), but the need for an improved technique to ascertain subjects’ Tante | Summary or ESP Scones Acconpino TO Bauer Carecontes tN Neve Puntisuep ESP EXPERIMENTS * Sheep Conflics Goat Hypoth- Dev- Avg. | No. of Pes. Avg. Ne. of pe Ava. | esis ation ae Runs iation Confirm- (25 tr.) Ran | QS tr: mas Ran ed** _ | +4252} — il | | + = = s 1 1 4! & s | 8 - 8 sf Paw | Beay | we

160 Journal of the American Society for Psychical Research others, The criteria of belief included (a) belief in the existence of ESP, (b) belief in the adequacy of the particular ESP test, and (c) belief in personal ESP. A detailed discussion of the problem of separating subjects into attitudinal categories can be found in Schmeidler and McConnell (5, Ch. 6), Mangan (2), and Van de Castle (7). Dr. Robert L. Van de Castle developed an incomplete sentence qersseoman® (7) for determining attitude toward ESP, rather than relying on self-ratings or ratings by the experimenter. From the results of his pilot study he realized that he did not merely have a scale from more to less belief: the middle group seemed to represent a conflict between belief and disbelief which, of course, is a qualitatively different motivational situation. He postulated that the ESP scores of the conflict group would be more erratic (the same subjects scoring high on some runs and low on others) and would therefore give higher “within subject” variance® than scores of subjects whose beliefs were more homogeneous. The disbelievers were grouped with the conflict subjects after Van de Castle found that they performed similarly in his pilot studies. He suggested that disbelievers also have a conflict between a conviction that “ESP is the bunk” and social obligations toward the experimenter. The conflict subjects are faced with an internal conflict relatively independent of the testing situation; the dis- believers’ conflict is thought to arise solely in the experimental setting and is therefore situationally labile. Van de Castle’s predictions (9) were confirmed in six repetitions (7) of similarly designed experiments (P = <.01). He also found certain consistent patterns in the levels of “between subjects” variance for the different belief categories. The strong believers (high sheep) and indecisives (conflict) had much higher variance than the moderate believers (sheep) and disbelievers (goats). Five out of six F ratios resulting from a comparison of between subjects variance for the belief groups were statistically significant at a level ranging from .05 to .001. The main problems investigated in the present study are the relationship between ESP scoring level and attitude toward ESP (Schmeidler’s findings) and the differences in variance levels within subject and between subjects (Van de Castle’s findings). In regard to belief criteria, we found criterion (a), belief in the existence of ESP, to be the most thoroughly explored and effective. It was adopted for the present experiment. We also used (c), belief in personal ESP, as a purely exploratory measure. We discarded (b), * For a discussion of the lication of the analysis of variance to this problem, sce Van de Castle (7, pp. 56-48). e Effect of Experimenter Differences 161 belief in the adequacy of the particular ESP test, because it seems to a concept under which too many theoretical, technical, situational, and sociological variables can be subsumed. "% SUBJECTS AND EXPERIMENTAL PRocEDURE i “The 246 subjects were persons attending a series of eight lectures (with accompanying slides) devoted to various aspects of ESP research, given in 1959-60 by the authors.* Those participating in three of the cight sessions were members of women’s clubs, three were church groups, one an educational group, and one a reshman class. Miss 5. Van Steenburgh assisted in the experiment ereas Van de Castle talked to his subjects on the opic of ESP for oot, gees five minutes, we found this an inade- uate compensation for their patience during a long test and we nc our lecture time to 40 minutes. We did not discuss Van de Castle's work in the lectures, nor the sheep-goat work in _ We used the Van de Castle-White method of testing ESP in groups (9). Each subject was given a concealed list of four columns ay ok: card symbols (100 in all) in random order (Van de Castle- White used ont columns). The stimulus symbols were randomized using the million-digit compilation of the Rand Corporation, A Million Random Digits. The random order of the ESP symbols was different for each subject. Each list was enclosed in an opaque manila envelope between two sheets of black construction paper. Two record sheets with carbon paper inserted between them were Henri on the outside of the envelope exactly over the target list letting the blank columns precisely match the columns underneath inside the envelope. The experimenters tested the Opaqueness in lighting considerably above the normal. It was foolproof in the given conditions of lighting and supervision. i after the lecture the target envelopes were dis- tributed to the subjects, together with a mimeographed instruction sheet. The experimenter read the instructions aloud and answered questions. The subjects were supervised by two persons. Upon completion of the ESP tests, the envelopes were collected and the questionnaires (see Appendix) ascertaining attitude toward ESP were distributed. They consisted of Van de Castle’s incomplete Sentence questionnaire (e.g., “I believe that hunches often “7 and two five-point self-rating scales ranging from strong conviction * The experiment isi lanned with K.O. as the experimenter, This Hada ba charged afves be bocaic fl fos aaron trons: oon me of the experiment by introducing « second experimenter as another variable.

162 Journal of the American Society for Psychical Research about the existence of ESP (high sheep) to complete disbelief in it (extreme goat). Scale A rated the subject’s belief in ESP in general and Scale B the subject's belief in his own personal ESP ability. The incomplete sentence A ciceryri was administered ight after the completion of the test; then the subjects were os ae apa the and fill in self-rating scales A and B.° Scoring of the tests was done by an assistant and rechecked by another person. The incomplete sentence questionnaires were i ently judged by three persons: K.O., D.D., and Van de Castle, All the judging was done as a blind procedure, i.c., without the knowledge of the subjects’ names or ESP test scores. An effort was made to follow as closely as possible Van de Castle’s original checking procedure, which he has described as follows: The method of evaluation was based upon an over-all clinical impression of the subject’s attitude. All the individual subject's responses were first read over in order to form a preliminary impres- sion and then read in order to uncover any cies between individual items and total impression or between individual items themselves, sabes were eventually placed into one of four attitudinal categories: high sheep, sheep, conflict, or goat. In order to be considered a high sheep, the subject had to display a consistent attitude of moderate belief on all items or else display a high level of belief on most of the items with the remaining items containing neutral expressions of opinion. When a subject indicated moderate belief on most of the items with the remaining items containing neutral expressions of opinion, he was classified as a “regular’”’ sheep. Two different patterns of responses were included in the conflict cir ae If a subject indicated belief in ESP on some of the items while at the same time he indicated strong doubts and suspicion regarding ESP on other items, a rather obvious conflict situation was present and such a subject was labelled accordingly. The other of response pattern which was considered as representing a subject who a conflicting attitude toward ESP was not so apparent. This subject might also be called an evasive one, for he would refrain from making any decisive commitment throughout all ten items by using such camouflaging responses as “is interesting,” “is strange,” “is not well understood,” etc. Other clues which were consi as evidence of conflict were excessive erasures or crossed- out responses, not completing items, stereotyped use of the same adjective or its antonym, completely irrelevant responses, or long * It was decided to ascertain the subjects’ belief category after the ESP test in order to keep the of attitude active on a low level of conscious awareness. It was t that if beliefs and attitudes were made conscious by them they might lose some of their affective charge and force the subject to act out a more superficial and clearly conscious commitment. ct of Experimenter Differences 163 explanations to items that would contain anywhere from twenty words, Pat 72-4 <s Soacs, ean thee {pho gave iudieaions of a or ieving attitude toward ESP on all or most of the that in the latter case the remaining items were al in expression (7, pp. 46-47). cases there was complete disagreement among the three Since there was no opportunity for a judging conference to an agreement, the data of these five subjects were from statistical evaluations relating to the incomplete uestionnaire.® In some evaluations of the data we ited those subjects on whose questionnaires all three judges from those where only two agreed. he self-rating questionnaires were independently checked by t vo persons other than the authors. | RESULTS _ The previous studies summarized in Table | led us to expect at any extrachance results occurring in the sheep or goat cate- might be obscured in the pooled totals: in other words, the positive scores of believers might be cancelled out by the ‘negative scores of disbelievers. As later analyses will show, this as indeed the case. The total deviation of the 183 subjects whose a were included in the analyses was very close to mean chance ion (CR = .68, P = .50). , division of the data according to experimenters yielded ing results. K.O.’s subjects scored 82.2 hits above mean expectation (CR = 2.18, P = .03), while D.D.’s subjects scored 45.4 hits below mean chance tion (CR = 1.17, _P = .24). The difference between scores of the two experimenters ape with a chi square of 5.6, 1 d.f., P = .02. D.D.’s and K.0.’s subjects taken as a unit (2 x 2 table of subjects scoring dove or at and below chance expectation) yielded a chi square of | : : ef é : |

164 Journal of the American Society for Psychical Research intention to test the differences in sheer scoring levels; our rimary concern, was with: the. various. kinds. of interaction berween the subjects’ belief and ESP ing level. Our problem, in a nutshell, was to relate three aspects of results (a scores, within subject variance, and between subjects variance) to two experi- mental measures of belief in ESP and to one exploratory belief As stated before, this experiment was largely a repetition of sta’ re, experiment was ly a no the work of other investigators; we therefore had to use some of their evaluation methods in order to permit comparisons, ¢.g., analyses of variance as they were applied in Van de Castle’s experiments (7). Our statistical consultant considered multiple regression analyses more appropriate and safer, ¢.g., not involving the assumption of homogeneity of variance; we sell tan multiple regressions in order to obtain more reliable adequate information about our data. Although we feel that the multiple regression analyses pe the main weight, they should be inter- preted in context with the other evaluations. These various evaluation methods basically deal with the same trends; therefore the results, the P values, are often not additive. Customary caution in interpreting P values of multiple measure- ments is necessary. We will present our results in the following order: (1) ESP scores in relation to belief categories, (2) within subject variance and belief categories, and (3) between subjects variance and belief categories. Incomplete Sentence Questionnaire In accordance with the results of the incomplete sentence questionnaire, we classified the subjects of both imenters as follows: high sheep (strong belief in ESP), sheep (belief in ESP), conflict (indications of conflict or evasiveness), an (disbelief in ESP). All three judecs agreed on the classification of 47 of K.O.’s subjects and 53 of D.D.’s; two ju agreed on the classification of an additional 40 of K.O.’s subjects and 38 of D.D.’s.7 We ju ifications because of the small number of subjects in the high sheep and the goat classifications (Table 2). This seemed to be justined since the scores in the three- and two-judge classifications showed roughly the same pattern. * As stated above, the data of five subjects whose classification all three roy ge : statistical evubsatione deleting 8 thee Saison: The Effect of Experimenter Differences 165 ‘The patterns of average scores in the four belief classifications w See te different for the two experimenters (Table 2 and Figure 1). K.O.’s high sheep scored highest; then the sheep and conflict groups; and the goats lowest, whose scores were considerably ‘below mean chance expectation. The average scores of D.D.’s subjects give a slight curvilinear pattern. Curiously atip. the scores of the high sheep were below chance tion; those of sheep and conflict subjects at the chance level; while goats scored far below chance expectation. Taniz 2 Avenace ESP Scores Acconpinc To Bevery Catncorms No. Avg. Ave. >a Avg. Ran Ran fan 47 #521 6 4358 _ = 4 83.01 3. 442 =— _ 8 487 S 5.05 4 4 16 5.08 4 5.19 6 i8 16 SSS} 12 5.12 2 4.87 9 469 16 «5.17 9 4.47 er to assess the consistency of these scoring trends, we subjects into those who scored above mean chance ion and those who scored at or below. There was a different lis tion between K.O.’s and D.D.’s subjects who scored above cl expectation and those who scored at or below in the pooled hig and sheep category, with at least two judges agreeing 6¢ = 4.72, 1 d.f.,P = .03), but a similar distribution in the conflict ‘and goat categories. Twenty-three subjects in K.O.’s combined _ sheep category scored above chance expectation and 11 at or below. i goat scored above chance expectation and five at or below. This difference in proportions as assessed by the Fisher exact ility test was significant at the three per cent level. The same kind of test of D.D.’s data yielded a non-significant result. In correspondence Van de Castle commented that it would be ‘interesting to see the results of the subjects where all three judges ee Siem 8 Deane ote See Ducts None whether he was a believer in ESP, regardless of the intensity of the belief. Accordingly, such an analysis was performed. K.O.’s eligible combined sheep (25) had an average score of 5.42. D.D.'s _ eligible combined sheep (32) had an av score of 4.97, which is very close to mean chance expectation. In experimenters’ data ‘combined sheep scored hi than goats: for K.O., 5.42 versus 4.58; for D.D., 4.97 versus 4.42. yA . |

166 Journal of the American Society for Psychical Research > ww uw +. . - AVERAGE PER RUM pam a - . Li, 42 ‘ll E. > = = > . * e = > Fic. 1. buttal cones, to cememiae Bie of sub; a Rage tide y ego acim subjects iste ven belo et Self-Rating Scale A Self-rating scale A separated the subjects in an even more dramatic way (Table 2 and Figure 2). Distribution of ESP scores obtained by this measure was similar to that using the incomplete sentence questionnaire classifications. The scores of K.O.'s sub- jects were again in a nearly linear relationship to the belief cate- gories: the high sheep had a 5.65 average score; the sheep 5.20; ee My and the extreme goats (lowest self- rating category) 4. The scores of D.D.'s subjects also gave roughly the same eae as when separated by the incomplete The Effect of Experimenter Differences 167 entence questionnaire. We again separated the subjects who i ‘scot TEs cassans chisiess exreosaitcer feonl'thoues weoree not ato + below. A significant difference between experimenters emerged from th Dir aitwercen ses tained 1 d.f., P = .014). ces were in the combined 5-19 Ls BS a o =| 4.2 EXTREME GOAT “get: > Comparison of scores between the two experimenters of subjects in i t belief categories, as determined by self-rating scale A. K.O.’s subjects storing 3 Sir oh and below chan ation in the combined beli ef categories 5 p plus 4/versus:3 plus 2 us | were distributed in a definitely non-chance pattern ttern (y* = , 1 df., P = .002). A similar chi-square test of D.D.'s data

168 Journal of the American Society for Psychical Research gave x* = .64, which is insignificant. It appears that separating subjects by self-rating scale A reveals slightly more differentiated scoring patterns than separating them by the incomplete sentence uestionnaire, Figures | and 2 accentuate the experimenter These two belief measures (incomplete sentence questionnaire and self-rating scale A) were evaluated jointly in relation to ESP scores by performing an analysis of multiple regression. (Self- rating scale B was an exploratory measure and not used in this ee The regression was positive in K.O.'s data. The regression cient related to the incomplete sentence questionnaire was f = .8365 and related to self-rating scale A was § = 1.2024. The hypothesis that 8, =8, =0 was rejected (F = 10.26; nm, =2, my =81 d.f., P = <.005). The hypothesis that 6, =0 was confirmed (F = .29), but §, was found to be significantly ppiiseaichirs 0 (F eM s Ae = ms = 81 pang rey 01). paren ere is a significant multiple regression scores on the two belief measures in K.O.’s data, with the self-rating scale A differentiation contributing the lion’s share. The same kind of regression coefficients in D.D.’s data are B, = .2409 and 8, = —.7201. Neither the multiple regression as a whole nor the separate regressions §, and f, are significantly different from 0, Apparently D.D.’s subjects did not show a sig- nificant linear relationship between ESP scoring level and belief in ESP as measured in this experiment. We also tested the hypothesis that the two regressions were identical for the two experimenters; that is, we wanted to see whether the error variance was significantly reduced by fitting regression lines separately for K.O.’s and D.D.'s data. The test reached significance. Self-Rating Scale B Self-rating scale B was designed to measure subjects’ confidence in their own ESP ability. The distribution of ESP scores in relation to scale B is different for the two experimenters (Table 2 and Figure 3). K.O.’s subjects had two ESP score peaks: one in the highest belief rating (high sheep), CR = 2.75; the other in the middle rating (conflict p), CR =2.21. If these measures indicate anythi siiine unas arthange dictribenblgt de ates: wel might, we may be dealing with a different dimension of belief robably involved with the nality factor of self-confidence. Patera ieeenated the cont ined sheep from the conflict group and combined goats according to scores above and at or below mean chance expectation and performed chi-square tests similar Th Effect of Experimenter Differences 169 | 5.70 33 iffesens belie Se ne ty cob Be a to those performed with self-rating scale A, no significant differ- ] fee eieread either for the pooled data or for the data of the ‘ experimenters separated. Within Subject Variance Van de Castle compared scoring variance (note; not the means) between the eight runs of 25 trials by obtaining an individual sum of squares for each subject. According to Van de Castle, “It ‘involves taking each run score, finding the difference between it and the subject's average run score, then squaring this difference and summing all these differences” (7, p. 53). The same technique ‘was followed in the present experiment in four-run blocks.

170 Journal of the American Society for Psychical Research The main measure was the within subject variance in the pooled data of both experimenters, comparing combined sheep versus conflict and goats. Only those sabioste were used whose belief classification (incomplete sentence questionnaire) was agreed upon by at least two judges (Table 3, line one), On the data where at Tante 3 Maan Squases or Wrritiw Susyecr Vantancn or ESP Sconss Accorpinc To Betrer Cateconis* ‘ i between A and B (KO-DD data): F = 1.24; m, = 318, mn, = 216 .f., * Belief determined by incomplete sentence questionnaire, at Jeast two judges least two ju agreed an .05 level of significance was barely reached (F = 1.24, 318 and 216 d.f.). We also computed the variance of the three-judge agreement subjects, which reached the .05 level of significance (F = 1.40, 204 and 96 d.f.). The results of K.O.’s subjects where the three judges agreed were also significant at the .05 level. However, significance vanished for his subjects when the two-judge and three-judge subjects were pooled. moot srasner were eee oh a om where three judges a ut for subjects where at two judges agreed the reached the .05 level (‘Table 4). eee j Van de Castle suggested that we also work out the within subject variance on data where all three judges agreed that the subject should be classed as some kind of sheep, regardless of intensity of belief, or should be placed in the combined conflict and goat group, without distinction between them. The within subject variance of subjects separated in this manner did not reach significance because of a slightly smaller F value and fewer degrees of freedom (F = Ns 234 and 171 d.f.). ¢ compared the average within subject variance run between the two experimenters and Van de Castle's original data (Figure 4.) This figure suggests that D.D.’s subjects followed Van de Castle’s original pattern more closely than did K.O.’s. In the goat classification neither D.D.'s nor K.O.’s subjects followed an de Castle's pattern, but there were only 19 subjects classed as goats in our group. This lack of similarity in scoring level in the Effect of Experimenter Differences 171 Tante 4 . Mean Squares of Wrruiw Supyecr Variance or ESP Scones Acconnine To ExpeniMenters AND Pootep Better Careconins* hedost_| High Sheep eo + Sheep F 3 | 2. ae 4,27 3+2 | 3. DD 3 a DD 2 3h DD 3+2 3 + DD 3 3. 0+ DD 2 3. KO + DD 3+2 3. q * Belief determined by incomplete sentence questionnaire, at least two judges Tee Significant at the .05 level of confidence. _ goat category is not surprising. Van de Castle’s own goat subjects fea variances fluctuating more widely from series to series than did the sheep or conflict subjects. It is evident that the patterns of ‘K.0.’s and D.D.’s subjects are more similar in the within subject _ variance comparisons than in comparisons of mean scores in the different belief categories. In order to see whether our comparatively slight success in as the within subject variance could be explained tf differences in judges, we computed the ive F ratios for three judges. They were almost identical: Van de Castle, F = 1.22; D.D., F = 1.18; K.O., F = 1.22. Apparently the combination of : judges increased the F ratio slightly over the single judge approach. ‘It is clear that the present experimenters were able to use Van de Castle’s incomplete sentence questionnaire technique with an effici equal to that of the original experimenter. The F ratio measure Van de Castle used was marginally signif- icant in our data. However, he saw a possible loophole in the ‘measure and used an alternative evaluation as follows: As was mentioned earlier, however, the mean can be heavil influenced by a few ically high or low scores, and since this ratio resulted from a ratio of two mean sums of squares, it is possible that this significant F value is due to a few atypically scoring subjects. In order to check upon the consistency of these scoring patterns, the chi-square technique was employed. The over-all mean sum of

172 Journal of the American Society for Psychical Research squares of all 194 subjects was computed and found to be 29.787, and the number of subjects having sums of squares scores above and below this over-all mean was tallied ... A 2 x 2 table was sct up containing the number of subjects scoring above and below the over-all mean for the combined groups . . . (7, p. 55). 5.00 4.8 4.8 DD ae 4.50 = “ ra = 3} g 8 MEAN SQUARES bg 8 2.00 1.50 1.6 Fic. 4. Within subject variance in Van de Castle's experiment and in the ief determined by incomplete present series, belicf sentence questionnaire, at least two judges agrocing. In the present experiment we computed a 2 x 2 table of subjects whose variance was above and at or below the mean for the experi- ment, with high sheep and sheep combined, and conflicts and goats combined. The results were very close to chance expectation (x* = .17, 1 df.). Therefore, the F ratio measure of the within subject variance must be interpreted with great caution. However, Van de Castle's argument does not apply to our regression analyses described later. ‘The Effect of Experimenter Differences 173 Within subject variance was also computed on subjects ‘according to self-rating scale A (Table 5), We combined the data of high sheep and sheep, and those of the conflict and goat cate- gories. The results of both K.O.’s and D.D.’s subjects when pooled were very close to chance expectation (mean square for combined Tanie 5 Mean Squanes or Wrriin Susyecr Variance or ESP Sconss AccorpInc To Beier CaTecories* Sheep Goats | &3 +445 t 2.78 2.79 4.29 KO 4.59 | 417 2.83 DD 4.51 4.75 5.27 * Belief determined by self-rating scale A. ** Significant at the .05 level of confidence. sheep, 4.05; mean square for combined conflict-goat group, 3.97). However, when we separated the subjects according to exper- imenters, the variance of D.D.’s subjects roughly followed Van de Castle's pattern, with the mean squares in the five belief cate- gories being 2.76, 4.51, 4.75, 5.27, and 4.29 (not. statistically significant sccoesling: to Van de Castle’s method of analysis of variance), K.O.’s subjects revealed just the opposite pattern from the one obtained with the incomplete sentence questionnaire and “was significant at the .05 level. The mean squares in the five belief categories—4.59, 4.17, 2.83, 2.78, and 2,79—show more ‘variance with more belief, a pattern which is contrary to our “expectations as based on Van de Castle's findings. This reversal ‘of the variance pattern in K.O.’s subjects might reflect subtle differences in the assessment of belief: the incomplete sentence ee uneare and self-rating scale A might have measured different dimensions of belief. Because of the level of statistical certainty (P = .05), the chance explanation of this unpredicted trend is not excluded We evaluated the within subject variance in joint consideration with the incomplete sentence questionnaire results and the self- tating scale A scores by ‘orming multiple regression analyses. The variance of each subject’s four runs in relation to his own “mean was the dependent variable Y. The incomplete sentence * score (where at least two judges agreed) was the independent variable X,; and the self-rating scale A score was the independent

174 Journal of the American Society for Psychical Research The regressions went in opposite directions in K.O.'s data. The regression coefficient related to the incomplete sentence question- naire was 8, = —1.9665; but the coefficient related to self-rating An F test of the hypothesis that 8, = 8, =0 and that 8, =0 gave non-significant results. The hypothesis that 8, =O was rejected at the .05 confidence level (F = 3.96; n, = 1, ny =81 ot) Apparently the joint consideration of the two belief measures in K.O,’s data did not conform to the patterns predicted on the basis of Van de Castle’s findings. In D.D.'s data both regressions went in the predicted direction. The regression coefficient related to the incomplete sentence questionnaire was 8, = —.1912 and related to self-rating scale A was 8, = —2.3270. As in K.O.’s data the f, coefficient was numerically larger than f,, but unlike K.O.’s data it went in the direction expected from Van de Castle’s hypothesis. An F test of the hypothesis that 8, = 6, = 0 came close to the 05 confidence level (F = 3.00; m, =2, nm, =85 df, when Fy, = 3.11). The hypothesis that 8, = 0 was accepted (F = .02). The F test of the hypothesis that , =O again gave an F value close to the .05 level of confidence (F = 3.86; 2, = 1, m, = 85 df. when F,, = 3.96). On the whole, the multiple regressions per- formed on D.D.’s data gave results somewhat similar to Van de Castle’s findings. As in the case of multiple regressions on ESP scores and belief variables, we also tested the hypothesis that the two regressions were identical for the two experimenters; that is, we wanted to see whether the error variance was significantly reduced by fitting regression lines separately for K.O.’s and D.D.’s data. The test reached significance at the .05 confidence level (F = 2.80; n, = 3, ny = 166 d.f.), Between Subjects Variance Van de Castle found certain differential patterns of between subjects variance within the sh classifications. In this experiment it was a surprise to find that the between subjects variance was significantly different in K.O.'s and D.D.’s subjects, with the variance of D.D.’s subjects about one-third higher than K.O.’s (F = 1.60; 87 and 83 d.f.; P = .02), Because of the apparent dissimilarity of variance, the between subjects variance was evaluated separately for the subjects of both experimenters (Taste 6).2 We computed six F ratios for each experimenter’s * Every Mean Square (MS) of Table 6 (which represents variability in each belief category: high sheep, sheep, conflict, and goat) mm compared with the MS in The Effect of Experimenter Differences 175 subjects (Taste 6A). One of the F values in each experimenter’s data reached the .05 level of significance which is quite close to chance expectation.” ‘Tanta 6 Anaysis or Variance Tante ror Evacvatine Dirrerences neTween Supyects® ‘OTAL Scores WiTHtn Dirrewent Becier Careconres* KO versus DD: F = 1,60, P = .02 * Belief determined by incomplete sentence questionnaire, at least two judges agrecing. Tassie 6A F Ratios sxerween Bevier Carecontes sown In Tance 6 DD Sheep Conflict | Goats 1,00 1.86 2.01" 1.08 1.85 en 1.34 | 1,07 1,43 *P= .05 We wanted to compare our between subjects variance with Van de Castle's results, but this could not be done accurately since he had twice as many trials for each subject. We found that his other belief category, Thus we obtained six comparisons for each experi- . oop hag from which we computed the same number of F ratios presented in * In a comment on the manuscript Van de Castle that there to bea similarity between his and D.D.'s pattern if the runk order of magn. Se tude of the variances is taken into account: “If you have four scores (1, 2, 3, 4), there are 24 possible different arrangements that these numbers can r in; and out of 24 such possible permutations D.D. and [ agreed exactly in position on the order of between subjects variance with both of us having the follow order: high sheep (1), conflict (2), goat (3), and sheep (4). Thia result by itself would be significant the .05 level."

176 Journal of the American Society for Psychical Research between subjects mean squares were about 2.7 times higher than ours; therefore we divided his values by 2.7. These adjusted values, of course, form a basis for only a very crude comparison (Figure 5). bs ~ ° 5.0 : Fic. 5. Between subjects variance in Van de Castle’s experiment and in the present series, belief determined by i sentence questionnaire, at least two judges agrecing, Van de Castle’s data adjusted by the factor of 2.7. In this Figure some similarities in between subjects variance atterns can be seen in Van de Castle’s and D.D.’s data. The fiebeks subjects variance in K.O.’s data does not follow Van de Castle's pattern at all. Following Van de Castle’s suggestion, we also computed between subjects variance in the pooled data of both experimenters, The differences between high sheep versus sheep, and high sheep versus goats were significant at the .05 level. However, the real probability values snight he quite different because the homogeneity of variance Do ba Nae of the statistical test used was grossly violated. riginally it had not been planned to evaluate the between subjects variances according to self-rating scales A and B, but sub- The Effect of Experimenter Differences 177 sequently we decided to compute it for scale A. The results are shown in Tables 7 and 7A. Three out of ten F ratios in K.O.’s data were significant, but two of them represented different patterns from those found by Van de Castle with his self-rating ‘technique. None of D.D.’s F ratios reached significance. Taste 7 ANatysis of Vaniance TAnie yor Evacuatine Dirrenences serween Supyjects’ Tora Scones wirinin Dirrenent Beuixy Carucories* KO versus DD: F = 1.49, P = .05 * Belief determined by self-rating scale A. Taste 7A F Ratios perween Baume CatTecontes sown ms TABLE 7 *P = .05 +P = Ol The correlation between the three questionnaires which were intended to measure belief in ESP was also of interest. Therefore a correlation was made of the incomplete sentence questionnaire results with those of self-rating scales A and B. High positive correlations were found with the following coefficients: r = .72, 176 d.f., and r = .75, 176 df.

178 Journal of the American Society for Psychical Research The ESP scores of K.O.'s subjects had a low but significant positive correlation with self-rating scale A scores (r = .30, P = .01). D.D.’s scores had a very slight positive correlation which was not statistically significant (r = .05). Other correlations were not performed on the data. Discussion The present experiment involved the usual precautions against ordinary perception of the stimulus material as worked out by Schmeidler (5), supplemented by Van de Castle and White (9), and used by Anderson and White, Rilling, and others in school experi- ments (8). These precautions have withstood cross-fire and are generally considered to be adequate. There were two independent records; two copies of the target order and two copies of the subjects’ calls. They were checked twice by two assistants who did not know the belief ratings at the time of checking. A number of subjects had to be eliminated from further evaluations because of incomplete ESP tests or questionnaire responses, but elimination was according to a set of objective rules and carried out without knowledge of scoring level on either ESP tests or belief question- naires (see Footnote 6). Although the present experiment was essentially a repetition of the earlier work of Schmeidler and Van de Castle, the procedures were not identical with theirs. The number of runs per subject in our experiment was four as compared with eight in Van de Castle’s experiments and an average of nine in Schmeidler’s; and attitudes toward ESP were ascertained after the ESP testing rather than before. These differences are mentioned because they might have been important in affecting scoring and variance levels. We will first consider the ESP scores in relation to the measures of belief in the existence of ESP. From inspection, as well as from statistical analysis, it is apparent that the effects are much more clearly expressed in K.O,’s data than in D.D.’s, although the disbelief (goat) categories might form an exception (Figures | and 2). Moreover, the hypothesis brought out in previous work (2, 5) that a decreasing belief in ESP is accompanied by a roughly linear pattern of decline in scores is demonstrated by K.O.'s subjects, The ESP scores of D.D.’s subjects show a slightly cur- vilinear distribution in relation to the belief variables; however, the trend was not strongly enough expressed to justify a test of curvilinear regression. The main differences between K.O.'s and D.D.’s subjects are in the strong belief (high sheep) category, where K.O.’s subjects scored significantly above and D.D.’s The Effect of Experimenter Differences 179 subjects below mean chance expectation (Figures | and 2). ignificant differences between K.O.’s and D.D.’s data emerge y in the combined sheep categories, but not in the combined conflict and goat categories when we take each subject as a unit, scoring either above or at or below chance expectation, in a 2 x 2 table evaluated by chi-square analysis. The same trends prevail dless of whether the incomplete sentence questionnaire or rating scale A is used as the measure of belief. The multiple _ regression analysis also underscores experimenter differences, i.c., a roughly linear relationship between ESP scores and two belief measures in K.O.’s data and no such relationship in D.D.’s data. How can we explain these differences between the two experi- menters in view of the fact that the talks both gave were based on the same outline and illustrated with the same slides? We can speculate that the conflict and goat subjects responded to the talk in the same fashion for both experimenters, while D.D. irked the sheep and reversed their motivation. An exploratory analysis of the subjects’ emotional arousal® indicated that K.O.’s combined shee scored on about the same level regardless of degree of arousal. In contrast, subjects in D.D.’s sheep groups performed in a curvilinear fashion: those moderately aroused scored above mean chance expectation whereas the not aroused and the highly aroused scored below. The goats in both experimenters’ groups scored below the chance line regardless of their emotional arousal (Figure 6). Have we any clues as to why these different reactions should have occurred? A few impressions can be put on record. The assistant (S.V.S.) who helped in the experiments and heard both lecturers remarked that whereas K.O.’s speech was presented in a relaxed, humorous, and confident manner, D.D. was rather tense, less secure about the subject matter, and very seriously intent on ing the audience to accept his point of view. In self-observations by both experimenters it was discovered that K.O. loves to give talks and thoroughly enjoys such occasions; D.D., on the other hand, did not enjoy giving talks at that time and became nervous when confronting an audience. Both experimenters are definitely convinced of the existence of ESP. If there is any difference in As a it-mortem loratory measure K.O., using a blind reborn pvakiaasd'adl enbijéots an to emotional arousal on the basis of their incomplete sentence questionnaire responses. Emotionally: statements were tabulated according to the frequency and intensity of the emotion expressed. contradictions in statements were also counted. ‘The subjects’ emotional arousal was judged on a three-point scale. In order to segregate the emotionally aroused sub; with sharp distinction, the benefit of the doubt was given to the class (arousal 1), and as a result more than half (93 subjects) were placed in this category.

180 Journal of the American Society for Psychical Research 5.0 4.5 w aw = . g e oO wu °o MEAN SQUARES be a 1.5 Fic. 6. Within subject variance of the pooled data of both experimenters according to levels of emotional arousal. P level of belief between the two, D.D.'s belief would be more intensely positive than K.O.’s. Is it ible that the experimenter who enjoys the lecture session for the fun of it gets good scores, while the one for whom it is a difficult task fails to get positive scores? This could be one variable to consider in further explora- tions. If we could finally specify the relevant factors in experi- menters which make for successful ESP testing, a major skirmish in the battle for repeatable experiments would be won. Personality and motivational factors would seem to need consideration in experimenters as well as in subjects, Another major concern of the present experiment was to test Van de Castle's hypothesis that within subject variance would be lower in the combined sheep than in the pooled conflict and goat subjects. With the F test that Van de Castle used, our results seonplees sentence questionnaire ‘The Effect of Experimenter Differences 181 ‘were confirmatory at the .05 level of significance. However, this finding must be taken with caution since when we used Van de ‘Castle's chi-square measure, taking each subject as a unit, we obtained chance results. The 2 x 2 table of subjects in both pooled belief categories having variance either above or below the mean was the basis of the chi-square test. The results of the chi-square analyses were not significant when using both belief measures wet self-rating scale A). reversal of the between subjects variance trend (sheep, high; _ conflict-goat, low) in K.O.’s data in relation to self-rating scale A was completely unexpected. If this reversal is a real effect and not just chance coincidence, it would seem to accentuate differences ; een the two belief measures. The incomplete sentence question- naire and self-rating scale A are highly correlated (r = .72, 176 d.f.); therefore, they seem to measure some belief factors similarly. _ But it is possible that one of them might be a sensitive measure of some factors which the other measures poorly. In view of our results, the two instruments might have registered differently the dimensions responsible for stabilizing ESP (within subject variance), am registering similarly those dimensions which affect scoring A multiple regression analysis was performed on the within subject variance. This analysis cl indicated the difference between the two belief measures as well as the differences between the data of the two experimenters. The multiple regressions approached significance only in D.D.’s data, where the regression coefficients of both belief measures had similar signs. Generall one. D.D.’s variance results were more in conformity wi an de Castle’s findings than were K.O.'s. We also compared the within subject variance of Van de Castle and the two present experimenters (Figure 4). The widest diver- gence is apparent in the goat category; the most uniform trend pears in the conflict category. In Van de Castle’s own experiment (), the goats’ within subject variance was also very unstable, varying from series to series from 12.88 to 40.69. His conflict p was remarkably consistent throughout his seven series, with the variance ranging only from 30.04 to 38.99. We might speculate that the motivational conflict in goats is entirely dependent on the test situation. Van de Castle entertained the idea that the conflict of the disbeliever is between a firm conviction that ESP tests are useless and a waste of time on the one hand, and a social obligation to cooperate with the experimenter on the other. If this is the case, the social obligation would be felt quite differently in relation to different experimenters, and the conflict, therefore, would depend _ largely upon the experimenter’s personality and approach.

182 Yournal of the American Society for Psychical Research The conflict ere eubyects seem to have an inner conflict between their own doubts and beliefs; that is, conflicting value systems. If this is so, the conflict would be relatively stable and less dependent on the immediate, external situation. Figure 6 illustrates the emotional arousal factor in relation to within subject variance in the different belief ar (data of both experimenters pooled). It was a surprise to find that the variance was negatively related to the arousal intensity; that is, more arousal, less variance. This trend is distinctly expressed in the combined sheep category, and to a lesser degree also in the pooled conflict and goat category. The high sheep and goat cate- gories taken singly do not show a linear decline; this may be due to the very small number of subjects with extreme beliefs split among three arousal categories. The emotional arousal dimension was introduced as a post- mortem analysis and therefore was not evaluated statistically, But if this trend is a real effect, we could speculate that emotional arousal, to the extent that it is expressed in verbal responses, might stabilize the ESP scoring level, at least in the combined sheep categories. In spite of the fact that K.O.’s sheep scored above and D.D.’s below mean chance expectation, the between subjects variance decreased quite similarly in both experimenters’ data. It would appear that emotional arousal stabilized both psi hitting and psi missing. Between subjects variance in Van de Castle’s data indicates a definite pattern: very high scoring variation in the high sheep category, moderately high in the conflict group, and very low in the sheep and goat categories, In other i strong believers did not respond in a uniform manner to Van de Castle as an experi- menter and had scores scattered from very high to very low. In contrast, moderate believers and disbelievers responded as a homogencous group and had little variation in total scores (Figure 5 D.D.'s between subjects variance followed the same pattern as Van de Castle’s, with the magnitude of the variance being in the same rank order, but of the six comparisons only one reached the .05 level of significance (Table 6A). K.O.’s between subjects variance showed no resemblance to either D.D.'s or Van de Castle's patterns, Because of the low F values obtained, repetition of Van de Castle’s findings in relation to between subjects variance is not claimed. If the trends in our data represent something more than chance coincidence, we might speculate that, like D.D., Van de Castle annoyed some of the high sheep and accentuated the motivational conflicts of the goats. This hypothesis is supported by The Effect of Experimenter Differences 183 another unpredicted finding in the data of the present experiment: D.D.’s between subjects variance is significantly higher than K.0.’s. It would be interesting to survey past ESP experiments to see whether low between subjects variance is consistently associated with significant above-chance scores and high between subjects variance (where psi-hitting might be cancelled out by i-missing) is an adjunct of non-significant scores. The scores of the incomplete sentence questionnaire correlated highly with those on both the A and B self-rating scales. However, only self-rating scale A and the incomplete sentence questionnaire seemed useful in separating psi-hitting from psi-missing subjects. Multiple regressions on K.O.’s data indicated that these two ‘measures used jointly were more effective than cither one alone. Self-rating scale B might not be so much a measure of the subject's belief in personal ESP ability as such as a measure of the interaction between such belief and his manner of self-description, i.e., boast- fulness, overcautious modesty, and conformity. The incomplete sentence questionnaire was more effective than self-rating scale A in relation to between subjects variance. The findings of this experiment indicate the great complexity of the motivational factors in ESP. ‘The decisive importance of the subject’s attitude toward the experimenter and the test situation was revealed in a comparison of K.O.’s and D.D.’s results. How- ever, no direct measure of such attitudes was obtained. To explore this factor modern tools of psychological scaling, ¢.g., Osgood’s semantic differential method (3), could be applied to subjects and experimenters in future research. Complex research designs also require complex multivariate evaluation methods, such as have been introduced here in a rather modest way. REFERENCES 1, Adcock, C, J., and Quartermain, D. “Some Problems in Group Testing of ESP.” Journal of Parapsychology, Vol. 23, December, 1959, 251-256. 2. Mangan, G. L. A Review of Published Research on the Relation- ship of some Personality Variables to ESP Scoring Level. Para- psychological Monograph No. I. New York: Parapsychology Foundation, Inc., 1958. 3. Osgood, C. E., et al. The Measurement of Meaning. Urbana: University of Illinois Press, 1957. 4. Schmeidler, G. R. ESP in Relation to Rorschach Test Evaluation. Parapsychological Monograph No. 2. New York: Parapsychology Foundation, Inc., 1960,

184 Yournal of the American Society for Psychical Research 5. Schmeidler, G. R., and McConnell, R. A. ESP and Patterns. New Haven: Yale University Press, 1958. 6. Solley, C. M., and Murphy, G. Development of the Perceptual World. New York: Basic Books, Inc., 1960. 7. Van de Castle, R. L. “Differential Patterns of ESP Scoring as a Function of Differential Attitudes toward ESP.” Journal A.S.P.R., Vol. 51, April, 1957, 43-61. 8. ——. “A Review of ESP ‘Tests Carried Out in the Class- room.” International Journal of Parapsychology, Vol. 1, Spring, 1959, 84-102. 9. Van de Castle, R. L., and White, R. R. “A Report on a Sentence Completion Form of Sheep-Goat Attitude Scale.” Journal of Parapsychology, Vol. 19, September, 1955, 171-179. AS.P.R. 880 Fifth Avenue New York 21, N. Y. APPENDIX Incomplete Sentence Questionnaire 1; I believe that Honchies often ..............20.i-ceseccccccesecnnescores POeeeeTeOtOererrerrerrrrr itt i rir tit ri iri ire it titre t i retire) SERRE EERE RETR EEE ERE EE EEE EEE EEE HERE EEE FETE SE EEE EEEEEE EES EEEEESOESESEEEEEEEEESS OUT ECTRRTOETOCOOOSOOSOCEOOCOOOOOC COS IS re te ii RRR RE ERE AREER EERE EE EERE EEE RE EERE EE EEE EEEE EEE EE SEE EEEEEEEEEEEEEEEEEE ESOS EEEEEES EERE EERE EEE EOE EEE EEE TEETER TEETER EE EEE TEETER EEE EEE ETE EEE ESET EE HEE SEES EET E EET EEE EERE EEE EE EER EEE EEE EEE REE EEE EEE EEE EEE EEE EEE EEE EEE HEE EEE EE EEE EEEE® SORE E EEE EEE EEE EEE MEH EEE HEHEHE EEE EEE EERE HEHEHE EEE EE EEE HEHEHE EEE ES ‘The Effect of Experimenter Differences 185 SEER EERE REE EE EEE EERE EEE EEE EEE EER EE EEE EEE EEE EH EEE EEE EEE EEE EEE EER EEE EEE EE EE SEER EE REE EEE EEE EEE EEE EEE EEE EEE EEE EEE GHEE HEHEHE EEE EEE EEO ene eene _ Self-Rating Scales A and B We would like to know your opinions on the following questions. Please check that answer which ae applies to you. Be as accurate and reliable as possible. A. Do you think some people are sometimes able to obtain information by ESP? Sure about it —, this is a possibility —, it might be so or not — it seems unlikely —, it is impossible —. B. Do you believe that you are sometimes able to get impressions by means of ESP? Sure about it —, this is a possibility —, it might be so or not —, it seems unlikely —, it is impossible —. PRINTED IN BRUGES, BELGIUM, BY THE ST CATHERINE PRESS LTD.