Note to Web Browsers: This review appears in Correlation 15(1), Northern Summer 1996, along with a review of The Tenacious Mars Effect by H. J. Eysenck. Clicking on endnote numbers in the text (e.g. [1]) will take you to the endnote and clicking on the note number itself will take you back to your original position in the main text.

And why beholdest thou the mote that is in thy brother’s eye, 
but considerest not the beam that is in thine own eye? 
     - Matthew, 7:3 

The conclusion reached by the CFEPP (Comité Français pour l’Étude des Phénomènes Paranormaux) in regard to the Mars effect is clear enough, as it is set apart from the rest of the text and in italics on page 32, so the casual reader is sure not to miss it: “We therefore conclude that the most careful analysis of 1066 French sports champions shows no evidence for the existence of the ‘Mars effect.’ “ It is also clear that the Committee feels its own study is well and thoroughly done, since it refers to its own work as “rigorous,” and that its efforts have also produced an explanation for the Mars effect: Gauquelin was biased in his selection of athletes. However, the astute reader, noticing that the difference between the date of the protocol signed by Gauquelin and the Committee to initiate the experiment (pages 43 through 45) and the date of the book is some fourteen years might begin to wonder whether “rigorous” is not simply a pleasant euphemism for “slow.” Furthermore, while there are many holes in the Committee’s account of what it did over fourteen years that call into question both its good faith and its scientific credibility, the evidence offered against Gauquelin is based solely on two letters produced within two months - in the heat of an adversarial situation created by the Committee itself no less - and their consideration of Gauquelin’s possible bias in regard to his own work lacks both the depth and objectivity (not to mention rigor) of an earlier analysis of the same problem by Ertel[2].

Twenty pages is devoted to outlining what the Committee did, and an additional thirty pages lists the data and six tables, while nearly thirty-nine pages is given to a detailed dissertation based on two lists of suggestions made by Michel Gauquelin for improvement in the sample as presented to him in a preliminary report. This tortuous analysis of Gauquelin’s proposals (none of which seem to have been followed by the Committee itself) contrasts with a lack of candor about crucial points in their own work, some of which we only find mentioned more or less in passing in an appended commentary by J. W. Nienhuys, in which he excuses the lapses of the Committee, continues making the case against Gauquelin’s suggestions (though some are included in a list of 67 proposed additions to the CFEPP’s data) and then launches an offensive against Suitbert Ertel. The word offensive can be read here both as noun and adjective, as this portion of the book contains gratuitous personal attacks.

In the report of an experiment, the authors should provide the reader with answers to basic questions such as:

• Who was in control of the experiment and its various steps ? 
• What was done, and in what order? 
• In what measure did the experiment follow or depart from its original protocol, and did any such departures adhere to the spirit if not the letter? 
• What procedures were put in place to keep bias out of the process? 
• What procedures were put in place to make sure the calculations were correctly done? 
• What were the results?

Though we can’t exactly be sure who was in charge, or when, the point about how closely the experimenters kept to the protocol can be answered more substantively, at least if we read the record and ignore the Committee’s self-laudatory interpretation of it. Evidence is scattered throughout the book that the CFEPP needlessly violated a central point of the agreement which expressly states it is there to prevent the experimenters (several of whom had expressed decidedly negative views about both the Mars effect and Gauquelin) from influencing the first of two crucial steps in the experiment, the determination of the criteria that would serve as a guide in the gathering of the sample. Note the difference between what the protocol specifies with regard to data selection and what actually happened. From the protocol, page 44, emphasis mine: “In order to prevent all possibility of subjective bias, the criteria defining ‘great champions’ (performances, compensations received, etc.) will be established diligently for the CFEPP by a commission of sports specialists, notably journalists. The list will then be drawn up according to these criteria, applied in a rigorously impersonal fashion, using existing directories and sports annuals. The CFEPP will investigate the date, place, and time of birth of the champions selected.”

The thrust of this, which is clear enough, is that the place of the Committee in regard to data selection and collection should be mainly clerical, limited to gathering only those data which meet criteria supplied by others. What actually happened is recorded on page 17: “Protocol demanded that, for the first phase of the experiments, Michel Rouzé contact several sports journalists, asking them to establish criteria by which a list of the most famous sports champions in France could be selected. To avoid all possibility of bias, they were told only that it was for statistical research, without any mention of the Mars effect of Michel Gauquelin. This objectivity unfortunately brought with it a lack of motivation on the part of the journalists, and they declined the CFEPP’s invitation for a group meeting. Their responses were thus gathered by mail or telephone conversations. It was therefore not possible to entrust them with the establishment of the criteria for the test sample, but several of them recommended that we use the 1973 edition of Bernard Le Roy’s Encyclopedic Dictionary of Sports....”

This account, which is self-serving on the face of it, entitles the reader to ask whether the lack of motivation was greater on the part of the sports writers or Rouzé. Notice that the only criterion supposedly elicited from the journalists was “look it up in LeRoy,” so what follows this passage is the account of how unnamed members of the Committee themselves then established the criteria by going through that book. This was entirely unnecessary, and the excuse offered for Rouzé taking this tack is very, very weak, for the simple reason that one can think of a variety of means by which journalists (or even other parties with some interest in or expertise on sports) could have been polled short of making phone calls or requiring them to meet in someone’s office all at once. For example, why not list the sports at issue (they are all given in Gauquelin’s published volumes) and ask a group of journalists - or athletes in a local university, or professionals involved with these sports - to list in rank order two or three markers for high achievement in each one? Even if something like that should prove unproductive, other means should have been sought for keeping determination of criteria for the sample out of the hands of anyone associated with the Committee. Objectivity lies not in the intent, but in the follow-through, and there is none of the latter evident in the account of Rouzé’s actions.

More problematic than this infringement, however, is the fact that at the same time that some on the Committee[4] were deciding on the sample, other unnamed members of the CFEPP (as Nienhuys, not the Committee, informs us on page 121) were going through at least the first of Gauquelin’s three previous publications of his data on European sports champions and sending out inquiries to birth registries. In fact, the second and third publications of sports data, which would have been available to them as well, contain full details of the bulk of his champion samples, including sources, birth information and sector positions for Mars[5]. Added to this is the well-documented fact that between the signing of the protocol in 1982 and the issuing of the preliminary report, the only communication between anyone on the Committee and Gauquelin took place at a personal meeting arranged with Michel Rouzé (through the intervention of Paul Kurtz) at Gauquelin’s request, in 1985, something which is only hinted at for the reader in Nienhuys’s commentary (page 120). Thus, while in the throes of unnamed “organizational problems” (Nienhuys’s term) which led them to forego communication with Gauquelin (who, along with Ertel, nonetheless was communicating with them), the Committee was on the one hand vesting itself with responsibilities for data selection explicitly denied it by the protocol, and on the other hand examining data gathered under explicit criteria by their adversary while simultaneously making inquiries to registry offices. Neither the Committee’s nor Nienhuys’s account gives us any reason to presume these groups or individuals were not in contact with each other, nor even, for that matter, that they were not one and the same. It was incumbent upon them to let the reader know how they avoided something which would have been a problem even for an investigator who was actually doing what they really were not - adhering to a protocol signed by an adversary in good faith - but instead we are only provided with self-serving statements about their “rigorous” investigation and their “spirit of cooperation” in providing Gauquelin with information about their study years too late (page 29)[6]. If organizational problems made it possible for one member of the Committee to keep Gauquelin in the dark for a full eight years without the knowledge of the others, then equally well it was possible for one or another of what may have been several different researchers to have affected both the selection and collection process in a way that would produce the desired outcome, even if others involved were unaware of it - and even if they themselves did not intend to do so. The Committee, in other words, was in a position to shuffle Gauquelin’s data looking for criteria (or including or excluding specific sports) that fit their own wishes for the outcome of the experiment, so something more to the point than adjectives such as “rigorous” is required[7]. Where they do get into details, it is often over information available to the reader in standard references, such as the nearly two pages (23-25) spent on the vagaries of time in France. This could have been summarized in a paragraph, with the wasted space devoted to matters more directly related to this particular experiment.

At a minimum, a group offering a report of this type, particularly when it devotes such a great amount of space to attacking its adversary for supposedly biased post-hoc suggestions, should be meticulous in informing the reader about what checks and balances were instituted in order to keep its own original selections free from bias, yet such details are given only in regard to calculation (page 25, section 8), and those relating to data selection tell us mainly that they violated both the letter and the spirit of the protocol. Anyone reading this volume should keep this point firmly in mind, as Rouzé’s obvious failure to let the protocol do its work is the primary reason much of the book is given over to arguments about which player or sport ought to be in or out of the data base. In the past, foreknowledge of an individual sport’s performance in regard to the Mars effect, information which was freely available to the Committee in Gauquelin’s publications, has been used effectively to lower Mars key-sector percentages[8] in a study of this type.

Their prize exhibit - 39 athletes for which Gauquelin noted differences between their data and his - comes from a letter (pages 76-78) which specifies that the attached information is “probably not complete...but it is already substantial.” The reader might be led to believe that the letter itself was the result of some 5 or 6 months of effort, since it is dated December 6, 1990 and Gauquelin mentions the CFEPP’s preliminary report of June 20, 1990. In actuality, the report was not sent to Gauquelin until September 13th by Committee secretary Claude Benski, and after forwarding from his Paris address it was not received by him until October 20th, in San Diego, though this fact is mentioned only in passing, in Nienhuys’s commentary. In his letter, Gauquelin appends a checklist showing 118 differences between his information and theirs, which includes 39 athletes for which he shows differing times and 79 others whose reason for inclusion is not particularly clear to me, though it seems they were among 373 of their original sample (1,439) for which the CFEPP had obtained no times. In making their case against the 39, they consider only half of the problem, pointing out that Gauquelin seems to be primarily interested in convincing them to add athletes with Mars in key sectors, ignoring others with differing times who do not have Mars in these places. The alternative explanation, that Gauquelin was proffering suggestions from a pool of names whose exclusion might indicate bias on the part of the CFEPP is merely brushed aside

Their case here is very simply that approximately 133 such cases of time disagreement can be found between Gauquelin’s data and theirs, and that of his 39, about half (19) have Mars in key sectors, while of the remaining group, all (94, page 33) show a key-sector Mars in their data, while fewer (84) show key-sector Mars in his data. Thus, it appears that making only his suggested changes would boost the totals by 19, while including both his suggested and “unsuggested” changes would increase the totals only by 9. Thus is Gauquelin damned in this instance over a difference of ten sector positions in a preliminary list. Unfortunately for the CFEPP’s point, in order to conclude that the Mars effect is due purely to Gauquelin’s selective bias we need to know the total picture, including answers (even if only based on summaries) to such questions as a comparison of eminence levels between the athletes in the CFEPP’s list (pages 48-75) and those not included in the final sample due to lack of full information. In other words, whether Gauquelin’s suggestions themselves are slanted to favor Mars in key sectors or not (sector bias), does the CFEPP sample list he is addressing show any evidence of eminence bias[9]? The best evidence of this might be the names of those not in the final sample. However, were it not for the CFEPP’s argument against Gauquelin’s suggestions, we would not even know any of these excluded names, and the fact that at the end of their section on Gauquelin and their final sample we still lack knowledge of over 300 names points to an important omission from the book.

Based on the information given by the CFEPP, and Nienhuys’s comments on their lack of organization, both interpretations - that the CFEPP’s study and Gauquelin’s suggestions are biased in one respect or another - seem equally likely, since both their study and the whole dispute between them is based on the procedure of advocates arguing criterion by criterion, athlete by athlete and sport by sport what should be in the sample and what not, with knowledge of sector placements for individual athletes and Mars effect percentages for various sports available to both sides. Interestingly, the solution to this, the use of citation counts, is actually endorsed at least twice by the Committee, first on page 19 in regard to mountain climbing and again on page 40 in arguing about whether or not pelote basque should have been included or not. Says the Committee in regard to this sport (emphasis mine): “The CFEPP had eliminated this discipline precisely because of its overrepresentation due to the artificiality of the title, ‘Champion of France.’ One can reintroduce it by correcting overrepresentation; for example, by retaining only champions that occur in both sources, as M. Gauquelin had recommended for five cycling champions.” Thus they used dual citation as an exceptional criterion in the case of mountain climbing, while refusing it in the case of pelote basque, but their argument for not using it as an overall standard (since only two reference works were used, “...it would have reduced the list to only a few hundred names....”) is specious, since nothing in the protocol required them to use only these two references[10]. More seriously, Ertel repeatedly offered use of his citation counts to them, over a period of years. The fact that a procedure they themselves endorsed in principle, and used at their convenience, later called into question their own results should not be lost on the reader[11].

This is an especially important point since the CFEPP themselves refuse even to consider Ertel’s work, something left to Nienhuys’s commentary. Unfortunately, while Nienhuys’s contribution at times offers some insight into areas of the CFEPP’s study that they do not cover, his comments are tinged with hot-button adjectives (“seems inordinately worried” “massive effort” “boldly proclaimed”) and denigrating rhetoric that presumes either base motives or ineptitude on the part of Gauquelin and Ertel. Particularly telling is an ad hominem attack on Ertel for pointing out an error in the most crucial calculation in the whole book, the probability for finding Mars in a key sector, a value which happens to be wrong (and significantly so) in a direction that favors a negative conclusion on the Mars effect. Of this, Nienhuys says on page 145: “Ertel does not believe this 18.2% result...He then states flatly that the CFEPP’s estimate must be wrong, and suspects a programming error on the part of the CFEPP, which is a curious remark from someone who makes easily detectable errors himself....” Two independent calculations, one by Mark Pottenger and one by Nienhuys himself, have since confirmed the accuracy of Ertel’s warning that this number was in error, so the contumely in this case has come home to roost.

Even though he makes an effort at addressing some of Ertel’s work in numerical terms, Nienhuys’s cynical tendency to personalize his arguments generally gets the best of him. His most derisive volley is also the emptiest, as he dismisses Ertel’s quantitative analysis of the CFEPP study as “rather like tea-leaf readings,” answering it mainly with qualitative comments about why this or that athlete or sport ought to have been included or not, and with what seem to me elementary misunderstandings of Ertel’s procedures. For example, he criticizes Ertel[12] For dividing “116 cyclists into three eminence groups,” calling this sample splitting, and saying that “in other cases he uses a five-group division.” In fact each of these three groups for cyclists and two other sports categories are coherent classes based on eminence coding used by the CFEPP, and they are given for purposes of comparison with Ertel’s citation counts. When using eminence ranks based on citation counts alone (not the same thing as is being done in the comparison singled out by Nienhuys), the number of ranks used for tests of significance varies with sample size (more ranks for a larger data base, fewer for a smaller data base), but this is based on statistical considererations and not on the kind of whimsy Nienhuys’s garbled criticism seems to charge. His comments on this and other matters have been rebutted elsewhere by Ertel[13].

This document provides an interesting cap on the Gauquelin-vs.-skeptics phase of the Mars effect controversy. Though the reader unaware of the history of the controversy may be swayed to take the Committee and Nienhuys at their word, that theirs are the true facts and Gauquelin and Ertel are (at best) in error, still there are quite a few frayed edges easily caught by the critical eye which raise questions about what was done. To elicit all of them here would require far more space than the editor can allow, but at least I hope I have signaled some of the most significant problems for those who wish to brave the book’s pages.

Kenneth Irving

Notes

[1] This is further explained in my contribution to The Tenacious Mars Effect, co-authored with Suitbert Ertel, Urania Trust, 1996.

[2] Raising the hurdle for the athletes Mars Effect: Association co-varies with eminence. Journal of Scientific Exploration, 2(1), 53-82. Ertel considered all of Gauquelin’s data, published and unpublished, using the independent criterion of how many volumes in a standard set of references cited each athlete, to rank over 4,391 sports figures in Gauquelin’s files. These citation counts apply a single criterion to all athletes, regardless of sport.

[3] The answer is in fact provided by Nienhuys’s remarks about the collection, correction and transcription of data - the third stage of the experiment. He says (page 120) “...errors were made at each stage, and the CFEPP’s organization was too diffuse to detect them.”

[4] Or perhaps just Rouzé himself? We are not told.

[5] These publications are alluded to indirectly as “other directories” on page 19, second paragraph.

[6] Though the protocol required the Committee to keep Gauquelin informed about five specific steps in the experiment (“...determining the criteria for the test sample, establishing the list of champions, establishing the control group, gathering information from the registry offices, and the program for analyzing the data...”) as they were completed, he was kept in the dark about at least three (the criteria, the list, and the gathering of registry information) until all were completed, though one would have to read very sharply indeed in order to even suspect most of this from either the Committee’s or Nienhuys’s account. The chronology of contacts between the Committee and Gauquelin is simple: protocol published in October of 1982; Gauquelin meets Rouzé to inquire about the progress of the experiment, September 1985; Gauquelin receives the report of a nearly completed experiment, October 1990. I leave it to the reader to judge just what kind of spirit of cooperation with Gauquelin this indicates and just what kind of spirit of cooperation with the reader the obscuring of these facts in their account implies.

[7] Note that I am not charging anything devious or conspiratorial here. The simple problem is that when data selection is not clearly kept separate from knowledge of the point under examination (in this case, the published Mars positions of sports groups and individual athletes), bias immediately enters the picture even when an experimenter considers himself to have no opinion, let alone in a distinctly adversarial situation such as this one. Read the mini debate between Gauquelin (page 78, 99-100), the CFEPP (39) and Nienhuys (133-134, 140- 141) on whether pelote basque ought to be included in the sample or not, keeping in mind that all sides were privy to the fact that the sport shows a high Mars effect in Gauquelin’s original data. Everyone gives fine reasons for why it should or should not be there, but when the dust has cleared, those who do not think much of the Mars effect have banished pelote basque, while the one who favors it has kept it. A similar pattern exists throughout the book.

[8] Again, see Ertel’s analysis in The Tenacious Mars Effect.

[9] Ertel’s analysis, previously cited, shows that in fact it is, as athletes not found in the final sample show an excessive number with one or more citation in Ertel’s references compared to the sample on which the CFEPP base their conclusions.

[10] LeRoy was of course suggested by sports journalists, while L’Athlège was said to have been requested by Gauquelin. Was this his only suggestion? It would be interesting to know.

[11] Again, outlined in The Tenacious Mars Effect.

[12] See pages AI-6 to AI-8 of The Tenacious Mars Effect.

[13] Currently, Ertel’s response can be found at http://www.rtd.com/~lippard/ertel-response.txt