York H. Dobyns says (Oct 30, '92): +quot;I'm no fan of the Mars effect: the evidence is qu

Master Index Current Directory Index Go to SkepticTank Go to Human Rights activist Keith Henson Go to Scientology cult

Skeptic Tank!

---------------------- Message --------------------------------- York H. Dobyns says (Oct 30, '92): "I'm no fan of the Mars effect: the evidence is questionable due to methodological flaws." Nevertheless he then dwells meticulously on consequences that would arise if the Mars effect were shown to be real. Apparently, the evidence is not questionable enough to remain aloof from it. My business today is to question the evidence for "methodological-flaws" arguments abounding in discussions here about the Mars effect. Most of these arguments have been brought forward by Jan Nienhuys. Newsgroup members seem to readily take his views as their own. What I would like to suggest is to read his messages more skeptically (should not be difficult in this circle). I am going to tell you why. ---------------------------- 1 ------------------------------------- J.N. Nienhuys 28 Oct 1992 > Later more "effects" were found with different professions. > But (to me) the main point is that Lasson already emphasized > one should take "famous" professionals. It's quite natural > that if the reality of this effect was suspected, they (both > Gauquelins) would try to determine how famous or good the professional > had to be. It is also conceivable that a psychologist receiving > his statistics training in the early '50's would not be aware of > artifacts resulting from cumulative biases introduced by this type > of exploration. News-readers, not being familiar with Gauquelin's procedure in detail will take it for granted that Nienhuys justifiably refers to some actual bias connected per se with collecting birth data of famous people. They will find it plausible that artefacts did accumulate due to poor statistical knowledge of the Gauquelins in the Fifties. "It wasn't their fault (how considerate we are), researchers at that time were not as sophisticated as we are today (how excellent we are)". The verdict is done. I feel obliged to defend Michel Gauquelin (he died last year and thus cannot defend himself) one of the most admirable figures (admirable regarding methodological conscientiousness) in frontier science fields. First, there is no bias at all associated with collecting birth data with preference for eminent people. On the contrary, collecting birth data without considering eminence must be considered as severely biased. As soon as Gauquelin had reported that planetary effects were stronger with famous than with average athletes (his first observation) any subsequent study testing the replicability of planetary effect was bound to select more excellent individuals from professional samples. Both, the American and the French skeptics did gravely (CSICOP) or appreciably (CFEPP) violate the eminence requirement. A study aiming at refuting some purported effect must show its absence despite having established most favorable conditions for the purported effect to occur. CSICOP/CFEPP apparently did not set up such conditions. They collected data in a way as if they feared the Mars effect might emerge (see data below). ---------------------------- 2 ------------------------------------- J.N. Nienhuys 28 Oct 1992 > Given the rampant habit of reporting "significance" without > model or hypothesis prior to the experiment (rampant at least > in social science and medicine), there is absolutely nothing special > or fraudulent about one psychologist not making the proper > distinction between exploration and testing. Jan Nienhuys here claims that the Gauquelins did not make proper distinction between exploration and testing. The facts tell a totally different story: M. Gauquelin published results of an exploratory study in 1955 based on French data. Since then most of his publications were of the hypothesis-testing kind. In 1960 he published his first hypothesis-testing study based on Italian, German, Belgian, and Dutch data. The book gives answers to the question: "Do previous French results replicate with non-French data? The chapters provide reports on hypotheses, subjects, methods of analysis, results, discussions as to whether the hypotheses had or had not been supported (etc.). On reading statements as those made by Nienhuys turning the facts upside down and on having to witness their acclamation by the majority of responding readers ("Congratulations, Jan!") I just feel sad, and doubts arise as to whether justice and fairness has any better chance to prosper in our science community than elsewhere in this foul world. ---------------------------- 3 ------------------------------------- How eminent were athletes selected by the skeptics research groups as compared to athletes selected by Gauquelin? Here we exclude athletes common in both samples, i.e., we only consider athletes listed either in the skeptics ("CSICOP-only", "CFEPP-only") or in Gauquelin's sample ("Gauquelin-only"). The eminence of an athlete is defined by the occurrence of his/her name in 18 reference sources. The CSICOP-only sample of U.S. athletes is compared with a Gauquelin-only sample of U.S. athletes (which G. collected later), the CFEPP-only sample of French athletes is compared with the Gauquelin-only sample of French athletes (which G. had collected earlier) (see Table 1). Table 1: Numbers of American and French athletes in skeptics and Gauquelin samples. American: Kurtz et al. (CSICOP) N = 216 American: Gauquelin .......... N = 162 French: CFEPP ............... N = 398 French: Gauquelin ........... N = 130 Table 2 shows citation counts in percentages of respective Ns for the skeptics' and Gauquelin's citation subsamples, as well as the differences "Gauquelin - skeptics." Table 2: Citation counts (percentages) --------------------------------------------------------------- CSICOP Gauquelin Difference GAUQ-CSICOP --------------------------------------------------------------- citations = 0 72.5 25.9 -46.6 citations = 1 24.9 22.8 - 2.1 citations = 2 2.6 29.1 26.5 citations = >2 0.9 22.2 22.2 --------------------------------------------------------------- 100.0 100.0 --------------------------------------------------------------- --------------------------------------------------------------- CFEPP Gauquelin Difference GAUQ-CFEPP --------------------------------------------------------------- citations = 0 71.9 36.9 -35.0 citations = 1 18.1 20.8 2.7 citations = 2 3.8 23.1 19.3 citations = 3 4.3 11.5 7.2 citations > 3 2.0 8.7 6.7 --------------------------------------------------------------- 100.0 100.0 --------------------------------------------------------------- Results 1) The majority of CSICOP-only athletes is much less eminent than Gauquelin-only athletes. 2) The majority of CFEPP-only athletes is much less eminent than Gauquelin-only athletes. Considering this difference alone the skeptics' studies cannot be taken as appropriate for testing Gauquelin's claim of a Mars effect. However, despite unfavorable preconditions a significant overall Mars G% deviation was observed at least for the French skeptics' sample. (This has been reported earlier). ---------------------------- 4 ------------------------------------- Another way of testing the presence of a Mars effect consists of calculating G% for each eminence subsample separately and to test for monotonic trend i.e., to test for the claim that G% increases with athletic fame. Results from such trend test is independent of the general G% level of the entire sample (=citation subsamples pooled) and may be considered as an alternative test of planet-birth relationships. Table 3 shows G% values for the skeptics and the Gauquelin samples, first U.S., then French athletes, entire samples, broken down by citation frequencies. Table 3: G% (key sector percentages) for the skeptics' and Gauquelin athletes' samples. ---------------------------------------------- CSICOP Gauquelin N=408 N=349 ---------------------------------------------- citations = 0 18.1 21.4 citations = 1 21.5 21.3 citations > 1 25.9 30.2 ---------------------------------------------------- CFEPP Gauquelin N=1076 N=2040 (publ. and unpubl.) ---------------------------------------------------- citations = 0 22.8 26.1 citations = 1 28.6 25.1 citations = 2 24.6 27.6 citations = 3 27.9 26.4 citations > 3 28.5 31.1 ---------------------------------------------------- Note: Less famous athletes (less citations) are much greater in number than famous athletes. Therefore, for small total samples of athletes famous subsample will be sufficiently large only by pooling individuals from a greater range of citation frequencies. Results: 1) A monotonic increase of G% with citation counts is present in the Gauquelin data, both samples. 2) A monotonic increase of G% with citation counts is present in CSICP's data (Kurtz et al.) 2) A monotonic increase of G% with citation counts is present in CFEPP's data (Benski) ---------------------------- 5 ------------------------------------- J.N. Nienhuys 29 Oct 1992 > There may be a file drawer effect. When Ertel asked Gauquelin > in detail *all* his data, not only the ones he had published, > he found that in the file drawer there were many athletes (unpublished) > that did collectively *not* have a Mars Effect (namely around 22% > born `under Mars'), and even did not conform to the average number > of people (17.2%), but that were collectively born `under Mars' > in a much smaller fraction. Their distribution over the sectors was > a kind of mirror image of the distribution of the published athletes. > Details: > French published athletes: 1357 of which born `under Mars' :306 > French unpublished : 683 89 > Nonfrench published : 1531 322 > Nonfrench unpublished : 820 133 > Readers with calculators (and knowing just a smattering of statistics) > are invited to draw their own conclusions. > JWN Jan Nienhuys here gives a partial summary of my published report about Gauquelin's selection procedure. His summary is partial for three reasons: First, Nienhuys would like to make his readers believe that discarding and not publishing cases must be regarded per se as bias. I discussed this point in my paper from which he took the information so he should know - and should have told his readers - that not publishing cases in Gauquelin's study was not at all bias per se. Gauquelin was not merely entitled, he even HAD to exclude from his athletes sample individuals of lower eminence rank after having found in 1955 that the Mars effect increased with eminence. He was no less bound to throw mediocre figures out of his sample than the skeptics in their replications were bound to do just that (as discussed above). Comparing unpublished with published Gauquelin atheletes regarding c i t a t i o n c o u n t s (see Table 3) we do find that the numbers of citations is considerably lower for unpublished as compared to published athletes. Gauquelin thus actually SUCCEEDED, by discarding cases, to improve the general level of success in his sample. Table 3. Citation percentages for published and unpublished Gauquelin athletes citations Published Unpublished Difference (N=2888) (N=1503) ----------------------------------------------- 0 46.1% 62.5% 16.4% 1 18.9% 29.4% 10.5% 2 16.0% 4.9% -11.1% 3 7.3% 2.9% -4.4% 4 3.3% 0.2% -3.1% 5 2.7% 0.0% -2.7% 6 1.1% 0.0% -1.1% 7 0.6% 0.0% -0.6% 8 0.1% 0.0% -0.1% --------------------------------------- 100.0% 100.0% --------------------------------------- Nevertheless, Gauquelin should have done this discarding (and not publishing) athletes with greater care. He should have discarded low achievers prior to collecting their birthdates. Or else, after having obtained their birth data, he should have asked some naive assistant to check biographical information and to discard low achievers by applying some reasonable criteria of success. Gauquelin did such selections himself, obviously overestimating his ability to base his judgment solely on the athletes record of successes without considering, at the moment of decision, his possibly knowing/remembering the respective person's planetary positions. I searched for this inflating bias and I found and published the result. When Gauquelin heard of my citation counts he welcomed this procedure as an objective alternative to what he was used to apply. He gladly opened his "drawers" to let me take all athletes data and do a reanalysis based on published plus unpublished athletes. In this study subsamples of athletes differing in numbers of citations were analysed seperately (similarly to what was shown above with Tables 2 and 3). A pronounced eminence correlation emerged. Secondly, Nienhuys informed sci-skeptics about the low overall G% of Gauquelin's unpublished sample as compared to his published sample inviting the readers to draw their own conclusions - and he could only expect his readers to draw wrong conclusions (which he apparently wants to see spread) with not informing them about essential details needed to draw correct conclusions (which he probably deters). Nienhuys did not inform readers about the UNPUBLISHED athletes' LOWER LEVEL OF SUCCESS. G% difference between unpublished (=less eminent) versus published (=more eminent) athletes MUST be obtained, if Gauquelin's Mars + eminence hypothesis holds. The observed difference is NOT CREATED but only INFLATED by Gauquelin's above mentioned seductions. A third neglect by Nienhuys must be set straight: He never informed readers about the effects on eminence correlation by my pooling Gauquelin's published with unpublished athletes. I just quote from my paper (with some linguistic amendments): "The crucial question remaining is as follows: Could the kind of bias noted in Gauquelin's procedure invalidate the outcome of the present study? Could an artefact carried over from original meterials raise the risk for wrong conclusions? ... [Gauquelin's] omission of athletes from experimental samples had two effects: (a) It served to inflate the level of kS-proportions [G%] overall. But also, (b), it weakened the eminence effect. Our merging of unpublished with published data did repair (i.e. lowered) the overall eminence level. But at the same time it served to repair the eminence slope, i.e., to make it steeper. Gauquelin's selection bias, therefore, does not weaken the conclusion that Mars'position and the athletes'births are statistically related. Paradoxical though it may seem, this claim has been corroborated due to this bias: Correcting for selection bias by pooling all data INCREASED empirical support for the stronger version of this claim: the data have overcome, IN SPITE OF DISTURBING EFFECTS OF BIAS, the higher methodological hurdle." ---------------------------- 6 ------------------------------------- Before carelessly attributing to Gauquelin's (and my own) work "methodological flaws", "file drawer effects", "rampant significance reporting", significance fetism", "lots of tests - one was significant", "no prior hypothesis", "post hoc interpretations", "anomaly-mongering" - look at the above Tables notably Tables 2 and 3. If you are suspicious of what is in the right columns, keep to the left columns with the skeptics results. This, hopefully, will foster thoughtfulness. Those scientists among ourselves calling themselves 'skeptics' should be more inclined than the remainder of us to reconsider skeptically, when faced with unexpected evidence, their own views. Once observational data demand it they should be ready to admit that they were wrong, at least as ready as "ordinary" scientists like Gauquelin who found and unearthed, with utmost rigor, in a dust heap despised by the community what might be considered - by an advanced posterity - as a real grain of gold. Gauquelin was faced one day with unexpected evidence, he was surprised by a replication failure in his "heredity" research (he could not replicate G% similarities between parents and their children), and he had this to say: "The samples [my former and my recent sample] seemed identical in every way, yet the first gave results strongly in support of the hypothesis while the second gave results almost as strongly contradicting the hypothesis ... Despite the disappointments, I am pleased about one thing: If my latest work has created doubt, then this is the best that can happen in science. As Bertrand Russell has written, "Not to be absolutely certain is, I think, one of the essential things about rationality". (Gauquelin's heredity hypothesis did not find empirical support in my studies either. His basic finding, however, (the one now replicated by CFEPP) is totally independent of the heredity construct which was part of Gauquelin's explanatoty model). The Dutch skeptics (de Jager, Koppeschaar) had already proclaimed in public to be able to "unmask" (as they said) the Mars effect as an unrecognized artefact caused by seasonal/ diurnal priodicities. It happened that they failed entirely. If they had stated their failure frankly and if Jan Nienhuys had reported about their failure in frank manner for us electronic readers I would compare him/them, regarding sincerity, with Michel Gauquelin. I am still ready to do so and to withdraw my suggestion to read Jan Nienhuys' messages more skeptically as soon as he would refrain from wrongly reporting about Gauquelin data and procedures and if he would show himself, instead, convincing signs of skepticism regarding his own stand.


E-Mail Fredric L. Rice / The Skeptic Tank