Misuse of statistics has become the stock-in-trade of many of the anti-technologists, pseudoenvironmentalists, social "engineers,'' and other aberrations with which our environment is currently polluted. This has become so widespread that statistics itself is maligned, with many people discounting its merit in the belief that any lie can be supported by statistics. This is false. Statistics is a beautiful and exact mathematical science of very great value. It can only be misused in the service of falsehoods when the audience lacks basic mathematical knowledge. Unfortunately, this lack is widespread. Even the scientific journals themselves are filled with statistical errors - especially since politically correct results determine tax-financed research funding. One of the most beautiful uses of statistics is in the science of statistical mechanics. There, the laws of probability and a few simple principles like conservation of energy are used to derive the entire science of thermodynamics. Misuse of statistics, however, is not so beautiful. Correlations:
The electric shaver example shows a type of statistical error that is very common. Most mathematical tests of correlation calculate a value of the probability that the two measurements under study are
not correlated with one another, Pnc. For example, Robinson, A. B. and Robin-son, L. R., Mech. Ag. Dev. 59 pp 47-67 (1991) gives Pnc for the correlation between urinary cystathione and age in 205 men as 0.025. Therefore the probability of correlation is 1-Pnc=Pc = 0 .975 or 97.5%.If cystathione were a pesticide and the Robinsons had a grant request pending with the Environmental Protection Agency, we might read headlines like, " Cystathione linked to premature aging.'' But wait. The Robinsons measured 51 substances in this experiment. Since they tried 51 times to get a correlation, about one substance is expected to have P
nc = 0.02 or less by random chance. They, in fact, found 10 (with even higher Pcs), so this paper is interesting. Had they found only one or two at 97%, this would be insignificant.The electric shaver "discovery'' involved the screening of large numbers of items until one was found for publicity. Without correction for the number of efforts to find a correlating item, the report is erroneous. Since thousands of pseudoscientists with lavish tax funding are searching for correlations between technology and evil, random chance is providing a plentiful supply of false correlation scare stories.
A second common error is in actual calculation of the correlational probabilities. The value of P
c usually must be at least 0.95 or greater before "statistical significance'' is claimed. In other words, unless there is less than one chance in 20 that the result is a random occurrence, it is not termed "statistically significant.'' Most reports calculate these P's with the assumption that the measurements of interest are distributed in accordance with a Gaussian or normal distribution function - the familiar bell curve. In fact, many measurements do not have normal distribution functions. This error often leads to falsely high probabilities of correlation.
Distribution functions:
It can be rigorously proven that, when the variations in a measured value are entirely dependent upon a large number of similarly sized independent variables, the measurements will be distributed as a normal distribution. The functional form of this normal distribution is illustrated in Figure 1 reproduced from Introduction to Probability and Statistics by H. L. Alder and E. B. Roessler, W. H. Freeman and Company (1977), p 115. Figure 1 shows a bar graph and normal distribution function for the expected frequencies of heads in tossing groups of nine coins 512 times. Since the factors that determine the variation in outcome of a single toss of nine are many and similarly sized, the distribution function is normal.If another set of coins that had a larger number of coins were used, then the distribution would be shifted to the right. In this case, both distributions would be normal. Assuming this, a calculation of the probability that the two sets of coins are different can be made with high reliability after a relatively few experimental trials.
Suppose, however, that the two different sets of coins are used together in a single experiment. The result will be a broadened distribution function that is not normal. If the sets were different enough, the experiment would even show a distribution with two peaks with a shape like two overlapping bells. In this case, the variation is not due only to similarly sized independent variables. The difference in number introduces a variable much larger than the others that give rise to the variable outcomes of individual tosses of sets of coins.
In most instances of alleged ill effects of technology, there is insufficient data to prove that the measurements under study have a normal

distribution function. This is not surprising because most experiments involving living things or other environmental phenomena are relatively difficult to perform. This limits the number of observations. Powerful statistical tests are often needed to extract significance from small numbers of experimental measurements. These tests are more likely to give values indicating significance if they assume distribution function shape, so tax-financed experimenters much prefer them.
When distribution function shape is unknown, the proper approach is to use nonparametric statistics. There are simple non-parametric methods of which the most common is the Wilcoxon Test.
Suppose that we know the age at death of 10 men (and only ten men, since it is imperative to use all of the data without selection), five of whom were members of the Sierra Club. The control subjects died at ages 78, 81, 83, 85, and 88, while the Sierra Club members died at ages 73, 76, 79, 80, and 82. The distribution of these men in order of age at death is, therefore, S-S-C-S-S-C-S-C-C-C. Since we do not know the shape of the distribution functions, the Wilcoxon test simply assigns a rank to each individual of S1-S2-C3-S4-S5-C6-S7-C8-C9-C10. The sum of the ranks of the Sierra Club members is 1+2+4+5+7 = 19. The test then calculates, for the case of blindly choosing 10 objects, five of one label and five of another in succession, the probability of a set having a sum of ranks less than or equal to 19.< /FONT >
For our simple case, this can be done by actually listing all possibilities and counting them. In this case, P
nc = 0.048, so there is a 95% chance that our experiment shows that Sierra club members do not live as long - providing there were no other systematic variables besides Club membership. There are, of course, other variables. For instance, those still clinging to Sierra Club membership in the 1990s, when it has allied itself with Greenpeace and Earth First and abandoned allegiance to its original goals, are likely to be people of below average wisdom. This may also shorten lifespan. Correlation does not prove causality.Nevertheless, we have done our calculation properly without assuming a normal distribution when the data set is too small to check that assumption. For much larger sets of data, Wilcoxon calculations are more involved. See the above-referenced book by Alder and Roessler and also the
Mech. Ag. Dev. article by Robinson and Robin-son for examples and references.(This reminds me of the story of the conservative retired Iowa farmer who suddenly announced to his friends that he had joined the Communist party. "Why,'' they asked, "had he done such a dishonorable thing?'' "Well,'' he explained, "I have been to my doctor. He says that I have just six months to live. I figure it is better that one of them should die than one of us.'')
Errors: Whenever possible, each quantitative scientific assertion should be accompanied by a quantitative estimate of its reliability or likely range of error. Where the error itself is an important part of the result, then the error of the error should also be calculated and reported. One sign of unreliable science is poor reporting of errors. Each scientist is obligated to estimate and report the reliability of his results.
|
|
Vol. 22, No. 6
Newsletter: Access to Energy Newsletter Archive Volume: Issues Issue/No.: Vol. 22, No. 6 Date: February 01, 1995 03:32 PM Title: Envy
Copyright © 2004 - Access to Energy Newsletter Archive
|