Just Doing Some Math

People familiar with my writing know I have discussed work by a man named Stephan Lewandowsky quite a bit. The short version of the discussion is he has behaved unethically, published false statements and, most importantly generated bogus results by misusing what is relatively simple mathematics.

I'm not the only person to say such, but the discussion has been spread out across many locations over several years. Today, I'd like to start working on collecting the information into a single resource by beginning with a discussion of the gross misuse of simple statistics.

Whatever one may believe about Lewandowsky and his behavior, the indisputable truth is the methodology he relied upon to publish several papers fabricates results because f how he misused it. Results he published are completely and utterly without merit.

Before I continue, I should point out Lewandowsky is hardly the only person to do what I will describe. The same fundamental error can be found in the work of dozens of researchers, if not more. I am focusing on Lewandowsky primarily because he has been the most inflammatory of these researchers (as far as I've seen), beginning with a paper titled:

NASA faked the moon landing-therefore (climate) science is a hoax: An anatomy of the motivated rejection of science.

This paper was Lewandowsky's attempt to paint global warming "skeptics" as conspiracy theorists. This is a common meme used to deride and dismiss people who challenge calls for strenuous action to combat global warming. Such people are often labeled science "deniers" who reject the "consensus" because they hold crazy beliefs. Strangely, attempts to justify such labels have relied on the misuse and/or abuse of science and math.

To demonstrate the problem with such claims, here is the data Lewandowsky relied upon to claim global warming "skeptics" believe NASA faked the moon landing:

The data shows there is a "statistically significant" positive correlation (r=0.13) between a person's stated level of agreement with these two statements:

1) The Apollo moon landings never happened and were staged in a Hollywood film studio.
2) The claim that the climate is changing due to emissions from fossil fuels is a hoax perpetrated by corrupt scientists who wish to spend more taxpayer money on climate research.

Thus Lewandowsky tells the world people who believe global warming is a hoax are prone to believe NASA faked the moon landing. However, that chart makes it clear such a conclusion is unjustifiable. Of 1,144 respondents, exactly 10 said they "Agree" (4) or "Strongly Agree" (6) with the claim NASA faked the moon landing. You can see and count those ten data points on the right side of the chart.

Now, the top half of the chart shows 134 responses from people who "Agree" (70) or "Strongly Agree" (64) with the idea global warming is a hoax. As you can see, only three of them agreed NASA faked the moon landing. A grand total of three people, out of a sample of 1,144, claimed to hold the position Lewandowsky uses as his headline.

Not only is the number of people endorsing Lewandowsky's headline position miniscule, we have no way to know those answers were genuine. The respondents to the survey used to generate this data were from internet blogs. The data shows those respondents overwhelmingly believe global warming is real and poses a serious risk. If a few of them decided to give fake answers to make "skeptics" look bad, it would have been easy to do. Lewandowsky dismisses this possibility, saying:

Another objection might raise the possibility that our respondents willfully accentuated their replies in order to subvert our presumed intentions. As in most behavioral research, this possibility cannot be ruled out. However, unless a substantial subset of the more than 1,000 respondents conspired to coordinate their responses, any individual accentuation or provocation would only have injected more noise into our data

This claim is mendacious. Lewandowsky claims the possibility of fake responses impacting the results is not plausible because that would require a conspiracy between "a substantial subset of the more than 1,000 respondents." Given only three respondents claimed to embrace Lewandowsky's headline position, that claim is laughable. Three people could have easily decided, independently of one another, to give fake responses. Alternatively, one person could have taken the survey multiple times (perhaps using internet services to mask the duplication) to give fake answers.

That said, the number of respondents who answered in this particular way is not what makes this claim mendacious. What makes the claim mendacious is the fact those respondents are irrelevant. Lewandowsky would have concluded global warming skeptics believe NASA faked the moon landing even if none of them said so. In fact, he would have drawn the same conclusion if not a single global warming "skeptic" had even responded to his survey.

The idea one could conclude global warming "skeptics" believe NASA faked the moon landing despite not having any data for global warming "skeptics" may seem strange. To demonstrate why it happens, here is the same chart as before, with a line superimposed to show what the correlation score means:

As you can see, the correlation score represents a line drawn through the data. The idea is you find whatever line best represents the relationship between two sets of data, and that line tells you how strongly they are correlated. The problem with this is you are trying to draw a line which best represents the data as a whole, meaning if most of your data is located in one quadrant, the data in the other quadrants doesn't really matter.

In effect, what Lewandowsky did was ask a bunch of people who believe in global warming if they thought global warming was a conspiracy or if they thought NASA faked the moon landing. Almost all of them said no. Lewandowsky reasoned, "People who believe in global warming think the moon landing was real, so since 'skeptics' are the opposite of them, they must believe the opposite - NASA faked the moon landing."

This isn't just a matter of images and interpretations. It is a mathematical fact. We can prove, mathematically, exactly which respondents were responsible for the results Lewandowsky got (and to what extent). This post is running long so I'll save an explanation of the math for another post. For now, here is a table showing the contribution of each possible type of response (1 = Strongly Disagree, 4 = Strongly Agree):

 CYMoon	 ClimChng count	 Contrib
1	1	892	0.6453
2	1	39	-0.3155
3	1	2	-0.0338
4	1	2	-0.0514
1	2	53	0.0247
2	2	20	-0.1044
3	2	1	-0.0109
4	2	2	-0.0332
1	3	65	0.0137
2	3	5	-0.0118
3	3	0	0.0000
4	3	0	0.0000
1	4	57	-0.0026
2	4	4	0.0021
3	4	1	0.0011
4	4	2	0.0033

If we take the sum of the contribution (Contrib) column, we get 0.1265, which is the same as the correlation score. As this table shows, 892 people said they Strongly Disaree (1) with the idea global warming is a hoax and Strongly Disagree (1) with the idea NASA faked the moon landing. Combined, these 892 people contributed 0.6453 to the "statistically significant" correlation.

The final correlation score was only 0.1265, but people who strong disagree with both propositions contributed 0.6453 to it. The reason for that is almost every other response type produces the opposite result. Respondents who only Disagree (2), not Strongly Disagree with both propositions knock 0.1044 off the correlation. Respondents who Disagree (2) with the idea NASA faked the moon landing but Strongly Disagree (1) with the idea global warming is a hoax knock off 0.3155 from the correlation score.

Whether we plot the data visually or examine it mathematically, the result is the same. There is an overwhelming amount of data from people who Strongly Disagree with two ideas. Lewandowsky cites this as proof people who agree with one of the ideas will agree with the other idea. That is completely inappropriate. It's nothing more than saying, "These two groups are opposites so they must believe the opposite of one another about everything."

The reason this happens is simple correlation tests like these assume the data being examined has a relatively normal distribution. For the tests to produce valid results, the data cannot be heavily skewed. Lewandowsky's data set is. That means these tests are inappropriate for his data set. Whether Lewandowsky knew it or not, applying tests like these to heavily skewed data sets can easily fabricate results out of nothing.

There is a lot more to cover, such as how Lewandowsky built upon his calculated correlation scores with Structural Equation Modeling (SEM) and examined other conspiracy theories, but for now, let's end this with one final thought. If we exclude all of the people who said they believe global warming is a hoax and all the people who said thye think NASA faked the moon landing, what do you think would happen?

Yup, that's right. The correlation gets stronger. It jumps from 0.1265 to 0.2654. Excluding all the data from people who claim to believe in these two conspiracies would only make Lewandowsky more certain people who believe one believe the other. HIs misuse of statistics would allow him to claim global warming "skeptics" believe NASA faked the moon landing with absolutely no data.

9 comments

  1. You write: "People who believe in global warming think the moon landing was real, so since 'skeptics' are the opposite of them, they must believe the opposite - NASA faked the moon landing." in quotes. But is it really a quote?

  2. Other objections:
    1. The data are ordinal/categorical so parametric statistics are inappropriate.
    2. The survey questions are compound ideas (moon landing didn't happen but images were staged; climate change is hoax and scientists are greedy for tax money). This type of wording complicates the respondent's decision-making. For example, someone who is unsure of the reality of climate change but objects to wasteful spending might go with the hoax answer out of anger about the latter part.

    Lewandowsky surely would benefit from a rigorous research methods refresher course. And a course in logic, too.

  3. Tom C Webster, no, which is why I said he "reasoned" that rather than "said" that. We can't actually quote a person's internal thoughts so even if we know the contents of those thoughts, the best we can do is paraphrase.

    Gary, there were a couple questions asked regarding climate change. If we look at the responses to other questions, I think it seems clear a number of the people claiming to believe global warming is a hoax don't actually believe it is a hoax - they just oppose some aspects of the "consensus" view (perhaps as little as, they feel money is being wasted combating it). There are tons of other problems with this survey/paper though. I could write 10,000 words and not cover them all.

  4. one thing that is oft overlooked, the three that said climate was a hoax, and the moon landing was a hoax.. All three answered all 4 of the climate science questions "correctly"... given the criticism of the survey, under the blogs that participated in the survey, entirely possibly, some locals ticked all the conspiracy boxes, in protest at such a stupid survey the nature/intent of which was so transparent to them.

  5. We could come up with any number of explanations for why some people responded the way they did (including the possible desire to influence the results to make "skeptics" look bad), but the reality is it doesn't matter. Three (out of over a thousand) people cannot justify conclusions or statements like those by Lewandowsky. Either he never looked at his data, or he intentionally published things he knew to be false/misleading.

    Whatever caused it, Lewandowsky's results depend entirely upon him running mathematical tests which were completely inappropriate for the data he had. His work was utterly without merit.

    What's worrying is not only did his work pass peer-review and get accepted by his peers (and much of the media), many other people have done the same thing as him. At least one person has been awarded a PhD because of it.

    It is difficult to find words to describe a situation where the field of research studying conspiracy theorists depends heavily upon a conspiracy of incompetence. That these people don't even understand the basics of the methodologies they use is dumbfounding.

  6. Whether or not he and Wood and others did not look at the data initially. IE merely incompetant.. Now, they have no such excuse. As it has been explicitly pointed out to them, yet still they defend it. I met Michael Wood in person, and he just hid behind statistical obtuseness.

  7. Oh, definitely. One of the more telling signs is they've stopped using the methodology now.

    As for Michael Wood, did you see the correspondence I had with him? It was one of the first things I posted here, and it pretty much defines willful obtuseness.

    (A new post was supposed to go up today, but internet problems are causing a delay.)

  8. 'Reasoned' can be used in place of 'said' if the quote involves some explanation. It is certainly plausible to interpret that as a direct quote.

Leave a Reply

Your email address will not be published. Required fields are marked *