People familiar with my writing know I have discussed work by a man named Stephan Lewandowsky quite a bit. The short version of the discussion is he has behaved unethically, published false statements and, most importantly generated bogus results by misusing what is relatively simple mathematics.
I'm not the only person to say such, but the discussion has been spread out across many locations over several years. Today, I'd like to start working on collecting the information into a single resource by beginning with a discussion of the gross misuse of simple statistics.
Whatever one may believe about Lewandowsky and his behavior, the indisputable truth is the methodology he relied upon to publish several papers fabricates results because f how he misused it. Results he published are completely and utterly without merit.
Before I continue, I should point out Lewandowsky is hardly the only person to do what I will describe. The same fundamental error can be found in the work of dozens of researchers, if not more. I am focusing on Lewandowsky primarily because he has been the most inflammatory of these researchers (as far as I've seen), beginning with a paper titled:
NASA faked the moon landing-therefore (climate) science is a hoax: An anatomy of the motivated rejection of science.
This paper was Lewandowsky's attempt to paint global warming "skeptics" as conspiracy theorists. This is a common meme used to deride and dismiss people who challenge calls for strenuous action to combat global warming. Such people are often labeled science "deniers" who reject the "consensus" because they hold crazy beliefs. Strangely, attempts to justify such labels have relied on the misuse and/or abuse of science and math.
To demonstrate the problem with such claims, here is the data Lewandowsky relied upon to claim global warming "skeptics" believe NASA faked the moon landing:
The data shows there is a "statistically significant" positive correlation (r=0.13) between a person's stated level of agreement with these two statements:
1) The Apollo moon landings never happened and were staged in a Hollywood film studio.
2) The claim that the climate is changing due to emissions from fossil fuels is a hoax perpetrated by corrupt scientists who wish to spend more taxpayer money on climate research.
Thus Lewandowsky tells the world people who believe global warming is a hoax are prone to believe NASA faked the moon landing. However, that chart makes it clear such a conclusion is unjustifiable. Of 1,144 respondents, exactly 10 said they "Agree" (4) or "Strongly Agree" (6) with the claim NASA faked the moon landing. You can see and count those ten data points on the right side of the chart.
Now, the top half of the chart shows 134 responses from people who "Agree" (70) or "Strongly Agree" (64) with the idea global warming is a hoax. As you can see, only three of them agreed NASA faked the moon landing. A grand total of three people, out of a sample of 1,144, claimed to hold the position Lewandowsky uses as his headline.
Not only is the number of people endorsing Lewandowsky's headline position miniscule, we have no way to know those answers were genuine. The respondents to the survey used to generate this data were from internet blogs. The data shows those respondents overwhelmingly believe global warming is real and poses a serious risk. If a few of them decided to give fake answers to make "skeptics" look bad, it would have been easy to do. Lewandowsky dismisses this possibility, saying:
Another objection might raise the possibility that our respondents willfully accentuated their replies in order to subvert our presumed intentions. As in most behavioral research, this possibility cannot be ruled out. However, unless a substantial subset of the more than 1,000 respondents conspired to coordinate their responses, any individual accentuation or provocation would only have injected more noise into our data
This claim is mendacious. Lewandowsky claims the possibility of fake responses impacting the results is not plausible because that would require a conspiracy between "a substantial subset of the more than 1,000 respondents." Given only three respondents claimed to embrace Lewandowsky's headline position, that claim is laughable. Three people could have easily decided, independently of one another, to give fake responses. Alternatively, one person could have taken the survey multiple times (perhaps using internet services to mask the duplication) to give fake answers.
That said, the number of respondents who answered in this particular way is not what makes this claim mendacious. What makes the claim mendacious is the fact those respondents are irrelevant. Lewandowsky would have concluded global warming skeptics believe NASA faked the moon landing even if none of them said so. In fact, he would have drawn the same conclusion if not a single global warming "skeptic" had even responded to his survey.
The idea one could conclude global warming "skeptics" believe NASA faked the moon landing despite not having any data for global warming "skeptics" may seem strange. To demonstrate why it happens, here is the same chart as before, with a line superimposed to show what the correlation score means:
As you can see, the correlation score represents a line drawn through the data. The idea is you find whatever line best represents the relationship between two sets of data, and that line tells you how strongly they are correlated. The problem with this is you are trying to draw a line which best represents the data as a whole, meaning if most of your data is located in one quadrant, the data in the other quadrants doesn't really matter.
In effect, what Lewandowsky did was ask a bunch of people who believe in global warming if they thought global warming was a conspiracy or if they thought NASA faked the moon landing. Almost all of them said no. Lewandowsky reasoned, "People who believe in global warming think the moon landing was real, so since 'skeptics' are the opposite of them, they must believe the opposite - NASA faked the moon landing."
This isn't just a matter of images and interpretations. It is a mathematical fact. We can prove, mathematically, exactly which respondents were responsible for the results Lewandowsky got (and to what extent). This post is running long so I'll save an explanation of the math for another post. For now, here is a table showing the contribution of each possible type of response (1 = Strongly Disagree, 4 = Strongly Agree):
CYMoon ClimChng count Contrib 1 1 892 0.6453 2 1 39 -0.3155 3 1 2 -0.0338 4 1 2 -0.0514 1 2 53 0.0247 2 2 20 -0.1044 3 2 1 -0.0109 4 2 2 -0.0332 1 3 65 0.0137 2 3 5 -0.0118 3 3 0 0.0000 4 3 0 0.0000 1 4 57 -0.0026 2 4 4 0.0021 3 4 1 0.0011 4 4 2 0.0033
If we take the sum of the contribution (Contrib) column, we get 0.1265, which is the same as the correlation score. As this table shows, 892 people said they Strongly Disaree (1) with the idea global warming is a hoax and Strongly Disagree (1) with the idea NASA faked the moon landing. Combined, these 892 people contributed 0.6453 to the "statistically significant" correlation.
The final correlation score was only 0.1265, but people who strong disagree with both propositions contributed 0.6453 to it. The reason for that is almost every other response type produces the opposite result. Respondents who only Disagree (2), not Strongly Disagree with both propositions knock 0.1044 off the correlation. Respondents who Disagree (2) with the idea NASA faked the moon landing but Strongly Disagree (1) with the idea global warming is a hoax knock off 0.3155 from the correlation score.
Whether we plot the data visually or examine it mathematically, the result is the same. There is an overwhelming amount of data from people who Strongly Disagree with two ideas. Lewandowsky cites this as proof people who agree with one of the ideas will agree with the other idea. That is completely inappropriate. It's nothing more than saying, "These two groups are opposites so they must believe the opposite of one another about everything."
The reason this happens is simple correlation tests like these assume the data being examined has a relatively normal distribution. For the tests to produce valid results, the data cannot be heavily skewed. Lewandowsky's data set is. That means these tests are inappropriate for his data set. Whether Lewandowsky knew it or not, applying tests like these to heavily skewed data sets can easily fabricate results out of nothing.
There is a lot more to cover, such as how Lewandowsky built upon his calculated correlation scores with Structural Equation Modeling (SEM) and examined other conspiracy theories, but for now, let's end this with one final thought. If we exclude all of the people who said they believe global warming is a hoax and all the people who said thye think NASA faked the moon landing, what do you think would happen?
Yup, that's right. The correlation gets stronger. It jumps from 0.1265 to 0.2654. Excluding all the data from people who claim to believe in these two conspiracies would only make Lewandowsky more certain people who believe one believe the other. HIs misuse of statistics would allow him to claim global warming "skeptics" believe NASA faked the moon landing with absolutely no data.