I don't use the word "fraud" lightly. I've long criticized people who do. Not only do I think it is wrong in principle, I think it is wrong from a strategic perspective. If you cry, "Fraud!" over every little thing, nobody will listen when you point out real fraud. It's called a sense of proportion. One's rhetoric should ramp up with the severity of what is being criticized.
I bring this up because I want to follow up on my last post which discussed a case of fraud involving $100,000. Or rather, it was a case of fraud where a person used the false promise of $100,000 to cheat people out of money. You can read that post for the details. It's a long post so I won't re-hash the details here. I'll just give a short summary.
Last year, a man named Douglas Keenan announced he would give $100,000 to anyone who could win a contest he had created. There were tons of red flags which should have made people suspect this was bogus, but despite that, several big names in the global warming "skeptic" movement promoted the contest. After people spent some time publicly discussing how they might try to win the contest, Keenan switched out the data set used in the contest for one the proposed methodologies would be less effective on.
This contest involved a $10 entrance fee. That makes what he did fraud. I've been pointing that out for the last year. Just 24 hours ago, Keenan admitted it. Of course Keenan did not say, "I committed fraud." What he admitted to is the underlying facts of the accusations I've been making. First, After Keenan changed his contest, he updated the web page for it to say:
The 1000 regenerated series were posted online four days after the Contest was announced—on 22 November 2015. (Each person who submitted an entry before then was invited to submit a new entry, with no fee.)
He didn't even offer to give a refund to people who paid to enter his contest before he changed it. Pretty messed up, huh? Who opens up a contest, takes entrance fees then changes the contest after the fact to make sure those who paid can't win it? Apparently Keenan does.
That's not what I want to focus on today though. What I want to focus on is the fact for the last year I've made the same point over and over: Keenan changed the nature of the data set used in his contest in a way which made his contest more difficult to win. I've received a not insignificant amount of criticism for this, largely in e-mail exchanges. The main example I've used has been the discussion the proprietor of the major "skeptic" blog Watts Up With That, Anthony Watts, and I had when I contacted him about this. I've quoted excerpts before because I felt it was necessary, but now I'd like to quote an entire e-mail he sent:
After doing further investigation, it seems Keenan has notified everyone that there is a restart to the contest due to the weakness in the PRNG. There was indeed a change. to both the data and key.
The change was announced on the Contest web page yesterday
Additionally, Keenan tells me he telephoned Andrew Montford to discuss it, and there is now a Bishop Hill post about it:
From what I can see, Keenan left a comment on each blog that had previously posted about the contest. He left one at WUWT:
Spot the trend: $100,000 USD prize to show climate & temperature data is not random
It seems to be he has done due diligence, made proper notifications about the key and the data being changed. Initially I got the impression he’s changed the key only, and the updated PRNG was only in the key, but it seems it was in both because both had a weakness that would allow somebody to win by exploiting that weakness rather than winning the contest on its terms.
If he had changed the data and didn’t notify anyone of the change to the data, I’d agree that would be terribly wrong. That’s what I thought you were complaining about. But, he didn’t do that.
That language you used in your post might be considered libelous under the circumstances.
Finally, you do NOT have my permission to publish my email.
I won't publish the rest of the e-mails Watts sent in which he repeatedly defended Keenan's actions and argued I was in the wrong. I don't like posting people's e-mails without permission, and I think this one is enough to show what the "party line" is. In case it is not, here is what Andrew Montford of Bishop Hill wrote in the post Watts mentions:
Doug Keenan has posted a note at the bottom of the notice about his £100,000 challenge, indicating that he has reissued the 1000 data series. This was apparently because it was pointed out to him that the challenge could be "gamed" by hacking the (pseudo)random number generator he had used.
Brandon Shollenberger emails to say that this is a terrible thing, but I can't get terribly excited about it. Presumably it doesn't make any difference to those who think they can detect the difference between trending and non-trending series.
He wrote this after I contacted him providing clear evidence Keenan had changed his contest to make it more difficult after people discussed how they would try to beat it (see the last post for details). He didn't respond to my e-mail. Instead, he went and published a blog post trashing me a bit while intentionally disregarding what I had said. I believe "willfully obtuse" is an apt description.
Two big names in the "skeptic" blogosphere were saying the contest wasn't changed in a manner that made it more difficult to win while ignoring clear evidence that showed it was. They went as far as to paint my criticisms as completely unreasonable and even kind of call for public ridicule. These "skeptics" just accepted it on blind faith with Keenan wrote:
The generation of the 1000 series relies on the generation of random numbers. That presents a difficulty, because current computers do not generate truly random numbers. There is a widely-used method of addressing the difficulty: use a computer routine that generates numbers that seem to be to random (i.e. fake it). Numbers generated by that method are called “pseudorandom”.
A computer routine that generates pseudorandom numbers is called a “pseudorandom number generator” (PRNG). PRNGs have been studied by computer scientists for decades. All PRNGs have weaknesses, but some have more serious weaknesses than others.
The Contest was announced on 18 November 2015. Shortly afterward, a few people pointed out to me that the PRNG I had used might not be good enough. In particular, it might be possible for researchers to win the Contest by exploiting weaknesses in the PRNG. I have been persuaded that the risk might be greater than I had previously realized.
The purpose of the Contest is to test researchers' claimed capability to statistically analyze climatic data. If someone were to win the Contest by exploiting a PRNG weakness, that would not conform with the purpose of the Contest. Ergo, I regenerated the 1000 series using a stronger PRNG, together with some related changes.
Keenan says he changed the random number generator he used. That shouldn't be an issue, except notice that last phrase "together with some related changes." Those "related changes" were, "I changed the contest in a way which makes it more difficult to win, and I'm just not going to say so." That's fraud. You don't get to run a contest taking people's money then make secret changes to the contest to ensure nobody can beat it.
Watts, Montford and others refused to admit these changes were made. Since evidence wouldn't convince them, how about Keenan's own words? I don't mean the vague words I quote above. I mean the stark admission he changed the statistical properties of the data set he created. That's what we got yesterday.
Yesterday, Keenan posted the answer to his contest along with the code he used for it. I'm not going to discuss the code today as he used Maple Worksheets. I find those terribly obnoxious to work with. We don't need the code anyway. If you want to see it, you can find the code for the original data set here and the code for the new data set here. The latter has this commentary at the beginning:
There are two important points here. First, there is absolutely no evidence the randomness used in creating the 1,000 series used for Keenan's contest was vulnerable to any sort of attack. In Keenan's preening about how nobody won his contest, he says:
Acknowledgements. I thank Andrew Montford, for advice in developing the rules of the Contest. I am also grateful to Mike Haseler, who presented strong evidence for a flaw in the PRNG that was originally used to generate the series (for details, see his blog post “The Doug Keenan Challenge”); he also appears to be the only person who appreciated an aspect of the generated series, namely that the polynomially-decaying autocorrelations are accurate (unlike seemingly all such series in the peer-reviewed literature).
But the blog post in question doesn't say anything like what he claims. The blog post only argues the randomness used for the encryption Keenan applied to an answers file was vulnerable. It says nothing about the randomness used in creating the 1,000 series. It appears Keenan is just making things up, both in the commentary on his code and the commentary on his website, to give himself an excuse for changing his contest to ensure nobody could beat it.
I think that's pretty important. I don't think it's as important as this part though:
To address the problem, the 1000 series were regenerated, using a different PRNG, on 22 November 2015. Some other changes also had to be made to the generating program, because otherwise, if someone had been able to crack the original series, they could have determined the generating program, and thereby cracked the regenerated series. (The main change to the generating program was to revise the parameters in the ARMA submodel.)
Whether or not you accept this excuse (you shouldn't), what matters is Keenan openly states he changed the process by which he generated the data set used in his contest. Importantly, the "ARMA submodel" he refers to determines the autocorrelation (basically, self-similarity) of the series he created. That is the central element in determining how difficult his challenge would be to complete. Here, we have him admitting he changed it.
Will Anthony Watts or Andrew Montford say anything now? I don't know. Douglas Keenan committed fraud by altering his contest in a way which made it more difficult to win. He attempted to hide this fact from people, making excuses about RNG systems that had nothing to do with the changes he made. "Skeptics" accepted it, willfully disregarding any evidence showing Keenan made changes that made his contest more difficult. Now we have Keenan's own words saying he made these changes. What more do we need?
I don't know. What I do know is I checked the original data set Keenan used for his contest. I would have beat it. I would have won $100,000 if Keenan hadn't intentionally changed his contest after the fact to ensure nobody could win it.
Of course, Keenan never would have paid me. Keenan's a lying cheat who committed fraud with the blessing of multiple big name "skeptics" in what was nothing more than a stupid, rhetorical stunt. People who promoted this fraudulent contest, like Andrew Montford, Anthony Watts and Ross McKitrick, should publicly acknowledge this.
As a final note, my last post mentioned Keenan took in $540 for this contest for 54 entries. After writing that post, the list of entries shrank to only 33. I regret not making a copy of the original list, but I would be interested in knowing why it changed.