A New Secret Skeptical Science Paper and a New eBook

Hey guys. Today's post is an interesting one. As you guys may know, I've been accused of hacking Skeptical Science on occasion, and while it isn't true, I have had a history of finding things they post in publicly accessible locations which they would like nobody to see.

I've done that again. This time, I found a "CONFIDENTIAL" manuscript (archived here) the Skeptical Science group has apparently submitted for publication with a scientific journal. I don't know if the manuscript has been accepted, rejected or is still under review, but the fact they posted it in a publicly accessible location when it was supposed to be kept confidential is rather amusing.

I also found a copy of John Cook's PhD thesis (archived here), which I find incredibly lackluster. If it can earn him a PhD, then I don't think PhDs mean much of anything. I imagine he'll update it and improve it before actually submitting it, but I can't imagine any way in which it could be made not to... well, suck. And that's not just because he's wrong in a lot of what he says in it. Even if I agreed with his conclusions, I'd still say it was unimpressive.

In any event, this latest discovery has given me the motivation and material to finish an eBook I've been wanting to publish for a while now. You can find it here:

It's a bit more personal than the last two eBooks I wrote, as I was directly involved in much of what it discusses, but I'd like to think I found a good balance to keep it from just being a mini-biography. I hope you'll agree. If you don't want to risk 99 cents to find out, you can download a free PDF copy here.

Now like my last two eBooks, this one is ~10,000 words long, so it shouldn't take too long to get through. Unlike the last two, it doesn't really cover any technical subjects so it should be easier to follow (though I'd like to think the others were easy enough to follow). It also doesn't cover everything as there are tons of topics and points I'd have liked to discuss but only so much room. I'd like to think I hit the most important points though.

Of course, with me having only recently discovered the latest paper by the Skeptical Science group, this eBook doesn't cover all the issues it might have. Because of that, I highly recommend people check out the paper themselves. The author list alone should prove it will be interesting:

John Cook1,2,3, Naomi Oreskes4, Peter T. Doran5, William R. L. Anderegg6,7, Bart Verheggen8, Ed W. Maibach9, J. Stuart Carlton10, Stephan Lewandowsky11,2, Andrew G. Skuce13, Sarah A. Green12, Dana Nuccitelli3, Peter Jacobs9, Mark Richardson14, Bärbel Winkler3, Rob Painting3, Ken Rice15

The normal Cook et al group is there, but so are people like Ken Rice, also known as the blogger Anders and Stephan Lewandowsky, famous for finding global warming skeptics are conspiracy nuts by taking basically no data and just assuming the lack of data proved his preconceived beliefs.

But what makes this paper truly remarkable is what these people say. For instance, while the Skeptical Science group had previously portrayed their consensus findings as being based on people having only read the title and abstracts for various papers, this paper now admits:

During the rating process of C13, raters were presented only with the paper title and abstract to base their rating on. Tol (2015) queries what steps were taken to prevent raters from gathering additional information. While there was no practical way of preventing such an outcome, raters conducted further investigation by perusing the full paper on only a few occasions, usually to clarify ambiguous abstract language.

The raters cheated. They looked at information they weren't supposed to look at when doing their ratings. They openly discussed having done so in their forums, with neither John Cook nor any other author of the paper speaking up to say it was wrong. And then, for years, they pretended this never happened.

But now, they insist everything is okay because the raters only cheated a few times. They offer no evidence for this claim, and it would be completely impossible to know the claim is true. Even so, they want to publish this under with expectation people should just trust them.

Similarly, they both acknowledge and distort another issue:

Raters had access to a private discussion forum which was used to design the study, distribute rating guidelines and organise analysis and writing of the paper. As stated in C13: "some subjectivity is inherent in the abstract rating process. While criteria for determining ratings were defined prior to the rating period, some clarifications and amendments were required as specific situations presented themselves". These "specific situations" were raised in the forum.

The raters didn't just talk to one another about clarifications and amendments. That's an obvious misrepresentation anyone who actually read what they said to one another in their forums would know is false. On a number of occasions, raters simply asked one another how they would rate papers, not saying a word about wanting any standards or guidelines clarified.

But even with that distortion in place, this admission is huge. The original Skeptical Science consensus paper stressed that the raters were independent of one another. That's a huge stretch given they were all members of the same activist group, were mostly friends with one another and were in direct communication with one another. It's an impossible stretch, however, once you admit they were talking to one another about how to perform the ratings they were supposedly doing independently.

What's perhaps most interesting, however, is Table 1 of this new paper. It lists a number of papers supposedly finding a consensus on global warming, and in it, there is a column for "Definition of consensus." This would have been a perfect opportunity to highlight and contrast the various definitions of the global warming consensus, explicitly stating what Cook et al had found. It doesn't. Instead of giving any explicit definition, they just copy the rating categories:

1. Explicitly states that humans are
the primary cause of recent global
warming
2. Explicitly states humans are
causing global warming
3. Implies humans are causing global
warming.
4a. Does not address or mention the
cause of global warming
4b. Expresses position that human’s
role on recent global warming is
uncertain/undefined
5. Implies humans have had a
minimal impact on global warming
without saying so explicitly
6. Explicitly minimizes or rejects that
humans are causing global warming
7. Explicitly states that humans are
causing less than half of global
warming

Intentionally not explaining what consensus definition you get when you combine these categories. This is interesting mostly because if one looks at the rest of Table 1, they see no other paper gets a 97% consensus without using a weak definition or arbitrarily limiting what portions of its results to use. Instead, you get values as low as 40% or as high as 93%. In many ways, this paper shows there is no meaningful 97% consensus.

Of course, its authors would never say so. They'll try to spin everything they find to support their consensus message, even if that means trying to excuse what were basically lies about the methodology of papers. Cook's PhD thesis is perhaps worse, with it repeating a number of falsehoods and even re-using at least one quote he knows fully well has been fabricated.

But to be honest, the thing I find most fascinating is I found these documents in the exact same way I found the Skeptical Science consensus paper's data. The Skeptical Science group called me a criminal who had hacked into their server to get that data. If what I did then was hacking, why would they still allow anyone to do it and find new material? Why are they posting "CONFIDENTIAL" material in publicly accessible locations then handing out the URL to that material?

It's mind-boggling. I'm sure some people will claim I've "hacked" Skeptical Science again, but come on! It's been over a year since I described exactly how I found the secret material last time. Why can I still find more secret material in the exact same way?! That the consensus message is being crafted by people this incompetent is dumbfounding.

Anyway, feel free to give my new eBook a look and tell me what you think. It's fine even if you want to tell me it is complete garbage. I think most writers tell themselves the same thing plenty of times about most things they write.

37 comments

  1. Cook's PhD thesis is quite a find. From page iii:
    “It has also been a pleasure collaborating with my co-supervisor Ullrich Ecker. His advice, death threats, and friendship throughout the course of my PhD have been deeply appreciated, despite his research efforts to undermine most of the backfire effects listed in the Debunking Handbook.”

    ????

  2. That was definitely not a well-written paragraph. I get he was making a joke about his supervisor threatening to kill him as a way of providing motivation or whatnot, but... yeah, the writing there is not good. He probably means his supervisor tried to disprove the backfire effects, or something like that, as one would expect actually undermining backfire effects to be something Cook would like.

  3. Yes, I took it as a joke, like the photos in Nazi uniform. If you suffer from irrational fears of death threats, right wing conspiracies financed by Big Oil etc., then making fun of your fears is maybe the kind of thing a psychoanalyst would recommend. Leaving the stuff around for others to wonder at - I don't know. It seems we're all of us - sceptics, prime ministers and presidents (yes, they're mentioned in the intro to his paper) - involved in Cook's quest to find himself.

  4. You know, I was wondering which "presidents" he was referring to. I know Barack Obama referred to the paper (not in that tweet from the not-Obama account, but in a speech something like a year later).

  5. It would appear John Cook has taken the documents down now. I can't wait to hear about how horrible I am for sharing links to this CONFIDENTIAL material from the people who posted it on their website in a publicly accessible location.

  6. "That the consensus message is being crafted by people this incompetent is dumbfounding."

    Not really. Most moderately aware people understand consensus is a by-product of scientific research, not a goal. These folks are polemicists at heart.

  7. On the kindle book, the quotation font size is quite small. On my cheep android phone, I have to switch the font to helvetica to be able to increase the quotation size. Otherwise it stays at an unreadable microscopic size, even when the rest of the text is expanded. But thanks for publishing it. It's made a lot of things about Cook's study clearer.

  8. Your ebook is interesting and held my interest throughout, despite for the most part it revisits pretty much the entire saga of the 97 percent and the moon landing conspiracist thing (Recursive Fury).

    The author list is the "who's who" and nearly the entire priesthood of global warming.

    On the legality of scripting a page retrieval, I doubt it is illegal per se in the United States. The word for your technique is crawling and is done continuously by every search engine on Earth but it is remotely possible they are breaking what a prosecutor thinks is the law. Crawlers hit my server pretty hard sometimes and I block most of them just for that reason alone. If a crawler ignores my ban and changes IP address, then it *might* violate the law if it also violates other provisions of CFAA. In my case it won't because I conduct no interstate commerce, thus it isn't a "protected computer". Security is mine to consider, deliver and implement.

    https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

    The CFAA is pretty clear on its requirements of intention, unuathorization, and more importantly, what happens next. Is it a "protected computer"? Being an Australian web server, probably not. Did you damage the computer? No. Do you traffic in stolen passwords? No. But law can be stretched like latex rubber to fit.

    https://en.wikipedia.org/wiki/United_States_v._Swartz

    United States v. Aaron Swartz, 2011. Aaron Swartz allegedly entered an MIT wiring closet and set up a laptop to mass-download articles from JSTOR. He allegedly avoided various attempts by JSTOR and MIT to stop this, such as MAC address spoofing. He was indicted for violating CFAA provisions (a)(2), (a)(4), (c)(2)(B)(iii), (a)(5)(B), and (c)(4)(A)(i)(I),(VI).[12] The case was dismissed after Swartz committed suicide in January 2013.[13]

    This case is instructive. Aaron Swartz had a JSTOR account and was apparently entitled to any document and all documents. It isn't clear to me what law he violated. He annoyed JSTOR and MIT. He was legally entitled to download from JSTOR but may have violated their terms of service if those terms specified download limits. Being threatened with 35 years in prison for violating terms of service seems a bit extreme.

    JSTOR's side of the story is also instructive. It appears they blocked this downloading numerous times as it violated their terms of service and at times the downloading was heavy enough to impact other users.

    http://docs.jstor.org/summary.html

    Good security is not to put private stuff on a public facing webserver (duh). How hard is it to put stuff like that on DropBox and share it with your friends? Easy as pie, and free, too (up to a limit of course). Putting a document in the public area of a webserver is, to me, prima facie evidence of intent to distribute to the public, especially since the Archive crawler found it and made a copy.

    Public webservers can have private areas; I have it on mine in the global configuration file (no accidental delete of the .htaccess file for instance and inadvertent release of family photos). But even there it is possible to have hardlinks or maybe even symbolic links that expose the private document on the public side.

    Public servers can be geographically restricted by the firewall as I do mine. No international access.

  9. You say you're baffled by Cook's failure to try and stop you publishing private information. I wonder if it can't be linked with the emails previously released dating from 2010 in which he expresses his admiration for Lewandowsky, whom he'd just met, and talks about placing spam bots on blog and “stirring the ants nest”. It's all about provoking sceptics into doing stuff he can complain about later.

  10. Richard Tol:

    Thanks Brandon. Your hacker skills are unsurpassed.

    I really hope that's a joke. I mean, I'd assume it is, but... some people really are calling me a hacker for this sort of thing, so >.<

    What did you make S1.4 on the quantity of abstracts? That entire section seems void of information.

    I think the section shows exactly why you were wrong on that issue, even making points I made to you. Not only was it poor behavior of you to raise the criticisms you raised because you knew or should have known what you said was false, but it was also foolish because it gave such easy ammunition to Cook et al.

    Claiming the index values indicate there were more papers than claimed was stupid, and I really wish you would drop that argument.l

  11. Interesting. There doesn't seem to be anything wrong with the HTML of the above comment, yet it is displaying incorrectly. I'm not sure what's going on, but I couldn't even change the HTML to fix it. Very bizarre.

    Anyway, Canman, I'm sorry to hear about the font issues you ran into. I'm not sure what's going on with that. The quotes shouldn't be significantly smaller than the rest of the text. The regular text is set at 12 while the quotes are set at 11. The italicization is probably playing a role as well, but still, it shouldn't have made that much difference. If you look at the PDF version of the eBook, it shouldn't have the same issue. I'll try looking at the eBook on a few different devices to see if they have the same problem. If so, I can increase the font size for quotes in the eBook version but keep it the same in the PDF.

    On the issue of hacking, you've largely covered the important points. There isn't anything wrong scraping a website in and of itself. It becomes wrong if you hit the site so much it disrupts the site, but that's why I put a delay in between each of my requests. Similarly, it is wrong if you try to bypass some sort of access control where the server admin tries to stop you from accessing a resource. In the case you cite, JSTOR actively tried to block the person's access to prevent him from scraping their server. John Cook didn't do anything of the sort. Heck, he even knew I had done this same thing before, and yet, he didn't take steps to prevent it from being done again. You can't claim someone overcame an access control if you know they're accessing resources and don't even try to stop them..

    Heck, Cook could have even just sent me an e-mail telling me to stop. That might have been enough to make my activity illegal (though someone else could still do it). It would have certainly been enough to make me not do it again.

    By the way, it's funny you bring up the idea of using Dropbox to share documents privately. Not only could John Cook have done that, there are many links in the list I generated which are to files stored with Dropbox. Cook was clearly aware of how to share things securely. He just didn't even try. I bet the reason was he wanted to be able to send links to the documents to people he wanted to see them.

    Anyway, I'm glad you enjoyed the eBook. One of the reasons I didn't publish this sooner even though I had started working on it months back is I was hesitant to write something in the first person like this. I was really worried using the first person perspective so much would make it unappealing. It worked out well though. I don't think it'd have been anywhere near as interesting if I hadn't wait until now, with me finding the new paper.

  12. Yes, Brandon, it was a joke. You are a lousy hacker.

    I don't think the index number issue is as innocent as you and they make it sound.

    Data were downloaded in two batches, the second batch after the first batch was rated. There was overlap in the two queries -- note that Cook agrees with me until this point -- so the missing index numbers may be the papers from the first batch that also appeared in the second.

    This would be fine if it were not that the second batch was rated differently than the first.

  13. That's... not accurate at all. The missing index values do not line up with two batches overlapping like you claim. Their timing indicates what actually happened is when the SQL queries were run to add the information to the database, some information was added more than once. This is a common thing when updating databases, as if an upload gets interrupted, it is very easy for extra material to wind up in the database.

    (Though it is also pretty easy to avoid this problem by using transactions. I can't fault John Cook for not doing so though, as I wouldn't have myself. It's just not worth the trouble when dealing with a single-user database involving no critical information.)

  14. "some information was added more than once. "

    I set a primary key, spanning more than one column if necessary, and disable error reporting in the case of overlapped data. The duplicates simply don't go in and in that particular application I expect a large number of duplicates so I don't need reporting.

    If an auto-increment field exists, it will increment monotonically (no gaps in the sequence). Gaps indicate records were deleted.

    But that's for exact duplicates. If a paper is rated multiple times and the results differ even slightly it won't be a duplicate.

  15. They used the autoincrement field as the primary key, which is pretty normal. Some papers got duplicated, and they were then removed from the database. That caused there to be some gaps. I've had the exact same thing happen to myself plenty of times. There are many ways to avoid it, but it's often easiest not to bother as you can just run a check for duplicate entries after you've finished the upload.

    Also, you can check for duplicates based off any columns you want, so it'd be easy to check based just off something like the paper title. You wouldn't need to look at ratings, especially since you'd want to remove duplicates before ratings were done.

  16. Did they have duplicates after rating ? If the ratings were different what happened then ?

    Apologies if I have misunderstood.

  17. Richard Tol:

    “since you’d want to remove duplicates before ratings were done”
    Indeed. But is that what they did?

    There is no way to know it is with absolute certainty, but there is absolutely no reason to think it isn't. There isn't the slightest shred of evidence to suggest it isn't, and the authors have clearly indicated it isn't. Suggesting they're lying about something with absolutely no basis is... lame.

    Morph:

    Did they have duplicates after rating ? If the ratings were different what happened then ?

    Apologies if I have misunderstood.

    According to the authors, there were no duplicated papers by the time ratings began.

  18. I just shared this article on Facebook, but had to go through a "test" to do so. Facebook somehow thinks your website is insecure or something.

    I wonder if Cook et al have somehow have had Facebook flag your site?

  19. It is possible to influence Facebook (and other companies') views of sites to cause things like that, but I wouldn't jump to any conclusions. There can be a lot of other factors involved, and even if people do manage to do it, it may well not have been intentional.

    I know one big part of it, at least with Facebook, is based on the poster's behavior. For instance, sharing links from only a single site is more likely to get you flagged than if you share a single link from each of many sites. Similarly, the more people interact with the links to post, the less likely you will get flagged. Because of things like that, it'd be interesting to have someone else share this on Facebook and see if they have similar problems. If not, it might just be something about your behavior Facebook was suspicious of.

  20. Well.. I typically share a wide variety of links including news, science, and culture. On average, about 5 or 6 "shares" per day.

    It's odd in that when I first shared the link, some reported receiving a warning when clicking on the link while others reported no warning.

  21. I don't mind an off-topic comment. I've been sick this week so I haven't had the energy to work on much anyway. It's bad timing, really, given I just published this eBook, but apparently illnesses don't care about people's schedules.

    Anyway, that link is interesting, though I'm not sure it should surprise anyone. When you have competition for grants that isn't based on any objective criteria, being able to BS well will be an important skill. People may like to think scientists are above such things, but... they aren't.

    One thing that does surprise me a bit is people would be that open about it. I think that shows to them, this isn't a serious thing. While it is dishonest, to them, it is the sort of institutionalized dishonesty everybody expects, and thus, it isn't really "wrong." I find that mentality bizarre myself, but it is a common one.

    Interesting, there is something of a parallel with the Cook et al consensus paper. They rated papers as endorsing the consensus as long as those papers even acknowledged CO2 is a greenhouse gas. There were many papers they rated as endorsing the consensus which weren't about global warming at all, but simply mentioned that CO2 is a greenhouse gas. For instance, a paper about manufacturing some polymer described a process in which CO2 is used as one ingredient. After mentioning CO2, it basically said, "Which is the a greenhouse gas causing the majority of the planet's warming," getting it rated as an endorsement of the consensus.

    The only reason that statement was added to the paper's abstract was because global warming is a "sexy" topic right now. The authors likely don't care about global warming, and they certainly don't care about it in regard to their work. They just figured mentioning global warming like that would help them get more attention.

  22. By the way, I spent quite a bit of time reading the abstracts which "endorse" the "consensus" on global warming a while back. It's hilarious how little they actually say on the subject. I think I should actually go ahead and implement my system to do a public access rating system for them, just so people can see for themselves how ridiculous it is.

    I did test such a system a while back, but I didn't follow through on it because it didn't seem worthwhile. It seems so obvious to me these rating these abstracts is meaningless, but maybe it would be useful for people who aren't familiar with the topic. I don't know. It would at least give a counter-argument to the Skeptic Science group saying, "Well you could just rerate the papers yourself!" whenever someone points out they lied and used a meaningless rating scheme as though that rebuts anything their critics say.

    I don't know though. It'd be quite a bit of work to do, and I'm not getting compensated for this stuff like John Cook is. I'd do this sort of work right if it could mean getting a PhD, but I think we all know there's no way one could get a PhD analyzing this topic without a heavy bias.

  23. so let me get this straight, you don't like people 'stealing' your articles but you've no problem doing exactly the same?

  24. Lou Maytrees, you might find it helps to write out whole thoughts rather than just make vague accusations nobody can possibly understand. For instance, how is anyone supposed to know what you mean by me "doing exactly the same"? Are you saying I've previously complained about somebody downloading files I posted in a publicly accessible location? If so, you're wrong. If not, what could you possibly be referring to?

    I could probably guess what you're trying to get at, but it'd be helpful if you'd just say it.

  25. Richard Tol:

    @Brandon
    Note that Cook and co also have a system for rating the abstract by the public at large. It has been up for three years or so. No results were ever released.

    Well... no. They never said or even suggested they would ever release results from that. They specifically said that feature was there so people could see what their own ratings would be. It was never described as being a research project used to collect data.

  26. Canman, those rankings are pretty cool, but I should caution you once the surge in sales from the Watts Up With That post wear off, odds are my rankings will plummet. I don't think there will be any real sustained level of sales. Though of course, if other sites or people advertise the book, that may cause another surge 😛

    Though actually, the better the book does in the peak times probably will affect how well it does from then on to some extent as a higher ranking increases the chance of a person stumbling upon it. And I could always try to advertise the book myself, though I have no idea how I would actually do that.

  27. @Brandon
    I seem to recall that they set this up because they wanted to write a follow-up paper.

    Anyway, they have the data, never released, and they have faced lots of questions about reliability. So, if the data gathered through the public rating system confirms their original ratings, then they would have every reason to release it.

  28. I have no idea what would make you recall that as I've never seen anything to suggest it is true. And no, they wouldn't release the data as they don't inform people using the system their data will be used, much less ask the people to give their permission to use the data. Not seeking permission to use people's ratings would seem to make it clear they never intended to publish a paper using those ratings... as they wouldn't be allowed to.

Leave a Reply

Your email address will not be published. Required fields are marked *