I'm going to try not to make this a "Gotcha!" post. I don't want to write a post just to say, "I told ya so." I hope you'll forgive me if there is just a bit of that to it though. A critic of my recent book was quite harsh in what he said about me and what I wrote, so the fact he's had an about face does encourage a bit of gloating.
I, of course, am referring to this post by the individual named Brandon Gates. It was a post which contained commentary like:
With the stage set, and my schadenfreude fed, I'll get on with it.
No wait, that's not the example I wanted to give. That was a remark I wanted to quote to hopefully earn some forgiveness for any pleasure I might show in this post. The actual examples are more like:
There aren't enough irony meters in the world....
Seriously, that's it. He expects what researchers say in private when putting together a study to stand, set in concrete, until the thing is complete, peer-reviewed and published.
For shit's sake. We'd still be using stone hammers if that's how scientific enquiry actually worked, if that.
The individual examples aren't important, though him doing things like seeming to suggest I engaged in criminal activity does tend to make discussions a bit awkward. What's important is the attitude present in that post seems completely absent in the newer post Gates has written, which he respectably links to in an update on the first post saying:
Since writing this post, I have reversed my position on the reliablilty of Cook (2013). The short version is that I no longer stand by its methods and don't have high confidence in its conclusions. The long version may be found here.
Which makes for an interesting about-face. I'll get back to this later though. Right now I'd like to try to focus on the productive part of our interactions. After all, that a strong critic of mine and I have managed to come to agree with one another on many points is a huge deal.
Before I say anything more though, a quick aside. The previous post Gates wrote is titled "The Difference Between Fraud and Farce." The new post is titled, "The Difference Beteween Fraud and Farce, Reflux." I've used this approach to titling posts before, and I'm not sure if Gates got the idea from me or just happened to come up with the same one, or if maybe he has a different idea in mind all together.
You see, when I titled my follow-up posts like this, I always used the word "Redux." Gates used the word "Reflux." I don't know if that's a typo or there's some meaning I'm missing. I know it's not important, but I wanted to make a note of it because I feel like I might just be missing some subtle joke, and it's bugging me. (I'm guessing it is just a "typo" since the title also says "Beteween.")
Anyway, on a topic which actually matters, Gates starts his post off with a summary:
Following are some issues that I have previously discounted, but which I now consider serious flaws. Detailed discussion of each, along with suggestions for improvements/alternatives, are in sections below the break:
AGW is inconsistently and therefore ambiguously defined across the eight endorsement categories. As well, it is vaguely defined in several endorsement categories.
The paper reports results in the abstract and body by combining dissimilar AGW definitions into consolidated endorsement buckets, and nowhere reports statistics at the higher detail level of the original eight endorsement categories.
I completely agree with both these points. I agree with much of what follows this, but I'd like to highlight a few particular excerpts that jumped out at me:
While it is arguable that the word "primary" often implies "most", in a complex system with multiple (often confounding) causal mechanisms, any single identifiable and quantifiable causal mechanism whose net percentage effect is greater than all others could be considered the primary causal mechanism.
As one point of contention in literature, which C13 attempts to address, is that human activity is not a dominant factor and/or accounts for <50% of observed warming trends over multi-decadal time periods, qualifiers such as "most" or quantified qualifiers, e.g. ">x%" seem more appropriate.
This is an issue I've mentioned before, but I've never focused on it because I know if I spend time discussing every issue I could find with the Cook et al. paper, it'd be too overwhelming (and would be portrayed as nitpicking). I'm glad Gates brought it up because I do think it is an important issue if one wants to properly figure out what levels of consensus there are on the various issues.
Categories 5-7 appear intended to be the mirror opposites of 3-1 (reverse order intended). However, as worded they are not the exact semantic opposites of their counterparts. Not only do these nuanced differences give more "wiggle room" for subjective interpretation by reviewers and the intended audience, it is inherently confusing due to the additional complexity.
Strictly interpreted, category 3 is not mutually exclusive with its counterpart 5. Were I to write, as in 3, "humans are causing global warming", it would be entirely valid for me to later argue that what actually intended to convey is that "humans are causing some global warming", or more specifically that "humans are causing <50% of global warming".
This is an issue I've spent some time discussing, and there's some interesting history stemming from it as an author of the paper used harsh rhetoric and basically said I was dishonest because I pointed out the lack of symmetry in the categories and consequently chose not to only compare 1-7, 2-6, 3-5. What makes that troubling (aside from the author being completely wrong) is the author himself has proposed making those sort of comparisons in private. That means he condemned me for doing what he himself had suggested be done.
That's all covered in the book though. Something not covered in the book is a post I wrote about this exact issue, titled "Why Symmetry is Bad." I wrote that post shortly after the paper came out, and I personally find it to be one of the most troubling aspects of this paper. The fact these categories are not symmetrical seems like such a fundamental design flaw I don't understand how it could happen. (It's worth pointing out both Gates and I have offered alternative categories for a rating system. I don't agree Gates's approach is symmetrical like he claims though.)
However, in the case of how C13 inconsistently defined AGW, I consider it inappropriate for the endorsement categories to have been consolidated at all. And more inappropriate that nowhere does C13 report statistics at the most granular level of the eight total endorsement categories.
I would consider the detailed reporting requirement even if the AGW definitions had been consistent because of the distinct qualitative difference between quantified and unquantified effects and explicit vs. implicit human causality.
I completely agree with Gates here. This is a point I've tried to draw attention to in pretty much all my criticisms of this paper.
As well, the C13 authors are known champions and defenders of "pro-AGW" climate literature who (rightfully so, in my opinion) advocate for policies designed to wean the world away from fossil fuels and toward less carbon-intensive alternatives. However, I think a compelling argument can be made that it appears the way C13 was designed and executed was to quantify the authors' own opinions about what literature says rather than be a dispassionate review of literature findings.
Or more simply, that C13 was designed -- deliberately or subconsciously -- to conform to some preconceived and too-broadly defined notion of there being a literature consensus that the letter "A" should precede "GW".
Yup. One point worth mentioning is the authors of Cook et al. made an online tool so the public could rate papers themselves to try to get an idea of the level of "consensus" and to see how their results would compare to that of the authors. They seem to not understand whether or not responses given under a particular rating system are biased doesn't mean much if the rating system itself is biased.
I think C13 is a flawed and should not have been published in its present form. While I would personally like to rely upon its results, I find that I cannot. Nor can I continue to defend its methods or findings as I have done in the recent and more distant past.
Since I am effectively calling C13 a bad paper, this raises the obvious question about whether I think the journal (IOPScience, Environmental Research Letters) should retract it. The facile answer to that question is that many "bad" papers are published and never retracted -- conventional wisdom holds that they simply don't get cited and fade into obscurity. However, according to ERL's own statistics, C13 has already been cited 32 times, though at least two of those citations are in later works by one or more of C13's listed authors.
I don't think I've ever called for this paper to be retracted, but I do agree with Gates that some serious action should be taken to address the flaws of the paper. He goes on to offer an alternative solution to retraction, and I think it would be a fine one, but the details of how the problem is fixed aren't that important. What is important is while Gates had previously defended this paper with vigor, he and I have now reached the point we both agree the paper is fundamentally flawed in a way that misleads the reader.
I also wish to make it clear that I my own anecdotal experience suggests that C13 is substantively correct to conclude that the majority of climate literature does indeed -- at least implicitly -- consider >50% of GMST increase since 1950 due to human causes (mostly in the form of CO2 emissions) to be essentially factual, if not a cause for concern with an appropriate call for reducing CO2 emissions by any and all reasonable means.
I'm not highlighting this as a point of agreement. I just want to point out my initial interest in this paper was largely cursory. My reactions to it have stemmed almost entirely from disbelief that a project so fundamentally flawed could get made, pass peer review, become widely publicized and be endorsed on a major level. If Cook et al had made a well-crafted study which reached similar results, I doubt I'd have had anything to say about it. (It's worth pointing out some other "consensus" studies I've looked at since examining this paper also suffer from significant flaws, which furthers my amazement at the situation.)
Addendum (3/23/2016 11:06 PM PDT)
I would be remiss to leave out that Brandon Shollenberger's latest e-book, and my subsequent discussions with him on his blog have been influential. My review of the book was hasty and and too-dismissive, which I regret. That said, I still cannot quite bring myself to endorse it either.
I think it also necessary to note how many of his arguments I have deliberately left out of my above critique of C13. Many of his arguments use materials and communications obtained from SkS servers which the C13 authors clearly would not have wanted published. I have read much of it, and it informs many of my above opinions even though I don't speak to them directly.
One reason why I left it out was because it's not clear to me that any or all of it was legally obtained, and I don't want the exposure.
The material from the Skeptical Science forum was definitely obtained illegally. Someone hacked into a server and stole it. It wasn't done by me though (despite Gates's apparent belief it was). I have no idea who did it, and it was years before I even obtained a copy of the material. By that point it had been shared by so many people there was no legal risk to anyone who might share it. That means it is perfectly safe to quote it as much as one might like. (I'm not criticizing his decision to not quote the material. I'm just trying to clarify things.)
In any event, this addendum to Gates's post is something I think he deserves credit for. I'd really like to end my post with it. I can't though. After I went to Gates's blog and looked at this post, I happened to check out another post of his. It deserves a more detailed response to highlight it's problems than I have space for in this post, and I may write about it later, but I want to highlight something that jumped out at me:
No consensus? Confused about what "consensus" means? Suck it Shollenberger. At least one oil company grokked it in the early 1980s. Wake up.
I had nothing to do with that post. I wouldn't have said a word about it at all. So while I'd like to end this post on a positive note about how Gates and I have reached an agreement on many issues, I can't. I tried, but I just cannot take the high road here. I mean, Gates was so wrong about this paper and so dismissive of what I said, that after seeing that I just have to be a little petty.
So Gates, you know that part where you made a huge fool of yourself by twisting into a pretzel to criticize me on points I was completely correct about? Yeah, suck it