What Should a Person Do?

For today's post, I'd like to ask people a question. I've been thinking about this question for the last few days, and I can't come up with a good answer. The question is this. Suppose a group of people published a scientific paper in which they said:

Powell’s main criticism of C13 is that 66.4% of the abstracts examined were rated as “no
position” and excluded from the consensus calculation. To count as an endorsement, C13
required that the abstract text refer to modern global warming or climate change and state, either
implicitly or explicitly, that humans are the main cause. It is true that many authors of those “no
position” abstracts may hold views that endorse AGW, but if the texts of their abstracts did not
provide evidence for this, no guess was made about their opinions. The “no position” abstracts
were therefore not used to calculate the consensus percentage.

Even though they knew beyond any doubt papers (or rather, their abstracts) were routinely rated as endorsing the "consensus" without expressing any opinion on what contribution humans might have had on global warming. Given this supposition, what should an onlooker do?

Before I go on, I should point out this quotation is taken from a pre-print of a paper. I have not yet seen the final, published version (as I don't care to spend $40+ out of my own pocket to do so). That means it is possible this text was changed before the paper went to print. This is, however, the text that was submitted for peer-review which it passed and ultimately accepted for publication by a scientific journal.

It is also text which is unquestionably false. That this was false was immediately apparent upon the (in)famous Cook et al. 2013 paper (which this new paper is referring back to) being published. Within a day of the paper being published, I noted:

A significant amount of information has come to light since I published that commentary, including remarks by authors of this paper while designing their rating system as well as remarks they made during the actual ratings explaining the reason for why they rated abstracts the way they did.

I'll cover some of that in this post, but it's a secondary issue. For today's question, I'd like people to suppose the evidence shows what I say, even if only as a hypothetical scenario. The reason is this question goes beyond this specific example. Whatever you feel about this particular situation, what should people do if they discover a group of people willfully publishing impoortant claims they know to be false in scientific papers?

I don't have an answer to this question. You can try filing a complaint with the journal that ran the untrue claims or file ethics complaints with universities and hope they will care about the dishonesty. They might. They might not.

You can try to draw the public's attention to the issue. That might work, but with how casually accusations of dishonesty are thrown around in polarized debates like the global warming one, who will pay attention? Will you be heard? Even if some people hear you, will it make any difference?

I don't have an answer to this question. Can you guys come up with one?


That's the end of the question portion of this post. What I'd like to do now is provide evidence of the authors' knowledge what they said in that paragraph was false. This won't be an exhaustive list of the available evidence, but it should give you some idea. Feel free to skip it if you are only interested in discussing this topic as a hypothetical.

To begin, it's important to look at what categories were used to try to classify papers as endorsing or not endorsing the "consensus" on global warming. In the words of Dana Nuccitelli, author of both Cook et al. (2013) and the newest paper:

1. Explicitly endorses and quantifies AGW as 50+% cause of the observed warming (or consistent with the IPCC, or something similar)
2. Explicitly endorses but does not quantify AGW
3. Implicitly endorses AGW (by definition does not quantify)
4. Neutral
5. Implicitly minimizes AGW (i.e. says the sun is playing a big role)
6. Explicitly minimizes AGW (less than 50%, less than IPCC, less than consensus, etc.)
7. Explicitly says there’s no anthropogenic effect on climate/temperature
For ‘humans are causing most of the warming’, #1 qualifies as an endorsement, while #5 through 7 are rejections.

For ‘humans are causing warming’, #1 through 3 are endorsements, while only #7 is a rejection.

Notice how Nuccitelli suggested examining two different positions that could be considered the "consensus" on global warming. One position was, "Humans are causing warming" without any quantification, meaning nothing more than humans are contributing some amount to global warming. The other position was, "Humans are causing most of the warming." This is the position the new paper claims was used. Nuccitelli understood the value of examining different consensus statements as he knew there would be differing levels of "consensus" on them. That is why he said:

They're not endorsing "the consensus position" - we have to demonstrate what the consensus is. They're endorsing the AGW theory in general without being specific about the human contribution to AGW.

This is the assumption problem I was talking about earlier. We can't assume that just because a paper says "anthropogenic global warming" that they agree the human contribution is >50%, but they have explcitly endorsed that humans are contributing. Thus they go in category #2.

The way I see the final paper is that we’ll conclude ‘There’s an x% consensus supporting the AGW theory, and y% explicitly put the human contribution at >50%’.

If we plug the results of Cook et al. (2013) into the formulation Nuccitelli envisioned, what we get is:

There’s a 97% consensus supporting the AGW theory, and 45% put the human contribution at >50%.

This is in the words of Nuccitelli himself, the person who came up with the rating system used for this study of the "consensus" on global warming. The only thing I have done is plug the numbers into the variables he gave. There should be nothing controversial about that.

Why didn't Cook et al. (2013) say there is a 97% consensus humans are contributing to global warming and a 45% consensus humans are the main cause? I can't say. While it's obvious saying there is only a 45% consensus humans are the main cause of global warming would have... caused some problems, whatever discussion that went into the decision not to publish that result isn't publicly available.

What is publicly available is discussions various members and authors of the Cook et al. (2013) study, seven of whom are co-authors of the newest paper (the one addition being Ken Rice, the blogger known as Anders). He is the only author of the recent paper who was not present for discussions of the rating system.

He was also not a participant in the ratings themselves. As such, we do not have access to any rater remarks from him. We do, however, have the remarks of other authors of the paper in which they explained why they rated specific papers the way they did. One author, Sarah Green, submitted a remark explaining she rated a paper as as endodrsing the consensus because:

'implicated in GW' is weak endorsement, but mitigation linked to climate = implicit

The paper in question, "Effect of encapsulated calcium carbide on dinitrogen, nitrous oxide, methane, and carbon dioxide emissions from flooded rice", includes this line in its abstract:

Emission of CH4, a gas implicated in global warming, can also be substantial in flooded rice.

CH4 is methane, a greenhouse gas. This paper takes note of the fact methane has been implicated in global warming. Given humans put methane into the atmosphere, I have no problem thinking this paper endorses the idea humans contribute to global warming. However, there is not even the slightest hint of quantification in this paper's abstract, and Green's remark during the rating process didn't suggest there was.

Andy Skuce, the lead author of the recent paper, submitted as his rater remark this line from the abstract of another paper:

,"addition of methane to the atmosphere warms the planet"

As his reason for saying it endorsed the consensus. Again, I have no problem with saying this is an endorsement of the idea humans contribute to global warming. It is just not an endorsement of the idea humans are the main cause of global warming.

Nuccitelli was right when he discussed separating out different "consensus positions." It might be useful to know how many scientific papers have indicated humans contribute to global warming in years past. It could certainly be useful to compare that to how many papers indicate humans are the main cause of global warming. That's what Cook et al. (2013) should have done. It should have said:

There’s a 97% consensus supporting the AGW theory, and 45% put the human contribution at >50%.

To count as an endorsement, C13 required that the abstract text refer to modern global warming or climate change and state, either implicitly or explicitly, that humans are the main cause.

And how should a person respond to them saying it?

(For those who want to examine what these people said and when, we have access to message board posts from the time leading up to the study and part of the period in which the study was conducted. A public copy of the forum is available here. Individual ratings, with remarks, can be found via this index.)

10 comments

  1. ...what should people do if they discover a group of people willfully publishing important claims they know to be false in scientific papers?

    Fair-minded editors who value debate above ideology would publish a short, succinct, fair-minded letter criticizing a flawed method. They wouldn't have to agree with the letter, only hold it to reasonable standards for any submission. Does the Bulletin of Science, Technology & Society have a fair-minded editor? I don't know. The response to submitting such a letter would tell you something.

    Or you could just walk away shaking your head.

  2. Gary, I agree with you about what a responsible editor would do. I don't know anything about that journal though. It might or might not behave responsibly. It's also possible the journal required the authors change that text before publishing the final version. I doubt it, but I do know it required them cut a lot of stuff out of the abstract (the pre-print's abstract is about twice as long), so I'm hesitant to say anything with certainty until I get a copy of the final version. I wonder if the authors of the paper would send me one. I know they're allowed to.

    Anyway, if this was in some inconsequential field nobody would pay attention to, I'd probably just send a letter to the editor noting the problem and leave it at that. I don't know if I can do that little in this case though. The fact this paper is going to be shared with and read by quite a few people and likely cited by even pundits/politicians makes me feel obliged to try a bit harder.

  3. There was one word changed in the published article for this paragraph: beginning of the third line "required" became "stipulated"

  4. Thanks Paul. I don't understand that change (how would the authors stipulate to an idea they are trying to test?), but it doesn't affect the issue this post is about. The statement is still wrong, the statement still exaggerates their findings, and the authors knew this when they wrote it.

  5. "The statement is still wrong, the statement still exaggerates their findings, and the authors knew this when they wrote it."

    - I think this covers it. You could also include JC's reaction to the infamous Obama tweet about CC being real, man made and
    dangerous as evidence of the professional character of the lead author. Personally I think the actual definition of what
    'The Consensus' *actually* is is deliberately nebulous to allow the sort of stats conflation in the MSM you have regularly written
    about.

    If they'd said:

    We examined ~12464 abstracts.
    Of those abstracts, 8264 contained no discernible position on AGW, 934 explicitly endorsed AGW with
    65 explicitly stating that man was responsible for greater than 50% of the observed warming trend.

    It'd probably get worse if we examined what a lot of these papers were actually about.

    Would the MSM have publicised these 'findings' in quite the same way?

    "0.52% of a convenience sample of published scientific abstracts agree. The observed warming
    signal in the GST record is mostly(>50%) attributable to anthropogenic factors".

    Not quite as catchy.

  6. This whole "consensus" genre of papers is just so silly. One wonders why someone like ATTP, who seems to be at least reasonably intelligent, could be so eager to hop onto the garbage scow. Is anyone aware of a similar set of papers in any other field? For example on dietary recommendations. I suspect that these other fields don't need to "manufacture" a consensus and perhaps couldn't do so anyway because the fields are so big and diverse.

  7. Quick note. I've been sick since for the last week or so after a minor respiratory reaction to a gas leak. It was nothing serious, but it has kept me from doing much for several days. I'm hoping this won't delay the publication of my eBook (which I plan to submit to Amazon for screening tomorrow), but I do have some formatting and image stuff to get done for it yet. Finalizing stuff like that may keep me from being very active for a bit longer.

    I'm hoping to post a semi-preview of one section as a blog post tonight, get the eBook submitted to Amazon tomorrow and work on the ethics complaint I plan to file for this paper over the next week or two. I don't think it'll accomplish anything, but who knows, maybe some journal/university will actually care about researchers willfully misrepresenting their work to exaggerate its findings.

    For something on topic, JonA, you say:

    If they'd said:

    We examined ~12464 abstracts.
    Of those abstracts, 8264 contained no discernible position on AGW, 934 explicitly endorsed AGW with
    65 explicitly stating that man was responsible for greater than 50% of the observed warming trend.

    An interesting thing about this new paper is it repeatedly refers to 11,944 papers/abstracts. The actual number is greater than that by several hundred. The authors have simply, and consistently, disregarded data for abstracts/papers they filtered out. I've pointed out this problem since they first published their data, data which excluded all the entries they filtered out. The authors didn't even publish a full list of papers they used for their study.

    One category for exclusion was a total joke as any paper which was "not climate related" was supposed to be filtered out even though hundreds, if not thousands, of papers not climate related were included. That's wrong, and if the authors had actually filtered out all papers not related to climate, their "consensus" result would have changed by quite a bit. Even if we ignore that though, it's not okay to hide data. If hundreds of papers were filtered out and people can't examine them to make sure that filtering was proper, that's a serious problem.

    The work going on in the study of the "consensus" on global warming is a train wreck. Practically everything in these papers by John Cook and his associates has been done wrong.

  8. > The work going on in the study of the "consensus" on global warming is a train wreck.
    > Practically everything in these papers by John Cook and his associates has been done wrong.

    I think even a moderately intelligent person, let alone anyone even passingly familiar with statistics
    would agree with this. Given this, the only reasonable (likely) conclusion was that it was a deliberate effort
    to misrepresent the state of the field to gain political traction. Alternatively, the authors are all completely
    incompetent and just got away with it.

  9. JonA, there are some things which I think were definitely deliberate (like the misrepresentation pointed out in this post), but I wouldn't be surprised if incompetence explained many of the misdeeds. It doesn't make a whole lot of difference to me. Whatever the cause for the Skeptical Science group consistently saying untrue things, the real problem is that they get away with it.

Leave a Reply

Your email address will not be published. Required fields are marked *