Consensus Chart Craziness - Part 3

Our last post continued examining a very strange chart from a recent paper by John Cook of Skeptical Science and many other people. The chart ostensibly shows the consensus on global warming increases with a person's expertise. To "prove" this claim, Cook et al assigned "expertise" levels to a variety of "consensus estimates" they took from various papers. You can see the results in the chart below, which I've added lines to to show each category:

4_13_scaling_example

As you can see, the "consensus estimates" are all plotted one next to the other, without concern for how the categories are spaced. The result is Category 4 doesn't exist in the chart, and Category 5 covers more than half of the chart. This creates a vastly distorted impression of the results which, if displayed properly, would look something like:

4_13_scaling_proper

The last post in this little series shows there is even more wrong with the chart. Cook et al openly state the "expertise" values they used were assigned in a subjective manner, with no objective criteria or formal guidelines. In effect, they just assigned whatever "expertise" level they felt like to each "consensus estimate" then plotted the results in a chart where they chose not to show the expertise levels they had chosen.

The arbitrary assignment of these "expertise" levels is an interesting topic which I intend to cover in more detail in a future post as a number of the decisions they made are rather bizarre, but for today, I'd like to discuss something else Cook et al did. Namely, I'd like to discuss how Cook et al arbitrarily chose to exclude some results while including others.

This topic has come up before, but for today I am not going to look at which papers Cook et al did and did not include in their analysis. Instead, I'm just going to look at which results in some of the papers they examined they chose to shows and which ones they didn't show.

Doran & Zimmerman 2009

There are 16 points in this chart, taken from a total of ten different papers. For today, I'm going to look at two of those papers. The first of these papers is Doran & Zimmerman 2009. It is the source of data points DZ1, DZ2 and DZ3. DZ1 is shown as having the second lowest consensus value at 46.6% in Category 1. This data point is said to come from "Economic Geologists."

This is a peculiar entry. I hadn't heard of "Economic Geologists" before, and I couldn't think of any reason they would be signled out for a survey on global warming. The other two esitmates from Doran & Zimmerman 2009 are for "Meteorologists" and "Publishing climate scientists," groups whose expertise has a clear bearing on global warming. Economic geologists though? Very weird.

To try to figure out what is going on, we should read Doran & Zimmerman. It says:

With survey participants asked to select a single category, the most common areas of expertise reported were geochemistry (15.5%), geophysics (12%), and oceanography (10.5%). General geology, hydrology/hydrogeology, and paleontology each accounted for 5–7% of the total respondents. Approximately 5% of the respondents were climate scientists...

Clearly, economic geologists were not singled out for this survey. Why then, did Cook et al choose to single them out for reporting? A possible answer can be found in Doran & Zimmerman:

In our survey, the most specialized and knowledgeable respondents (with regard to climate change) are those who listed climate science as their area of expertise and who also have published more than 50% of their recent peer-reviewed papers on the subject of climate change (79 individuals in total). Of these specialists, 96.2% (76 of 79) answered “risen” to question 1 and 97.4% (75 of 77) answered yes to question 2.... The two areas of expertise in the survey with the smallest percentage of participants answering yes to question 2 were economic geology with 47% (48 of 103) and meteorology with 64% (23 of 36).

Doran & Zimmerman singled out economic geologists as having the lowest "consensus estimate." There were only 103 of them though, out of a total of 3,146 people. That means economic geologists made up only ~3% of the respondents, and they gave the lowest "consensus estimate." And they are the ones Cook et al singled out for inclusion in their paper, labeling this group as expertise Category 1.

Cook et al also show results for meteorologists (36 people) and results for publishing climate scientists who answered a particular question affirmatively (77 people). That totals up to 216 people. 3,146 people took this survey, and Cook et al only showed results for 216 (~7%) of them. And they conveniently picked one group (economic geologists) whose opinion holds no special value other than being the most extreme example they could show of non-experts having a lower consensus estimate than experts.

One could perhaps understand this decision as Doran & Zimmerman didn't publish results for each group in their paper. They only published the results for the three groups Cook et al used. Cook et al could perhaps have been forgiven as they might simply not have had access to any other results they could have included.

If this were simply a case of not having access to results, Cook et al should have warned people their sample was non-representative, but otherwise could be forgiven. However, the author list of the Cook et al paper is:

John Cook, Naomi Oreskes, Peter T Doran, William R L Anderegg, Bart Verheggen, Ed W Maibach, J Stuart Carlton, Stephan Lewandowsky, Andrew G Skuce, Sarah A Green

Peter T. Doran was the lead author of Doran & Zimmerman 2009. That means the authors of Cook et al (2016) had access to the results for each and every group that responded to the Doran & Zimmerman survey. They just chose to not include 93% of those responses, picking out only the ones which most supported their views.

Stenhouse et al 2014

Another three data points for this chart come from Stenhouse et al 2014. This includes the data point with the smallest value, S141 at 46.2% assigned to expertise Category 1. The group for this data point is "Non-publishers (climate science)." The other two data points are S142 for "Publishing (other)" and S143 for "Publishing climate." They are assigned to Category 3 and 5 respectively, with "consensus estimates" of 80.5% and 87.9%. These are taken from this table in the paper:

table

The values can be found in the first three columns by summing together the first two entries in each column. These entries are for responses to the prompt:

Is global warming (GW) happening? If so, what is its cause?

One strange aspect of this is the two answers summed together to get these results are, "Yes; Mostly human" and, "Yes; Equally human and natural." Combining these two categories means the consensus represented by these data points is not that humans are the main cause of global warming, but merely, that they are one of the main causes.

Cook et al don't explain why they chose to group these two categories, nor do they explain how that consensus position compares to that examined in other studies. This seems strange, as if they had only looked at the consensus given by Stenhouse et al (2014) on the idea humans are the main cause of global warming, their results would have dropped from 46.2%, 80.5% and 87.9% to 38%, 71% and 78%. That is not a trivial change.

There is a bigger issue though. These consensus estimates are only for three columns. This table contains nine columns for consensus estimates from various groups (plus a tenth for all groups combined). Cook et al simply chose not to report results for six of these groups.

Three of those groups are listed as having a field of expertise of "Meteorology & Atmospheric Science." Cook et al showed results for meteorologists from Doran & Zimmerman, so clearly, the views of these groups should be relevant (they are certainly more relevant than those of economic geologists). Why didn't Cook et al report them?

I don't know. What I do know is these categories include 61 people who have mostly published on climate change in the last five years, 501 people who have published some work on climate change in the last five years and 641 people who haven't published on climate change in the last five years. The consensus estimates given for these groups on the idea humans are the main cause of global warming is 61%, 57% and 35%. If we include the second answer, that natural effects have caused as much global warming as humans, those numbers go up to 71%, 67% and 46%. And that doesn't even address the remaining four columns, which provide information about views of people who are neither climate scientists nor meteorologists.

There is a wealth of information here. Cook et al ignore most of it. I have no explanation for why. What I do have is a problem. Cook et al rated the climate scientists of Column 2 as expertise Category 3, the same as meteorologists in Doran & Zimmerman. Why? Because the climate scientists hadn't published predominantly on climate change in the last five years. That got them rated as equal in expertise to meteorologists. The climate scientists of Column 3 were rated as expertise Category 1 because they hadn't published on climate change at all in the last 5 years. That put them equal in expertise to economic geologists.

Given that, what would Columns 4-6 be rated as? They are for people whose field of expertise is Meteorology & Atmospheric Science. Would they all be rated as expertise Category 3 like the meteorologists of Doran & Zimmerman, regardless of how much they had published on climate change in the last five years? Would a meteorologist who hasn't published on climate change in the last five years be put in Category 3 along with climate scientists who have only published a bit on climate change in the last 5 years? Or would this meteorologist be put in a lower category, unlike the meteorologists from Doran & Zimmerman 2009?

And what about the people whose field of expertise isn't climate science, meteorology or atmospheric science? It would take a little bit of arithmetic to separate out that group's responses given Columns 7-9, but it'd be easy enough to do. And even if it weren't, Edward Maibach is an author on both the Stenhouse et al and Cook et al papers. He could have obtained the results for those groups directly.

So why didn't he? Why didn't Cook et al show these results? I don't know. What I do know is if these results had been included, Cook et al would have had to change the expertise levels assigned to at least some of their data points if they wanted to have any sort of consistency. If they had done that, included these six extra results from Stenhouse et al and included who knows how many more results from Doran & Zimmerman, there's no telling what their chart would have looked like.

83 comments

  1. It's as if the newly added co-authors did not read the paper closely. Or did not care that it was of very low quality.

  2. I suspect the new authors were just added to the author list to pad it and make it look more impressive (and give them another paper for their resume). I doubt people like Peter T. Doran put any meaningful amount of work in on this paper.

  3. Ah, so "economic geologist" is simply a commercial geologist, working is seismic surveys, etc?

  4. Brandon:

    I suspect the new authors were just added to the author list to pad it and make it look more impressive (and give them another paper for their resume).

    That could well be true (Ken Rice is another one we could suspect hasn't really contributed much).

    I know these guys don't like to hear this but it is considered unethical to allow your name to be added to a paper that you did not make a significant contribution to. See for example this:

    The authorship of research publications should accurately reflect individuals’
    contributions to the work and its reporting.

    If you didn't really contribute, you shouldn't be on the author list.

  5. Carrick, yup. Interestingly, John Cook himself noted "skeptics" and "deniers" tend to write papers with fewer authors than people on the other side. I have to wonder if that's not because they rarely write these papers with 10+ authors where most of the authors didn't do anything more than sign their name to the paper. The worst example of that is the EOS paper something like 13 people co-authored as a response to Soon & Balinaus. The Climategate e-mails show most of them didn't do anything for the paper. They were just included to pad the number of people as a sign of solidarity. (Disturbingly, each criticism levied against Soon & Balinaus would have been more appropriately leveled against authors of the EOS paper. The solidarity they showed on the arguments was purely an illusion as many of the authors were being 4hugely hypocritical.)

    I don't know if Anders (Ken Rice) contributed much to the paper, but I do know at least some people in the Skeptical Science group have been pushing him to be more a part of their effort. He might actually be an active contributor to their group now. That doesn't mean he did enough work to justify an author credit on this paper. Given how little content the paper actually has, I suspect only two or three people actually should have been listed as authors.

    By the way, there was a topic in the Skeptical Science forum about how they should start publishing papers with a few members of their group on each paper to build up their resumes and get more papers out there. I should find a link to it because it was kind of interesting.

  6. Oh lordy. While skimming some topics half-heartedly looking for that thread, I found this one in which John Cook quotes Stephan Lewandowsky as saying:

    Bray also violated all internet survey methodological standards by not recording dates, times, and IP numbers of respondents (I know this from him personally). He thus has no way to check or verify the integrity of his data. In other words, the data are possibly (probably?) useless, although the published paper seems to include more data than the previous unpublished report which was entirely compromised as just stated.

    Now, all that said, the results are not particularly distressing from our perspective, and he correctly identifies that there is a large segment of the scientific community who think that the IPCC understated the problem.

    Overall, though, this study should not have been published without the authors demonstrating the integrity of their data—I doubt that they could.

    I don't know how I missed that before. It's priceless.

  7. Carrick,

    That could well be true (Ken Rice is another one we could suspect hasn't really contributed much).

    I know these guys don't like to hear this but it is considered unethical to allow your name to be added to a paper that you did not make a significant contribution to.

    Given that you are an academic yourself and should know better than to malign others without evidence, on what basis do you have this suspicion? Oh, hold on, you're commenting on a blog that has accused the authors of being liars, so it seems you have no problem with people making unfounded accusations of unethical behaviour in others. Carry on.

    Just to be clear, making an unfounded accusation of unethical behaviour (implied or otherwise) is - IMO - itself unethical. Maybe bear that in mind in future?

  8. Anders, please realize saying we can suspect something to be true is not accusing anybody of anything. As for your remark:

    Oh, hold on, you're commenting on a blog that has accused the authors of being liars, so it seems you have no problem with people making unfounded accusations of unethical behaviour in others. Carry on.

    There is nothing unfounded about the accusations I've made on this site. Ignoring the clear explanations people provide as justification for the accusations they make does not make those accusations unfounded. The explanations exist even if you choose to pretend they do not.

    That also applies to the multitude of problems I've pointed out with this chart. You can choose to ignore these, but that won't make them go away.

  9. Brandon,
    You explicitly called the authors liars. Let me explain a very basic concept to you. People who say things with which you disagree are not necessarily liars. People who get things wrong are not necessarily liars. Even if you believe very strongly that they have lied, it doesn't make them liars. A lie is intentional not a mistake. By calling the authors liars you have - IMO - excluded yourself from any further discussion about this paper with the authors. I probably shouldn't even bother pointing this out as I think I have done so before and you still appear to not understand this basic concept.

    Carrick, on the other hand, is - as far as I'm aware - an academic, As such he should be aware that accusations of unethical behaviour are serious. If he wants to suspect me of not having contributed, and then follows that with a claim that such behaviour is considered unethical, then he is implicitly making that accusation. I don't particularly care, but I do regard it as unethical to make such claims. This would seem rather ironic given that Carrick appears to think he is in a position to question the ethics of others. I fully expect him not to get the irony of this situation.

  10. Anders, I have no idea why you choose to write:

    You explicitly called the authors liars. Let me explain a very basic concept to you. People who say things with which you disagree are not necessarily liars. People who get things wrong are not necessarily liars. Even if you believe very strongly that they have lied, it doesn't make them liars. A lie is intentional not a mistake.

    I'm obviously aware of this. I rarely call people liars. I only do so when I have a clear and compelling reason which I provide along with my accusations. That people can choose to ignore the explanations and just whine about how I mistreated them by pointing out their dishonesty was in fact dishonest doesn't magically make their dishonesty not-dishonest.

    By calling the authors liars you have - IMO - excluded yourself from any further discussion about this paper with the authors. I probably shouldn't even bother pointing this out as I think I have done so before and you still appear to not understand this basic concept.

    If people feel being accused of dishonesty means they shouldn't respond to anything the person making such accusations say, they can choose not to. That's their call. However, there is nothing inherently wrong with calling a person dishonest so unless one can show the accusations were baseless or at least exaggerated, it's only going to appear to be a poor excuse used to avoid addressing things one doesn't want to address.

    Carrick, on the other hand, is - as far as I'm aware - an academic, As such he should be aware that accusations of unethical behaviour are serious. If he wants to suspect me of not having contributed, and then follows that with a claim that such behaviour is considered unethical, then he is implicitly making that accusation.

    This is contrary to what you've said in the past. On many occasions you've used constructions like Carrick's, and when people complained about your accusations, you responded by saying you didn't actually say it. I'm not sure what to make of that. What I am sure of is there is strong evidence some of the authors of this paper did not make any significant contributions to it. Given that, suggesting you might be one of them is hardly remarkable.

    I don't particularly care, but I do regard it as unethical to make such claims. This would seem rather ironic given that Carrick appears to think he is in a position to question the ethics of others. I fully expect him not to get the irony of this situation.

    If Carrick's suggestion is correct, there is clearly nothing unethical about him making it. If Carrick's suggestion is wrong but it is true other authors made no significant contribution to the paper, then his suggestion was reasonably-founded and also not unethical. The only reason it'd be unethical for Carrick to say what he said is if he had no basis for it at all, which isn't the case.

  11. Anders:

    Given that you are an academic yourself and should know better than to malign others without evidence, on what basis do you have this suspicion? Oh, hold on, you're commenting on a blog that has accused the authors of being liars, so it seems you have no problem with people making unfounded accusations of unethical behaviour in others. Carry on.

    Just to be clear, making an unfounded accusation of unethical behaviour (implied or otherwise) is - IMO - itself unethical. Maybe bear that in mind in future?

    Nice amount of bluster there, Anders. You might even fool your pet cat with that approach.

    Had you even a remedial reading capability, you would see I never made any accusations. Brandon made one suggestion, which I admit is very plausible:

    Here is the sum total of what I actually said:

    Brandon:

    I suspect the new authors were just added to the author list to pad it and make it look more impressive (and give them another paper for their resume).

    That could well be true (Ken Rice is another one we could suspect hasn't really contributed much).

    I know these guys don't like to hear this but it is considered unethical to allow your name to be added to a paper that you did not make a significant contribution to. See for example this:

    The authorship of research publications should accurately reflect individuals’
    contributions to the work and its reporting.

    If you didn't really contribute, you shouldn't be on the author list.

    As Brandon points out, and is obviously, this is a very short paper with very little no technical content.

    It's not even remotely plausible that it required so many authors to produce so little content.

    Moveover, technically it's a crappy paper. The one figure is deliberately misleading.

    So it's very plausible, as Brandon suggested (and I agreed), that there is authorship bloat here.

    Your background is astrophysics. You obviously aren't a practicing social scientist. So what exactly did you do on this paper? Read it for technical content? (That's not usually enough to get yourself anything more than an acknowledgement.)

    Since we're on the topic, the technical content paper is completely awful.

    I'm wondering if you even read it, or if your academic standards are really that low.

  12. I edited that sitting in an eye-doctors office. Some language changes:

    ======-

    As Brandon points out, and is obviously, this is a very short paper with very little or no technical content.

    [...]

    Since we're on the topic, the technical content of the paper is completely awful.

    =======

    I don't have any problems stating there is a consensus on climate change. I stated on Anders blog that I thought it was plausible that Cook 2013 understates the amount and strength of agreement.

    I realize that's a theme that Cook and Lewandowsky have both echoed. Too bad they are unable to competently make a reasonable argument.

    But this paper..about half of it is just a review. Including several paragraphs singling out Richard Tol for criticism. (Somebody has butt hurt here it seems.)

    There were really 17 authors, all of whom made a contribution significant enough to warrant authorship?

    Not very plausible.

    And that figure... Wow.

  13. Carrick,
    Thanks, that's a most useful couple of comments - although probably not in the way you might think. I'll leave it at that.

  14. I'll point out that Anders did not admit that he falsely characterized me initially.

    I'm still interested why it took 16 authors to write what is mostly a date entry paper.

  15. Brandon---Regarding Soon & Balinaus and the response, I assume you mean this paper.

    I'm not actually that familiar with that saga (in particular the climategate emails you were describing), so if you were up to it, it'd be a nice topic for a blog post.

    It required 13 authors to generate two pages of text and two figures.

    So arguably there's about twice the content of the paper that Anders (Ken Rice) was so hot under the collar that anybody would dare think was a bit over padded in authorship.

  16. Carrick, that's the one. I've written a little about it in the past (such as in this post and this one), but I haven't spent a lot of time on the matter. Perhaps I should have as it's remarkable for the demonstrable hypocrisy involved. This post by Steve McIntyre highlilghts a bit of it, but the Climategate e-mails show things are even worse than one might expect.

    Basically, everything the authors of the paper say about Soon & Baliunas is just as true for their work. To some extent, authors of the paper knew this. Similarly, they recognized S&B were right on some issues but refused to acknowledge such because they didn't want to give S&B credit. And perhaps most relevant to this discussion, the e-mails show a number of the authors signed onto the paper without contributing much of anything to it, with a couple not contributing anything at all. The best source I know for a discussion of this stuff can be found here:

    https://climateaudit.org/2011/11/25/behind-closed-doors-perpetuating-rubbish/

    But there is more material if one is willing to dig into the e-mails themselves. I might try to do so after I finish with this series on the Cook et al paper (or more specifically, its graphic), but no guarantees.

  17. Thanks Brandon.

    I read S&B along time ago without the benefit of the "cultural reference" provided by the group of 13.

    Again, my memory is pretty muddy on this, but Isn't part of what got people so hot under the collar was not what S&B in the paper, but what they said in their press release? (This is just from memory, and could be faulty.)

    I'm guessing that Michael Mann was hot and bothered over the suggestion that there really was a MWP. This was a while before before people started admitting that MBH had too little variance.

  18. Over on his own, Anders argues that Shollenberger's current comments are invalid because Brandon was naughty once.

    I'm sure that as a boy Ken Rice has nicked a cookie from the jar. We therefore should ignore his papers on disc accretion.

  19. Richard,
    That isn't really what I said, but I suspect you know that? I guess that this is the comment you're referring to.

    Feel free, of course, to ignore my papers on disc accretion. I can't really see why it would make much difference whether you ignored them, or not.

  20. This is what Anders said:

    IMO, anyone who starts with accusations of dishonesty, should not expect those he’s accused of lying to then engage with him. This is especially true if that person appears to then ignore their own lapses in integrity. Shollenberger is simply a mouthy blogger who has strong opinions and is virtually never swayed.

    Note, he doesn't say what my "own lapses in integrity" are. His constant resorting to vague accusations lets Anders smear me without having to actually make an argument or take a stand that anyone could judge. The reality is I don't think me calling the authors of this paper dishonest isn't why Anders won't respond to what I've said. He's come here and commented several times despite the accusation. Given what he's said, including how he's responded to me, the most likely explanations I can come up with for this are: 1) He knows I'm right about things and doesn't want to acknowleddge it; 2) He just wants to troll. I think the latter is more likely. His behavior in the last thread really does make it seem like he enjoys trolling.

    Regardless, that thread over there is depressing. I'm baffled Richard Tol is still promoting this same stupid argument. Basically, Tol is arguing authors' self-ratings of their full papers should match the Skeptical Science group's rating of the abstracts or it's wrong to compare the two. That's nonsense. It was nonsense a year ago when I spent a fair amount of time discussing it to no avail. The two data sets reflect different things so there is no reason to expect them to match. On the other hand, it is perfectly reasonable to compare reflections of different things to one another. It's why we can compare results from different papers which used different data sets. Tol is just being an idiot.

    I still say Cook et al could not ask for a better ally than Richard Tol. His constant stream of inane complaints allow them to pretend to have addressed the problems with their work simply by dealing with his absurd claims (he used to argue their sorted data set shouldn't show any pattern because unsorted data wouldn't have any). The reality is it is easy to see author self-ratings do nothing to address the central problem of this paper - that the rating system is horribly flawed. As long as you group Cook et al's Categories 1-3 together, you are guaranteed to get a ~97% consensus. It doesn't matter who does the ratings.

    The problem with this, which Tol's inane criticisms help Cook et al hide, is Categories 1-3 should not be grouped together. Categories 2 and 3 do not endorse the idea humans are the main cause of global warming, merely that they cause some amount of it. Given Categories 2 and 3 are agnostic on the idea of quantifying man's contribution, we cannot say whether they would rate the human contribution as greater than or less than 50%. That means Categories 2 and 3 endorse the idea humans cause some amount of global warming, but only Category 1 endorses the idea humans are the main cause of global warming.

    This basic point, which even the authors of the paper discussed in their forum (long before their paper was published), applies to the rating system they used as a whole, no matter who is applying it or to what. That people rating their full papers with this flawed rating system get similar results to people rating the abstracts of papers with this flawed rating system does nothing to suggest the rating system is okay. And if people like Richard Tol would have focused on real issues like this rather than the many imaginary ones he has come up with, things would be very different.

  21. Anders you haven't stated that you did contribute substantially to the paper, is that the case ?

    I have no reason to doubt, only asking.

  22. Brandon,

    He just wants to troll.

    In the sense that my intent is not to have any kind of serious discussion here, this is probably a fair assessment. Not really much point in doing anything else here.

    I still say Cook et al could not ask for a better ally than Richard Tol.

    Agreed. I think you help too.

    Morph,
    If someone wanted to know the answer to that question they could have asked politely, not suspected that I hadn't and that not doing so is regarded as unethical. If you have no reason to doubt, you have no reason to ask. You are aware of the "when did you stop beating your wife" fallacy? Also, given that your Twitter profile says "embra" I'm assuming you could ask me in person. You're welcome to do so.

  23. Morph,
    I realise, but you're on forum where the initial introduction of that issue was not. As I said, unless I'm mistaken, you could ask me in person. You are welcome to do so. That's a genuine and polite invitation.

  24. Anders:

    In the sense that my intent is not to have any kind of serious discussion here, this is probably a fair assessment. Not really much point in doing anything else here.

    In that case, my recommendation is: Leave. Going to people's blogs with no intent of even trying to have a real discussion is rude and obnoxious.

    If someone wanted to know the answer to that question they could have asked politely
    I asked politely.
    I realise, but you're on forum where the initial introduction of that issue was not.

    I believe in treating people the same wherever I go. It's sad you don't. That where a person asks a question determines whether or not you should respond to it is a bad sign as to what your goals are.

  25. Brandon,
    Just a thought. Maybe don't go around calling other people liars and maybe don't seriously discuss if others are behaving ethically or not, especially if you have no evidence whatsoever to suggest that they have not. Tends to tick people off, seems rather rude and obnoxious, and makes it rather unlikely that they'd have any interest in having a serious discussion with you. Worth a try?

  26. Anders:

    Just a thought. Maybe don't go around calling other people liars and maybe don't seriously discuss if others are behaving ethically or not, especially if you have no evidence whatsoever to suggest that they have not.

    I don't do either of these things. As I said above, I only do these sort of things when I have clear evidence to support what I say. Suggesting a person not make accusations without evidence while ignoring the evidence they provide is not going to accomplish anything.

    Tends to tick people off, seems rather rude and obnoxious, and makes it rather unlikely that they'd have any interest in having a serious discussion with you. Worth a try?

    If I had no evidence for the things I've said, you might have a point. That's not the case though. Pointing out dishonesty and wrongdoing while providing clear evidence which supports what you say is not rude or obnoxious. It may "tick people off" to have their wrongdoing pointed out, but that people would rather be able to hide what they do wrong doesn't mean people are obliged not to point it out.

  27. Brandon,
    It was just a thought. I didn't expect you to seriously consider it. Carry on.

  28. Anders, it was a thought predicated upon false premises. Posting it was no better than saying, "You should maybe consider not accusing people of dishonesty while freebasing Venusian cliffs for an audience of adoring Martians " I'm not going to do that, but I'm also not going to spend time seriously considering why I shouldn't do something I would never consider doing in the first place.

  29. I don't have a huge amount more I want to say about this but:

    • It is a logic fallacy to claim that somebody must be perfect before they can criticize someone else.

    • If a thing is true, it's true regardless of who said it.

    • When you put your name on a paper, you need to be able to defend why your name was on the paper. That's part of having a robust transparency in science. You need to be able to vigorously defend the paper, and be able to provide what your contribution was to the paper. You don't get to treat any of this like they are state secrets.

    • Claiming that people must ask you politely, in person, before you will divulge information you should freely be providing is a total copout.

    • It's completely reasonable to wonder why a paper that nearly 50% data entry, contains one badly defective figure, and has at most two pages of technical content required 16 people to write. It's reasonable to wonder what an astrophysicist who studies accretion disks brought to the table here.

    • It's within Anders rights to decide not to defend the paper, or to not defend his position on the paper. It's not within his rights to attack people for questioning what his role on the paper was, because this sort of questioning is within the normal practice of science.

    • It's totally within mine and anybody else's rights to wonder if there was not a substantial amount of authorship padding on this paper.

    •  It's totally within normal scientific practice to view Anders refusal to explain what his contributions were on this paper in an unfavorable way. He should provide that information when asked.

    [Okay, it's not really a point of wondering. I would guess that practically anybody who looks at this paper isn't going to question whether there was authorship— they are going to assume it actually happened.]

    I should say I've asked people what their role on a paper was. I've also been asked. It is a question that comes up.

  30. No edit key, no preview key. Needs new prescription for glasses. :-/

    [Okay, it's not really a point of wondering. I would guess that practically anybody who looks at this paper isn't going to question whether there was padding of the authorship— they are going to assume it actually happened.]

  31. I've been thinking about adding a preview feature along with a few other improvements to the blog. I know I ought to, but finding motivation for stuff that is largely cosmetic is tricky given I have two relatively large projects I'm working on right now related to the site (research for a new eBook and creating a resource I want to post on the site), and I still ought to be working on the death by police project I had started on (reviewing a database that is junk in a number of ways).

    I know I could just install a plugin and it'd probably be fine, but I hate adding third-party code to my server without reviewing it pretty thoroughly.

  32. I've gone ahead and installed a plugin to see how it works. I like it because the preview it gives is live, so you don't have to click on any buttons or anything like that. I actually had it installed before but never enabled it beyond some basic testing because I wanted to make sure it'd be stable. I know its styling is a bit off and it has issues with assigning comment numbers, but I think it should be good enough. Let me know if you have any problems with it. I can always look at tweaking the plugin to fix smaller things, and I can disable it if there are any larger problems.

    Oh, and just so you know, it requires Javascript to work. If you, like me, prefer to have Javascript disabled by default, you may need to whitelist this site if you want an Edit feature. I don't like forcing people to use Javascript to use the site, but with the Edit feature just being a bonus, not a necessity, I think it should be okay.

  33. Oh, sorry if I caused confusion. There's no edit feature with this plugin because having a live preview removes most of the need for an edit feature. I could add both features, but allowing users to edit their comments requires a plugin accessing/modifying the database. I'd have to thoroughly review the code for such a plugin before I added that to ensure it's done properly. A live preview just adds some Javascript functionality to the page people load, meaning it raises far fewer security concerns.

  34. Oh, and one thing I would recommend watching out for is performance. Having Javascript constantly running on a page consumes CPU cycles. I don't think this plugin should have any meaningful processing load, but I don't know for sure. If you guys notice the page being slow, let me know.

  35. Oh, OK Brandon, thanks, that clears it up. I was looking for an "edit" button such as appears at Lucia's after posting.
    I agree that preview removes most of the need, especially for HTML typos, omission of "http" in URLs, etc. On the other hand, one has to *think* to check the preview before posting. Plus sometimes you just don't "see" what you wrote until a few minutes later.

  36. Carrick,

    I should say I've asked people what their role on a paper was. I've also been asked. It is a question that comes up.

    Apologies, I didn't realise that you'd been suspected of not contributing to papers on which you are an author, by people who then point out that doing so is generally regarded as unethical. That's never happened to me before. If you are seriously interested in asking me a question in a normal scientific way, feel free to email me.

  37. HaroldW, I agree edit functions can be nice for catching mistakes you miss at first, but I've never liked how a person can respond to a comment only to find it has changed after they hit Post. We all make mistakes. There's no harm in just writing another comment to acknowledge one if it matters enough.

    Edit functions probably wouldn't bother me except I've had so many experiences where someone made a substantial edit to a comment without indicating it. It's so easy for it to be used in ways that cause problems.

  38. Anders, I've not had people suspect me of not contributing, mostly because I don't put my name in a list of 16 authors without having a visible and obvious role (like organizing a measurement campaign).

    But nonetheless I've been asked what my role was, and when asked, and I've always told them. I have an ethical obligation towards transparency to do so.

    Typically, when people get asked these sorts of questions, the asker is merely working out who did which part of the research. For example, if three people had a numerical model, whose model eventually got used helps you understand the research better.

    That said, I don't agree with the notion that you get to decide what the motivations are for why a person is asking a question. Motivations shouldn't matter for your response, just whether the question was itself appropriate or not.

    As I see if, if the question was appropriate---e.g., the question "what was your contribution to this paper?" is totally appropriate to me---you owe the person a response. It's part of what you signed up for when you agreed to put your name on the paper.

    If you had a legitimate role on the paper, as far as I can see, there's no reason to refuse to divulge this information. In fact, refusing to divulge information that you are ethically obligated to share is generally going to be viewed very negatively.

    We have here a paper with 16 authors with virtually no technical content. This certainly looks like one of those "letter to the editor" papers (like Mann's response to Soon & Baliunus) where people put their names on the paper more as a "I approve this message" than because they legitimately contributed to the paper.

    Those sorts of papers have no business in a science journal. Science is not a democracy. The result is no more likely to be true with 16 authors, two of whom did any work, than just having the two authors. People still do it. Editors tolerate it. But that doesn't make it right.

  39. Carrick,

    I've not had people suspect me of not contributing

    Until you've just done so, nor had I. Maybe doing so isn't really that normal? Certainly something I think I would rarely, if ever, do. My normal assumption is that others have behaved appropriately. Assuming otherwise without any actual evidence would - IMO - reflect more on the person doing the judging than on those they're choosing to judge. I also don't think that the number of authors, however long, short, simply, complex, the paper might be, counts - by itself - as evidence. So, I really don't think you're in a position to lecture me, or anyone, about appropriate ethical behaviour. Having said that, if you really would like to ask me a question, feel free to email me.

  40. Anders, Putting your name on a paper with 15 other authors, with very little technical content to spread around, then refusing to divulge what you've contributed that elevates you to a right to claim authorship… this makes you look really bad whether you like it or not.

    And yes, it actually does count on how long the paper is. Simply agreeing with a result does not automatically qualify you as being a legitimate contributor.

    Were I in your shoes, I would come out with an explanation of why this apparently simple paper needed a team of 16 crack scientists to launch.

    That's the last I'm saying on this.

  41. Carrick, glad to hear it's helpful!

    Anders:

    I also don't think that the number of authors, however long, short, simply, complex, the paper might be, counts - by itself - as evidence.

    The length and complexity of a paper determines the maximum amount of work which can go into the paper results. The total amount of work involved determines the amount of work each author could have done. If the length and complexity of a paper is small enough and/or the author list long enough, it can be impossible each author made a substantial contribution to the paper.

    Of course, that the results of some of the authors who signed onto this paper was inappropriately represented, if not simply misrepresented, is also pretty good evidence some authors did not make a substantial contribution. The only other conclusion is they did review what the paper said about their work and conclusions yet chose not to speak against the inappropriate representations.

  42. I find this sort of remark troubling:

    Having said that, if you really would like to ask me a question, feel free to email me.

    I can't imagine dictating where people should and should not talk about issues. As long as the topic is relevant and something one intends to discuss, I think they should be willing to discuss it wherever they're participating. Going to sites and telling people if they want to talk to you about things they have to do so elsewhere just seems rude.

    Which I guess isn't that surprising given Anders has already said he doesn't aim to have a serious discussion here. I guess he wants to come here and make comments that contribute nothing while telling people if they want to have a real discussion they need to go elsewhere...?

  43. Just as a heads up, I've received half a dozen alerts in the last few days because code I have running to monitor traffic for things like bots detected what it felt was suspicious activity. It appears someone has been reloading pages every few minutes for whatever reason, and that got flagged. I don't get that pattern of behavior, but since I don't think it is anything nefarious, I'm going to adjust some parameters. (If the page refreshing had been much quicker, the person would have been flagged as a bot and blocked instead of just having an alert sent to me.)

    Anyway, I don't think it should have any effect on users, but if I mess something up, comments might start landing in spam. If your comment doesn't show up right away, that might be why.

  44. Yeah, no. Don't post people's e-mail addresses here without their permission. I'll either redact them or delete such comments in the future.

  45. As a pure point of data the first use of the word "liar" in this thread occurred April 28, 2016 at 2:36 pm and was posted by a user under the handle of "...and Then There's Physics"

    Brandon (or admin with publishing power) used the STRING "liar" in head post as a part of the word "peculiar". A person suffering from conspiritorial idiotication or whatever the term of art is might have reacted to the subtle neuro-lingistic programming and taken offense. A neuro-normal reader, I believe, would not.

  46. On other threads, here and elsewhere (notably Jurry Cudy's) I have never noticed our host exhibiting tact in avoiding, or reluctance in using, the word "liar". Often in circumstances where I have responded by assuring him that neuro-normal readers allow for more "slack" in casual writing and phrasing than he typically expects of himself. For instance, Brandon accuses Mark Steyn of being a liar where I would infer much of Steyn's writing uses rhetorical devices to make a point humorous and memorable before making it strictly accurate.

    Perhaps, Professor Rice, you can address how your (and others') topic publication/chart here is intended to be understood, and by whom. Is it generally the case that data points are presented in categories side by side along the ordinal axis, creating an image within a category of increasing value? Is it generally the case your audience will expect more authors per page in certain disciplines than in others? Is it generally the case that authors-in-common in both a current paper and a paper cited within the current paper are bashful about sharing data from the earlier paper with their co-authors, or that co-authors are afraid to ask for such data, or that one team may decide that using "unpublished" data is some sort of abuse?

    Is there any way to assume your expertise in this paper is a matter of misunderstanding, in the same way Brandon apparently fails to understand, or at least appreciate, Steyn's humor?

  47. Pouncer, I'm afraid you're wrong on my use of the word "liar." I did use it in an earlier thread, even in its title.

    For instance, Brandon accuses Mark Steyn of being a liar where I would infer much of Steyn's writing uses rhetorical devices to make a point humorous and memorable before making it strictly accurate.
    ...
    Is there any way to assume your expertise in this paper is a matter of misunderstanding, in the same way Brandon apparently fails to understand, or at least appreciate, Steyn's humor?

    Given the many false things Mark Steyn has said, I don't know offhand which instance you're referring to. What I do know is Steyn either says things he knows are false or puts stupendously little effort into understanding things he makes serious accusations over, to the point he makes wildly untrue claims basedin no facts or evidence which are completely untrue. He also distorts and misrepresents quotes, as well as misattributing quotes to people who never said them and in at least one case fabricating a quote. And he does this while sneering at people for believing he might not represent quotes accurately.

    By the way, I assume you weren't referring to this thread of mine, but it is the first one which comes to mind when I think of Mark Steyn and lying. I challenge anyone to read it and think Steyn behaves in an honest manner.

  48. By the way, the reason I called the authors of this paper liars is simple and clear. A draft version of Cook et al (2016) cites Powell (2015) as making two central points:

    1) Cook et al (2013) were wrong.
    2) Richard Tol is wrong.

    Based upon these two points, Powell (2015) provided a consensus estimate which Cook et al (2016) argue is invalid. The final version of Cook et al (2016) cites Powell (2015) as making only one point:

    1) Richard Tol is wrong.

    The final version removes all mention of Powell (2015) saying Cook et al (2013) were wrong, as well as any mention of Powell (2015) having provided a consensus estimate. Despite this, Cook et al (2016) claims to examine "the available studies." This is clearly not true. Cook et al were aware of Powell (2015), as well as the fact the study it was based upon provided a consensus estimate and claimed Cook et al (2013) was wrong. They also must have thought Powell (2015) was a valid source since they relied upon it in their paper to support an argument they wanted to make - that Richard Tol is wrong.

    Intentionally citing one set of favorable results from a study while excluding other disfavorable results one is aware of is dishonest. Doing so whilie claiming to examine "the available studies" is lying. Anders may not like that I call this lying, but that's exactly what it is. If he thinks it isn't, he is free to challenge anything I've said. But simply whining that I said something he dislikes won't change anyone's mind about anything.

  49. @Brandon
    In fairness to Cook 2016, it may well be that they had written a response to Powell because they expected it to be published in ERL; and removed all reference to Powell when they learned that paper was rejected.

    Powell's paper is rubbish on par with Oreskes but without the Harvard affiliation.

  50. Richard Tol, if that were true, I wouldn't have criticized Cook et al (2016) over this matter. It's not true though. The final version of Cook et al (2016) cites Powell (2015) as proving you are wrong regarding the strenght of the consensus.

  51. @Brandon Shollenberger

    By the way, there was a topic in the Skeptical Science forum about how they should start publishing papers with a few members of their group on each paper to build up their resumes and get more papers out there. I should find a link to it because it was kind of interesting.

    You mean stuff like this?

    Ultimately, we would work in groups of say 5-10 authors per paper and hopefully have for each article some scientists who have experience with the peer reviewed literature

    I love that "hopefully ... some scientists" is ruefully dreamed about along with the utter implicit imperative that whatever happens the inflated numbers of 5-10 of "we" is the main criterion 😉

  52. tlitb1, I believe that is from the thread I had in mind. Do you have a link handy?

  53. The peer review process would appear to have a gap regarding the assignment of authorship. While it is possible that, in at least some cases, there could be unrevealed explanations for what appears to be a flaw in peer review, an explanation is not automatically an excuse for lax standards.

    It could be argued that the author of a previous study, upon which the current multi-author study relies, has sufficiently "contributed" to the current work to justify co-authorship. Yet that principle would seem to open most papers to claims for credit by un-named "co-authors" who published earlier, related work. Therefore, one reasonably expects all authors to contribute in some direct manner (providing new analysis of older databases, for example) rather than just providing access to the results or the database itself (the authorship/acknowledgement distinction).

    It is also reasonable, IMO, to apply common sense to the interpretation of such situations. In "real life," appeals to authority are common (and effective). Citing a larger number of "authorities" is considered even more effective. "9 out of 10 doctors agree…"

    This brings us back to the original purpose of the Cook papers which appear primarily intended as climate communications tools rather than social science research. Applying that filter to the observations about the papers does clarify some of the mystery surrounding why 16 authors would be allowed to sign on.

  54. tlitb1, thanks for htat link. Especially since it indirectly led me to re-visiting a thread that is relevant to Anders getting upset about me claiming Cook et al lied. I don't know if people responded to the thread after the forum was released, but this is interesting to me:

    As to the main question, Peter Gleick has obviously created "a bit of a stir" to say the least. Despite a lot of attention and discussion, it's my belief that the really interesting questions regarding Gleick and his transgression have gone largely unexplored; deprecations have been naive from the perspective of ethics and morality and with a complete, mile-wide miss by AGU as they've expressed themselves in their hastily written (and primarily fear-driven, I believe) press release and President's statements.

    Meanwhile, once again Professor Richard Lindzen has come up once more as a topic at RealClimate, thanks to his having once more delivered an underhanded accusation of scientific miscondut to climate modelers as well as delivering what might charitably be described as continuing to deliver slanted and truncated descriptions of climate science to yet another audience.

    To me, an interesting way of approaching this is to take integrate the two cases into an overarching examination from the perspective of a more sophisticated treatment, not focusing on the personalities and instead looking at their relative moral positions.

    There's a terrific book on the topic of when and how we may or may not lie, by an expert, Sissela Bok. "Lying: Moral Choice in Public and Private Life" explains that sometimes we may arguably be be morally compelled to lie, that doing otherwise is less "right" than the transgression of lying itself. How do Gleick and Lindzen fall on the scale of evaluation?

    Putting this into practical applicaton, AGU thoughtlessly demanded that its members adhere to "excellence and integrity" in all their activities having to do with the public and science communication, yet is not to be found censuring Richard Lindzen for his very long history of misleading the lay public on climate science. This is rather strange; Gleick's activity was peripherally connected with AGU and was not about science communications whereas Lindzen's bad work with the public is precisely in the domain described by AGU.

    This is Doug Bostrom, a central Skeptical Science team member, minimizing the illegal activity of Peter Gleick in which he committed mail fraud to obtain documents from the Heartland Institute, and when what he received wasn't "sexy" enough, forged another document in an attempt to deceive people Additionally, Gleick was a member of the very ethic panel Bostrom refers to while suggesting the AGU should have taken less interest in his illegal and dishonest activity. In this context, he even says it is sometimes right to lie, a point he has made elsewhere in defending/minimizing Gleick's wrondgoing.

    Accusations of dishonesty aren't as surprising when leveled against groups whose members defend dishonesty.

  55. opluso, there are many cases of people being made authors simply because their data was used. It's sometimes a condition people place on sharing their data. It's unethical, but it does happen. In this case, however, it appears the authors didn't even share their data. Cook et al (2016) only uses (some of the) results published in the papers themselves. None of the underlying data is used. This is particularly noteworthy for the Doran & Zimmerman paper as Cook et al (2016) only winds up displaying results for 216 of the 3,146 (~7%) respondents. If Peter T. Doran had at least contributed the full data set so Cook et al (2016) could use all of the data, not just ~7% of it, it would have made his contribution to the paper at least somewhat useful.

    Richard Tol, as far as I can tell that graph is BS. Many of the data points appear to be created by using comparisons that are completely inappropriate.

  56. Richard Tol, if and when you provide sufficient detail for people to actually check your work, I'll be specific. Until you actually provide the details necessary for people to know what you've done, there's just no reason to spend time trying to figure it all out. There are a few data points in the chart I can identify with certainty which are completely inappropriate, but I'm not going to waste my time trying to identify each point because you chose not to disclose necessary information for your work.

  57. As always, I ask people not to refer to me by my initials as I find the association they have unfortunate.

    As for your response Richard Tol, don't be silly. If you do not tell people where your data points came from in sufficient detail so they can examine your work, you are the problem. That people might choose not to spend much time examining unverifiable work is hardly surprising or remarkable.

  58. Brandon: You used "BS", not I.

    The graph uses the data in my published ERL comment. In most cases (10), Cook used the consensus rate and sample size including don't knows. In some cases (3; Cook, Rosenberg, Verheggen), Cook used the consensus rate and sample size excluding don't knows. In one case (Oreskes), Cook used the consensus rate excluding don't knows and the sample size including don't knows.

    Cook 2016 also seems to have misread the consensus rates in Stenhouse. Stenhouse says 52% and 78%, Cook says 73% and 93%.

    Carlton's text and tables disagree on the consensus rate: Tables say 90.4%, text 91.9%. Cook uses the number from the text.

  59. Anders offer an explanation for the Stenhouse' 78% v 93%:

    93% = 78% (humans contributed more than half) + 10% (human and natural causes about equal) + 5% (humans made some contribution)

    Cook's category 7:
    "7. Explicit rejection with quantification – Explicitly states that humans are causing less than half of global warming"

    In other words, someone who says "less than half" counts as a rejection in Cook and as an agreement in Stenhouse.

  60. Richard Tol:

    The graph uses the data in my published ERL comment.

    One generally provides a link to the location of the data being plotted when displaying charts. Regardless, that table shows you plot things like the consensus value for people who are "unconvinced of anthropogenic climate change" and the consensus rate for people "convinced of anthropogenic climate change." That is completely inappropriate. You can't filter out everybody who disagrees with a central premise of the "consensus" being measured then use the consensus rate of the people who remain.

    Similarly, many of the data points in your chart exist solely because you double, triple, or quadruple count the same data, causing you to arbitrarily give results from some studies more focus than others. Strangely, while you flood your table with many subsets from the same papers, you also exclude some obvious subsets with no stated reason. Plus you say things like:

    Cook 2016 also seems to have misread the consensus rates in Stenhouse. Stenhouse says 52% and 78%, Cook says 73% and 93%.

    Which strongl;y suggest you haven't even read Cook et al (2016) as it clearly explains how it gets these numbers. If you have read the paper, then you must just be willfully changing the definition of "consensus" from what was used to a different one to claim Cook et al (2016) "seems to have misread" something.

    Interesstingly, the values you cite aren't even the ones plotted in the Cook et al (2016) chart I've been criticizing. I planned to bring this up in a later post when I compare "consensus" positions of these papers, but an odd quirk is Cook et al (2016) used a different consensus position for Stenhouse et al (2014) in the body of their paper than they did in their Apendix and consequently their chart.

    Carlton's text and tables disagree on the consensus rate: Tables say 90.4%, text 91.9%. Cook uses the number from the text.

    Aye. The text claims 98.2% of people who believe temperatures have risen believe humans are the main cause, but Q4 gives 96.66%, not 98.2%. I'm not sure what's with that.

  61. Richard Tol:

    Anders offer an explanation for the Stenhouse' 78% v 93%:

    I don't know why you would turn to Anders for an explanation provided in Cook et al (2016) (and the paper it cites for the result). Simply reading the papers you're discussing would suffice.

    93% = 78% (humans contributed more than half) + 10% (human and natural causes about equal) + 5% (humans made some contribution)

    Cook's category 7:
    "7. Explicit rejection with quantification – Explicitly states that humans are causing less than half of global warming"

    In other words, someone who says "less than half" counts as a rejection in Cook and as an agreement in Stenhouse.

    There was no option for :"less than half" in the Stenhouse et al (2014) survey. I don't know why you think there was.

  62. Yes Richard Tol, I am aware of that. None of those fit your description of "less than half." One is more than half, another is half, and a third is "I don't know."

  63. Richard Tol:

    "mostly natural" is less than half,

    Yes, and "mostly natural" was not included in the consensus value.

    as is "some"

    No. Saying you believe humans have caused some portion of the observed warming, but you don't know how much is not saying humans have caused less than half of the observed warming.

  64. Like most words, the word "some" can be used in more than one way, but a common definition for it is "an unspecified amount or number of." That is how it was used here. Nothing suggests it was used to mean "a small share," a definition of "some" I've never even seen.

    (I have seen it defined as "at least a small amount or number of people or things," but the phrase "at least" in this definition makes it clear the amount can be more than a small amount.)

  65. I suspect the new authors were just added to the author list to pad it and make it look more impressiv

    Are there actually people out there who are gullible enough to think more authors = better research? On a 4-page paper?

    Haven't these people ever suffered through a movie that had three directors, four producers and a screenplay that changed hands six times?

    Whatever happened to the age of the auteur?

Leave a Reply

Your email address will not be published. Required fields are marked *