2012-02-22 12:29:46Sorry John
Tom Curtis

t.r.curtis@gmail...
112.213.180.137

Under current instructions I am being forced to rate far too many papers as neutral when it is almost certain they accept the concensus, but do not mention "anthropogenic" or "GHG".  The result will be that the the ratings will underestimate support of the consensus to a level that amounts to distortion, IMO.  Of the last five papers I have rated, 3 where rated as neutral as per instructions, but which I believe to clearly have supported AGW.

 

In light of this, I feel I can no longer participate in this project.  I wish you and everybody else invovled good luck, and look forward to the final results.

2012-02-22 12:53:01Underestimate consensus
John Cook

john@skepticalscience...
130.102.158.12

Tom, I know how you feel as I felt some frustration when I first started rating papers, just knowing that some of them would endorse in the full paper but not in the abstract. Then I had an epiphany that set me at peace with rating papers neutral even though I knew they probably weren't. The epiphany was that the "rating by abstract" was only the first step in the TCP campaign.

By looking at only the abstracts, we get a sense of the level of endorsement - call it an imperfect proxy for consensus in the same way that sea ice extent is an imperfect proxy for ice volume or surface temperature is a noisy proxy for heat content. What I expect to find, from my initial reconnaissance, is the # of endorsements rising exponentially over time while the # of rejections flatlines. In other words, a strengthening consensus and a growing gap between mainstream science and denial.

However, I also expect to establish quantitatively by two independent means that our estimate of the strengthening consensus is an underestimate - both by comparing our "abstract rating" to the "scientist self-rating" and your idea of a subsampled "full paper rating". That means the result of a strengthening consensus will be robust and if it can be criticised, only for the fact that it underestimates the level of consensus.

Now my hope is that the message of a strengthening consensus makes a strong impact and a big splash and plan to network and schmooze this message out with every means at our disposal, including Peter Sinclair doing a video about the results and collaborating with Google to visualise our data (this collaboration has already begun). A strong impact will justify us going to the effort of launching "phase 3" of TCP which is publicly crowd sourcing reading the full papers of all the neutrally rated papers, to determine more accurately which papers endorse the consensus. As the crowd sourcing gradually sifts through the papers, the level of consensus will incrementally increase and we will slowly build over time a definitive, quantitative measure of consensus in the peer-reviewed literature.

By dragging this out over time, and dribbling new updates and announcements, we also get to repeatedly beat the drum of a strengthening consensus. This project is not intended as a one-off launch but a long-game strategy with the end goal being the term "strengthing consensus" achieving public consciousness. It's the ultimate counter-narrative to the increasingly used denier meme "the consensus is crumbling" or "scientists are mass-exodusing to skepticism".

The psychological research tells us that a key - a deal-breaker if you will - to the public accepting climate change is an accurate perception of the scientific consensus. If the public don't perceive a consensus, they won't support climate policy. But we know not only is there a consensus, it's getting stronger. This is a strong message and it is rarely presented and never quantified to my knowledge. So my hope is SkS can have a deep and lasting impact on the public perception of consensus which will make the path to climate action easier.

2012-02-22 13:43:22
Tom Curtis

t.r.curtis@gmail...
112.213.180.137

John, I understand your reasons, and wish you every success with the project.  I just cannot bring myself to continue rating as neutral papers I feel endorse the consensus.  Nor can I continue rating papers if I am using a different rule to everybody else.

On the plus side, my contribution to date has been minor, and you are well ahead of schedule, so I do not think I will be missed.

2012-02-22 14:41:19Clarification
John Cook

john@skepticalscience...
130.102.158.12

What do you mean by papers that clearly support AGW but don't according to our rules? Can you give a general example (not necessarily specific wording but the general gist)?

2012-02-22 15:25:44
dana1981
Dana Nuccitelli
dana1981@yahoo...
71.137.108.231

An implicit endorsement doesn't necessarily have to mention the words 'anthropogenic' or 'GHG'.  That's why it's implicit rather than explicit.

2012-02-22 15:47:40
Tom Curtis

t.r.curtis@gmail...
112.213.180.137

If you mouse of the instructions, it says that if a paper merely metions global climate change or global warming, that is not sufficient to imply anthropogenic global warming.

To me that means if it merely mentions CC or GW, then it must be classified as neutral; however my admitedly brief experience is that nearly all papers on impacts or mitigation mention CC or GW without specifying either anthropogenic, or GHG.  Hence a neutral classification even though I believe it is clear the auithors support the IPCC consensus.

2012-02-22 16:04:57
dana1981
Dana Nuccitelli
dana1981@yahoo...
71.137.108.231

That depends what else it says.  For example, I think if it talks about the impacts of impending global warming (or something similar), that's a clear implicit endorsement (as predictions of impending warming are contingent on AGW).  Though I'm waiting on a confirmation from that in the other discussion thread.

2012-02-22 16:06:03Mitigation papers
John Cook

john@skepticalscience...
130.102.158.12

There is a key issue here, Tom - we are clearly distinguishing between what scientists think/believe and what the research says. We do not concern ourselves with what scientists think in this study. This is not a survey of scientist opinion a la Doran et al 2009. All we are measuring is what the peer-reviewed research says. Specifically, the words they use in their published research. And that's powerful. When Rick Santorum talks about climate change being based on some dodgy studies, we can point to our database and say we have X studies across Y countries over 21 years - this is a substantial body of work, based on empirical evidence and published in peer-reviewed literature. We can supply quantitative numbers, we can use infographics to show the mountain of evidence. The emphasis will be on the research, not scientist opinions.

Note: if a mitigation paper is about reducing CO2 emissions, then it implicitly endorses AGW - why would you need to reduce CO2 emissions unless humans were causing CC/GW?

However, for other types of papers that are not about mitigation but mention CC or GW, then yes, for sure many of the authors support the IPCC consensus without saying so in the abstract. Why would they need to - the abstract is a very short, terse summary of what's novel about their research so why would they need to state the obvious? Similarly, many papers WILL endorse AGW in their full paper but not in the abstract so on that note, we will underestimate the level of consensus by limiting ourselves to the abstract. 

Consequently, we will very clearly communicate in our results several key points:

  • Neutral papers don't mean the scientists are neutral or that their research results are inconclusive about AGW - just that the focus of their research isn't specifically about the question of "are humans causing global warming?"
  • Depending on the results of our 2 independent checks, we will communicate that our result underestimates the level of consensus and yet still shows a strengthening consensus.

So you're right. We are underestimating the level of consensus. This will both strengthen our result and give us somewhere to go with Phase 3. These are positive things.

2012-02-22 16:24:53
John Cook

john@skepticalscience...
130.102.158.12

BTW, Tom, I hope you do continue to contribute to TCP because your input is greatly valued. I understand if not. I'm a big believer in this as a long-term project - while SkS has had a great impact over the last half-decade, much of what we do is reactionary and short-term. This is an attempt at a proactive, long-term project and I hope will deepen our impact and make some history along the way with the unique, crowd-sourced nature of what we're doing and the diverse ways we'll disseminate the results.

2012-02-22 19:59:24
Riccardo

riccardoreitano@tiscali...
2.33.129.146

I had the same problem with the rating but now I'm ok with it.
We all know that the opinion of the authors of many of the paper rated "neutral" is probbably in support of the AGW theory. If I want to know for sure I have to ask them, not deduce their opinion indirectly reading their paper. On the contrary, if I want to know what the science says I have to read the papers.
Mitigation papers clearly tell me that the starting point is the validity of the AGW theory, no questions about it. If a paper deals with the impact of temperature on something or analyses changes already occured in the environment "due to global warming", I can't tell. As for what I was looking for they do not provide informations, they don't count.
So, I think of the neutral rating as if I'm deleting the paper from the database or rating it irrelevant or undetermined. This is correct.

2012-02-22 20:56:27
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

My opinion even on mitigation papers are that they are not necessarily endorsements. For example, I have encountered several papers that take policy pressure as their starting point instead of validity of AGW theory. There are also papers that discuss some technology almost completely unrelated to the issue but they mention on the side that this technology might also help reduce GHG emissions without taking any stand to the question if there is a relation to temperature. This we currently have to rate as implicit endorsement.