2012-02-20 10:34:53Rating starts now! Post thoughts.comments on specific papers here
John Cook

john@skepticalscience...
130.102.158.12

Rating has now started - well, I've set the clock to start midnight tonight and we have 2 months to get through the ratings. The reason for the deadline is because I want to get our paper submitted before the IPCC AR5 deadline, on the slim chance that our paper might get included. But if the first journal we submit to rejects our paper, we'll have to resubmit which starts the process over again. So the quicker we rate papers, the better our chance of an accepted submission. So I don't recommend rushing the rating but I do recommend some small sense of urgency. You rate here:

http://www.skepticalscience.com/tcp.php?Action=Rate_Papers

As you encounter papers that you have questions about, not sure how to categorise it, post your questions in this thread and we'll clarify things as we go.

2012-02-20 12:31:54
Tom Curtis

t.r.curtis@gmail...
112.213.173.31

Giving the objective to being as close to double blind in methodology as possible, isn't in inappropriate to discuss papers on the forum until all the ratings are complete?

2012-02-20 12:39:51Well, discussing general issues more than specific papers
John Cook

john@skepticalscience...
130.102.158.12

Eg - "how do we categorise papers about carbon cycles - are they impacts or methods?"

or

"This paper talks about the 'atmospheric greenhouse effect' - is that an explicit or implicit endorsement?"

If any issues need clarification, I'll update our Guidelines which are available by mousing over the "mouseover to see guidelines..." text at http://www.skepticalscience.com/tcp.php?Action=Rate_Papers

The idea is that while we hashed out lots of definitions and guidelines in the initial discussion, as they say, battle plans all get thrown out the window once the battle starts (the saying is much more eloquent than that). Eg - there will be specific instances or types of research that we will need to determine what level of endorsement they entail.

2012-02-20 14:20:42For example
John Cook

john@skepticalscience...
130.102.158.12

I'm encountering a number of papers that discuss the global warming potential of other greenhouse gases like methane and CFCs. Does accepting the gw potential of other greenhouse gases impliticly endorse the consensus?

2012-02-20 18:15:15
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

I think if paper implies that greenhouse gases cause warming, then it is implicit endorsement of AGW. Sure there is the possibility that authors think that humans can't affect atmspheric GHG content but I think it's a quite slim chance.

2012-02-20 18:44:55
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

Having seen one GWP paper now myself, I add that in my opinion, if global warming potential paper is just a laboratory study and doesn't suggest that the gas in question has increased in atmosphere, then perhaps better to put it to neutral. Perhaps we should add a note on these papers ("global warming potential", for example), so we can revisit the issue to see how much of these cases there were.

2012-02-20 18:51:10
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

Has the quantification of >50% contribution from AGW has to be done with numbers? One paper said AGW was dominant forcing during last few decades, so I'm wondering if that can be counted as >50% quantification. I placed a note on that paper about this issue.

2012-02-20 18:52:51
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

Damn, I should read those guidelines more carefully, it says there that "dominant" is counted as >50% quantification. John, how can we correct our ratings afterwards?

2012-02-20 22:40:38
John Cook

john@skepticalscience...
121.222.175.176

Returning to the issue of greenhouse gases that aren't CO2, I'm leaning towards neutral - it's just a bit of a stretch saying a paper about mitigating CFCs is endorsing AGW. The goal isn't to plump the # of endorsements - it's okay if we play it safe and underestimate the # of endorsements.

Good idea also to add a note - will try to get into the habit of doing that.

Oh and Ari, you've really hit the ground running. Trying to set up an insurmountable lead too big even for Rob H?

2012-02-20 22:46:56
Ari Jokimäki

arijmaki@yahoo...
194.251.119.198

I doubt that this lead will last long.

2012-02-21 05:08:20
Riccardo

riccardoreitano@tiscali...
192.84.150.209

Taking advantage of the time zone, I'm third! :)

I noticed that the biology papers rarely mention anthropogenic warming and they end up in the neutral category. Though I'm sure that the interest on the impact of the changing climate is due to AGW.

One paper mentioned a forest as a sink of CO2, CH4 and N2O but did not say anything about the warming climate. Though, they quoted the global warming potential. I was in doubt. I guess I should not say what my final decision was.

2012-02-21 07:57:09
Riccardo

riccardoreitano@tiscali...
188.152.84.205

btw, no one sketic paper in 100 rated.

2012-02-21 09:17:13Dude, spoilers!
John Cook

john@skepticalscience...
121.222.175.176

Although 0 rejections in 100 papers is hardly a bombshell :-)

My general policy - if in doubt, rate neutral and sometimes record a note in the text box.

2012-02-21 10:47:58
Riccardo

riccardoreitano@tiscali...
188.152.84.205

This paper

"Applying The Science Of Communication To The Communication Of Science"

has no abstract. I didn't rate it but left a note.

 

Update: this one too

"Ad Hoc Committee On Global Climate Issues: Annual Report"

2012-02-23 19:34:48
Ari Jokimäki

arijmaki@yahoo...
194.251.119.197

I just ran into a mitigation paper that explicitly said that global warming and its attribution to humans are both debatable. Yet, according to the rules I had to rate it as implicit endorsement. I made a note on that.

2012-02-23 19:57:04Sorry Riccardo
John Cook

john@skepticalscience...
121.222.175.176
Leaving a note without a rating doesnt save anything to the database. The system only records an entry if you rate the paper.

There are, I think, around 100 papers with no abstract. Perhaps what should we do is crowdsource the remaining abstractless papers. There will probably be a few papers where we simply can't find the abstract. Perhaps those should be removed from the analysis. A few papers out of 12,000 is not significant.

Ari, my inclination is to update the guideline to "mitigation of CO2 emissions are implicit endorsement unless the 'vibe' is neutral". But there's a discussion with Riccardo on the other thread on whether we change direction mid-stream or not that is yet to be resolved.

2012-02-23 20:34:56
Ari Jokimäki

arijmaki@yahoo...
194.251.119.197

I have left some notes on some mitigation papers that I think would be better as neutral, so we can capture some of those afterwards.

For the papers without abstract I have done following: classify them as undecided and rate them as neutral + leave a note of missing abstract

2012-02-24 07:01:27
Andy S

skucea@telus...
209.121.15.232

I occasionally struggle with the disntiction between "implicit" and "neutral". From now on, whenever I see uncritical reference to Global Circulation Models or GHG/Global warming potential, I'm going to assume that that's implicit endorsement.

I'm still classifying most "impact" papers as neutral.

Mitigation papers are nearly always implicitly supportive of AGW, almost by definition.

Correct me if I'm wrong.

2012-02-24 07:03:19
Andy S

skucea@telus...
209.121.15.232

duplicate deleted

2012-02-24 08:41:14
Riccardo

riccardoreitano@tiscali...
2.33.129.146

I have problems with the more stringent definition of implicit endorsement when the authors use IPCC scenarios in GCMs. I think that using them is already an implicit endorsement, no need to explicitly endorse them.

 

Andy

I had a similar problem. My decision was that reference to GWP alone is not enough; it is if it's a paper on mitigation, which is often the case.

 

ADDED:

on emore thought on implicit endorsement. I tend to assume that when people talk about future/predicted/expected global warming they're implicitly endorsing AGW. Otherwise they have no way to now what the future climate will be.

2012-02-24 09:57:54
dana1981
Dana Nuccitelli
dana1981@yahoo...
64.129.227.4

Riccardo - I said the same thing (talking about future warming is implicit endorsement).  Ari disagreed.  John is on the fence.  I think we'll just have to hash it out in the disagreements at the end.

2012-02-24 10:06:59On the fence
John Cook

john@skepticalscience...
121.222.175.176

But leaning Ari's way. Better safe than sorry, I think. Perhaps take on a case by case basis. This isn't easy.

2012-02-24 18:53:56
Ari Jokimäki

arijmaki@yahoo...
194.251.119.197

It seems we have two threads in parallel both discussing almost same things.

2012-02-24 20:10:55
Riccardo

riccardoreitano@tiscali...
192.84.150.209

There we discuss the problem at large, here I just reported a potential problem asking for advice on how to deal with it. We need to follow the same set of rules.

2012-02-25 06:26:10
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

I'm having trouble with categories for some papers. Specifically:

1. papers on statistical analysis of trends (T or other) [methods?] that go on to conclude something [impacts? mitigation?]. 

2. Climate modeling [methods?], with analysis of impacts on regional/local weather or influenced by El Niño, monsoons, etc. [impacts?]

3. (similar to #1) sensitivity analysis: how much does change in X (fertilizer addition, ocean DMS emission, policy Z, turbine design, etc) affect projected warming? Some, but not all, are mitigation.

2012-02-25 07:55:43
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

There may now be enough ratings to see if we are statistically more or less on the same track. e.g. What is my rating distribution relative to the others? Or to the whole sample so far?

Or would that skew the results?

 

2012-02-25 09:43:41
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

Since I'm rolling along here, I'm concerned about whether I should be rejecting more papers as "not related to climate". I've only rejected a few as irrelevant (about equal to the number I've classified as deniers).  

That means that I rate studies on bird nesting or insect genetics, as long as they mention global warming in the abstract. I generally rate them "neutral" under "impacts".

 

2012-02-25 09:53:11
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

And I'm calling all life cycle analysis/impact articles "mitigation", and mainly "implicit endorsers", which agrees with Andy S above.

2012-02-25 10:04:45
John Cook

john@skepticalscience...
121.222.175.176

I've rated very few papers as "not related to climate" - they always manage to get climate in there somewhere. Stuff about bird nesting often involve some kind of climate science as it's the climate that drives the species impacts.

Re comparing our ratings to each other, Glenn's objections aside, there are only 5 papers with 2 ratings - so not many examples to directly compare against each other. Not sure if comparing average ratings will be that instructive at this point.

Re papers that involve methods but go on to conclude something, this is the relevant guideline:

If a paper describes methods but no actual results, assign it to methods. If it goes on to results, then assign it to whatever the results are relevant to (eg - impacts/mitigation/paleoclimate)

Re your question 3, hmm, will depend on the emphasis. If it's about mitigating X, then obviously mitigation. But if it's just looking at how X affects Y, possibly methods. If it doesn't obviously fall into mitigation or impacts, then methods is like the "everything else" box that all other papers get thrown into.

Oh and one other thing - you went past me overnight but just sneaked past you again this morning :-)

Update: scratch that,  you smoked me while I was typing this comment!

2012-02-25 10:42:57
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

And on the entertaining side:

"climate change has been implicated in mass mortailies of mail..."

2012-02-25 11:31:11
Sarah
Sarah Green
sarah@inlandsea...
173.174.112.26

John-

I'm done for a while, though; dinner time here. May not get back to it for a few days (despite the rather adictive nature of this activity).

 

2012-02-25 11:31:25
Andy S

skucea@telus...
209.121.15.232

I've been finding a few mitigation papers that I've been grading "neutral", especially when they talk about ozone or SO2.

I'm only grading mitigation papers "as implicit" if they talk about reducing GHGs and warming/climate change in the same breath or talk about GWP, but that's probably most of them.

There are so many papers that mention "climate change" and I just know that they mean anthropogenic CC, but since this distinction is not relevant to the paper they leave the key word out, so they get a "neutral"

2012-02-25 11:37:47Addictive nature of this activity
John Cook

john@skepticalscience...
121.222.175.176

LOL, I have this game on my iPhone, Temple Run, which I've gotten addicted to but I try to restrict it to just when I'm waiting for the bus, sitting at the train platform, during ad breaks. At least with TCP addiction, we're also building a long-term campaign to establish the scientific consensus in the public consciousness.

2012-02-26 06:05:47Trial and Error
BaerbelW

baerbel-for-350@email...
93.231.133.55

Okay, I'd like to help with this activity but am not sure how feasible that is given that I don't really read scientific papers on a regular basis. I have now rated 10 papers and would appreciate a second set of eyes to verify that I'm not completely off base with my ratings before I continue. John, is this something you can do as you are able to see who rated which papers?

2012-02-26 12:08:54
rustneversleeps
George Morrison
george.morrison2@sympatico...
174.91.126.125

The following paper does not display an abstract: Common Threads - Research Lessons From Acid-rain, Ozone Depletion, And Global Warming

2012-02-26 12:12:24Abstract free papers
John Cook

john@skepticalscience...
121.222.175.176

rust, I think for now, I will exclude abstract free papers for the moment and set up a page allowing us to crowd source tracking down abstracts from papers that WoS failed to include.

2012-02-27 03:54:19
Sarah
Sarah Green
sarah@inlandsea...
67.142.177.20

I've run into a handfull of papers with no abstract. I've put "NO abstract" in the comments for these.

Opinion, comment, or corresponance (letters) articles don't have abstracts in many publications. It may be appropriate to leave those out.

2012-02-27 04:09:30
Sarah
Sarah Green
sarah@inlandsea...
67.142.177.20

I'm calling papers on N in agricultural soil "mitigation" and mostly "implicit", as they are generally about changes in N2O and/or CH4, CO2 emissions.

2012-02-27 04:25:07
Sarah
Sarah Green
sarah@inlandsea...
67.142.177.20

I'm still stuck on modeling papers. 

Re papers that involve methods but go on to conclude something, this is the relevant guideline:

If a paper describes methods but no actual results, assign it to methods. If it goes on to results, then assign it to whatever the results are relevant to (eg - impacts/mitigation/paleoclimate)

For example, a modeling study that concludes that ENSO needs to be incorporated more explicitly in climate models. That idoes not fit in impact/mitigation/ or paleo. 

i guess I'll stick in methods.

2012-02-27 05:24:17
Sarah
Sarah Green
sarah@inlandsea...
67.142.177.20

I'm pretty sure I've reviewed some of the same abstracts twice. Either:

1. the ones I get are random and not filtered to avoid those I've already seen.

or

2. Some abstracts are in the database twice.

or

3. Some abstracts are really really similar (which is another problem altogether).

2012-02-27 07:04:31
logicman

logicman_alf@yahoo.co...
109.150.152.138

content free (no abstract):

Tropical Cyclones: Meteorological Aspects

2012-02-27 07:08:09
logicman

logicman_alf@yahoo.co...
109.150.152.138

another empty one

Global Warming Costs, Benefits Debated

2012-02-27 07:25:29
Andy S

skucea@telus...
209.121.15.232

When there is no abstract I classify it as "neutral" and usually the category is "undecided" unless it's obvious from the title. Then I put the comment "No abstract" in the "Notes" box.

Like Sarah, I sometimes get a "deja lu" feeling. But I'm not sure if that's real or just a side-effect of reading hundreds of abstracts. I'll maybe note the title when it happens so that John can check the database.

2012-02-27 08:05:17
Sarah
Sarah Green
sarah@inlandsea...
67.142.177.27

Abstract truncated:

Complex Climate Controls On 20th Century Oak Growth In Central-west Germany

Full version here:

http://treephys.oxfordjournals.org/content/29/1/39.short