2010-08-24 20:58:45New rebuttal 121: Poptech list of 800 peer-reviewed skeptic papers
John Cook

john@skepticalscience...
124.187.125.135

I haven't written a rebuttal on this but just wanted to open up discussion on the best way to structure a 'definitive rebuttal' to the Poptech's list of 800 papers (apparently its been updated from 450). I think it's important to have a rebuttal because Poptech is now a regular at Skeptical Science and is frequently linking back to his blog post. So it would be handy to have a rebuttal to refer to, to keep the discussion on that topic contained in the one place rather than have him polluting other discussion threads with his off-topic comments.

So I've been thinking about how best to rebut the list as its a tricky thing to do. The fact that he's listed so many papers makes it impractical to go through the whole list, individually picking over each paper. So I think there are several approaches, or perhaps a combination of approaches to take - with the overall goal being to make his 800 paper list actually a teachable moment:

  • Rather than try to say none of the papers are skeptic, what about making a bee-line for the real peer-reviewed papers that are genuinely skeptical and highlight those? Point out that one paper says global warming is caused by El Nino, another says its CFCs, another says cosmic rays. Point out that there is no internal consistency among the many skeptic arguments - the only thing they have in common is they oppose man-made global warming but they give no coherent picture on what's causing global warming. Poptech's list is not about finding truth, it's a rhetorical technique to confuse and distort the truth.
  • Another approach might be to explain that science isn't about assembling lists of papers, it's about assessing the findings of each paper and integrating them into the full body of knowledge. Eg - each is a piece in a giant jigsaw puzzle and we need to extract meaning from each piece of research in order to build an understanding of climate works. Again, the theme that we need to work towards a truth, not employ propaganda.
  • Then there's the approach of other blogs who have individually nitpicked some of the papers on the list to show they're either not peer-reviewed or not skeptical. Here's what I've found so far (let me know if there are other critiques and I'll add them to the list):
    • Greenfyre's first post on the 450 list: mentions how Pielke himself has distanced his papers from the list, disses E&E as a non-peer reviewed journal, lists many skeptic papers that have been refuted, lists strawmen arguments (eg - papers that Poptech thinks refute AGW but don't) and so on. A great post, lists many different categories and sorts all the papers into these categories
    • Greenfyre's second post: More on Pielke, more on E&E, points out papers listed twice, a quote from a lead author of one of the papers and some quotes from Poptech
    • Roger Pielke's blog post where he distances himself from the list

So I thought I'd open it up for discussion. If any time throughout the discussion, any of you wordsmiths want to tackle writing the rebuttal, whether it be basic/intermediate/advanced, be my guest! In fact, even that might be something to discuss - perhaps the advanced can get into the nitty gritty Greenfyre style but the lower levels have less details about individual papers and more general principles.

2010-08-24 21:33:43Credibilty of journals?
BaerbelW

baerbel-for-350@email...
93.231.181.141
Whenever I see a group of 450 or 800 entities - whatever they might be - I'm trying to sort them into more manageable sub-groups. I'm not an expert but I for example notice that quite a lot of the papers seem to have been published in "Energy & Environment" which - from what I've read - isn't as credible as for example "Nature" or "Science". Could it therefore help to count how many papers made it into which journal and then contrast that to how well their respective peer-review process is considered to be? There might be other meaningful groupings which could be - and perhaps already were - applied.
2010-08-24 22:04:11
Ned

ned.flounders@yahoo...
129.170.23.6

John, that's a good idea.  Some further thoughts:

 (1) One approach might be to highlight a handful of papers that illustrate different types of flaws in the list.  For example, one case where a paper was only published in E&E or other junk journal, one where a paper was published in a real journal but has since been discovered to be flawed (e.g., McLean et al), one paper that is OK but where Poptech is misinterpreting or exaggerating the conclusions to make an ordinary paper seem "skeptic", etc.

(2) Another idea -- have a team of a few people look at a random sample of the list (say, 10, 20, or 40 papers total).

(3) Another approach that might conserve effort is to pick out five or six egregiously bad papers from the list, detail their shortcomings, and then say "With junk like this on the list, it's not worth the effort to examine all 800 papers."  If Poptech objects, ask him to identify the specific papers that he thinks are really, really good and we can then respond to them at an individual level.  In other words, refocus the debate away from an abstract "800 papers" and towards the quality (or lack thereof) of individual papers.

Should this rebuttal also deal with Poptech's (frantic) insistence that E&E really, really, really IS just like all the other journals and deserves to be taken just as seriously?  A lot of the papers on the list are from E&E.  Pointing out that it's a junk journal will probably lead to an extended angry outburst from Poptech.  But taking the E&E papers seriously and trying to individually diagnose the flaws in them could conceivably take a lot of time and effort.  

He also has some other lists, like one of the names of some large number of people who he claims are "skeptic scientists".   I pointed out problems with that list in this comment:

http://www.skepticalscience.com/news.php?p=4&t=288&&n=189#13475

and he gave his response in this one:

http://www.skepticalscience.com/news.php?p=4&t=288&&n=189#13528

Basically, he counts anybody who has even a B.Sc. or a M.S. as a "scientist" regardless of whether they have ever done any actual science in their life (for example, "Steve Milloy, B.A. Natural Science, M.S. Health Science").  Plus, a lot of the people on his list are dead, in some cases for many years -- is it really fair to assume that someone who was skeptical about climate change back in the mid-90s, when the satellite data erroneously showed a cooling trend, would still be skeptical today when  the evidence is much stronger?

2010-08-24 23:10:28Energy & Environment
John Cook

john@skepticalscience...
124.187.125.135

Some further thoughts on Ned's further thoughts:

  1. I like the idea of using illustrative examples. Greenfyre's first post is good in that it lays out a number of different categories (not peer-reviewed, refuted, not actually contradicting AGW). So perhaps the structure of the various rebuttals could take the form:
    Basic: mention the various categories of papers
    Intermediate: mention the various categories with 1 or 2 examples
    Advanced: mention categories with a longer list of examples
    Of course, it would be important in all that not only do you refute the listed papers but more importantly, you have to set a narrative at the start of each rebuttal (eg - the 'no internal consistency' or 'rhetorics vs science' themes) and conclude by reinforcing the narrative.
  2. I would say before we reinvent the wheel, that we look at what Greenfyre and others have done as they have addressed many of the papers in the list. Perhaps some zealous person would be keen enough to build a spreadsheet of the paper list but they would be a hardier soul than I. Is this the kind of thing that can be collaboratively done in Google docs?
  3. Re picking 5 or 6 papers, I would use this approach for the intermediate version. Whether we go on to do a more thorough Advanced version is optional.
  4. I'm thinking an indepth look at E&E might be suited to either the intermediate or advanced level. I wonder if making it all about E&E would be a distraction though - the central issue here is the rhetorical arguments employed by Poptech - the strawmen, the listing of refuted papers. It's important not to get bogged down in details but to expose his rhetorical techniques.
  5. Re Poptech's lists of skeptic scientists, well, I guess that's a whole other rebuttal.
2010-08-25 03:41:18
Ari Jokimäki

arijmaki@yahoo...
91.154.102.68

I started converting this list today to Excel-sheet. It's easier to sort in different ways there. It's quite slow to convert though, I have handled only about 150 items of the list so far.

One thing I will be looking is the temporal distribution of the papers published in proper journals. Initial feeling on the publication years suggests that there are more old papers than new ones.

One point about this list is that 800 papers is not that much. I have some climate science journals in my google RSS-feed reader and it's not uncommon that there's about 50 new papers per day (that's 800 papers per about two weeks). Most of these are AGW supporting papers. One could for example see how many papers Journal of Climate has published in some year and see how many papers are in this from that year to estimate the percentage of these papers from the total.

2010-08-25 06:02:57Comment
Robert Way

robert_way19@hotmail...
142.162.177.145
- A good strategy would be to categorize each by journal in excel. That would take time but specifically one can use the ctrl-f button and search every paper listed in science, nature, nature geoscience, geophysical research letters...etc... all the reputable journals and list the number of ones in reputable journals. Secondly one can probably find a program online for counting how many times a particular sentence is shown on a website (then search energy and environment for example).
Finally, I've looked at a few of the papers and know the conclusions are miscontrued. I can detail which ones. Finally I know one of the professors very well on the list. Professor (Paleoclimatology) Andre Viau. I could ask him whether or not he would be willing to comment ( he has in the past been frustrated by having his paper included on the list).
2010-08-25 10:08:43Go for the Weakest Link
MattJ
Matthew Johnson
mej1960@yahoo...
64.105.34.176

Hi, John-

I think you have already identified the weakest link in his convoluted chain: the claim that all 800 are in fact, 'peer-reviewed'. There is no need to go down the whole list and show that he has misinterpreted the facts of the case on each one. His whole argument is based on, "Look: they say there are none, but I have 800 here to prove them wrong".

That is, once you cast legitimate doubt on his claim of 800, you cast legitimate doubt on his whole claim. But this has already been done with"Greenfyre's first post on the 450 list".

The hard part comes if/when he removes those few you look at, and then claims victory, claiming "all criticisms refuted". I don't think there is any way to tackle this in one post. They will simply tweak their argument and return to crowing victory over and over.

It is an example of the fallacy known as "special pleading" combined with "moving the goalpost". In fact, Fallacy Files lists the latter as a 'subfallacy' of the former.

Still, I can't help but wonder if this illustrates the fundamental weakness of the peer-review process.  Whenever legitimate scientists claim such-and-such is "not peer reviewed", anti-scientists can set up their own "peer-review". They then simulate the process well enough to fool even some experts, (Freeman Dyson?) not to mention laymen.

AGW denialists are not the only ones. I am thinking of Chiropractic as well (Creationists have been less successful at this simulation).

2010-08-25 12:14:10Comment
Robert Way

robert_way19@hotmail...
142.162.177.145
Id really like to see the intermediate to advanced versions be kicked in first. This guy is annoying me with his constant posting. I looked at his website includes 134 citations for Energy and Environment for one... Regardless. We have to do this right and not give him any way of taking away from our arguments. We find his responses to previous rebuttals and use them to our advantage.
2010-08-25 15:40:17
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

Robert: "- A good strategy would be to categorize each by journal in excel."

Yes, my sheet includes a column for journal.

2010-08-25 15:55:15Ari
John Cook

john@skepticalscience...
124.187.125.135

Ari, if there's a way for the community of authors to assist you with your spreadsheet work, please let us know.

In the spreadsheet, can you sort the papers into different categories? I'm thinking something like what Greenfyre does in his first post, categories that are something like this:

  • Not peer-reviewed
  • Refuted papers
  • Misrepresenting the paper's results
  • Contradicting each other
  • And so on...

In fact, we possibly could even include a pie chart in the rebuttals, especially for the lower levels. I think the key message from our rebuttal is not nitpicking whether this or that paper is peer-reviewed. We don't want to get bogged down in details. I think Poptech's list is actually a great embodiment of the whole skeptic movement in the one page and an opportunity to use him as both a cautionary tale and a teachable moment. In the one blog post, he employs strawmen, misunderstands fundamental science (past climate change disproves AGW), papers contradict each other, cites refuted old science, etc. The key take-home point is that Poptech's list is not about increasing our understanding of climate, it's about spreading confusion through a message with no internal consistency or coherency. I'm working on the intermediate rebuttal and that's the general approach I'm taking - to reframe the argument rather than get into a bickering about Energy & Environment.

Re Robert's suggestion of having the intermediate/advanced versions first, I totally agree. I'm thinking we either roll them out one at a time or possibly launch all 3 at once. Eg - publish the 3 levels of rebuttals and I publish a blog post that kind of summarises all three in some way, pointing to each one to show what you'll find. Whichever has the most impact, I guess. Kind of like the idea of doing all 3 at once and making that the story of the blog post, it's somwhat of a new paradigm in rebuttal/science communication.

2010-08-25 17:21:13
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

"Ari, if there's a way for the community of authors to assist you with your spreadsheet work, please let us know."

Well, I can send you the unfinished spreadsheet, and we can assign different sections to different people. However, I can finish this in a couple of days. I can even rush it a bit if you feel it's important to get finished fast.

"In the spreadsheet, can you sort the papers into different categories?"

Yes, but of course only after the papers have been categorized. We just need to add a category column to the sheet. Current columns I have are:

- Number (I'm adding this for each item for a easy-to-use id)
- Name of the paper
- Authors (short form - several authors are given as Firstauthor et al.)
- Publication year
- Journal
- Status (published/submitted)

2010-08-25 21:14:43
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

I have already handled 600 papers, so I expect to finish the spreadsheet tomorrow.

2010-08-25 21:48:25One or multiple columns for category?
BaerbelW

baerbel-for-350@email...
109.85.5.228

Hi Ari,

here is a suggestion about this part:

"In the spreadsheet, can you sort the papers into different categories?"

Yes, but of course only after the papers have been categorized. We just need to add a category column to the sheet. .."

Wouldn't it be better and more flexible to have one column for each category? I have a hunch that there might be papers which fall into more than one category and that could be more easily shown if all applicable categories could simply be checked (and the paper with most categories checked, wins!). If later additional categories come up, it would only make the addition of columns necessary.

I'm not an Excel-expert but I've dabbled a bit with things like pivot-tables and -charts, so please let me know if I can help with that.

Cheers
Baerbel

2010-08-25 21:53:50
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202
Yes, Baerbel, we can do it that way too. :)
2010-08-25 21:55:48Ari's 600 papers
John Cook

john@skepticalscience...
124.187.125.135
Ari, you are a machine! What do you think of the idea of featuring the spreadsheet in the Advanced version? I'm not sure what form the advanced rebuttal should take, how detailed it should get. Perhaps it could be considered a summary of the spreadsheet. Eg - consider the spreadsheet the raw data, the rebuttal is like a paper expounding on the results.
2010-08-25 22:07:06File-Repository?
BaerbelW

baerbel-for-350@email...
109.85.5.228

Could it make sense to have an upload/download area - which is only available for authors / translators - for items like this spreadsheet  while things are worked out? It could be similar to uploading pictures but we'd also need a directory to go to where we can then find the items. Not sure how feasible this is security-wise, though....

Cheers
Baerbel

2010-08-25 23:23:19
Ari Jokimäki

arijmaki@yahoo...
91.154.102.68

Sorry, I think I wasn't clear. The spreadsheet I'm creating is so far just a conversion from the HTML-formatted list to Excel format. When I say that I have handled 600 papers, I mean that I have converted the format of those papers to spreadsheet format, so I haven't included any rebuttals to each paper and the spreadsheet will not contain any new information compared to the original HTML-formatted list. Just to be clear. :)

2010-08-25 23:25:57
Ari Jokimäki

arijmaki@yahoo...
91.154.102.68
The inclusion of the spreadsheet might be good at least because the numbering in it offers an easy way to refer to individual papers.
2010-08-26 18:21:13
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202
The spreadsheet is now ready. I sent it to John. Maybe he can come up with a distribution scheme for it.
2010-08-26 19:19:32Spreadsheet by Ari
John Cook

john@skepticalscience...
124.187.125.135

I've uploaded the spreadsheet to the website which you can all download (but don't publicly distribute the URL):

http://www.skepticalscience.com/Peer_reviewed.xls

An incredible effort by Ari!  I guess the next step is to sort them into various categories. Greenfyre gave a number of categories:

  •  NOT peer-reviewed
  • Known to be wrong (refuted by other papers)
  • Straw Men (misrepresenting or misunderstanding the results of the paper)
  • Trivial (nothing to do with AGW skepticism)
  • Dated (comments that are no longer relevant)
  • Do NOT support Denial
  • Mutually exclusive (papers contradict each other)

Anyone have any other thoughts on sorting/how they might be categorised?

2010-08-26 20:23:24
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

Oh, by the way, I added the column "subject" where I included the subsection of the list as given by the author of the list. I added this because it seems that there are some subsections that can be handled as an entity rather than individual papers. For example, at the end of the list there are several subsections that are about politics or economy which as such don't have anything to do with climate science and are therefore rather irrelevant.

2010-08-26 20:25:14
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

I also added couple of additional options to the status column: "comment" means it's not a full paper but just an additional comment on already existing paper, "double entry" - there was at least one paper two times.

2010-08-26 20:42:40Uploading the spreadsheet yourself
John Cook

john@skepticalscience...
124.187.125.135
BTW, Ari, if you want to upload the spreadsheet yourself, just go to the Author Upload Form which uploads files into the pics/ directory. Then you can post the URL here so anyone can download an updated file.
2010-08-26 20:51:24
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

I think we have to divide this work to separate threads because if we start commenting on all the 800+ items than we'll end up with very long thread very fast. So perhaps a thread per 50 or 100 items?

2010-08-26 20:54:18Spreadsheet with some pivot-tables
BaerbelW

baerbel-for-350@email...
109.84.87.221

 

Wow, Ari, that list looks great!

As a starting point, I added a filter to it and a couple of Pivot-tables which quickly show things like these:

  • how many papers were published in which paper (132 in Energy & Environment has the most with 6 of them statused as "comments")
  • which authors provided how many papers (Idso shows up - at least 42 times)
  • journals published x-number of papers by author y
  • how many papers from which year show up

 

There might be other groupings which could help to get a handle on the data and to try and focus on eg. "heavy-hitters"

The file is on its way to Australia as I type this (or more likely already there)

Cheers
Baerbel

2010-08-26 21:19:23
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202
Hmm... the author search will not give correct results with this spreadsheet, as there's usually only the first author showing (for example, I know that Idso is a non-first author in some papers).
2010-08-26 21:36:46Author search
BaerbelW

baerbel-for-350@email...
109.84.87.221

Hi Ari,

the list can be filtered where the author-field contains "idso". It then does show the name as sole author, or in a combination either as first or second author. So I do get these:

Idso
Idso & Balling
Idso et al.
Idso & Mitchell
Idso & Kimball
Idso & Idso
Kimball & Idso
Graybill & Idso
Balling & Idso

Is this what you meant?

Cheers
Baerbel

2010-08-26 21:40:48
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202
No, I meant that there might be papers that are by Author et al. where Idso is one of the et al. so in that case Idso has participated in writing the paper but that does not show in the spreadsheet.
2010-08-26 21:53:31Who is "et al"?
BaerbelW

baerbel-for-350@email...
109.84.87.221
Okay, now I understand. Would the authors collected under "et al" only be listed in the paper or are there other sources where this information could be retreived from? I guess that it would mean quite some legwork to get all those names for the 332 papers showing up with "et al" in the author field. How "important" are the authors listed under "et al" to judge a paper's  reliability?
2010-08-26 22:06:15
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

Most authors are in the original HTML formatted list. I just didn't see it necessary to copy all the authors to the spreadsheet. Now I see that there would have been a need for that too. However, there are plenty of papers where even the original list gives just "et al." so for those one should dig up the paper's abstract page for all the authors.

I personally don't see the author angle being very important unless one wants to pursue the percentage of papers from oil company propagandists or something like that. :)

But if it feels important, I can start adding all the authors to the list.

2010-08-26 22:43:16
Ari Jokimäki

arijmaki@yahoo...
91.154.102.68

On the categories, there probably are some papers that are review articles and not original research articles. These might or might not be properly peer-reviewed but at any case they don't present original research.

The Energy & Environment has been mentioned as non peer-reviewed journal. Additionally there's the arXiv which is just a preprint server. If you gain an access to arXiv you can upload non-peer-reviewed material there and this has been used by contrarians in many branches of science to publish non-scientific material as if it would be scientific.

2010-08-27 01:47:17Comment
Robert Way

robert_way19@hotmail...
142.162.191.40
The thing about classifying E and E as a non-peer reviewed is that you have climategate emails in which tom Wigley says it is in fact a peer reviewed journal. Plus all the major databases still list it. A good way to go about it would be to first list how many comments and addendums or corrections. Then sort by Journal title out of the remaining ones so we know how many E and E, GRL and so on. Then what can be done is we list all the obscure journals and then we have to have people go through the list and start reading through some. I think on skeptical science all we should have is a summary of the results but if the readers want to read through the results themselves we have a supplemental section which contains the comments for each paper. It would have to be a huge collaborative effort but it might be worth it.
2010-08-27 10:17:17Uploaded Baerbel's amended spreadsheet
John Cook

john@skepticalscience...
124.187.125.135

I've uploaded Baerbel's updated spreadsheet with the pivot tables:

http://www.skepticalscience.com/Poptech-peer-reviewed-with-Pivot.xls

Note to Baerbel and any others who update the spreadsheet: feel free to upload your amended file directly to the website via the Author Upload Form which uploads files into the pics/ directory.

Re authors, I don't think it necessary to list all the authors - "et al" is sufficient. Going into that extra detail is much work for little return.

Agree with Robert Way - as our rebuttals, we should publish a summary but people can download the spreadsheet if they want. My thinking is we publish the 3 rebuttals simultaneously. The basic rebuttal just speaks generally about the tactics used in the Poptech list. The intermediate goes in a bit more detail, giving examples of each tactic. The advanced lists more details still, perhaps more facts and figures. Something like this anyway. I'll be posting an Intermediate Rebuttal sometime soon, will start it in a new thread. That can be a useful starting point that we can structure everything else around perhaps.

I think this forum is an ideal place where a number of people can collaborate to put together a definitive rebuttal of Poptech's list so we should take advantage of the number of motivated people available here.

2010-08-27 13:54:37
Jim Meador

jimm58@gmail...
67.101.212.216

{Back from a camping trip. Amazing to see how much has happened...}

One thing I suggest for this rebuttal is to somehow acknowledge the usefulness and value that healthy skepticism brings to science in general, and to the climate change debate in particular. For instance, the skeptics uncovered a few errors in the IPCC 4th report. Thanks to them, there is now a better report. The goal is to have a perfect report, and the skeptics have helped!

The fact that some skeptical papers are published is a good thing in that it undermines the "group think"/"self hypnosis" argument that my brother frequently brings up. ("They never allow stuff that disagrees with their theories to be published" and so forth...) Climate science, like any science, should not be afraid of papers that don't fit the accepted theory, so long as they meet basic quality criteria. In some ways the fact that there are skeptical papers published SUPPORTS that the science is robust, and has really examined all the angles.

So among all these 800 papers, there must be a few that pass muster (???) and we should acknowledge the value of that.

The other thing for this rebuttal is to balance the number 800, or however many are actually legitimate, against the  number of papers that support and validate that climate change is real. How many is that? Are there something like 3000 papers cited in the IPCC report, and there must be many many others that could have been cited. This is like the Oregon Petition "31000 scientists" thing. The number 800 (or whatever is the count of legit skeptic papers) standing there by itself looks big...it needs to be put in perspective.

 

2010-08-27 14:48:18Numbers game?
davidpalermo

davidpalermo@earthlink...
68.6.71.54

Hello all... I am new here - thank you John.

I am thinking that in addition to going through all 800 or so papers it might be worthwhile to try and find out how many peer-reviewed papers have been published that support the evidence for AGW.  

Maybe there are 30,000?  Maybe 100,000.  if so, then it becomes a numbers game and quite possibly could be put into perspective like a consensus.  There are always going to be credible scientists who disagree with some of the studies but the body of the evidence suggests that AGW is happening etc...

Another thought which may have been mentioned earlier is to use the Doran/Zimmerman Consensus study: (http://tigger.uic.edu/%7Epdoran/012009_Doran_final.pdf)  

Why do the 97% of published climatologists agree that AGW is happening?   My guess is because the evidence is pointing to AGW.

Why aren't the 97% of published climatologists skeptical because there are "800 or so" papers casting doubt on the whole thing?

I don't know any climatologists but it sure would be great to ask a few about this. 

I have to admit, I am curious as to how many of the 800+ papers are really skeptical of AGW. 

2010-08-27 16:12:49
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

The list is divided to several subsections (column "subject" in the spreadsheet). Here are my thoughts on them:

- AGW (the name I gave for the first unnamed section of the list): probably most relevant. It would be good to check each paper here.

- An Inconvenient Truth: Two papers in this category could probably be placed to category "trivial" depending on what the trivial category actually is. John has described it above as "nothing to do with AGW skepticism". These papers have lot to do with AGW skepticism but they seem to have nothing to do with AGW as such. However, I haven't seen the contents of these papers so I cannot be sure.

- Antarctica: default category here is straw man, but it would be good to check if any of these papers actually discuss some evidence from Antarctica and use that to conclude that AGW is wrong (I doubt any of them do that, though :) ).

- Arctic: Same as Antarctica.

- Clouds: relevant subsection. Would be good to go through paper-by-paper.

- CO2 lags Temperature changes: All papers but one here belong to the straw man category. The one exception is number 299, Kuo et al. (1990), published in Nature. This paper needs to be addressed properly as it reports a few months lag from modern records.

- Coral reefs: probably trivial for all papers (possibly straw man).

- Deaths: probably trivial for all papers (possibly straw man).

- Disease: probably trivial for all papers (possibly straw man).

- Droughts, floods: probably trivial for all papers (possibly straw man).

- Ecological: probably trivial for all papers (possibly straw man). However, some papers might cover the subject so thoroughly that they might actually have some relevance (but looking at some paper titles there makes me doubt it very much).

- Glaciers: straw man for all papers (just report of some individual glaciers that haven't yet reacted to AGW).

- Greenland: probably straw man for all papers (ice sheet growth in the interior of Greenland ice sheet is not unexpected thing, IIRC).

- Gulf stream: probably trivial for all papers (possibly straw man).

- Hockey stick: probably trivial for all papers (possibly straw man).

- Hurricanes: probably trivial for all papers (possibly straw man).

- Medieval Warming Period - Little Ice Age: straw man for all papers.

- Ocean Acidification: probably trivial for all papers (possibly straw man).

- Permafrost: I'm not very familiar with this subject but probably trivial for all papers (possibly straw man).

- Polar bears: probably trivial for all papers (possibly straw man).

- Sea level: seems to be straw man category, with lot of papers with only regional analysis, but perhaps some papers make a global analysis making them more relevant.

- Species extinctions: probably trivial for all papers (possibly straw man).

- Storms: probably trivial for all papers (possibly straw man).

- Tornadoes: probably trivial for all papers (possibly straw man).

- Weather Stations: first seems to be trivial, but there might be some papers that claim there haven't been any global warming, so those have to be considered as relevant.

- Wildfires: probably trivial for all papers (possibly straw man).

- 1,500-Year Climate Cycle: most likely straw man but might contain some relevant papers (claiming that current warming is just a part of some natural cycle).

- Cosmic rays: some papers here are relevant because they suggest alternative hypothesis to AGW. At least one paper actually seems to go against the AGW skepticism (the 612, Erlykin et al., 2010).

- Solar: some papers here are relevant because they suggest alternative hypothesis to AGW.

- Climategate: difficult subsection to categorize. Trivial?

- IPCC: we move to police-related subsections. Irrelevant to the science of AGW so perhaps trivial.

- Kyoto Protocol: same as with IPCC.

- Socio-Economic: same as with IPCC.

- Stern Review: same as with IPCC.

- Rebuttals to Published Papers: I marked these all as comments, but perhaps some of these are full papers. No common category as these span wide scale of subjects.

2010-08-28 06:47:28The meaning(s) of "straw man"
Jim Meador

jimm58@gmail...
67.101.212.216

I suggest using a different term that "straw man". It has several meanings, and the one I am familiar with is not the one we are using here.

I suggest "Misrepresentation" instead.

2010-08-28 06:56:23Going paper-by-paper
Jim Meador

jimm58@gmail...
67.101.212.216

Well, I just went to look at the first paper on the list..."Loehle 2007" along with the comments on it. It took a long time to read through it all. I saw a lot of rhetoric, but also some serious debate about methods, etc. I would summarize the critcism as

"Simplistic analysis. Methods do not meet normal publication standards. Some data used are not temperature proxies. The reconstruction is not global." (However, I don't really feel qualified to bless this summary as 'correct'. I could easily be accused of cherry-picking criticism of the paper.) 

Going paper by paper this way will take a lot of time. How about a sampling method, for instance looking at every 10th paper?

-Jim 

2010-08-28 16:42:07"Peer-rebuttals" available?
BaerbelW

baerbel-for-350@email...
93.231.190.116
Instead of trying to interpret the papers ourselves to assess their content wouldn't it be possible to find "official" rebuttals from climate scientists for them? The papers listed could then be contrasted by how many refutations they received. I am not sure however, if this is a meaningful comparison - unless this is perhaps also done for a sampling of "proper" papers where I guess the number of refutations would be "somewhat" lower (but I might be wrong!).
2010-08-28 17:10:53Kobayashi Maru
doug_bostrom

dbostrom@clearwire...
184.77.83.151

May I suggest checking the comments census at "Poptech?" The "Google Scholar Illiteracy in the PNAS" post of August 5th-- the most recent entry at Poptech--  has attracted zero (0, nada, zilch) comments. The "800 Peer-reviewed Papers..." post of July 25 has garnered 16 comments other than "Andrew" (poptech) himself. Other posts are are similarly non-resonant.

Doing a post or rebuttal on this guy's ridiculous collection of papers will actually promote his site, significantly increase his traffic, that's my humble input. A lot of work for effectively no return, or even negative return.

2010-08-29 02:06:00Latest thoughts
Jim Meador

jimm58@gmail...
67.101.212.216
I have been thinking about this more. Since the argument is basically one of numbers, that there are a lot of papers, the rebuttal should focus on the numbers game, and point that there are many many more papers supporting AGW. Also I think there is some merit to point out that a great majority of the papers are published in "E&E" which is often cited as having lower quality standards than other publications. (It would be good if there was some way to quantify the E&E quality standards.)
2010-08-29 04:54:42E&E quality
nealjking

nealjking@gmail...
91.33.114.10

E&E is not considered a peer-reviewed scientific journal.

There is an index of journal-quality, that has to do with how often articles from a particular journal get cited. I'm not even sure E&E would even qualify to have a rating. It's called the impact factor: http://en.wikipedia.org/wiki/Impact_factor .

Probably a good reference librarian can figure out how to access the IF for E&E.

 

2010-08-29 05:58:18Comment
Robert Way

robert_way19@hotmail...
142.162.188.138
Another thing that we should consider is determining how many papers actually deal with attribution? Lets remember that is the most important thing about this all. Authors can nitpick all they like about different problems with AGW supporting data (such as proxies and such) but how many studies in the 800 are actually attributing warmth to something else or are poking holes in the attribution of CO2
2010-08-31 17:31:20Numbers game...
davidpalermo

davidpalermo@earthlink...
68.6.71.54

"Since the argument is basically one of numbers, that there are a lot of papers, the rebuttal should focus on the numbers game, and point that there are many many more papers supporting AGW"

 Jim - I agree.  I proposed the very same thing a few days ago. Read my earlier post above.

The challenge is how to find all those papers in support of AGW and count them?

2010-09-01 05:59:06A subset of "all"?
Jim Meador

jimm58@gmail...
67.101.147.143
How many papers supporting AGW were published in 2009? Is it more than 800? Maybe we would not have to count them all to put up a rebuttal based on numbers...i.e. all the skeptical papers published in history (800?) are equal to the total supporting AGW published in 2009, or in 4 months in 2009, or whatever. Of course this argument also needs to include something about quality and impact. All papers are not created equal... 
2010-09-01 19:46:30
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

I checked some 2009 numbers for Journal of Climate:

Published articles: 399

2009 articles in Poptech list from JofC: 3

Percentage of "skeptical articles": 0.8 % (and 2009 happens to be most skeptic-article-full in JofC, there are less "skeptic" articles in all other years so this number is more like an upper limit than an average).

It takes Journal of Climate little more than two years to publish 800 articles.

Looking at other journals in similar manner would give us some feel of which journals are skeptic-friendly.

2010-09-03 14:40:28Doing the numbers Journal by Journal
Jim Meador

jimm58@gmail...
67.101.147.248

Ari, that is very helpful. It seems that doing the Pro/Con AGW paper count on a Journal by Journal basis could be a powerful rebuttal, particularly if the Impact Factor for each journal was included as Neal suggested.

2010-09-12 02:55:06
Ari Jokimäki

arijmaki@yahoo...
91.154.102.68
It seems that this faded to the background. What do we do with this? Are people keen on going through whole list so that we first debunk groups of papers and then paper-by-paper for those left after the group phase? I already made some suggestions on some groups of papers above. Might be a place to start.
2010-09-12 11:17:56Next step
John Cook

john@skepticalscience...
121.222.93.62
I'd started an intermediate rebuttal which kind of hit it in a broad brush method. Or maybe it's better as a basic version. I'll post it next week - perhaps that could lay the basic skeleton and then we build up the more advanced levels with the extra details.