2011-05-22 05:09:50Database of peer-reviewed papers: classification problematics
Ari Jokimäki

arijmaki@yahoo...
91.154.109.46

I started writing an article on the problematics of classifying papers for the peer-reviewed paper database. It's not ready yet but I decided to mention it already so people can give comments and suggestions.

2011-05-22 07:33:15
Rob Painting
Rob
paintingskeri@vodafone.co...
118.93.0.102

Looks good so far Ari!.

2011-05-22 08:58:10
Alex C

coultera@umich...
67.149.101.148

If I may, perhaps a post in which multiple contributors to the database give their own philosophies as to how categorization should work?  For example, I don't agree with your final summary that papers that suggest CO2 affects climate (or even affects the current climate) should be necessarily placed as pro-agw, since it's often been that we put papers that say the sun affects our current climate into neutral as well if they don't suggest that the sun was the main contributor to the warming.

I think such an approach, either way, might help to emphasize the "work in progress" aspect of the database, the ambiguity of many papers and how the classification rules have been for us so far, so on.  A "Rob says [this], Ari says [this], John says [this]" type post, maybe.

I also think that we should also touch on the issue of categorizing based on stance of the paper on AGW, or on the basis of pertinence to skeptic arguments (aside from the fallacies).  This could probably be covered at least in part in the third category of problematics you have there, or could fit in with the format I suggested above.

Or, we hash out a set rules guideline right now (well, for this post) and say that while we were less organized before the deliberation that occurred to make this post, there will be work done to categorize papers already in the database to these rules.

2011-05-22 14:39:48
Ari Jokimäki

arijmaki@yahoo...
91.154.109.46

All good points, Alex. I'm going to incorporate them to this post. One of the reasons for bringing this up so early was that I wanted other point of views into the mix, so everyone having disagreement with my opinions feel free to argue.

Also, I haven't yet looked back on our previous discussions on the paper classification but I'm going to. There are more relevant points and some good examples in our previous discussions.

2011-05-23 20:59:47
Ari Jokimäki

arijmaki@yahoo...
192.100.112.210

I added a few paragraphs discussing "it's the sun" argument.

Alex: "For example, I don't agree with your final summary that papers that suggest CO2 affects climate (or even affects the current climate) should be necessarily placed as pro-agw, since it's often been that we put papers that say the sun affects our current climate into neutral as well if they don't suggest that the sun was the main contributor to the warming."

I changed my wording relating to this. Here's the changed sentence: "My opinion is that the papers discussing past climate changes and suggesting that carbon dioxide can have a strong role in climate (and maybe even suggesting explicitly that their results support AGW theory) are “pro-agw” papers."

2011-05-24 04:13:40Wrote a few paragraphs for local v. global, etc.
Alex C

coultera@umich...
64.88.86.200

Suggestions welcome of course:

"The issue of local v. global studies arises when a certain study concludes either a predicted response from AGW is (is not) happening, or explicitly concludes that AGW is (is not) directly observable or having an effect; but such study is of a specific location, and not global trends.  It is not necessarily safe to assume that any effect observed in one area can be extrapolated to the entire globe, and as AGW theory most often deals with global trends, or trends dealing with whole biomes, we have generally agreed that local studies be placed as “neutral,” regardless of main conclusion.

We have also applied this same logic to research dealing with the effects of AGW on certain species: studies that focus on one plant species and how it will respond to AGW, for instance, are classified as “neutral” as well, as generalizations among plant species (especially) are illogical to make.

Of course, one question that arises is, “Where can the line in number of local publications be drawn so that a trend can be extrapolated?”  While it is a good question, it is our opinion that that is not the point of the classification system.  Papers can be referenced collectively to identify a widespread effect that extends beyond the local observations of each paper, but the limited extent of the observations of each individual paper subsequently limits the conclusions of each individual.  As this database is to categorize individual papers by the explicit conclusions of the authors, and not groups of papers by the implicit agreement among them, it is our opinion that “lines” as above not be considered, and local/species-specific papers be categorized as “neutral.”

We would also like to remind those using this database, though, that papers are also associated with related skeptic arguments to AGW, so papers can still be de-facto grouped together for careful analysis."

2011-05-24 19:04:30
Ari Jokimäki

arijmaki@yahoo...
192.100.112.202

Excellent Alex, perhaps we'll make a group effort of this post now. Few thoughts:

We could insert an opening paragraph to your thing discussing it generally, i.e. how papers dealing with parts of a whole should be classified. After that we would enter to the examples you are using. We could also cite some other examples of this same thing. Perhaps long time series vs short one could be similar thing (for example a paper could be presenting temperature measurements from short period of time when the phenomenon in question might call for longer time series).

There's at least one thing I would add to your writing. Local papers can be pro-agw or skeptic, if they explicitly argue so. Same with single species papers. Also, when does local or regional become global? How many species must a paper study so that it could be considered meaningful for whole ecosystem?

2011-05-25 19:21:46
Ari Jokimäki

arijmaki@yahoo...
192.100.112.210

I added your text, Alex, and framed it with my own paragraphs.

2011-05-26 20:23:57
Ari Jokimäki

arijmaki@yahoo...
192.100.112.210

Ok, first draft of this thing is ready (link in the opening post). Now you can also comment on language errors and all that.

2011-05-28 02:24:35
Alex C

coultera@umich...
67.149.101.148

>>>are based on logical fallacies, especially on logical fallacy known

<<<are based on logical fallacies, especially on the logical fallacy known

>>>It is presented as if AGW theory would claim that climate hasn’t changed before and then the straw man is attacked by pointing out a climate change from the past and claiming that climate has changed before so mankind can’t be the cause for the current climate change.

[Bit long and a run-on, perhaps:]

<<<AGW is misconstrued as claiming that climate has not changed in the past, and then this straw man is attacked with examples of past natural climate change, with the erroneous conclusion then being that humans cannot be causing this climate change.

>>>After all, in peer-reviewed literature logical fallacies are being avoided.

[I don't think a present participle tense is appropriate here, I think this sounds better:]

<<<After all, logical fallacies are avoided in peer reviewed literature.

>>>But what papers are “neutral” and what papers are “pro-agw” for the “climate’s changed before” argument?

<<<But what papers are “neutral,” and what papers are “pro-agw,” for the “climate’s changed before” argument?

>>>There are some difficult arguments from classification point of view, "it's the sun" for example. 

<<<There are some difficult arguments from a classification point of view, such as the "it's the sun" argument. 

>>>Theory of AGW doesn't suggest that there's no role...

<<<AGW theory doesn't suggest that there's no role... [or, "The theory of AGW..."]

>>>...role for sun in Earth's climate. So, how do we classify papers for this argument? [combine these sentences w/comma]

>>>A paper suggesting that Sun has no role in Earth's climate whatsoever certainly seems to go against this argument but it also would be against mainstream climate science, so in rational classification system it could not be pro-agw paper but it clearly is not skeptic paper either as it goes against the "skeptic" argument.Such paper should probably go to neutral bin. However, if we just concentrate on paper's take on the argument in question, then such papers would be "pro-agw" papers.

[grammar issues, but this sentence also goes logically against your previous statement that those papers that sneak by the peer review process should still be classified as skeptic (the "climate's changed before" ones) - should only pro-agw be exempt from scientifically contradictory papers?  That seems to be self-serving, or inconsistent.  Anyways...]

<<<A paper suggesting that the Sun has no role in Earth's climate whatsoever certainly seems to go against this argument, and while this would go against mainstream climate science, it would be classified as pro-agw. [end, continue with next sentence - maybe just combine what's left of this paragraph with the next, since this is short now.]

>>>There are papers saying that sun has had certain warming effect during last decades.

<<<There are papers saying that the Sun has had a warming effect during last several/few decades.

>>>The size of this effect varies by paper. How much warming should a paper suggest as the contribution of sun to the recent warming so that we could classify it "skeptic" paper?

<<<The size of this effect varies by paper, so how much warming should a paper suggest as the contribution of the Sun to the recent warming so that we could classify it "skeptic" paper?

>>>...taking over and will dominate in future, is that pro-agw or skeptic paper?

<<<...taking over and will dominate in the future, is that a pro-agw or skeptic paper?

>>>This is compatible with AGW theory which says that only in recent decades anthropogenic CO2 has become dominant climate forcing.

<<<This is compatible with AGW theory, which says that only in recent decades has anthropogenic CO2 has become the dominant climate forcing.

>>>Such papers should go to neutral bin unless they also suggest that CO2 doesn't affect climate (or affects generally just weakly) in which case they should go to skeptic bin.

<<<Such papers should go to the neutral bin, unless they also suggest that CO2 doesn't affect climate (or affects only weakly), in which case they should go to skeptic bin.

>>>If a paper suggest large role for sun in past climate changes but also emphasizes that CO2 can have large role too, then the paper should go to pro-agw bin.

<<<If a paper suggests a large role for the Sun in past climate changes, but also emphasizes that CO2 can have a large role too, then the paper should go to the pro-agw bin.

>>>There are lot of papers that discuss certain parts of an issue but not the whole issue. There are papers discussing certain weather events but not their climatical significance. Some papers report regional studies while "skeptic" argument on that issue relates to global situation. Studies on single species are rarely conclusive from the whole ecosystem perspective. These are jus few examples of papers that address only partly the whole that is relevant to certain "skeptic" arguments. As a general rule, such papers should be put to neutral bin. However, an exception to this rule is a paper that addresses only one part of the whole but claims explicitly to be relevant for the whole also. Fro example, if a paper reports warming in Finland and claims that it means global warming is happening, then we should put that paper to pro-agw bin.

<<<There are a lot of papers that discuss certain parts of an issue, but not the whole issue. There are papers that discuss certain weather events, but not their climatic significance; some papers report regional studies, while the "skeptic" argument on that issue relates to the global situation; studies on single species are rarely conclusive from the perspective of the whole ecosystem. These are just a few examples of papers that address only part of the whole that is relevant to certain "skeptic" arguments. As a general rule, such papers should be put to the neutral bin. However, an exception to this rule is a paper that addresses only one part of the whole but claims explicitly to be relevant for the whole also. For example, if a paper reports warming in Finland and claims that it means global warming is happening, then we should put that paper to the pro-agw bin.

>>>We have also applied this same logic to research dealing with the effects of AGW on certain species: studies that focus on one plant species and how it will respond to AGW, for instance, are classified as “neutral” as well, as generalizations among plant species (especially) are illogical to make.

<<<I also think this same logic should be applied to research dealing with the effects of AGW on certain species: studies that focus on one plant species and how it will respond to AGW, for instance, are classified as “neutral” as well, as generalizations among plant species (especially) are illogical to make. [to make it consistent with the POV you've been speaking from, which has been first person]

>>>While it is a good question, it is our opinion...

<<<While it is a good question, it is my opinion...

>>>...it is our opinion that “lines” as above not be considered...

<<<...it is my opinion that “lines” as above not be considered...

>>>...and local/species-specific papers still be categorized as “neutral.”

>>>If such argument would not be straw man, then we should classify papers according to that argument.

<<<If such arguments are not straw man, then we should classify papers according to those arguments.

>>>Prime example of this is the situation with polar bears. There is the argument "polar bears are increasing". For this argument, papers suggesting that polar bear populations are declining are "pro-agw" and papers suggesting that populations are increasing are "skeptic". However, also for this argument studies on one population only are usually "neutral" as they don't deal with the global situation of polar bear populations.

[first, do you think this might have been covered earlier with the reference to biomes, or no?]

<<<A prime example of this is the situation with polar bears. There is the argument "polar bears are increasing;" for this argument, papers suggesting that polar bear populations are declining are "pro-agw," and papers suggesting that populations are increasing are "skeptic." However, also for this argument, studies on a single population only should be considered "neutral" as they don't deal with the global situation of polar bear populations.

>>>It's not a problem as such as the database allows several arguments per paper.

<<<In and of itself, it's not a problem as the database allows several arguments to be assigned per paper.

>>>Database currently allows only one classification per paper so this is a problem.

<<<The database currently allows only one classification per paper, so this is a problem.

>>>Most obvious method is to select the argument which the paper addresses most. Sometimes that doesn't help, though.

<<<The most obvious method is to select the argument which the paper addresses most, but sometimes that doesn't help.

>>>Well, these are things that need to be thought of.

<<<These are issues that need to be thought of.

>>>...further development is needed and especially we need to set common ground rules for the classification i the future.

<<<...further development is needed, and we especially need to set common ground rules for the classification of papers in the future.

>>>Currently we are classifying these papers in gut-feeling bases. That is not good in the long run. [delete sentences, one before is a good end note.]

2011-05-28 02:25:36Wow that's a lot now that I submitted it....
Alex C

coultera@umich...
67.149.101.148

Mainly just small typos.  I made more important comments in brackets.

2011-05-28 16:02:22
Ari Jokimäki

arijmaki@yahoo...
91.154.108.219

Thanks a lot, Alex. I have made corrections as you requested. I think this might be ready now (if there's no other comments), but I probably need to add a reference to John's introductory post on the database as it also touches these classification matters.

Alex: "should only pro-agw be exempt from scientifically contradictory papers?"

A fair point. However, I'm not discussing a contradictory paper here, but a paper that claims there's no role of the Sun in Earth's climate. While the claim is ridiculous, it is not contradictory or a straw man - it just makes a claim that seems ridiculous but might even have some evidence and logic backing up the claim (it's hard to tell as this is a hypothetical situation :) ).

Alex: "<<<...it is my opinion that “lines” as above not be considered..."

I corrected these, but it is in paragraph written by you so they are not exactly my opinions, but perhaps they are better this way in order not to create confusion.

Alex: "first, do you think this might have been covered earlier with the reference to biomes, or no?"

No, it's ok like this. It's a separate issue I'm discussing there and I only use an example that is from biosphere to illustrate the point.

2011-05-28 17:48:33
Rob Painting
Rob
paintingskeri@vodafone.co...
118.93.14.212

Thumbs up from me.

2011-05-28 18:43:24
Alex C

coultera@umich...
67.149.101.148

Looks good.  My recommendation for changing my wording to "my" could work the other way around too if you'd like, or if you think it'd be better the other way - I was thinking consistency in POV, so I'm fine if you either want to keep it the way it was, or keeping it all "our," whichever you'd prefer.

RE "A fair point." That's fine too.

2011-05-28 19:05:43
Ari Jokimäki

arijmaki@yahoo...
91.154.108.219

"My" is better because rest of the article is written from my perspective.