2011-06-01 05:38:43Basic ideas for improving the database; Goals
Alex C

coultera@umich...
67.149.101.148

So, with the launch completed, there seems to be general approval and also several "complaints" and suggestions for change.

Which we expected, right?

I think that there were several good basic ideas brought up in the Comments section of both the release page and Ari's problematics post, and I thought that if we are going to be improving the database then we might as well start brainstorming future goals we'd like to have for it.  I'll start, and think that we should aim also for:

- Classification strictly adhering to Naomi's method - does it support AGW, or no?

- Getting authors and journals listed separately.

- Whether a paper applies to local, hemispherical, past, global, etc. climates or situations.

What do we want our database to do?

 

Also, and I think if we're going to publicly state this we should have a venue for it: how can other users submit suggestions?  Do you think the comments page of either post will be sufficient?

2011-06-01 08:27:024 things
John Cook

john@skepticalscience...
121.222.31.47
1. Yes, been thinking we really need to do this using Naomi's classification. Good news is we have already done this. Any paper classified under the "it's not us" myth will either endorse or reject the consensus. Right? So we just need to double check that all the papers under this category are properly classified and we can think about how we could present this. It's a simple matter for me to rejig Paul's visualization to only show papers under this classification.

There are a number of papers that use detection and attribution to find human fingerprints in various elements of climate (beyond my usual human fingerprints) so should check they're already in there).

2. Yes, need to list authors and journals. I do have an XML file to bring a lot of this data in - will do this soon then we can build from there.

3. Re categorizing, hmm. I can think of one easier way to do this - instead of classifying every single paper, we classify the climate myths into different categories (eg - cryosphere, ocean, etc) and papers under each category will be then classified. Bit inflexible but lower maintenance then doing every single paper.

4. Another idea I like - keeping track of which papers have been rebutted in the peer reviewed literature. I could just set up a database of 'rebuttal pairs' where you match a paper to it's rebuttal. This could be a really practical resource - when someone says "hey, here's a skeptic peer reviewed paper", they can look up our database to see if it's been rebutted yet.

2011-06-01 16:52:38
Ari Jokimäki

arijmaki@yahoo...
192.100.112.211

Is there a way to import lot of papers in one go? I'm thinking that it would be nice to be able to arrange for example all of the papers in this January 1988 issue of Journal of Climate to some file and them import them to database. It would be simple (although still somewhat laborous) to transfer all the papers of all relevant journals like that. We could create a temporary buffer argument for imported papers and then we would crowd source the papers in that argument to their proper arguments after the transfer.

2011-06-01 19:39:30Bulk import
John Cook

john@skepticalscience...
121.222.9.229
No facility to bulk import yet. Would be complicated though - it would need a way to map arguments to each post. A bit complicated. Ari, I may get to that sometime but it's such a young one, there are some other programming tasks higher on my priority list for now (I've finally got through the backlog of work that built up over the book launch whirlwind so finally getting back to some coding shortly).
2011-06-01 20:38:07
Paul D

chillcast@googlemail...
82.18.130.183

My brother asked a few people to comment about it and one didn't know what AGW meant.

2011-06-01 20:58:02
Ari Jokimäki

arijmaki@yahoo...
192.100.112.211

John, in what kind of format database is now? Would it be possible to manually edit the database "file" and make bulk import like that? If there's such a file, could you perhaps send it to me, so I could ponder the situation a bit?

2011-06-01 21:20:11Database format
John Cook

john@skepticalscience...
121.222.9.229
It's in MySQL format. I can give you PHPMyAdmin access if you want to get in there, noodle around. I trust you, Ari :-)
2011-06-02 14:40:44
Ari Jokimäki

arijmaki@yahoo...
192.100.112.211

Let's pass on that for now, I'm not familiar with SQL. I just thought that there could have been a possibility to take the database file out, add bunch of papers to it manually, and then add the file back in.

2011-06-02 16:52:06
Paul D

chillcast@googlemail...
82.18.130.183

PHPmyadmin does allow the importing of data, but I can't remember if data can be appended.
I think the biggest problem would be that the data would have to be the same format as the existing database tables.

Mysql did have a tool for migrating data but I haven't used it, I think it is complicated.