2011-01-03 15:39:17Redirecting links
John Cook

john@skepticalscience...
60.231.58.57

Nick Kocharhook has been emailing me examples of bad links littered throughout our rebuttals which of course then spread into the translations. It's quite a stale link epidemic and he suggested a way to resolve the issue - have a "links database" where we maintain all the links used throughout our rebuttals. Then when we change a link (because it moves or goes bad), we change the centralised links database and the change will propagate throughout the blog posts, rebuttals and translations.

Now while I was talking to Nick, I realised we already have a links database, displayed at http://www.skepticalscience.com/resources.php. So what I propose is the following system where we use redirect URLs like www.skepticalscience.com/link_Hansen_et_al_2008 and this automatically redirects to the appropriate URL somewhere on the NASA website. Then if NASA move their files around (which they have done once, the blighters), then we change the database once rather than having to go through all our various articles, looking for links.

So this is the procedure I propose for transitioning to this new system:

  • For the existing links database, I add a new field FILENAME. This works the same as blog post filenames - it becomes part of the URL. So for example, if I use the filename "Hansen_et_al_2008", then the URL will be http://www.skepticalscience.com/link_Hansen_et_al_2008
  • Next I create a PHP page linkredirect.php and update my .htaccess file so that it rewrites any URLs starting with link_ and sends them to linkredirect.php which then finds the final URL in the database and sends the browser there
  • That's the easy part. Then we have to go through our rebuttals/translations/blog posts and convert them to redirect URLs. It doesn't have to happen all at once - I suggest it be a gradual thing and we crowdsource it so anyone who sees a bad link can fix it using this new system, plus we use this new system when writing new articles. So what I was thinking we do is I create a form that only Authors can access that let's you add new links or edit old links. Whenever you add or edit a link, the webpage will look for the final URL in all the rebuttals, translations and blog posts and replace them with the new redirect URL. So as we add links, the system will do the bulk of the heavy lifting for us - it's just a matter of maintaining the link database.
  • Note - it won't be essential that we use this for EVERY link on the website. Mainly for links likely to be used more than once - but I think we should use it for all peer-reviewed links (which comprise most of our links anyway).
As well as improving the bad link issue, it also will build our database of links, especially populating it with lots of peer-reviewed papers. Big fan of this concept - it's already an impressive database of links but this will help fill it even further. So thoughts, comments?
2011-01-04 04:09:09sounds good
dana1981
Dana Nuccitelli
dana1981@yahoo...
38.223.231.252
Sounds like a good idea to me.
2011-01-06 10:41:27OK
k9

nick@kocharhook...
84.9.36.188

That's great that some of the infrastructure is already there for this.

One thing I was wondering was if it would make sense to group the links in any way, and possibly to provide metadata about them. In the scheme I described to John originally, there would be, e.g. a /paper/Author_2008 link and then a /data/NOAA_dataset link. Then you could just go to /data/ to see a list of the links to data sources.

But perhaps that's not too important. It would be possible to tag the resource links in the current setup just fine, if we decided we wanted to do that.

I like your idea of crowdsourcing the conversion. That will be the hardest part anyway, and it doesn't have to happen all at once.

There should also be some mechanism for periodically checking the links we have to make sure none have broken on us. This is definitely made easier by having them all in a DB.

In conclusion: how can I help? :-)

-Nick Kocharhook 

2011-01-06 13:00:28Re grouping the links
John Cook

john@skepticalscience...
60.231.58.57

The links are currently grouped by skeptic argument. This is compulsory. Whenever someone submits a new link, they *have* to select at least one skeptic argument. So we're steadily adding more peer-reviewed papers to each skeptic argument. This has been a big resource for me and I hope others.

It would be possible to add metadata to the papers but it's not a big priority at this point. But I will say that I've been talking to Mila at zvon.org - that website that created a database of all the references in the IPCC AR4. We're discussing sharing data between each other so I may be able to get hold of some of his peer-reviewed meta-data and incorporate it into our links database. Will keep everyone posted on this development (don't hold your breath, as I said, it's not a high priority).

So I have two questions before I start coding this feature:

  1. Do we need to group the links in any way other than the current setup, grouping by skeptic argument? 
  2. Is using the URL format http://www.skepticalscience.com/link_Hansen_et_al_2008 sufficient? You mentioned having /paper/ or /data/ in the URL. I was thinking keeping it simple - just a link_ rather than other options.
Once these 2 questions are settled, I'll start coding then let everyone know when they can start crowdsourcing the conversion.
2011-01-07 03:12:07Comment
Robert Way

robert_way19@hotmail...
134.153.163.105
I think it might be preferable to have paper or data included in the url. Hopefully its not too much work.