2011-05-15 00:29:34Of Averages & Anomalies
Glenn Tamblyn


Finally finished the monster - here it is


This a very large post. I have written it initially as one big post but will be split in two before release.

My intention with this is to provide in part 1 a study of how to assemble a climate temperature record without falling into the traps. In the process I am trying to highlight that most peoples, probably unconscious, ideas about how it is calculated may lay behind much of the public angst about the reliability of the record. Thus it is quite long since I am trying to explain in detail why but step by step so I don't actually attack peoples pre-conceived ideas. What I hope for is that I can get by in to the ideas step by step and by the end hopefully have people questioning their assumptions without actually rubbing their noses in it. If I can lead the horse slowly to water I might be able to get him to drink.

Then in part 2 I go through many of the common claims about the supposed problems with stations, biases etc and try to show why they are wrong. I cross-link to previous SkS posts looking at the evidence that the record is robust, but I am trying to shed some light on why that is the case. Again it is detailed and thus long because I am trying to use both arguments and examples to get people thinking more carefully about the why's of this.

Since I haven't split this in two yet there are some unmade links from Part 1 to Part 2 that I can't make till then.

I have not made any reference to the Watts/Pielke paper. There is one link to surfacestations.org and a brief reference to possible impacts from bad stations on shorter term results, Daily, Seasonal etc.

Once we are happy with it I suggest that the two parts be published close together. This means we might be able to partition the comments - underlying analysis method to part 1 and debate about the problems in part 2.

Opinions Please...


2011-05-15 12:15:51
Rob Painting

Glenn, will get back to ya, this will take some 'absorbing' on my part!.

2011-05-15 20:25:16
Rob Painting

Glenn, before we go any further, I think you're going to have to chop this down into 3 or 4 installments. It's just way too long. I enjoyed it, but I don't think casual readers are going to have the staying power, which would be a shame because you've done a great job in explaining this in 'accessible' language. Stopping Part 1, prior to the GISTEMP heading, might be a good place. 

2011-05-16 00:57:28
Mark Richardson

Splitting it up into more pieces and running it over a week is the way I'd go. I also think you could cut the length down by thousands of words without losing any detail (to test it, I edited your first 1.2k words down to about 650 and I think it still said the same stuff :p ).


Perhaps release it as a series of bitesized chunks but also publish the whole thing as a single URL?


The first part could be split into 2-3 quite easily. The problems and what a site measures. Anomalies. GISTemp & Others.

2011-05-17 05:43:54This might sound weird...
Robert Way


Hey Glenn,
this might sound weird but do you want to write a paper on this subject? I've been working on it for a long time and I've thought about writing a paper on the differences and advantages/disadvantages of different methods. I was going to use an isolated, climatically sensitive region to test each method on. I've actually conducted the analysis and have a comparison of the RSM, CAM and FDM methods run. I also used the Bias Minimization Method used by Tamino (adapted) and am thinking about trying to run RomanM's adaptation also. The results are pretty obvious, CAM significantly lowers the amount of stations you can use. RSM lowers a bit, FDM keeps the most besides BMM which produces the best result and most accurate.

Either way I think it would be useful to point out in your post that RomanM's method is likely the most accurate (or tamino's)

2011-05-20 12:25:33I have been off 'the air'
Glenn Tamblyn


Sorry to not reply sooner guys. I go away on holidays to a quiet country town, the wireless connection on my laptop dies and the town is so quiet it takes three days to get a replacement.


I agonised for some time over the length of this post - its been germinating for a while. It is the length it is because I am aiming it a specific audience. Much of the content here at SkS is pitched at a certain technical level and the people who actually post seem to operate at that level - Pro-AGW & Skeptics. What is unknown is how many of the lurkers are at that level. As the popularity of the site grows we are likely getting lurkers from a larger range of understanding levels.


What I am interested in doing is targetting an audience that is almost scientifically uneducated. And the import thing to realise is that people who don't have science as the basis of their view of the world don't just go through life ignorant. They construct a sense of the world and how it works that seems consistent to them from their restricted knowledge base. This doesn't mean they are stupid. Just unknowledgeable.


However, if you are trying to present ideas that don't fit their constructed narrative of 'how the world works', simply presenting a few correct facts isn't enough. If these 'new' facts don't fit their narrative they will be rejected. Like the 3 little pigs. If one has a house of sticks it seems OK to him. Then you come along with a few bricks. and try to replace some sticks. Sure the bricks seem better, but actually your bricks are making his house unstable - get rid of those bricks, your going to break my house!'.


So you have to introduce your bricks very slowly, conservatively, getting the little pig himself to become interested in bricks. But he will be uncertain and distrustful at first. Softly Softly Catchee Piggee.

So I am using a very conversational voice and a lot of inclusive language - 'we' etc. Also by posing the problem as questions first before moving to answers I am allowing the reader to go, 'hmm, he's got a point' May be time to start thinking about how good my sticks really are' You don't get someone to change their view by telling them they are wrong. You have to let them discover that they are wrong all on their own. One advantage of the Internet is that the anonymity of it allows the lurkers to not have to handle embarrassment over being wrong - there is nobody to know.

Another important factor is how to present information to people who aren't proffessional 'information processors'. Doctors, Lawyers, Engineers, Scientists, Managers. Many professions involve routinely assimilating information presented in written form. If they can't do that very well they don't make it in those professions. And I suspect most of the authors here in the forum fall into that category.


However, most people aren't like that. Although my background is in IT & Engineering I made a change a few years ago and my wife & I run a cafe in a country town (not the best decision I have ever made but thats another story). And I can assure you the vast majority of my customers couldn't handle the content or the use of language here at SkS. At times when it is quiet in the cafe I will work on my laptop on Climate Science related stuff. Regular customers, people I like will occassionally look at what I am doing and say something like 'that looks pretty technical!'. I explain what it is and they give me what I can only describe as 'the look' I have just painted myself as someone who doesn't live in their world, the world of 'ordinary people'.

So some aspects of writing style that are think is needed is not just conversational, but anecdotal, wry, repetitive, almost as if your having a beer with some mates, shhoting the brease. And slowly, safely you let the piggie play with the bricks.


So I understand your point about brevity in wording Mark and for an audience of 'information processors' absolutely valid. Brevity and conciseness are important for them because you are making them process info too slowly otherwise. But for a more general lay audience I think that is actually too brief. Look at how good politicians speak, Measured, simple concepts and repitition. And ipaying attention to the pacing of what they say to match their audience.

That is why my post is rather long-winded. It is intentional.

And given the subject matter, about peoples misperceptions about how surface temperatures are done it is exactly those people we need to reach. I suspect that the majority of people involved in surfacestations.org for example are good decent people who honestly think that the issues with poor station siting in the US, which are real, could well be distorting the reported temps. Let someone see a weather station next to an AC unit and most peoples reaction, from their constructed sense of how the world works would be - "Shit, thats BS, thats got to be distorting it hugely". Not just misunderstanding, but a perception that 'fits' how they think the world works. Even the distinction between the impact of that on Meterological uses of the data and Climatological ones will go straight over their heads.

That said their is the alternative problem of length and reader attention span. And the Net is worst for this. On a recent panel discussion on 'the Future of the Book' here in Oz, one of the panelists described the response he got to a survey of people in the publishing industry to a questionaire on the subject. He sent it by email to lots of people. Hardly any responses. Then sent it as a printed query - higher response rate. Then he sent hand written letters and this generated the highest response rate of all. In an ideal world you would get presentations like this printed, delivered to peoples homes, and let them read it with their morning cornflakes. Reread a bit tomorrow, and slowly absorb it.

Unfortunately we are working with the Net. So we have to cope with attention span/text length issues. ut if we use brevity to solve that we miss the audience that matters.

For that reason I am loathe to make the text briefer - there then is much less point posting it. Rather I would break it up. I had originally intended 2 posts - one on how its done and why, the other on the impact of the way its done on the supposed issues. If I break it further my instinct is what Rob suggested - break the first half in 2 at GISTemp. So the narrative of the first half of this is here is the problem (averaging temps) aqnd here is the answer (averaging anomalies). Without that the narrative of the first section is all problems and no answers - 'your telling me my sticks are no good, so what should I use instead A***hole?' The reader may not go onto the answer. The second part I will probably divide in two as well - problems with the network of stations - marching thermometers etc - then problems within stations - AC Units etc. Hopefully by tackling the 'less' contentious 'problems', at least as far as the general population is concerned and making reasonable observations on those subjects, when I get to the inside station problems at the end the readers will be less defensive.

Hopefully I'll have a reworked multiple post version early next week. Holidays over today, back to the salt mines over the weekend.


2011-05-20 12:44:01Writing a paper
Glenn Tamblyn




Yeh, that might be interesting. I'm not sure where that might go but lets take a look at it. As you can see from my posts. my focus has been on trying to explain the basic concepts to a lay audience rather comparing the details of competing methods but it would be interesting to look at it.

I must warn you my time available for anything that involves 'a body of work' can be very erratic.

One thing that has always struck me about the problem of calculating a global (or any regional) temp series is whether there are other branches of mathematics that could contribute.

Think of the temperature (or temperature anomaly) record as being like a Fakir's Bed of Nails. Square structure whoes x/y coordinates are Lat.Long (OK you have to adjust for the surface being a sphere). Then there are nails at various points on the 'bed'. They aren't distributed evenly and their length is variable. Each nail is a single station. And over time the length of it ''nail' changes. The problem is how do we determine the shape of of a temperature field based on these nail points and how it varies over time? Could we 'lay' a flexible sheet' over the nails and see what shape it assumes? Is that the temp field? How 'flexible' is the sheet. If it's like cling film it will fall down around each 'nail' and our sheet will look all dimpled. A stiffer sheet will have smoother contours. Getting a 'valid' shape then depends on the balance between the separation of the nails and the stiffness of the sheet. The 'stiffness' is essentially the degree of 'teleconnection' in the data, and thus determines the spacing needed between the nails.


Is there a Topologist in the room?


Rob, email me on glenn@thefoodgallery.com.au

2011-05-26 15:32:13Ready to go if no more comments
Glenn Tamblyn


I have split this into 4 posts

Of Averages & Anomalies - Part 1A. A Primer on how to measure surface temperature change

Of Averages & Anomalies - Part 1B. How the Surface Temperature records are built

Of Averages & Anomalies - Part 2A. Why Surface Temperature records are more robust than we think

Of Averages & Anomalies - Part 2B More on why Surface Temperature records are more robust than we think


If no one can see any problems, they are ready to go. I would suggest 1 per day. I will add a summary post after them that can also reference BEST and the Watts paper

2011-05-28 09:57:30
Rob Painting

Only gotten through parts 1a & b, but looks good so far. Also, it's great that you've completed such a thorough examination of this issue, "skeptics" always cherry-pick and confuse matters, and this series of posts will be a very  useful reference for the genuinely inquisitive.    

2011-05-28 14:13:40
Dana Nuccitelli

Well, I don't know about doing one per day consecutively.  There are other posts that need to be published too - the schedule can get pretty crammed!  But we'll try to keep them reasonably close together.

I went into Part 1A and made some minor formatting and punctuation-type edits.  Also, you said "so" a whole lot, so I took some of those out.  Hope you don't mind :-)

2011-05-28 18:00:13
Glenn Tamblyn


Whatever is possible Dana, I understand the schedule is tight. If poss, keeping 1A & 1B reasonably close together, with a gap to 2A & 2B.


All grammatical and punctuation fixes greatfully accepted. At school I loved Science & Math but Emglis - well that was like crawling over broken glass. I have changed my view in the decades since but what doesn't stick early seldom sticks well later.

2011-05-29 02:40:46
Dana Nuccitelli

Sounds good glenn.  I'll post 1A now, and 1B hopefully in the next day or two.  Tomorrow might be possible.

2011-05-29 03:14:58
Dana Nuccitelli

Okay 1A is posted.  I made the same type of minor edits to 1B, and put it on the schedule for tomorrow (hopefully it won't get bumped for something time sensitive).  I also changed the URLs to 1A and 1B and 2A and 2B (instead of 1, 2, 3, 4), and updated all your links (including the comment above).  We probably can't do 2A and 2B until late next week, because there's a lot of other stuff on the schedule.  I've got them on there for Friday and Saturday, Aussie time.