What is Correlation? Part Two

Last week, I demonstrated how the mathematics which go into calculating "correlation scores" is relatively simple. Today, I'd like to look at some of the steps involved to better understand what correlation scores actually mean.

In our previous example, we had four (x,y) data points. We wanted to find how the x and y values correlated. Having done so, we went a step further an examined how each (x,y) data point contributed to that correlation. The result was this table:

     x  y contribution
[1,] 3  2   0.39686270
[2,] 5  4   0.05669467
[3,] 7  6   0.01889822
[4,] 9 10   0.51025204

As we can se, x-y = 1 for three of the four data points. The one data point where that is not true is the fourth data point. For that data point, x-y = -1. That makes it an "outlier," a point notably different from the rest.

Notice how the outlier gets the greatest weight of all the data points (.51 as opposed to .40, .06 and .02). One might wonder, why does this happen? To understand, we can look back at how these contribution scores were calculated:

contribution = (x - mean(x)) * (y - mean(y)) / (3*sd(x)*sd(y))

You can go back to the previous post if you need a refresher on how we got this formula. The important thing to understand is for each data point, we subtract the average value of that variable. For each x, we subtract mean(x) and for each y we subtract mean(y).

That is in the numerator (the top portion of a fraction). As everyone knows, the larger the value in the numerator, the larger the result will be. Similarly, everyone should understand the closer a number is to the average value, the smaller your result will be when you subtract the mean value.

What this tells us is data points which are less like the rest will tend to get more weight in correlation calculations. There are further complications. Outliers will also affect the mean values you calculate. They will also increase the standard deviations the numberator is divided by. Additionally, since the x and y values (less their means) are multiplied together, having only one of the two values in a pair be an outlier will have a different effect than if both points are outliers.

Understanding those details can help us quantify the effect of outliers. We don't need to understand them to ounderstand the concept though. The concept behind outliers getting more weight in correlation calculations is the assumption the data has a normal distribution. To understand what that is, here is an image from Wikipedia:

This image shows three different curves, each with a "normal distribution." Each curve has a single peak, with values tapering off at the same rate in either direction. How steep the peak is or how quickly the values taper off on either side can vary, but normal distributions will all have this same general shape. For correlation calculations, it is assumed the data has such a shape.

The reason outliers get more weight in the calculations is we expect there to be far fewer of them. We don't want to give each of the 100 points near the peak of the curve the same weight as the one or two points we have near the thin "tails" of the curve.

Of course, that assumes the outlier is a valid data point. Often, outliers are not. When an outlier is caused by some sort of data error, it can be bad to give it extra weight. In fact, the larger the data error, the more weight you'd give the outlier. Issues like that mean we cannot give a perfect answer to every question just by looking at the math.

But looking at the math can help us learn a lot about our data sets. An outlier which doesn't fit on a normal distribution curve will cause problems for our calculations, but that's not the only time our data will fail to have a normal distribution. To demonstrate, let's look at these results from a question on a survey:

This is clearly not a normal distribution. 1,145 people took this survey. For the question seen in this example, there were asked how much they agree with a statement, with 1 being "strongly disagree" and 4 being "strongly agree." The idea they were asked to agree or disagree with? NASA faked the moon landing.

Of course people nearly universally disagreed with the idea NASA faked the moon landing. That the results don't have a normal distribution is to be expected. The question is, what effect would this lack of a normal distribution have on any correlation scores? To try to get an idea, let's look at another histogram of responses to the same survey:

The distribution of this data is a bit closer to a normal distribution, but it is still clearly "skewed" toward one side. Why is that? Well, people were asked to agree or disagree with the idea the federal government knew the attack on Pearl Harbor was going to happen but let it happen so the United States would join World War II. That isn't as crazy as the idea of NASA faking the moon landing.

Looking at these two histograms, you can probably guess there would be a positive "correlation score" between responses to these two survey items. Let's check. We start by assigning our variable:

x = surv$CYMoon
y = surv$CYPearlHarbor

And check the results with the built in correlation calculation function:

[1] 0.2252181

Telling us there is a strong correlation between people's agreement/disagreement with the idea NASA faked the moon landing and the 1940s federal government allowed the Japanese fleet to bomb a United States military base in order to justify the United States entering into a war.

But what does that correlation score mean? Does it mean people who believe in one conspiracy theory are inclined to believe the other? That's easy enough to check. Let's make a table showing how people responded to both survey items:

x     1   2   3   4
  1 420 524 112  11
  2   2  49  17   0
  3   1   1   0   2
  4   0   2   2   2

A total of 10 people responded with a 3 (agree) or 4 (strongly agree) with the idea NASA faked the moon landing. Of them, four disagreed with the idea of the federal government intentionally allowing the Pearl Harbor attack to happen. Only six people claimed to agree with both conspiracy theories.

If only six people claimed to agree with both conspiracy theories, why then do we find a positive correlation between the answers to each survey item? To find out, let's calculate the correlation scores as we did before. I don't want to show a table of all 1,145 responses. Instead, I'll shameless crib off some code from Steve McIntyre, proprietor of Climate Audit to make a cleaner table:

Stat= data.frame(CYMoon=c(1,2,3,4,1,2,3,4,1,2,3,4,1,2,3,4),PearlHarbor=c(1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4),count=c(  with(surv,table(CYMoon,CYPearlHarbor)) ))
Stat$dot= (Stat$CYMoon-m[18])*(Stat$PearlHarbor-m[15])
Stat$normdot= (Stat$CYMoon-m[18])*(Stat$PearlHarbor-m[15])/(sd(surv$CYMoon)*sd(surv$CYPearlHarbor))/(N-1)
Stat$contribution= Stat$normdot*Stat$count

The math is the same as before, but McIntyre's code is cleaner. If you'd prefer to use the previous formula and generate a table of over 1,000 lines, that'll give the same results: You could also clean up the code I've written to make it more modular. Whatever you do, you'll get the same final results (I'm excluding the fourth and firth columns as they are intermediary steps):

   CYMoon PearlHarbor count contribution
1       1           1   420  0.098971237
2       2           1     2 -0.005269441
3       3           1     1 -0.005505087
4       4           1     0  0.000000000
5       1           2   524 -0.036637985
6       2           2    49  0.038306391
7       3           2     1  0.001633446
8       4           2     2  0.004970258
9       1           3   112 -0.042054369
10      2           3    17  0.071370195
11      3           3     0  0.000000000
12      4           3     2  0.026691422
13      1           4    11 -0.007491562
14      2           4     0  0.000000000
15      3           4     2  0.031821024
16      4           4     2  0.048412587

Like in our previous example, if we sum the contribution scores:

[1] 0.2252181

We get the correct calculation score. Now that we know that, let's check where this correlation score came from. The largest contribution score is in the (1,1) coordinate pairing, where 420 people said they "strongly disagree" with both conspiracy theories. This contributed 0.099 of the 0.225 total correlation. Another 0.032 and 0.048 came from four people who claimed to agree with both conspiracies. Another 0.071 came from 17 people who said they disagreed with one conspiracy but agreed with the other.

This tells us of 1,145 people who took this survey, a correlation between two survey items stems primarily from 4m20 people who strongly disagree with both conspiracies, 17 people who agree with one but disagree with the other, and four people who agree with both conspiracies.

Let's try the same with a different pairing of conspiracies. This time, we'll use the moon landing conspiracy and the idea global warming is a hoax. For the sake of space, I'll skip some of the steps. Here is a frequency table showing how people responded to the two items:

table(surv$CYMoon, surv$CYClimChange)
      1   2   3   4
  1 892  53  65  57
  2  39  20   5   4
  3   2   1   0   1
  4   2   2   0   2

The histogram showing how many people agreed global warming is a hoax:

The correlation score between the two items:

cor(surv$CYMoon, surv$CYClimChange)
[1] 0.1265264

And finally, the table showing how this correlation score arises:

   CYMoon ClimChange count contribution
1       1          1   892  0.081511056
2       2          1    39 -0.039846588
3       3          1     2 -0.004269590
4       4          1     2 -0.006495765
5       1          2    53 -0.008748525
6       2          2    20  0.036911683
7       3          2     1  0.003856235
8       4          2     2  0.011733771
9       1          3    65 -0.027398354
10      2          3     5  0.023564379
11      3          3     0  0.000000000
12      4          3     0  0.000000000
13      1          4    57 -0.038643707
14      2          4     4  0.030320669
15      3          4     1  0.015838294
16      4          4     2  0.048192843

892 people said they strongly disagreed with the idea NASA faked the moon alanding and strongly disagreed with the idea global warming is a hoax. That contributed 0.082 to the total correlation score of 0.127. Two people said they strongly agreed with both ideas, contributing 0.048. One person claimed to strongly agree with one idea and agree with the other, contributing 0.016.

Of 1,145 respondents, a total of three people claimed to believe NASA faked the moon landing and global warming is a hoax. 892 people found both ideas laughable. That is what the math indicates. What then, should one say about this math?

A competent researcher would notice this data is so heavily skewed any correlation scores we might calculate are meaningless as the assumption of a normal distribution in the data is violated to an extreme. That's not what was done though. This is the title of a paper published based on this survey:

NASA Faked the Moon Landing—Therefore, (Climate) Science Is a Hoax

That paper got its authors a ton of attention and caused them to be "experts" in their field. That's a shame. In Part Three, I will prove these authors would have gotten the same results they published if nobody taking their survey had claimed to believe NASA faked the moon landing. They would have gotten the same result they published if nobody taking the survey had claimed to believe global warming is a hoax.

These authors published results which didn't require any data that supported them. They did so because they used naive correlation tests on data with a heavily skewed distribution, violating the requirements of those tests. They then chose not to examine their data to find out where their results came from despite it being very easy to do.

And they're not the only ones. Many "scientists" are doing the exact same thing.


  1. Hopefully this post turned out okay. If you have any questions or corrections, feel free to let me know. In the meantime, you can find the data file for the paper this post discusses here. This data file includes more data than the authors published. It turns out the authors chose not to report results for a number of questions they asked on the survey, and they chose not to publish the data for those questions either.

    I don't think authors should get to pick and choose what results and data they let people know about as that is just asking for people to cherry-pick results that fit a particular perspective. That's a matter for another day. For now, I just want to make sure all the data is available for people who might be interested.

  2. Very interesting. Thanks.

    I don't suppose you'd consider doing a similar exposition on the law of large numbers?

  3. Thanks JonA. I'm not sure what you have in mind though. The law of large numbers pretty much just says the more times you repeat an experiment, the more likely you are to get an average close to the correct result. I'm not sure how much there is to say about that.

  4. It nominally reduces the standard error of the mean. However, I think it has the same requirements on a normal distribution (I.I.D)
    as your correlation example. I often see tLoLN widely applied to temperature data and wondered if it really was valid to do so.
    Temperature data is generally regionally correlated; measurement error is a combination of random bias and systematic error. It's
    relatively common to see average temperatures reported to a precision which is greater than the resolution of the measurement

  5. Ah. That's a more complicated topic than just the "Law of Large Numbers." Technically, that law is a mathematical law, meaning it only applies in the sense of sample spaces of objectively definable populations. When you move to the real world, there are many reasons it can be wrong. As an obvious example, it doesn't matter how many times you measure something with a thermometer which is miscalibrated to always read one degree high. Every measurement will be one degree high. Doing a million measurements will not magically make the bias caused by the miscalibration go away.

    I don't know of a way to discuss that as a concept in detail while remaining interesting. It's true simply throwing more data at a problem will not make it magically better in the sense of that doesn't do anything to address the possibility of systematic biases/methodological problems, but discussing things like that requires examining each situation individually. I might be able to do a posts on a particular case, like how this "law" applies to the modern temperature record, but I think it would need to be a module within a post (or series of posts) rather than something that stands on its own.

    I actually have a lot I would like to say about the modern temperature record, and this issue would come up in it. I don't know that I"ll ever find the time and interest to write it out though. The math interests me, but there has been so much garbage published (on both sides) about the issue I find it difficult to want to talk about the subject.

    Though now that I say that, there is an interesting play between that law and the subject of the post The highlight correlation for Lewandowsky's paper is only .127. The amount of variance that explains (as given by r^2) is 1.6%. That is, even if one believes the correlation Lewandowsky found means something, the size of the effect he focuses on explains less than 1/60 of the variance in the data. That's minuscule, but it is "statistically significant" because there were so many respondents. Which shows why "statistically significant" doesn't mean "significant" in any sense a layperson would imagine.

    Anyway, I'll keep the idea in mind.

  6. Thanks for the detailed reply Brandon. I think a disinterested discussion of the statistical
    methods used in modern temperature record products would be quite useful for the
    non-subject matter experts trying to keep up. However, I can well understand your lack of
    desire to produce such a thing :-).

    I seem to encounter a lot of people with similar views to myself - not necessarily sceptical
    of AGW itself but very sceptical of the various data analyses and models. For instance,
    Tony Heller has a very interesting scatter plot of CO2 vs. temperature adjustments here:


    This correlation could of course be spurious but it does look suspicious.

  7. See, that people refer to Steven Goddard is part of why I don't want to talk about this issue. Goddard is a liar and a hack. He's made colossal mistakes in the past, including ones fundamental to his conclusions. His discussions of adjustments tend to involve blatant cherry-picking and/or completely invalid mathematical procedures. On top of this, he secretly edits his posts when he realizes he's made errors in order to cover them up. Combine that with him being an arrogant prick who refuses to have anything resembling a real discussion with anyone who disagrees with him, and he is pretty much the worst person anyone could cite.

    That people like Goddard are happily embraced by so many Skeptics, to the point his (completely erroneous) work gets promoted on the floor of the United States Congress, makes me think there'd be little point in me discussing the subject. I've criticized the modern temperature record many times, but why would anyone listen to anything I have to say? Either I'll get dismissed as a conspiracy nutjob like Goddard who makes things up, or I'll get ignored because the issues I point out aren't as "sexy" as what fanatics are throwing out.

    For the record, that chart shows the (claimed) effects of adjustments to USHCN, a temperature record made only for the United States. The United States has always been an outlier in its adjustments for a variety of reasons. The effects of adjustments for it are nothing like the effects for other areas. Whatever one thinks of them, it's important to remember the United States makes up only a tiny portion of the planet.

  8. Thanks Brandon. I generally don't believe in dismissing something simply based on source. It's
    why I'm sceptical of the claims of the 97% for instance. Tony Heller/Steve Goddard may indeed
    be all of the things that you say - I don't have enough experience with him or his work to
    quibble. That's why a lot of us look to people like you who seem to have the ability to dispassionately
    analyse claims like these. I thought the linked graph was pertinent to the subject of your post (correlation).

    I take your point about the USHCN - however, isn't the USHCN the dominant source of temperature
    data contributing to global temperatures?

  9. I don't believe in dismissing something purely because of who said it, but I do think the more often a person is wrong, the more hesitant people should be to repeat what he says. I don't ask people to ignore things just because Steven Goddard says them, but I do ask people to acknowledge his behavior and how that impacts the credibility of what he says. This is the same point I've made with other people, and it all comes back to the same point. As long as the Skeptic crowd doesn't hold people accountable, people won't trust it. That's true of any group.

    Anyway, USHCN isn't the dominant source of data for global temperatures in any meaningful sense. The United States does have the most densely sampled temperatures (though a significant amount of that data is not included in USHCN), and it does have a number of the longest temperature records. On the other hand, the United States makes up only a small portion of the globe. It's influence on the temperature record is roughly proportional to that size. Excluding data for the United States would have a minimal impact on the temperature record (except in earlier periods where there is little data in general). Particularly, data for temperature data for oceans has a far greater impact (while having far more potential problems).

    Since you ask though, I will bring up a funny thing about how Goddard made a number of his claims. One of the major mistakes Goddard made, one he defended as not being a mistake, was to combine each temperature record together by simply averaging them all. He didn't do things like account for the baseline temperature of Alaska being different from the baseline temperature in Arizona. That created issues when stations dropped out for no reason, but it also created a hilarious problem. By simply averaging all stations together, Goddard gave equal weight to each station regardless of where it was located. That meant if 10 temperature stations were located in the same city while only one temperature station existed in another, the one city would get ten times the weight.

    Normally people would use something like grid cells to try to give weight to their data based on area rather than number of temperature stations. I think Goddard eventually started to, but for quite a while, he didn't. And he said it was right not to. It was silly. That the state of Pennsylvania may have recorded and shared more data than the nation of Russia doesn't mean it should get more weight in the global temperature record when it is ~0.1% the size.

  10. Back to correlation. I hadn't noticed, but the effect on the correlation coefficient (r) of changing a y-value is asymmetric. Take your toy data set, and vary y(4) from its "natural" value of 8. [That is, the value at which r=1.] Obviously that's a maximum for r. A decrease in y(4) from this optimal value, reduces r more than an increase in y(4). I've plotted the effect here.

  11. HaroldW, you may not have noticed it, but that's not actually surprising. Remember, the denominator in the formula uses the standard deviation of both x and y. In the formula for the standard deviation, you both square and take the square root of values. That means linear changes in the data set will not produce linear results.

    Looking at the normal distribution curve shows this is what we should expect, on a purely conceptual level. Moving one unit toward the center of the curve does not have the same effect as moving one unit away from the center.

    That said, the shape of your graph is interesting. I hadn't thought about things that way. I get why it has the shape it does, but it's not something which would have occurred to me on its own.

  12. Hi Brandon.. why should we trust that the dataset contains the 'correct' actual data from the kwik surveys?
    It is just a spreadsheet that Lewandowsky has created from the original data (he has chopped out a number of responses,etc)

    Silly question?

    well he has archived the dataset you provide at Bristol.. (moved from it's UWA location)

    He has also archived an extended dataset for the the exact same number of respondents, it also includes the Iraq Question, age and other variables

    The problem..

    There are over a hundred differences in the responses... between the two datasets, where they should be exactly the same..

    an example:

    In the CO2Tempup column, in the 'extended' data set there are no 1s, it is all 2 3 4.
    The 1s in the original data set, LskyetalPsychSciClimate.xls, have been turned into 2s in the extended.

    (or vice versa, Roman (that Mcintyre has) obtained a dataset from another co-author ages ago, which seems to be the extended dataset, it had Iraq War answers)

    We now can't trust either dataset, because of the different responses, but why should we even trust the same responses...
    We now need to see the raw unadulterated data, as provided from Kwik survey.

    but I've been asking the journal, and Lew and the University for that for over 5 years)

  13. Barry Woods, I do not believe I have ever heard someone make those claims about the data before. Do you have a copy of the data set you say has been so altered? I wouldn't be able to convince the university to release it to me as I am not an "academic," and I have no desire to file an FOI request.

  14. Hi Brandon.
    If I give it to you, then I run potentially the risk of Prof Lew using that as a reason for Universities not release data under FOI. I was refused in the 1st instance following there data request procedure, despite copying them the invitation from the Chief Editor of the journal that I submit a comment to the journal. Lew has been lobbying not to release data without consideration of who people are and their motivations (his red flags nonsense) and that data will be used to 'attack' scientists.
    I do believe another academic is soon to bring this to the attention of the journal/University..

    I spent a few months going through the date request procedure and appeals... I then filed a FOI request (just an email, with a link to the Bristol data repository location, and I stated that I understood it had already been released to another party under FOI, and they sent it to me no questions asked a few days later.

    If you still don't want to FOI, watch this space for a while, and see what becomes of events. (if nothing happens, I'll think again about sending it to you) FYI -A UK professor (who FOI'd it) pointed out the differences to me.. these:

    "In the CO2Tempup column, in the 'extended' data set there are no 1s, it is all 2 3 4.
    The 1s in the original data set, LskyetalPsychSciClimate.xls, have been turned into 2s in the extended."

    are his words. (but it could be the other way around, or both incorrect)

    We don't know which data set has been "altered", it could be either, or even both 'wrong'
    The extended one is the more complete one, possible that is the 'original' and the other one is different, or vice versa.. Both data files are just .csv files, that Lew has processed the original raw kwik survey data, where he stripped out any meta data, timestamps, IP addresses, referring urls (if captured) other lifestyle questions and a couple of hundred of responses, as he described in the paper. I want to see the original raw data, how can we trust either of these files to be the correct ones.

  15. I don't think you sharing data you obtained under FOI could be used to avoid sharing data requested by FOIs in the future. As far as I know, FOI responses don't carry any restrictions on disseminating what gets released. Then again, people and universities have pulled all sorts of nonsense in trying to avoid releasing data, so maybe it's possible.

    As for an FOI, the university is in the United Kingdoms. I live in the United States. Can I even file a request? I think they're limited to residents of the country. I'm not sure. If I can file a request and it's not too much trouble, I'll go ahead and do it. I have no stomach for chasing data though. I've spent enough time trying to get people to release data which should already have been released. At this point, if people aren't going to release their data, I'm inclined to just ignore what they write.

    Which I know is exactly what they want.

  16. JonA

    1. Heller's plot. Ive searched his code for how he created it. No luck.
    2. It gives a result that is what you would expect IF
    A) the true temperature is correllated with C02
    B) the raw data is biased cool.

    Think about it. take a relationship where Y = f(x,...) you will of course find a correlation between y and x.
    Now, suppose that your y measurements have a bias, such that to correct them you have to add z

    Now... what do you expect about a correllation between z and x?

    in other words, heller's chart shows that the corrections are largely correct, because 1). We know temperature should roughly
    correlate with c02, and 2) if we have to add 'z" to the temperatures to correct for known bias, then we can expect that z
    will correlate with x.

    The other point would be that adjustments for africa go the exact opposite was as america.

  17. You know, normally I'd be inclined to roll my eyes but otherwise ignore a nonsensical claim like this by Steven Mosher:

    in other words, heller's chart shows that the corrections are largely correct, because 1). We know temperature should roughly
    correlate with c02, and 2) if we have to add 'z" to the temperatures to correct for known bias, then we can expect that z
    will correlate with x.

    But one of the errors he makes is surprisingly relevant to the topic of these posts. I think it might make a good example for a post. I mean, yeah, Mosher's entire argument is a non-sequitur since he relies upon unstated assumptions being assumed as fact. That makes it silly on its face. However, try looking at his argument under the assumption his premises are true. We posit:

    1) Temperatures are expected to (roughly) follow CO2 levels, which have been rising.
    2) There is a cool bias in a data set. Corrections to this will warm the data, resulting in an adjustment series with an increasing trend.
    3) These two series correlate, showing the adjustments to the data are largely correct.

    If we're arrogant and obnoxious enough, we might be able to convince people this makes some sort of sense. It doesn't though. I think I'll ask readers to try to figure out why in my next post on correlations.

  18. I've only just realised this post has updated comments. Brandon's post on
    Sept 2 at 11:49am, in response to SM's, strikes straight to the heart of the
    concerns I have. SM argues that of course there is a correlation as CO2
    is the primary driver of the temperature change - a clear case of circular
    reasoning in my opinion. To my sceptical eye, one also can't rule out the
    possibility that the temperature adjustments are conforming to the output
    of some model which assumes a priori what the temperature *should* have
    been based on CO2 concentration. I don't think the last 20 years temperature
    change would correlate to CO2 anywhere near so well. If the model doesn't
    work over the entire time series then the model is wrong.

    SM also can't have looked very hard for TH/SG's data and code as it's all over
    his site. Python and C++. I believe he uses GHCN raw temperature series
    as his data source. Please, critique away.

  19. JonA, I'm going to make two separate commen ts as there are two separate issues here. Steven Goddard doesn't post code for figures like this. He's posted code for a general temperature reconstruction once, code which showed his methodology and let people confirm he was doing it all wrong. (No, if there are 30 temperature stations in one city, you should not give them the same weight as if there are 30 stations in the entire state of Alaska.)

    That said, Steven Mosher's comments on the lack of code can be rather bizarre. One time, he made multiple comments lambasting me because I didn't provide code for a chart I posted. I had provided the data used in the chart. All I didn't provide was code that... showed how I took a column of data and made a picture of it.

    I'm all for sharing data and code. Goddard doesn't come close to meeting the standards I expect of people. At the same time, I wouldn't take the fact that Mosher complains about data/code being unavailable as indicating there is actually a problem.

    Edit: To be clear, Mosher said he examined the code Goddard shared and couldn't find anything indicating how this chart was created. It's not as though Mosher is claiming to be unaware of any code having been shared. He's saying the code that was shared doesn't cover this chart.

  20. Second comment, there are many problems with what Mosher said, but the main one is found here:

    in other words, heller's chart shows that the corrections are largely correct, because 1). We know temperature should roughly
    correlate with c02, and 2) if we have to add 'z" to the temperatures to correct for known bias, then we can expect that z
    will correlate with x.

    Now, let's take it as a given United States temperature measurements are biased as Mosher says. Let's take it as a given we expect there to be a cooling bias in the measurements that grows over time. Taking that as a given, and recognizing CO2 levels have increased, we could say we expect the bias correlates (negatively) with the rise in CO2 so the adjustment to fix the bias correlates (positively) with the rise in CO2.

    In this situation, as CO2 levels go up, we should expect the adjustment for biases in the US temperature record to go up. Mosher says that proves the adjustments are largely correct. But why? What kind of correlation would there be if I adjusted the US temperature record by 1 to 20 degrees, with the number chosen increasing over time? A positive one.

    That's right. I could make outlandishly incorrect adjustments to the US temperature record and get the same correlation Mosher claims proves the adjustments are correct. That's because correlations look at the relative changes. Adding 1 to 20 to the US temperature record would have the same effect as adding .01 to .2 to it.

    There's a lot more wrong with what Mosher said, but that's the error I find most interesting given the topic of this post. Correlation scores cannot tell us adjustments are correct, largely or otherwise. When using correlation tests, a series that goes from 1 to 10 is the same as a series which goes from 10 to 100. The absolute magnitude of the values is irrelevant.

  21. Thanks for the detailed responses Brandon. I agree wrt the sample bias issues you
    raise (areal weighting) - I did try and raise this issue with SG/TH but simply
    received a cryptic Donald Knuth quote in return. I'll try just asking him how he
    generated the chart.

  22. JonA, I don't know what I'm supposed to take from that. Steven Goddard's response is so off point I'd call it dishonest. Notice how he doesn't provide a link to what he claims to be responding to? It was important that he didn't. Try looking at what Nick Stokes said. He talked about the lack of sourcing for two figures Steven Goddard has frequently promoted. As you'll see if you check the link, both figures in question use (supposedly) official, published results. They don't involve any calculations on Goddard's part. The code Goddard shows has absolutely nothing to do with the figures Stokes referred to.

    For Goddard to mock Stokes because he ostensibly couldn't find something in Goddard's code which created an entirely different figure is absurd. Goddard's response has absolutely nothing to do with what Stokes said. Anyone can see this just by comparing the figures Stokes discussed with those Goddard discussed. They're entirely different. I don't know what a person could be expected to learn from this.. When one compares what Stokes said (in context so they know what he was referring to), Goddard's response seems deranged as he insults Stokes based on things that have absolutely nothing to do with what Stokes said.

  23. Sorry Brandon, I didn't post that link to bring attention to any back and
    forth between Stokes and SG/TH. The linked post explains how the CO2 vs.
    adjustment series is generated (rather how SG/TH generates it). Essentially
    I'm following up on my post (6) which first mentioned it on this thread (as the
    subject was correlation) and SM's post (16) which queried the code and data
    source for the plot.

    SM has expressed the opinion that this correlation shows that the adjustments are
    'correct' as commented in this thread and re-iterated somewhat more succinctly
    recently at JC's blog:


    I think it's one of potentially many explanations and that statistics like this don't prove
    anything. I think it certainly needs looking into which was really my only point.

  24. I get there may be information to gain from Goddard's post, but I can't ignore how the post is complete garbage. I don't think that post could be any worse than it is.

    Anyway, based on that post Goddard only just now updated his code. While Goddard mocks Stokes for having been unable to find it, he doesn't actually say where Stokes could have found it before this latest update. That his code now shows how he made that plot doesn't do anything to address people's concerns about being unable to find it in his code before. I can't rule out the possibility that code was available somewhere before, but I've downloaded the code Goddard linked to in the past, and I couldn't find the code for that figure. Combine that with the fact Goddard doesn't say where a person could have found the code before yesterday, and I suspect Goddard never shared the code before yet is mocking people for having been unable to find it.

    It'd hardly be any more nonsensical than the rest of that post.

Leave a Reply

Your email address will not be published. Required fields are marked *