Splicing, Right or Wrong

An issue I discussed in my recent eBook is how Mark Steyn, widely admired figure within the Skeptic community, claims Michael Mann spliced instrumental temperature records onto his (in)famous hockeystick to cover up the fact proxy temperatures (estimated from things like tree rings) were going down. I discussed this because that claim is entirely false.

However, that Mann did not do this does not mean other people have not done it. I was recently surprised to see it is, in fact, an accepted practice within the paleoclimate community these days. This surprised me because years back when a user said:

Whatever the reason for the divergence, it would seem to suggest that the practice of grafting the thermometer record onto a proxy temperature record – as I believe was done in the case of the ‘hockey stick’ – is dubious to say the least.

Apparently holding the same incorrect belief as Steyn (misinformation tends to spread when nobody corrects errors like Steyn's), Mann responded:

No researchers in this field have ever, to our knowledge, "grafted the thermometer record onto" any reconstruction. It is somewhat disappointing to find this specious claim (which we usually find originating from industry-funded climate disinformation websites) appearing in this forum.

Mann said this in late 2004 so I can't fault him for being unaware of what would happen after 2010, but given the response Mann shows to this accusation, I find it strange this practice would be an accepted one a mere ten years later. Plus, I thought it was interesting nobody has pointed out any recent examples of it happening despite at least one being easy to find.

Before getting to a recent example, I want to revisit a bit of history. Mann's (in)famous hockey stick papers contained this figure (MBH Figure 7):

Some people, including Steve McIntyre, claim this shows Mann spliced instrumental temperatures onto reconstructed temperatures. The caption for it does say:

Relationships of Northern Hemisphere mean (NH) temperature with three candidate forcings between 1610 and 1995. Panels, (top to bottom) as follows. “‹Å“NH’, reconstructed NH temperature series from 1610–1980, updated with instrumental data from 1981–95. “

Which certainly supports that interpretation. However, the "splice" is clearly identified in the figure's caption, and if you look closely at the top chart of the figure, you can see the instrumental data is plotted differently than the reconstructed data. It takes a careful eye to see, but it is there. One might be concerned about using spliced data like this for calculations like those used in correlation coefficients in the bottom chart, but at least the splicing is made rather clear. That's not the case in some examples.

A paper which came out a bit later than Mann's contains a troubling example of splicing instrumental data onto reconstructed data. This paper is Crowley (2000). It's freely available here, though unforunately, I was unable to find a free copy of Crowley and Lowery (2000) which it takes the series in question from. Here is Figure 1 of the paper:

In the caption for it, we see:

Fig. 1. Comparison of decadally smoothed Northern Hemisphere mean annual temperature records for the past millennium (1000–1993), based on reconstructions of Mann et al. (Mn) (11) and CL (12). The latter record has been spliced into the 11-point smoothed instrumental record (16) in the interval in which they overlap. CL2 refers to a new splice that gives a slightly better fit than the original (12).

The author, Thomas Crowley, openly states he spliced instrumental temepratures onto the reconstructed temperatures. If you dig into the two papers, you can find the splice begins at ~1870. That means ~120 years of instrumental temperature data were spliced on, and unlike when Mann did it, they were plotted without any visual distinction. Interestingly, Crowley fails to note he has spliced data onto Mann's reconstruction as well, causing the red line in this chart to track the blue line better and further into the future than it would have otherwise.

Now, that was a 2000 paper. Mann was the lead author of a 2003 paper in the Eos journal with 12 other authors. It included this figure:

One might think the inclusion of the line labeled "Crowley and Lowery" would indicate Mann was familiar with the work done on and around this reconstruction and thus, familiar with how instrumental data was spliced onto reconstructed data.

I'm not bringing that up as a "Gotcha!" though perhaps such would be justified. I bring it up because it there is an interesting factoid most people have never noticed. That yellow line labeled "Crowley and Lowery" is not actually taken from any work done by Crowley and Lowery. Instead, it is... Mann's (in)famouse hockey stick. Somehow, this figure includes the same temperature reconstruction from Michael Mann (and two co-authors) twice, once labeled as Crowley and Lowery.

I have no idea how that happened. The error in the paper has never been corrected. It hasn't even been acnkowledged. Mann published a 2004 paper in which he said:

The Crowley and Lowery [2000] series shown
here replaces an incorrect version of the series shown in similar previous comparisons [e.g., Mann et al., 2003a].

But somehow I don't think it's reasonable to say that's acknowledging the error. If Mann publishes his reconstruction while labeling it "Crowley and Lowery," that is not publishing "an incorrect version" of Crowley and Lowery. This isn't important for the issue of how acceptable paleoclimatologists feel splicing instreumental temperatures onto reconstructed temperatures is, but it's too funny not to bring up.

With that diversion out of the way, let's look at another publication from the same general time period. In 1999, Mann was involved in the creation of a figure for the cover of a World Meterological Organization (WMO) report. This figure:

Shows three paleoclimate reconstructions that are remarkably coherent in the modern period. It turns out the reason they agree that much is the people who made the figure spliced instrumental data onto the reconstructions and pretended they were one and the same. This was particularly egregiouos for the green line as Keith Briffa's reconstruction had also been truncated to 1960 because it supposedly produced bad results after that. That means data was deleted and then replaced with instrumental data... and it was impossible for an onlooker to know.

That was a pretty extreme example of splicing instrumental temperatures onto reconstructed temperatures. It has received a fair amount of attention. Given how negative that attention was, one might expect researchers would avoid splices like these in the future. At a minimum, you might think they'd do what Mann did in 1998 and plot the two types of data in a manner which made them visually distinctive.

However, I recently read a 2012 paper which I hadn't paid much attention to before. I was trying to track down what tree ring series had been used in another paper, and that paper cited this one for some of its series. I can't find any listing or archive of the data as used in the paper (I plan to contact the authors in the next few days). The authors say:

The tree-ring chronology network available for reconstruction of summer temperatures is shown in Fig. 1 as a series of filled red triangles. It is comprised of 323 annual tree-ring chronologies previously identified as potential predictors of the Palmer Drought Severity Index (PDSI) over monsoon Asia (Cook et al. 2010a) and 99 new treering records contributed by members of the PAGES Asia2k project, for a total of 422 chronologies.

This is a bit strange as even this paper notes:

Although neither property is desirable for our specific application, and almost certainly limits the quality of the temperature reconstruction that is currently achievable, this new tree-ring network is actually denser than the 327-chronology network used to successfully reconstruct drought over monsoon Asia (Cook et al. 2010a).

The Cook et al. 2010 data set contains 327 series. Why the 2012 uses only 323 of them is unexplained. I don't know what, if anything, that might mean. Perhaps a reader can shed some light. In the meantime, the reconstruction for this paper can be seen here:

Looking at this figure, there is nothing to suggest instrumental data has been spliced onto anything. The caption notes it:

Fig. 9 Asia2k summer temperature reconstruction from tree rings for temperate East Asia with 1 RMSE uncertainties (a) and the changing number and average replication of the chronologies used (b). The annually resolved values are in red, the 10 year low-pass spline filtered values are in dark blue, and the uncertainties are in pale blue. The time period covered is 800–2009 C.E., but the 1990–2009 data are instrumental data for comparison to the past estimated temperatures

As does the text:

The full period covers 800–2009 C.E., with the data since 1990 being instrumental for easy comparison of recent warming to earlier reconstructed temperatures.

For which the authors deserve credit. It's not like this was completely hidden as in the 1999 WMO report. Still, I can't help but wonder, why would you ever publish a chart showing instrumental and proxy temperatures on the same line with nothing to distinguish them? In 2004, Michael Mann thought it was "disappointing to find [the] specious claim" this happens on his website. Only, the claim is clearly true. Respected Paleoclimate researchers do this.

I don't have a concluding thought on all this. Personally, I think splicing series together like this is a bad idea. I don't know that it is necessarily "wrong" though. If I'm being honest, I'm more curious about those 323 series.

25 comments

  1. Wrong suggests guilt. Misleading is probably a more relevant term and does not imply moral judgement.
    I would take the definition of data splicing to be like it is for ropes, two different series joined together. They are combined to give an unbroken line. Whether they are distinguishable or differentiated isn't really relevant. A good example of splicing would be sea level rise, like here:
    https://i1.wp.com/climateadaptation.hawaii.gov/wp-content/uploads/2015/11/Brief-1-Figure-4.png
    There is no explanation as to why the kickpoint is at the join. There isn't even overlap to show if the information conveyed is measuring the same thing.

  2. Does "wrong" suggest guilt? Getting an answer wrong on a test doesn't indicate guilt, but I'm not sure if "doing something wrong" does. I am sure this is wrong though:

    I would take the definition of data splicing to be like it is for ropes, two different series joined together. They are combined to give an unbroken line. Whether they are distinguishable or differentiated isn't really relevant.

    it is certainly relevant if people can tell two series are spliced and where. Knowing what portion of a series comes from what type of data can be essential in interpreting what the results show. There are many other issues which can matter, but that is one which certainly matters. The figure which caused this post may not be morally wrong, but the decision not to make the different types of data distinguishable is certainly relevant to people who might view the figure.

  3. There is on important issue you left out in your discussion on the graphs; context. In the middle of a scientific paper, a annotated graph with the explanation underneath is correct. However, many of these graphs are stripped out then used in presentations to politicians or the general public, and their original captions are gone. That is morally wrong. This is often a deliberate policy so as the message isn't diluted.
    The prime example of this is the hockey stick as the backdrop to the public announcements on TAR. That was a single line with pseudo error bars. It is also the case I linked to on sea levels. How many people would have actually known where the graph came from, let alone looked up the references and read the text? Information for the public needs to be clear and unambiguous, with things properly referenced. There also needs to be caveats explaining why the data came from different sources.

  4. I can't agree that the original hockey stick graph was displayed out of context in a way which changed it's meaning. I think you're right about that happening and being a serious problem, but in that particular case, no context could help. The graph was bogus, and the lead author on the study intentionally lied about it on numerous occasions, including in the IPCC report which made it famous. Shameless self-promotion, I've written a short eBook which demonstrates this. In simple terms, the graph was created via an inappropriate methodology which was heavily biased toward producing the desired results (though I don't think this was known by the authors at the time), and even so, the authors had to lie in order to maintain those results. No amount of context could fix that.

    Otherwise, I agree with what you say.

  5. That's not a 2006 WMO report.
    I think the 'trick' part is they are intending to deceive. Even with the IPCC chart, putting a red color at the end hides all the others underneath.

  6. MikeN, you're right. I'm not sure why I typed 2006. It was 1999. 2006 was the year of the NAS report, but I shouldn't have mixed those two up. It wouldn't have even made sense since by 2006 the Team were showing additional reconstructions. I'll get that fixed.

    Related to what you said on the other topic, there are lots of details and nuances as to what season/data to use. For instance, trees tend to grow during specific seasons (not always the same season based on species and climate). A tree's growth might only be affected by temperatures of a few months, not of an entire year. At the same time, temperature changes for those months might correlate well with annual temperatures. How do you tease out all the details? There's no clear answer. Generally, what you (should) do is just use the same months in the instrumental data as you expect to be relevant in the proxy record. That may not give you as much information as you would like (ideally, you'd reconstruct annual temperatures), but it means you're comparing like-to-like.

    Of course, comparing like-to-like may not mean as much as one would think. It's a little difficult to tell since I can' find any listing of what data was used, but from what I can see, it appears a significant number of precipitation proxies were used in the two recent papers I mention in this post. The Team was quite vitriolic about that being wrong (despite the original hockey stick and several other reconstructions of the time doing it too). I'm not sure why it'd be okay now. What I do know is the 2010 Cook et al. paper makes no mention of trying to reconstruct temperatures, only precipitation. The 2012 paper which cites it makes no mention of trying to reconstruct precipitation, only temperatures. They do say this though:

    As mentioned earlier, we target here a reconstruction of past summer temperatures. While it is arguable that annual temperatures would be better to reconstruct for the purpose of assessing temperature change over eastern and monsoon Asia, it is also the case that the direct effects of temperature on tree growth are largely confined to the photosynthetically active warm-season months, thus making summer temperature a logical and more causally defensible choice for reconstruction.

  7. I've slightly updated the text to this post to clarify my meaning in reference to MBH98's Fig. 7. I didn't mean to indicate Steve McIntyre (and others) have overstated what was done. My point was to highlight that Mann "spliced" two series together in a clear manner, indicated both in the text and the figure itself. That is different than other examples where splicing was revealed in text or not at all.

    I was trying to avoid implying what Mann did was wrong simply because "splicing" has come to have a negative connotation to some people. Hopefully the updated text makes my intended meaning more clear.

  8. No one seems to have a problem with it, but I always get the feeling something is wrong when I see Briffa declaring his reconstruction is for the time period of the year that has the best correlation between tree rings and temperature.

  9. I think I asked before, but how often did Mann use reconstructions as proxies in his 98 and 99 papers?

  10. MikeN, I'm not sure you asked (me) that before. It's an interesting question. I don't have an answer offhand. One interesting aspect of paleoclimatology is there is no clear definition of what a "reconstruction" is. Individual proxy records (like an individual core sample taken from a tree) will often get combined together to form a "chronology." These "chronologies" will often be taken as individual "proxies." Those "proxies" will then be used to form a reconstruction. However, sometimes many "chronologies" will be combined together first to form a single "proxy" which can then be used in reconstructions. And in some cases, individual series are combined to form a 'reconstruction" without dealing with the "chronology" step.

    There are more details and nuances, but the point is it is all very muddled. There is no clear definition on what counts as a "reconstruction," and often, you'll see a "reconstruction" used as a proxy in another "reconstruction." And as I allude to in this post, it's often not clear just what data is used in which proxy/chronology/reconstruction. (This lack of clarity is hamstringing the next post I had planned as knowing which data was used for certain reconstructions/papers is vital to discussing a point I think is important.)

    That all said, I'll give you the best answer I can give. Of the relevant data, Mann used no reconstructions. There are two proxies relevant to MBH (the combination of MBH98 and MBH99). The first is a single tree ring chronology known as the Gaspe series. This is a series based on only a handful of tree ring cores. The data for it is so sparse the series is based on only one tree ring core in its earliest years, and for the years 1400-1404, Mann simply fabricated data to extend the series far enough back to be included in his reconstruction.

    The second relevant proxy was the NOAMER PC1, a proxy created by running a large number of tree ring chronologies from western North America through a process called principal component analysis (Mann's implementation of this process was faulty and biased toward producing hockey sticks). Interestingly, the NOAMER tree ring network included the Gaspe series, but it did not extrapolate the series back to 1400 as Mann did when he used Gaspe as a standalone proxy. In any eventt, NOAMER PC1 was the first "principal component" created via that process.

    Neither of those two proxies have been labeled "reconstructions." There are 20 other proxies which extend back to 1400 in the MBH papers (and 110 which are used for any time period), and some of those may be considered "reconstructions." I don't know offhand. None of those proxies matter though. The original hockey stick depended entirely upon the Gaspe and NOAMER PC1 proxies.

    As a final note, the fact Gaspe was used as both a standalone proxy and as part of the large NOAMER tree ring network shows there is no clear standard for what a "proxy" must be. That's true even if we ignore the duplication of the series in the MBH data set. A single tree ring chronology can be considered a "proxy" on its own, or it can be combined with a hundred other tree ring chronologies to create a couple proxies (in this case, principal components). That gives a lot of room to pick and choose.

  11. Brandon, although you criticize Steyn for falsely accusing Mann for splicing it seems that your post largely convicts the "hockey team" of the practice. I would put forth that Mann was the leader of this team and thus could be given some responsibility here, especially for the 1999 WMO graph. I say Bravo for a well constructed post. I would only add that splicing is not acceptable here particularly since the properties of uncertainties of the two types of data are dissimilar.

    One should be highly curious and skeptical as to how the tree ring and other non-instrumental proxies would fair on their own during the late 20th century. And, if they do poorly tracking instrumental data that is highly relevant to their validity as proxies. Even if they do track well with the instrumental plot it does not validate the proxies that were chosen specifically because they were "tested" for qualification of use by their ability to track 20th century instrumental plots.

  12. Ron Graf, no. Just no. Mark Steyn made a specific claim about what Michael Mann did in his original hockey stick. He has made it multile times.
    It is false. I get we could change the subject to some other issue and say that issue is somewhat similar and in that case Mann did something kind of like what Steyn said, We're not going to do that though.

    One of the favorite tricks of the Team was a tactic popularized at Real Climate. What they would do is respond to criticisms of the hockey stick by saying, "But you still get a hockey stick if you do X." It was not okay when they did it to defend Mann. It is not okay when people to do it to defend Steyn. If people want to discuss the WMO chart and how it came to be, I am happy to. I can even provide the e-mail correspondence that went into the process (including some e-mails not contained in the CG1 and CG2 releases). It's an interesting issue.

    But what Steyn said on this issue was wrong. What Steyn said on a number of other issues was wrong. As long as people refuse to squarely deal with that, they shouldn't expect to be taken seriously. That Steyn responded to a libel suit by making a series of false (and in some cases defamatory) claims is ridiculous. I get why people may want to defend his right to free speech, but anyone doing so should be willing to acknowledge he's saying a lot of untrue things.

  13. "What they would do is respond to criticisms of the hockey stick by saying, "But you still get a hockey stick if you do X." It was not okay when they did it to defend Mann. It is not okay when people to do it to defend Steyn. "

    I disagree that one needs to hold a political satirist up to strict scientific standards as one needs to with peer reviewed science.
    Claiming that flawed methods are irrelevant to science is wrong.
    Having a slight confusion of chronology of the methods of deceit if there truly was scientific trickery can very easily be irrelevant.

  14. Ron Graf, cut it out. Your bias is beyond obvious here. It's not worth commenting if you're going to behave like this. We get it. People can find excuses to justify anything they want to themselves. No matter how terrible the latest Trump gaffe, you can find someone to go on Fox News to cover for him. When I explicitly refer to behavior on Real Climate, you can change the subject to peer-reviewed literature to pretend I am making some comparison.

    You can wave your hands and try to downplay things all you want, but you will convince absolutely nobody. Everyone who might agree with you will already make the exact same excuses. Everyone else will just lose respect for you. Portraying repeated and severe misrepresentations as "slight confusion of chronology" is pathetic.

    If you want to take this approach to defending Steyn, you can. I'm sure you can find all sorts of excuses to justify the rampant misquotations, use of fabricated quotes taken from random internet commenters and any number of other misdeeds of Steyn's I have pointed out. And all that will do is show everyone you don't care if people in your tribe do wrong as long as it doesn't interfere with the tribe's business.

    Which to be fair, wouldn't make you special. From what I've seen, that's true of most people.

  15. Pingback: Homepage
  16. I'm confused. You discuss the WMO chart, but this was explicitly produced with Phil Jones's using 'Mike's Nature Trick' of splicing instrumental and smoothing.
    Is there another Nature paper that used this trick?

  17. Only half here as some stuff came up unexpectedly that was kind of big news for me. I'll have a post up tomorrow explaining, but I wanted to give you a quick response MikeN.

    Put simply, Phil Jones may have thought he was doing the same thing as Michael Mann, but he wasn't. If I send someone an e-mail, "I prevented that reporter from his story with Donald Trump's trick of murdering the man," that doesn't mean Trump is a murderer. There is a great deal of information, context and detail surrounding Mann's "trick." Putting the slightest effort into understanding what Mann did would be enough to know what Steyn said, repeatedly, was untrue. That he could find some single line somewhere which makes it sound like Mann did something doesn't mean it is okay for Steyn to ignore the volumes of discussion and mountains of evidence which show Mann did not.

    There isn't the slightest chance Ron Graf would tolerate this sort of error were it made by someone on the other "side." There isn't the slightest chance Steyn would either. Steyn would talk for 15 minutes about how stupid a person would have to be to make an error like this, laying it on as thick as he could. It's a colossal mistake which shows Steyn has no actual understanding of what was and was not done with the hockey stick.

    The more Steyn talks, the more he makes it seem like he's just bloviating - behavior libel law is designed to penalize/prevent. Heck, there are a number of people Steyn quotes (or claims to quote while actually using someone else's words, something nobody seems to mind) who I would support if they sued him. I honestly believe Steyn either knows things he said in his book are false or intentionally avoided looking at anything which might cause him to.*

    *That, or he had an intern or the like do most of the work on the book. In that case, he might have hired someone who was utterly incompetent.

  18. I guess that wasn't really a quick response. Oh well. That not a single person other than myself has spoken out against Mark Steyn's book grates on me so. It's made even worse by the fact I know many people recognize Steyn's book is filled with issues. The excuses they give are the exact same ones people who refused to speak out against Michael Mann gave.

    It's pure partisanship, and that's part of why I've not posted as much. Why post if I see people agree with me when I say things they like to hear and hate me when I say things they dislike? What difference can it make? I mean, do I have to just hope I have some impact that isn't discernible from people's (public) reactions?

  19. I just realized I forgot to point out something rather important MikeN. Phil Jones said he performed a "trick" of adding real temperatures to the reconstructed series, but that isn't what Mark Steyn describes. According to Steyn, Mann deleted several decades from his reconstruction that went down then appended the modern temperature record.

    So even what Phil Jones says doesn't support Steyn'd depiction. Steyn just conflated things which were done to entirely different reconstructions. The best I can figure is other people made the same error first, and he just accepted what they said without question. He certainly didn't do any research or find any evidence which supported his depiction.

  20. This has been something that's bothered me. I've seen people say that Mann's reconstructions have problems after 1980 just like Briffa's after 1960, but never seen any evidence of this.
    Steve McIntyre asserted to a court that Mann spliced in his 98 hockey stick.

    I keep asking about the reconstructions as proxies, because Gary Funkhouser so adamantly argues that it would never happen. He was always against Mann's hockey stick, but hates McIntyre more and adopts all the Mann talking points. I am trying to see if a major error has gone unnoticed.

  21. I'm not sure what you're talking about MikeN. The MBH reconstruction ended at 1980 so I don't know why people would say it had problems after that point. I also haven't ever seen McIntyre claim there was any splicing involved in that reconstruction, unless you count Mike's Nature trick where instrumental temperatures were spliced onto the end to allow the smoothing to function the same at the endpoints as everywhere else.

    Who is Gary Funkhouser, anyway?

  22. Have you read McIntyre's brief in the Steyn v Mann case? I misinterpreted your comments above as saying it is wrong to claim splicing of any sort in Mann's paper. Just noticed you already credited Steve for I think the same thing as in hi brief.
    Funkhouser is a scientist who I've seen in a particular forum. Makes a brief appearance in FOIA mails.

  23. I read it, but it's been a while so Ican't recall any specifics of it. Do you happen to have a link to the brief handy?

  24. Pingback: iraq Qanoon

Leave a Reply

Your email address will not be published. Required fields are marked *