Does Anyone Else Think This is a Problem?

A bad storm hit Thursday, and I'm still dealing with the aftermath (fortunately, there was no real damage just a huge mess to clean up) so today's post is going to be a but light. I wouldn't even make one today except this issues blows my mind. Yesterday I saw Richard Betts tweet about a new paper of his, with this tweet in particular catching my eye:

This tweet surprised me because the IPCC didn't publish a single PDF for TCR, a matter Betts even discussed a couple weeks ago. What the IPCC did was publish PDFs from a dozen or so studies, not picking one or attempting to combine them into a single PDF. This means there was no single PDF Betts could have chosen to use in his paper as "from IPCC AR5." There were a dozen or so he could have picked from, but...

To show what I mean, here is an IPCC figure showing PDFs for TCRs which were given by various studies (also shown are PDFs of ECS):

If you don't know what PDFs are, they are probability distribution functions, effectively curves which represent the probability a range of scenarios being correct. TCR is basically the planet's short-term climate sensitivity while ECS is basically its long-term climate sensitivity.

The IPCC looked at the PDFs given by various studies, decided there were too many issues in figuring out how to properly combine them and decided it would not publish a singular PDF. Instead, it simply gave a likelihood range. For TCR, the IPCC said there was a 5% chance TCR is lower than 1C per doubling of CO2 and a 5% chance it is greater than 2.5C per doubling.

As the figure above shows, even with those limits on what we consider likely, there is a lot of room for differences in PDFs. Some PDFs are normally distributed (both halves of the curve have the same shape). Some PDFs have longer/wider tails on one side or the other. Some PDFs peak at different values, with those peaks representing their "best estimate."

Given such a wide range of possibilities, how does one pick a single PDF to use? The IPCC didn't provide one that was the obvious one to use. This means Betts and his co-author Doug McNeall would need to pick one of the dozen presented by the IPCC. But which one? How could they choose which one was the "right" or "best" one to use for their paper?

I asked Betts on twitter to clarify which PDf he used but didn't get an answer. The question was zomewhat redundant anyway as I figured out the answer myself. The paper says:

Uncertainties in the ECS and TCR have been systematically studied and quantified, and probability distribution functions (PDFs) providing estimates of their likely ranges have long been a key outcome of the assessment reports of the IPCC. However, no comparable systematic probabilistic assessment of implications of these uncertainties for the relative strength of the radiative and biological impacts of CO2 has yet been produced. To allow for such a systematic approach, we invert the PDF of the TCR estimated from observational constraints in the IPCC’s Fifth Assessment Report (AR5)18, to create PDFs of CO2 concentrations that would occur at the time of passing GWLs of 1.5 °C and 2 °C (Fig. 1), if CO2 were the only agent of radiative forcing (see Supplementary Information).

Now, a careful reading of this might give you a clue as to what they did. On Twitter, Betts said the paper uses "a PDF of TCR from IPCC AR5." The paper doesn't say that though. The paper says it uses "the PDF of the TCR estimated from observational constraints in the IPCC’s Fifth Assessment Report." That doesn't claim to use a PDF from the IPCC, but rather, the "PDF of the TCR estimated" by the IPCC.

This is a subtle difference readers couid easily miss. It's only if a reader digs into the Supplementary Information that they can find a clear explanation of what was done. It says the authors used for this PDF of TCR:

1) A Gaussian distribution, fitted to have very similar 5th and 95th percentiles as the IPCC AR5 estimate (1.0 and 2.5°C) [i.e. TCR ~N(1.75, 0.452
)].

A Gaussian distribution is jsut a normal curve. The parameters listed show this curve peaks at a TCR of 1.75C, falling off at a rate given by the second parameter, ultimately resulting in a PDF whose 5% and 95% percentiles are "very similar" to the IPCC's.

Am I the only one who sees a problem with that? The authors of this paper made up their own PDF, with no study or physical basis for it. They simply decided since this one curve they made fit the 5%/95% percentiles of the IPCC estimate for TCR, they could describe it as "the PDF of the TCR estimated from observational constraints in the IPCC’s Fifth Assessment Report" in their paper and "a PDF of TCR from IPCC AR5."

Even if you don't think representing a PDF you made up yourself as being from the IPCC is misleading due so the exact word choices you use making it possible for people to figure out what you meant, this is certainly misleading. The paper specifically says it uses "the PDF of the TCR estimated" in the IPCC report. That is false.

There is no single PDF when all you have are the 5% and 95% percnetile ranges for a value. There are hundreds of PDFs one could come up with whose 5%/95% percentiles would match the IPCC's 1C/2.5C values. As just one example, why did the authors assume their PDF needed to have a normal distribution? Plenty of PDfs people have come up with for TCR don't.

And even if it were normally distributed, why did they decide their PDF had to peak at the height it did and fall off at that particular rate? As the IPCC figure shows, some PDFs peak higher and are skinnier while others peak lower and are fatter. How did Betts and McNeall decide their PDF would not only have a normal distribution but also a specific height and thickness?

They don't say. Nowhere in their paper do they even make it clear what they did. Instead, they make false claim to be using the PDF for the TCR estimate of the IPCC report even though there are hundreds of possible PDFs which would fit those estimates.

Even in the Supplementary Information, where they disclose what they acutally did, they offer no explanation. They provide no explanation as to how they chose this particular PDF. They offer no explanation as to why anyone should believe that PDF is "right." If the authors just wanted to use it as a hypothetical scenario to consider, that'd be fine, but they never represented it that way.

The authors represented a PDF they created themselves, without any study or physical basis underlying it, as being taken from the IPCC report when it was not. Does anyone else think that's a problem?

2 comments

  1. You wrote, "How did Betts and McNeall decide their PDF would not only have a normal distribution but also a specific height and thickness?"

    I don't understand the last part. A normal distribution with 5-95% points at 1 and 2.5 K/doubling has to have a mean at 1.75 and a deviation of (2.5 -1.0) / (2 * 1.645), where 1.645 is the (rounded) z-score corresponding to a 5% one-sided tail. [I get a deviation of about 0.456, not 0.452 of the paper, but that's a minor difference.] Once one decides to match the 5-95% points and chooses a normal curve, the height and thickness are determined.

    I agree with you that there is little reason to assume a Gaussian pdf. For example, the pdf of Otto et al. is skewed toward lower TCR: the paper gives a 5-95% range of 0.9 - 2.0 with a median of 1.3. Also, my reading of the IPCC section is that the 1.0 & 2.5 K points were selected as convenient round numbers, rather than 5%/95% points obtained by a mathematical procedure. Looking at the graph which you cited, without the blue curve -- "Harris et al. (2013)" -- one might easily place the upper end of the interval closer to 2.0 K.

    Myself, I would ignore the "IPCC"-derived results and concentrate on the "R16" results, which have (to my mind) a more plausible distribution shown in figure S3.

  2. HaroldW, thanks for catching that. You're absolutely right. As I mentioned at the start of this post, I'm swamped with real-life stuff. That led to me being a bit sloppy. When I saw "Gaussian" I thought "bell curve" even though normal distributions are just one type of bell curve. If you assume a normal (Gaussian) distribution, then the width of the curve is fixed by that assumption (and your SD).

    Hopefully that mistake of mine doesn't detract too much from the post. The thrust of the question still remains. Even if the authors thought a bell curve shape was appropriate (meaning there was no asymmetry), why would they assume the bell curve must follow a normal distribution? More importantly, how could they justify making that assumption while claiming to use the PDF from the IPCC report? There is nothing in the IPCC report which justifies such an assumption.

    Imagine if someone created their own PDF for ECS which peaked at 2C and had a long right-tail. Now imagine they claimed that PDF was the PDF from the IPCC report. Every climate scientist in the world would say that was wrong. Most would say it was dishonest. How is that any different than what Richard Betts and Doug McNeall did? How can anyone possibly justify what the authors of this paper said/did?

Leave a Reply

Your email address will not be published. Required fields are marked *