2011-09-08 03:54:03Roy scrambles-- it is bad and ugly and wrong
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Sigh...here we go again.  "They are stupid to understand my brilliance", "they are finally seeing the light", "they do not know what they are doing"....not his words but the subtext is clear.

2011-09-08 05:10:43
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

I am just at first glance on this, but the first part of the discussion about energy budget equation their different answers for the relative magnitudes of ocean forcing : cloud forcing seems as though it could be summed up as "I disagree with Dessler (and Lindzen) about the value for the LHS of the equation".

But I also think he has made a simple math error... I am just checking it again...

2011-09-08 05:15:18
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Hi Rust,

"But I also think he has made a simple math error."

Are you perhaps referring to this?

"if I assumed a feedback parameter λ=3 Watts per sq. meter per degree, that 20:1 ratio Dessler gets becomes 2.2:1. If I use a feedback parameter of λ=6, then the ratio becomes 1.7:1. This is basically an order of magnitude difference from his calculation"

2011-09-08 05:25:26
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

Yes, but now I realize that he is correct there. I get exactly the same.

But it still remains that Spencer has the value for the LHS as 2.3W/m^2 (Levitus), whereas Dessler has 9.0 W/m^2. That is where most of the difference will lie. The other two values matter as well, but that's the big discrepancy.

2011-09-08 05:43:05
dana1981
Dana Nuccitelli
dana1981@yahoo...
64.129.227.4

In his paper, Dessler pointed out that his value was consistent with the ocean heat exchange result from Douglass and Knox (13 W/m2).  Looks like we've got skeptic pitted against skeptic on this issue.  I'm pretty sure Douglass and Knox used the same data as Levitus too, didn't they?  Dessler says they used ARGO data from 2003 to 2008.

And of course rather than owning up to his cherrypicking, Spencer criticizes Dessler for....calling him on his cherrypicking.

And Spencer still can't grasp the fact that he's not testing climate sensitivity, he's testing how well models simulate ENSO.  He's still totally (wrongly) focused on sensitivity.

2011-09-08 05:59:25
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

Shoot, I tried to email Barry Bickmore and Arthur Smith... but Spencer responded first. Spencer is correct that Dessler did use an assumption for λ between 1 and 6 W/m^2/degree K. (His very words in is paper say this...)...

As far as I could tell, he did this so as to exhaust all plausible values for climate sensitivity λ, which includes the Planck response along with all positive and negative feedbacks all bundled in... 

2011-09-08 06:08:41
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust,

Help me here.  As per Eqtn. 1 in Dessler 2011, Ts refers to "surface temperature", right?  Dessler calculates dTs/dT by "subtracting each month's global average surface temperature from the previousmonth's value".  However, Spencer in his latest blog uses dOHC/dT for the 100-m mixed layer, which will of course have a much lower sigma than dTs/d and which is not conistent with the formulation which is for Ts (i.e., the surface temp.) in Dessler's paper, but doing so is consistent with using the heat capacity for the top 100 m.  Question is why does Dessler calc. "c" for the top 100 m?  I'm missing something, b/c I doubt Dessler would made a simple error.  Also, why did he not use the HadSST product or ERSST.v3?

Dessler uses the renalysis MERRA data to calc. sigma for dTs/dT (9 W/m-2).  He then corroborates this using ARGO data reported in DK09 which yields 13 W m-2-- but Dessler does not say where in the paper exactly, probably from Table 1 somewhere and going by the dates he seem to have used the data from von Shuckmann et al. (2003), and again which data exactly (i.e., over what depth?)?

One other potentialy important difference is that Spencer on his latest blog post is, for some reason, using at 3-month variations in OHC.  That would also reduce sigma(c*dTs/dt).  Oddly enough, SB11 used monthly data, not 3-monthly data.  Hmmm....did the goal posts just shift??

Anyways, as I see it Dessler needs to eloborate on some of his numbers-- that he did not do so in the paper is not surprising given that is was published in GRL which has stringent resircitions on length.

2011-09-08 06:14:25
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust and Dana,

Re the choice of lamda...Smith erred there.  What Dessler did, as far as I understand matters,  was place some constraints on the possible range of values of delFocean and delRcloud using observations rather than simply using assumed values. 

2011-09-08 06:19:23
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

OK, just checked in at Spencer's place.  Smith seems to be thinking along the same lines as you Rust. It would be really helpful if Andy Dessler could comment on this thread and clarify some matters.

2011-09-08 06:53:32
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

I think that Arthur has picked up on an error in the math. To get to Spencer's "1.7:1 and 2.2:1" ratios for the ratio of S(ocean)/N(clouds), it is apparent that he calculates N by simply subtracting λT from CERES. But the value for both CERES and λT are in "units" of standard deviation, and you can't do this. Bottom line, both N and λT must be less than CERES.


But you can naively do simple algebra on the equation and confirm that Spencer gets value for N that are larger than CERES ...

 

Shoot, this isn't going to format correctly, but you line up the labels with the numbers, here they are:

(LHS, CERES and T are from his "numbers", λ is the assumed 3 or 6, and everything else is straight algebra, with N = CERES + λT... (fixed - see later in thread)

LHS CERES N λT T S λ
S/N
2.3 0.56 0.794 0.234 0.078 1.74 3
2.191436
2.3 0.56 1.028 0.468 0.078 1.74 6
1.692607
2011-09-08 06:58:48
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

My point is that those calculated numbers are wrong, but I am almost certain that is how he got to the 1.7:1 and 2.2:1 ratios. (See the values for S/N in my table above...)

Spencer is now saying that this doesn't matter a whole bunch, but since N is the denominator, it matters a whole bunch...

2011-09-08 07:04:25
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Good catch Rust.

2011-09-08 07:05:13
nealjking

nealjking@gmail...
91.33.100.27

So what are the (wrong numbers | right numbers) ?

2011-09-08 07:08:10
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust,

Did yu mean to say N = CERES + λT?  That is, 0.794 = 0.56 + 0.234

2011-09-08 07:15:56
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

CORRECTION: where I say "N = CERES - λT" above it should read "N = CERES + λT" (plus not minus) HOWEVER I have the calc correct in my naive spreadsheet... (You can confirm by seeing that column 3 = column 2 + column 4)

@ nealjking... the correct derivation for "N" would be along the lines of:

std.dev(N) = sqrt(std.dev(CERES)^2 + std.dev(λT)^2) (fixed as per correction later in thread)

which I am not going to do here, but here's the thing... the WRONG numbers for "N" are my column 3 above (0.794 and 1.028 for assumed values for λ of 3 and 6 respectively.) But the RIGHT numbers for N must be less than 0.56.

Substitute any values for N less than 0.56 (in place of the WRONG N's) into the ratio above for S/N, and the ratio goes up substantially as well...

I think these to issues, the assumption for the LHS and this basic math error will get us back to Dessler's figures...

(Arthur Smith is on the case, I think...) 

2011-09-08 07:23:15
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

Yes @ Albatross, I was typing that fix as we crossed messages...

Can't be SURE this is what Spencer did, but it sure looks like...

2011-09-08 07:24:10
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Hi Rust,

"You can confirm by seeing that column 3 = column 2 + column 4",

that is how I figured the text was probably wrong.  No worries, a small, typo, you clearly know what you are doing.

"think these to issues, the assumption for the LHS and this basic math error will get us back to Dessler's figures...

(Arthur Smith is on the case, I think...)"

I think so.  Smith is also talking about the problem of using 3-month versus 1-month sigmas now, and may have found aproblem with Roy's calc. of sigma (N) and/or sigma (CERES). The big problem in explaining the difference seems to be the OHC data....maybe someone should ask Dessler.  At least we will get a coherent reply from him ;)

2011-09-08 07:41:25
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

And, er, I also put a minus instead of a plus in the std.dev(N) equation... but the (corrected) point stands... (I think!)

2011-09-08 07:46:10
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust I think that you and Smith have identified one problem, but that only gets us to a ratio for Ocean/Clouds of about 3.5 (by my estimate), which is about 7 times higher than Roy assumed, but still a long way from 20:1.

We really do need for Andy to explain in more detail what OHC ocean temperature data he used  fore the mixe dlayer and if it was indeed for the top 100 m, and I'm still confused (as explained at 6:08 am) by the value used for "C" and what Andy did in his equation, but that is probably the point, I am just confused ;)

Update:  I must say, this I am not comfortable withthe fact that it seems that none of the variables are constrained to within a narrow range.  I would not state that the ratio of 20:1, even if correct for these data, is THE value.  Just that ocean forcing is seemingly significantly larger than cloud forcing for the study period in question.

2011-09-08 08:01:29
rustneversleeps
George Morrison
george.morrison2@sympatico...
198.96.178.33

Yes, absolutely, that is what I referred to as the difference of value for the LHS. It's a factor of 4, so it's pretty obvious that's key. (And part of that may be the 3-month issue... which I think is a problem for Roy as well, even if he has stayed consistent... I am just going off the top of my head, but I think that on his blog recently he said that he thought the response time of T to the radiative forcing of clouds might be almost instantaneous, and wouldn't that argue against a 3-month s.d.??? That's just an intuition...)

Anyway, like you said, I am sure that Dessler will respond (although in an earlier online back and forth it was as if they were speaking different languages...)

2011-09-08 08:06:29
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust I hope that Andy does not engage Spencer over the internet or by email again...as you noted previously that exchange was a dismal failure.  I'm naively hoping that Andy might post somehting at SkS.  Or maybe one of us should just email him.

And yes, this whole three month (i,e., averages over a meteorological season) complication that roy has decided to introduce is rather perplexing.

2011-09-08 08:10:15
Alex C

coultera@umich...
67.194.184.102

>>>I would not state that the ratio of 20:1, even if correct for these data, is THE value.

Good point, I would think that giving the range would be best.  In the case that this is a best estimate though, a value of 20 v. 0.5 is outrageously off, by (with rounding) 2 orders of magnitude with a base ten scale (specifically, 1.6).  I would be personally surprised if the lowest range of certainty would come close to halving that magnitude difference.

2011-09-08 11:20:31Andy on blogs
John Cook

john@skepticalscience...
130.102.158.12

From what I've heard, Andy Dessler has said he's too busy to guest post on blogs this week.

2011-09-08 12:14:05
Rob Painting
Rob
paintingskeri@vodafone.co...
118.92.48.81

Alby - the supplementary tables for Douglass & Knox (2009) are here. At a quick glance seems to support what Dessler says - I'll let you do the calcs.

2011-09-08 14:38:17
Rob Painting
Rob
paintingskeri@vodafone.co...
118.92.48.81

Ran the numbers from Douglass & Knox. I get a different value for the standard deviation - but then I'm probably doing things wrong. 

2011-09-08 15:14:30
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Hi Rob,

Thanks for thaat link, I was unaware of it.

I looked at the numbers too.  Manually entered them into Excel.  There are a couple of issues here:

1)  I'm not entirely sure what Dessler means for one to do when he says "month-to-month change in monthly interannual heat content anomalies".  

2)  Even if I figure out what that means, these are 0-700 m data, and he used 0-100 m mixed layer in his study.  And the variabilitu of the 0-100 m data will be greater than for 0-700 m.  So the whole exercise may be moot.

For what it is worth, and probably nothing until we figure out exactly whcih data to used (as per 1 above), the sigma for all the monthly data is ~2.5x10^22  J. The sigma for changes from month-to-month (the sigma for the following series: July anom-June anom, Aug anom -July anom , Sept anom - August anom.......) is 1.78 x10^22 J/month (I think the units are right for the second calc. b.c I'm looking at the sigma for changes in Joules between adjacent months).  But he has 1.2 10^22  J/month.

This is all rather annoying, and Andy is understandibly busy, so we could be in the dark for a while. Talking of which, I am going to sleep.

2011-09-08 22:01:55
grypo

gryposaurus@gmail...
173.69.6.13

From Roy's latest post.  An update that may be important to our posts

UPDATE: I have been contacted by Andy Dessler, who is now examining my calculations, and we are working to resolve a remaining difference there. Also, apparently his paper has not been officially published, and so he says he will change the galley proofs as a result of my blog post; here is his message:

“I’m happy to change the introductory paragraph of my paper when I get the galley proofs to better represent your views. My apologies for any misunderstanding. Also, I’ll be changing the sentence “over the decades or centuries relevant for long-term climate change, on the other hand, clouds can indeed cause significant warming” to make it clear that I’m talking about cloud feedbacks doing the action here, not cloud forcing.”

I'm not sure if this effects us or not, but someone should probably contact Andy Dessler to see if we need to change or update our posts before "official" publication.

2011-09-09 01:45:48
dana1981
Dana Nuccitelli
dana1981@yahoo...
64.129.227.4

The only change would be if they agree on a different number regarding ocean heat transport vs. cloud TOA changes.  If so, we'll publish a correction, but it should at least be closer to Dessler's value than Spencer's.  Ideally they'll agree on a range of values based on the various data sets.

2011-09-09 02:09:09
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Grypo,

Thanks just saw that.  OK, so better minds than mine are working on reconsiling the data.  Good.  Now we just wait.

It will be interesting to see how this pans out, it could be that Dressler has to revise his paper before it goes to print, but i doubt that teh changes would be significant. The "skeptics" will be delighted of course and spin the death out of it, but ultimately it would have been a good thing.

Even if the ratio of ocean heat transport to cloud TOA changes is halved (i.e., 10), that still means that clouds are a small player compared to ocean changes, and that number would be in stark contrast to Spencer's assumption of a ratio of 0.5.  

2011-09-09 14:02:40
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

Arthur Smith notes that Roy Spencer has updated his response to Dessler... in response to Dessler's response to Spencer's response to Dessler!

Arthur is making more explicit the point I suggested upthread regarding what I believe is a math error. He articulates that the error makes a significant change in Spencer's "S/N" ratio (what a brutal choice of letters for the variables! was that on purpose???) ... I think this is unavoidable... (although I lose Arthur a bit on his final number...)

And the main part of the debate is indeed going to happen on the "LHS of the equation", which seemed apparent from the get-go... Dessler seems to be acquiesing to Spencer's dataset choice**...

But I was struck by this part of Roy's updated commentary:

"it appears (Dessler) put all of his R(cloud) variability in the radiative forcing term, which he claims helps our position, but running the numbers will reveal the opposite is true since his R(cloud) actually contains both forcing and feedback components which partially offset each other."

Isn't Spencer now claiming that the radiative feedback for clouds - which I thought in Dessler's simple energy budget model was part of the coefficient λ, a response function to ΔTs - should iinstead be combined in the forcing part of the equation - R(cloud)? Unless I am missing something, isn't Spencer effectively saying something akin to "clouds cause clouds"?

C * ∂T/∂t = ΔR(cloud) + ΔF(ocean) - λΔT (Dessler's Eq. 1)

I am just glancing at his comments and somewhat blind because all we are getting is Spencer's updates. We don't actually know what Dessler is saying or not. But it reads to me almost as if Spencer's position can devolve into something like OHC change causes OHC change, clouds cause clouds, and maybe even temperature change causes temperature change...

Since a key part of Dessler's argument was about the relative magnitudes of the standard deviations of forcings ΔF(ocean) and ΔR(cloud), then unless Spencer is suggesting something akin to the "radiative feedback from clouds in response to a radiative forcing from clouds" is positive, then wouldn't the s.d. of the "net" forcing (i.e forcing net of feedback) have to be less than that of just the forcing itself???. (Which is what Dessler seems to be alluding to (i.e. "helps your position") - and even Spencer seems to agree when he says the forcing and feedback "partially offset each other".)

Some of this seems too obvious to be right... I'm laying out how I read it, but I am really asking it as a bewildered question...

** How much of a difference this will make, I don't know but I would think that dana's point about the corroboration of the Argo data is a hint that it will not be that much... although Spencer's updates indicate that Dessler has said he is going to walk back somewhat from the ~ 20x number. (That could get ugly... even if it isn't substantial...)

2011-09-09 21:39:09
Tom Curtis

t.r.curtis@gmail...
112.213.162.57

I hate to put a dampener on things, but I think rust and Arthur Smith are wrong.

 

Specifically,  λΔTs is the change in the OLR due to a change in surface temperature.  All else being equal, an increase in surface temperature will result in an increase in OLR.  Thus if there is a cloud forcing such as Spencer supposes, an increase in cloud forcing will result in an increase in surface temperature, and hence in λΔTs.  Because high (low) values in λΔTs will match high (low) values in ΔR(cloud), and because net TOA flux is ΔR(cloud) - λΔTs (plus other forcings which don't concern us here), there will be less variance in net TOA flux than in either of the individual terms.

 

Of course, my statistical knowledge is woefully lacking, so I expect Rust or Arthur to shoot down this point in flames.  But it certainly appears to me that they are wrong, and I would like to see their guns fire before I surrender.

 

While on the subject, however,  λΔTs is a very dodgy way to represent climate sensitivity.  Specifically, once the equilibrium response to a (non solar) forcing is achieved, the net change in OLR is zero.  In this circumstance,  λΔTs can be defined as being equal to the forcing.  But prior to that time, there is no reason to think that  λ is constant.  Indeed, it seems probable to me that it evolves over time, and indeed, when you have short term and long term feedbacks, it must do so.  Hence  λ must be indexed by lag from the initial forcing.  That also means using three month data to determine the on month  λ as Spencer does is strictly speaking an error, although it is not impossible that it would yield a usefull approximation.

 

Further, while the other forcings can be ignored for the discussion, they cannot be ignored in determing ΔR(cloud).  Spencer determines ΔR(cloud) by the formular ΔR(cloud) = ΔTOA Flux +  λΔTs, but the correct formula is

ΔR(cloud) + ΔR(CO2) + ΔR(CH4) + ΔR(Solar) + ...  = ΔTOA Flux +  λΔTs  

Spencer can plausibly argue that the other terms, because they are poorly correlated with ΔR(cloud) can be ignored as noise.  But I suspect in using three month data to obtain C * ∂T/∂t he has actually used seasonal data.  That being the case, seaonal variation introduces a significant short term forcing that is directly correlated with his period of analysis and hence cannot be ignored. 

2011-09-10 09:15:58
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Rust,

Sorry for the silence...been busy.

I'm afraid that this particular riddel is beyond my expertise. 

I see that Spencer is now greatly inflating the importance of some of the editorial changes that Dessler will be making to the galley proofs, sadly his fan base are easily awed and misled and seem to think that the changes are of substance or somehow affect the results of his paper, which they do not of course-- at least for now.

More importantly though, Spencer seems to be suggesting that Dessler might decide to use the top 700 m ocean temperature data-- a poor choice for this type of analysis, b/c from what I understand, 100 m is far more realistic mixed-layer depth. Although Dessler may have already done that given his reference to Douglass and Knox (2009). He certainly should not cave and use three-monthly data!

Very confusing stuff and annoying that we are only hearing the shrill comments from Spencer on this.....Spencer should shut up until they have figured this out, rather than providing running commentary on his blog that is presented so as to score points.

2011-09-10 09:22:01
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Hmm, now Barry says on his site that he thinks that both Roy and Arthur may both be wrong-- but he does not elaborate (sigh).  Does Barry have access to this forum?  It sure would help to have his insights.

Updated for clarity.

2011-09-10 09:26:14
Alex C

coultera@umich...
67.194.19.152

Barry does, I have seen him post here if I remember.  I'm interested, you say both Roy and Arthur - Imma have a look at Barry's site...

2011-09-10 09:44:20
nealjking

nealjking@gmail...
91.33.103.240

If folks want to report on issues that haven't settled out, you're gonna get whipsawed.

2011-09-10 10:04:05
rustneversleeps
George Morrison
george.morrison2@sympatico...
99.232.158.68
There are arguments about the appropriate data series and models to use. AND I still think the math problem exists. I don't think Barry comments pertain on that. Bwtfdik. I think Arthur's point on the math stands. I'll scribble my inklings later, probably tomorrow.
2011-09-10 10:08:40
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Hi Neal,

Sorry, but I don't follow...

2011-09-10 18:53:22
nealjking

nealjking@gmail...
91.33.120.94

Albatross,

It's exciting to be involved in promoting breaking news, but breaking news is, by definition, not settled: Things can be revised and controverted on a daily basis. If we're publishing stories at such an early stage, there's a very good chance of being wrong-footed by events. If we don't want to be left with egg on our collective face, we may want to be cautious about getting involved so early.

2011-09-10 19:02:18
Riccardo

riccardoreitano@tiscali...
93.147.82.89

I have to admit my difficulties in folowing the discussion. It is pretty clear that the larger difference between Dessler and Spencer is the OHC data used. The error discovered by Arthur Smith has been admitted by Spencer, but it is small.
What I find difficult is reasoning on a heat balance equation which is foundamentally ill-posed.

The LHS of the equation is the energy change of the system; whatever you put here defines the system. The RHS should be written accordingly.

The temperature in the two sides of the equation must be the same. The very fact that in Spencer's equation they are different tells us that he needs a two box model, a thin surface layer and the mixed layer below. But the fluxes between the two layers and the equation for the second layer are missing. As it is, even if the forcing is given it can not be solved. In this (unphysical) situation, the meaning of the various terms and of the results is not clear.

To add to the confusion, the forcing terms can be anything but they can not depend on the temperature of the system; if they do, they're a response, not a forcing. While N (Spencer notation) might not depend on temperature, S surely does.

If Spencer wants to use a one box model, the average properties of the only layer must be used, including the Plank response. If we do, we have to halve the temperature on the RHS of the equation and triple or quadruple the LHS due to the thickness of the mixed layer (100 m instead of 25-30 m). The result, using Spencer numbers, would be a S/N ratio somewhere around 10.

I think that the weakness of Dessler paper is stategic more than scientific. He did not focus on Spencer's more blatant error thus leaving the door open to a probably endless discussion on the more appropiate dataset to use.

2011-09-11 03:47:59
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

I somewhat sympathize with Neal's most recent point. And Riccardo is correct that the LHS is the main point of discrepancy. And Albatross' frustration of being in the dark. Group hug!

Anyway, I wanted to try to make my point (and what I think one of Arthur Smith's is) more simply. And hey, if I'm wrong, I'm wrong and there's not much ego or reputation attached. I am a stockbroker by day job. My stats is rustier than I'd like and the atmospheric physics is a very part-time study.

Any, here is what I understand.

In Spencer and Braswell 2011 (On the misdiagnosis...) (SB11), they have an Earth energy budget equation like so:

Cp dΔT/dt = S(t) + N(t) − λΔT

In Dessler 2011, he has the same equation, with different nomenclature:

C * ∂T/∂t = ΔR(cloud) + ΔF(ocean) - λΔT.

These terms are the energy fluxes.

But in Spencer's online rebuttal, we see the following:

Where he has added his "Numbers", he has subsititued the pertinent standard deviation for each flux.

And, when he uses his assumption for a λ value, he can solve for σN and σS. But here's the thing. It seems evident that he does this "solving" simply by adding and subtracting the standard deviation terms.

I.e. as in my notes above, he seems* to solve for σN thus: σN = σ(CERES) + σ(λΔT). And, also, seems to solve for σS as σS = σLHS - σ(CERES). That can be confirmed by looking at the values in my "spreadsheet" above (wish it was better formatted.)

But here's the thing. You can't do that. You can't simply add and subtract standard deviations. Quick stats refresher:

Assume c = a + b, and a and b are independent variables.
Then, statistics says...
Mean(C) = Mean(A) + Mean(B)    Check! Correct!
and
Variance(C) = Variance(A) + Variance(B) Check! Correct!
but
StdDev(C) = StdDev(A) + StdDev(B)   ??? Bzzzt! Wrong.

The correct calculation is:

StdDev(C) = SquareRoot(StdDev(A)^2 + StdDev(B)^2)

One immediate implication of using the correct equation - and before even doing the calculation - is that if σ(CERES) = SQRT(σN^2 + σ(λΔT)^2), then σN must be less than the value for σCERES, 0.56.

My calc for his σS is 2.23 - but like I said, my math is rusty. So in a best case scenario, even (correctly?) using Spencer's own equation and values, the ratio of σS/σN must be at least 2.23/0.56 = 3.98. (Which is already ~ 8 times what he caculated in SB11.)

But that best case scenario (i.e σN = 0.56) would assume that the value for σ(λ∆Ts) would be zero, and we know that is not the case. So σN must be less than 0.56, and the ratio must be even higher than ~ 4. Arthur Smith is suggesting that the ratio must be at least 4:1 or 5:1. (I seem to get a different number than Smith for σS, but I generally don't argue with him, and in any event, my ratio can only go up, not down...).

Anyway, I am not going to take this much further, but I hope that is clearer.

One big caveat is that there is so little to work with in SB11 or Dessler's preprint. There is a possibility that Spencer's calculations in the background are correct, and that whether you use the correct equations or not you still arrive at the exact same two ratios he got - 1.7x and 2.2x. But I strongly doubt it.

Another big caveat is that I could be just plain wrong. But I'd genuinely appreciate a pointer as to where.

When I said upthread that "Some of this seems too obvious to be right.", I genuinely meant it. How can he make such a basic error? Now, he has been known to occassionally gets stats wrong in the past. But this?

Anyway, like I said, hopefully clearer, and if anyone sees something fundamentally wrong, let me know. And besides, the larger dispute is likely going to be on the LHS stuff anyway...

(P.S. I know I am carrying too many significant digits... hey, it's the stockbroker thing... you lose all math skills...)

2011-09-11 07:16:20
Riccardo

riccardoreitano@tiscali...
93.147.82.89

I think that both you and Smith are wrong. Spencer number are in units of standard deviation, they're not standard deviation themselves.Given that he's workng with variability of detrended (zero mean) data, this is the right way to compare the two quantities.

If you were right, the numbers would be positive by definition, which is obviously unphysical.

2011-09-11 07:27:29
rustneversleeps
George Morrison
george.morrison2@sympatico...
204.101.237.139
Standard deviations are positive "by definition". What is the unphysical part?
2011-09-11 08:33:14
Riccardo

riccardoreitano@tiscali...
93.147.82.89

Representing a forcing they can be, and are in the case at hand, both negative and positive.

2011-09-11 11:51:07
nealjking

nealjking@gmail...
91.33.120.94

I am not following this closely, but I think I see rust's point:

Consulting the fount of all wisdom (http://en.wikipedia.org/wiki/Variance#Discrete_random_variable):

Given two random variables, X and Y:

Variance(X - Y) = Variance(X) + Variance(Y) - 2*Covariance(X, Y)

If X and Y are uncorrelated, Covariance(X, Y) = 0 , so:

Variance(X - Y) = Variance(X) + Variance(Y), or:

Std-dev(X - Y) = sqrt[(Std-dev(X))^2 + (Std-dev(Y))^2]

Thinking about it in terms of uncertainty: if you subtract two uncertain numbers, the uncertainty doesn't get smaller, it gets bigger - unless you know some non-statistical relation between the two.

2011-09-12 06:19:46
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

@ Riccardo: "Spencer's number are in units of standard deviation, they're not standard deviation themselves."

No, they are in units of the underlying flux: "W/m^2". Which is one of the advantages of using standard deviation instead of other measures of dispersion, such as variance. The numbers that Spencer uses for the standard deviation (σ) of the LHS and CERES are 2.3 W/m^2 and 0.56 W/m^2. The number he uses for σΔTsfc is 0.078 °C, but λ is in units of W/m^2/°K, so the units for σ(λΔTsfc) is again in W/m^2.

They definitely are not in "units of standard deviations", i.e they do not represent, say 2.3 standard deviations from the mean, or somesuch, as you seem to imply. They are the actual standard deviations, and they in the units of the underlying fluxes, i.e. W/m^2.

@ Riccardo: "Representing a forcing (standard deviations) can be - and are in the case at hand - both negative and positive."

No, standard deviations are not negative. Standard deviation is the square root of the variance, which itself must be positive. Yes, one could solve for the negative root, but the standard deviation is a measure rof the magnitude of the dispersion of a variable. It is always expressed as a positive number.

See also my next post.

2011-09-12 07:24:20
Riccardo

riccardoreitano@tiscali...
93.147.82.89

My first point was explained quite badly. The standard deviation are used to scale the relative magnitudes of the two terms. The relevant paragraph in Dessler paper is from line 51 to 57.

Standard deviations are positive, forcings can be negative. Hence, it is impossible to express the forcing as a strictly positive number.

2011-09-12 08:44:58
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

I seem to be having some trouble communicating my point. Perhaps I am flat wrong, or perhaps it is SO obvious that it's just eliciting a big "so what?" shrug. But I think it is important, a significant mistake by Spencer.

So, partly for my own purposes, I am going to try to work from first principles and then try to show how and where Spencer goes wrong (on this specific point), and what it means.

Ok, let's assume that:

D = height of the household's dog in inches
C = height of the household's cat in inches
H = combined height of the household's two pets in inches
H = D + C
and that every household has one dog and one cat, and that D and C are independent variables.

Let's say that the variance of the of the dogs' heights is 64 in.2, and the cats' is 16 in.2.

Therefore the respective standard deviations are σD = √(64 in2) = 8 inches, and σC = √(16 in2) = 4 inches

Ok, but what is the variance and standard deviation of H?

I am not going to go through the derivation, but you can easily prove that:

Var(H) = Var(D) + Var(C)
Var(H) = 64 + 16
Var(H) = 80 in.2

But if we want to find out what the standard deviation (σ) of H is, we must take it's square root. Therefore:
σ(H) = √(80 in.2)
σ(H) = 8.94 inches

Ok, so substituting terms from above:
σ(H) = 8.94 inches = √(80 in.2) = √Var(H) = √(Var(D) + Var(C)) = √((σ(D))2 + (σ(C))2)
so
σ(H) = √((σ(D))2 + (σ(C))2) = √(8 in.2 + 4 in.2)

This is all basic statistics.

But now let's look at the at the three terms for σ:
σ(D) = 8 inches
σ(C) = 4 inches
σ(H) = 8.9 inches

Do they sum? I.e. does σ(D) + σ(C) = 8 inches + 4 inches = 12 inches = σ(H) ?

Obviously not, since we know that σ(H) = 8.9 inches.

Ok, so this is all basic statistics, but hopefully the examples shows that:

σ(H) = √((σ(D))2 + (σ(C))2)

and, importantly with respect to Spencer,

σ(H) σ(D) + σ(C)

Ok, so now turning to Spencer's online rebuttal to Dessler, upthread I show that he substitutes standard deviation values into his energy budget equation:
Cp dΔT/dt = (N(t) − λΔT) + S(t)
(I will henceforth refer to the "Cp dΔT/dt" term as LHS.)

And, to me, it seems very clear that he then goes on to solve for the unknown standard deviations for S and N by simply adding and subtracting. Which we know is incorrect. Follow me.

If,
Net radiative flux = (N(t) - λΔT) = CERES flux,
then also
σ(CERES) = √((σN(t))2 + (σ(λΔT))2) (see Neal's point above. Even if the term equation for the fluxes involves subtraction, the RHS here for the variances will be additive)

And, if the energy budget equation can be expressed as:

LHS = CERES + S(t)
then
σ(LHS) = √((σ(CERES))2 + (σS(t))2) as we have shown above as the proper equation.

With those equations, then if we know the standard deviations for LHS and CERES, we can solve for σS(t) as follows:
σ(LHS) = √((σ(CERES))2 + (σS(t))2)
2.3 W/m2 = √((0.56 W/m2)2 + (σS(t))2)
squaring both sides,
5.29 = 0.314 + (σS(t))2
subtracting 0.314 from both sides (which we can do, because we are back in variances...),
4.98 = (σS(t))2    
and, finally
σS(t) = √4.98 = 2.23 W/m^2. This is the term for the std.dev. for the ocean forcing.

Now, I could do the same thing to solve for σN(t) based on Spencer's assumptions for values of λ of 3 and 6, but even without doing that, we know that if

σ(CERES) = 0.56 W/m2 = √((σN(t))2 + (σ(λΔT))2)

then

σN(t) < 0.56W/m2 - i.e. the maximum possible value for σN(t) is 0.56 W/m2 (this should be obvious)

So, if we want to calculate the ratio of the standard deviations of respective forcings - σS(t)/σN(t) (or Dessler's σF(ocean)/σR(clouds)) - we get:

σS(t)/σN(t) > 2.23/0.56
σS(t)/σN(t) > 3.98

Now, in Dessler's paper, he finds that this ratio is ~ 20:1 (based on his assumptions for datasets and values to use.) And he states that Spencer found the ratio to be ~ 0.5:1.

So, there is still a gap for Dessler to close (which may be found in the dataset/value assumptions). But for Spencer it appears that his initial estimate for the ratio appears to off by a factor of ~ 8.

But wait!

In his online rebuttal, Spencer now himself calculates the ratios for assumptions of λ of 3 and 6 W/m2. His ratios for the respective σS(t)/σN(t) are 2.2 and 1.7.

But both of these numbers are less than 3.98, and we just showed that they must be greater than 3.98. How could Spencer have gotten these (wrong) numbers?

I'll show you how. It is shown in my post above (number 10 in this thread). He does it by simply adding the terms in the equation. Which we have shown he must not do if he is working with standard deviations.

E.g. for an assumption of λ = 3 W/m2/°K, he solves for σN thus:
σCERES = σN - σλT
0.56 = σN - (3)(0.078)
σN = 0.56 + .23
σN = 0.79

And he solves for σS thus:
σLHS = σCERES + σS
2.3 = 0.56 + σS
σS = 1.74

Those are BOTH wrong calculations, as we have shown from first principles, but he then uses the results to calculate the ratio (for case of assumption λ = 3) as:
σS/σN = 1.74/0.79 = 2.2

It's trivial to do the case for assumption λ = 6, and find the ratio in that case (using the same wrong math) is 1.7.

So, the actual ratio using Spencer's own value assumptions must be greater than 3.98, but he calculates values of 1.7 and 2.2.

If I am correct in all this, then I think this error is quite significant, and potentially quite embarrassing to Spencer.

It doesn't go all the way to explaining the difference in the ratios between Dessler 2011 and Spencer's online rebuttal, but it is a significant part. Furthermore, it's just a flat-out basic math error. This error would propogate through Spencer's calculations no matter what datasets or values he used.

And with that, I plan to grab some popcorn and wait for Dessler's next public response. And if I've made any errors, I still welcome critiques/corrections/education...

2011-09-12 09:08:29
nealjking

nealjking@gmail...
84.151.59.188

rust,

I pointed this out above (11 Sep 2011, 11:51 AM). In even greater generality,

from http://en.wikipedia.org/wiki/Variance#Discrete_random_variable:

\operatorname{Var}(aX+bY)=a^2\operatorname{Var}(X)+b^2\operatorname{Var}(Y)+2ab\, \operatorname{Cov}(X,Y),

So if X and Y are uncorrelated, Cov(X, Y) = 0, and your point follows. If Cov(X, Y) is not = 0, then there is some statistical relationship going on: These variables are paying attention to each other somehow.

To be honest, I haven't read the work that leads up to this question, so I can't give a judgment as to whether Spencer is calculating the standard deviation at all. But if he is, these are the applicable rules.

2011-09-12 09:38:34The post by Socratic as Spencer's blog is key to the dispute
Tom Curtis

t.r.curtis@gmail...
112.213.162.57

if the figures are correct:

 

"Hopefully my final post on this topic. While looking at what it would take to compute the mixing layer from WAO data, I found that it has already been computed on a global grid, and available from NOAA, here:http://www.nodc.noaa.gov/OC5/WOA94/mix.html

There are three criteria in use: 1) (most common definition) depth at which temp is .5 C lower than the surface; 2) depth at which the density is .125 standard deviations greater than the surface; and 3) depth at which the density is equal to what the density would be with a .5 C change. These three definitions give rather different results.

After downloading all the data and running global weighted averages (see my first post), the global average mixing layer depth for each definition was:
1. 71.5 meters
2. 57.2 meters
3. 45.9 meters

For quarterly data, using these mixing layer depths gives for the LHS of the equation energy change rates of 8.9, 7.1, and 5.7 Wm^-2 respectively.

For monthly data, these mixing layer depths give energy changes rates of 12.5, 10.0, and 8.0 Wm^-2 respectively.

Dr. Dessler’s use of 9 Wm^-2 with monthly data therefore seems about right (though his mixing layer is too deep); Dr. Spencer’s use of 2.3 Wm^-2 with quarterly data seems too small (in large part because his mixing layer is too shallow)."

http://www.drroyspencer.com/2011/09/the-good-the-bad-and-the-ugly-my-initial-comments-on-the-new-dessler-2011-study/#comment-25051

 

Note that the jump to three month data by Spencer is highly suspect.  His Spencer & Braswell 11 he used 30 day standard deviations to determine N.  Switching to a three month SD, with the consequent reduction in the LHS of the equation to rebut Dessler's criticism is misdirection at best.  

 

2011-09-12 09:40:48
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

Neal,

Yes, that is the general case. And yes, if there is a covariance, then yes, that could explain a difference in answers, but I think the likelihood of it leading coincidentally to the same 1.7 and 2.2 he seems to get by bodging it, well, I would doubt it. And furthermore, remember, he is using an extremely simple model here. Would he really be trying to calculate the covariance between the ocean forcing and the radiative flux, for instance? Again, I doubt it.

As to "I can't give a judgment as to whether Spencer is calculating the standard deviation at all." In Spencer's online rebuttal, he says:

"Using the above equation, if I assumed a feedback parameter λ=3 Watts per sq. meter per degree, that 20:1 ratio Dessler gets becomes 2.2:1. If I use a feedback parameter of λ=6, then the ratio becomes 1.7:1."

Dessler's 20:1 ratio is a ratio of std. deviations. So that implies to me Spencer is calculating a ratio of standard deviations as well. And he has to derive his σN and σS. So I think he is (mis)calculating them.

Thanks for the feedback.

2011-09-12 10:01:15
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

Thanks Tom Curtis for that pointer.

Yes, I think Socratic's point is key.

By the way, even using Socratic's three-month values for the LHS (5.7, 7.1 and 8.9), and continuing to use Spencer's suggested 3-month values for for variables on the RHS, and still not even driving down to find out an exact value for σN, those values imply ratios for σS/σN of:
for LHS = 5.7, σS(t)/σN(t) > 10.1
for LHS = 7.1, σS(t)/σN(t) > 12.6
for LHS = 8.9, σS(t)/σN(t) > 15.9

I think any of those numbers is still sufficient for Dessler point that σN is not relatively large enough to be a significant forcing still holds. 

2011-09-12 10:05:19
rustneversleeps
George Morrison
george.morrison2@sympatico...
65.95.187.30

Thanks Tom Curtis for that pointer.

Yes, I think Socratic's point is key.

For what it's worth, even using Socratic's three-month values for the LHS (5.7, 7.1 and 8.9), and continuing to use Spencer's suggested 3-month values for for variables on the RHS, and still not even driving down to find out an exact value for σN, those values imply ratios for σS/σN of:
for LHS = 5.7, σS(t)/σN(t) > 10.1
for LHS = 7.1, σS(t)/σN(t) > 12.6
for LHS = 8.9, σS(t)/σN(t) > 15.9

I think any of those numbers would still be sufficient for Dessler's point to still hold - that σN is not relatively large enough to be a significant forcing.

2011-09-12 13:45:41
Albatross
Julian Brimelow
stomatalaperture@gmail...
199.126.232.206

Interesting, but I'll decline from commenting further on the specifics of this matter....I get the feelig that I am talking through my hat.  Socratic and others here not :)

It will be interesting to see who is correct.  Maybe Dessler will inlcude a range in his update.