Gregory et al 2019: Unsound claims about bias in local weather suggestions and local weather sensitivity estimation

Reposted from Dr. Judith Curry’s Local weather And so forth.

Posted on October 18, 2019 by niclewis |

By Nic Lewis

The just lately printed open-access paper “How precisely can the local weather sensitivity to CO2 be estimated from historic local weather change?” by Gregory et al.[i] makes quite a few assertions, many uncontentious however others for my part unjustified, deceptive or positively incorrect.

Maybe most significantly, they are saying within the Summary that “The actual-world variations imply that historic EffCS [effective climate sensitivity] underestimates CO2 EffCS by 30% when contemplating all the historic interval.” However they don’t point out that this discovering relates solely to efficient local weather sensitivity in GCMs, after which solely to when they’re pushed by one explicit observational sea floor temperature dataset.

Nevertheless, on this article I’ll give attention to one explicit statistical situation, the place the declare made within the paper can readily be confirmed flawed without having to delve into the small print of GCM simulations.

Gregory et al. take into account a regression within the kind R = α T, the place T is the change in global-mean floor temperature with respect to an unperturbed (i.e. preindustrial) equilibrium, and R = α T is the radiative response of the local weather system to vary in T. α is thus the local weather suggestions parameter, and F2xCO2 /α is the EffCS estimate, F2xCO2 being the efficient radiative forcing for a doubling of preindustrial atmospheric carbon dioxide focus.

The paper states that “that estimates of historic α made by OLS [ordinary least squares] regression from real-world R and T are biased low”. OLS regression estimates α because the slope of a straight line match between R and T information factors (normally with an intercept time period for the reason that unperturbed equilibrium local weather state isn’t identified precisely), by minimising the sum of the squared errors in R. Random errors in R don’t trigger a bias within the OLS slope estimate. Thus within the beneath chart, with R taken as plotted on the y-axis and T on the x-axis, OLS finds the pink line that minimizes the sum of the squares of the lengths of the vertical strains.

Nevertheless, a few of the variability in measured T could not produce a proportionate response in R. That might happen if, for instance, T is measured with error, which occurs in the true world. It’s well-known that in such an “error within the explanatory variable” case, the OLS slope estimate is (on common) biased in the direction of zero. This situation has been referred to as “regression dilution”.

Regression dilution is one motive why estimates of local weather suggestions and local weather sensitivity derived from warming over the historic interval typically as a substitute use the “distinction technique”.[ii] [iii] [iv] [v] The distinction technique includes taking the ratio of variations, ΔT and ΔR, between T and R values late and early within the interval. In apply ΔT and ΔR are normally based mostly on differencing averages over not less than a decade, in order to cut back noise.

I’ll notice at this level that when a slope parameter is estimated for the connection between two variables, each of that are affected by random noise, the chance distribution for the estimate might be skewed relatively than symmetric. When deriving a finest estimate by taking many samples from the error distributions of every variable, or (if possible) by measuring them every on many differing events, the suitable central measure to make use of is the pattern median not the pattern imply. Physicists need measures which are invariant below reparameterization[vi], which is a property of the median of a chance distribution for a parameter however not, when the distribution is skewed, of its imply. Regression dilution impacts each the imply and the median estimates of a parameter, though to a considerably totally different extent.

To date I agree with what is claimed by Gregory et al. Nevertheless, the paper goes on to state that “The bias [in α estimation] impacts the distinction technique in addition to OLS regression (Appendix D.1).” This assertion is flawed. If true, this may suggest that observationally-based estimates utilizing the distinction technique could be biased barely low for local weather suggestions, and therefore biased barely excessive for local weather sensitivity. Nevertheless, the declare isn’t true.

The statistical analyses in Appendix D take into account estimation by OLS regression of the slope m within the linear relationship y(t) = m x(t), the place x and y are time collection the obtainable information values of that are affected by random noise. Appendix D.1 considers utilizing the distinction between the final and first single time durations (right here, it seems, of a yr), not of averages over a decade or extra, and it assumes for comfort that each x and y are recentered to have zero imply, however neither of those have an effect on the purpose of precept at situation.

Appendix D.1 reveals, appropriately, that when solely the endpoints of the (noisy) x and y information are utilized in and OLS regression, the slope estimate for m is Δy/Δx, the identical because the slope estimate from the distinction technique. It goes on to assert that taking the slope between the x and y information endpoints is a particular case of OLS regression and that the truth that an OLS regression slope estimate is biased in the direction of zero when there may be uncorrelated noise within the x variable implies that the distinction technique slope estimate is equally so biased.

Nevertheless, that’s incorrect. The median slope estimate isn’t biased because of errors within the x variable when the slope is estimated by the distinction technique, nor when there solely two information factors in an OLS regression. And though the imply slope estimate is biased, the bias is excessive, not low. Fairly than going into an in depth theoretical evaluation of why that’s the case, I’ll present that it’s by numerical simulation. I will even clarify how in easy phrases regression dilution may be seen as arising, and why it doesn’t come up when solely two information factors are used.

The numerical simulations that I carried out are as follows. For simplicity I took the true slope m as 1, in order that the true relationship is y = x, and that true worth of every x level is the sum of a linearly trending factor operating from zero to 100 in steps of 1 and a random factor uniformly distributed within the vary -30 to +30, which may be interpreted as a simulation of a trending “local weather” portion and a non-trending “climate” portion.[vii]  I took each x and y information (measured) values as topic to zero-mean unbiased usually distributed measurement errors with an ordinary deviation of 20. I took 10,000 samples of randomly drawn (as to the true values of x and measurement errors in each x and y) units of 101 x and 101 y values.

Utilizing OLS regression, each the median and the imply of the ensuing 10,000 slope estimates from regressing y on x utilizing OLS have been zero.74 – a 26% downward bias within the slope estimator as a result of regression dilution.

The median slope estimate based mostly on taking variations between the averages for the primary ten and the final ten x and y information factors was 1.00, whereas the imply slope estimate was 1.01. When the averaging interval was elevated to 25 information factors the median bias remained zero whereas the already tiny imply bias halved.

When variations between simply the primary and final measured values of x and y have been taken,[viii] the median slope estimate was once more 1.00 however the imply slope estimate was 1.26.

Thus, the slope estimate from utilizing the distinction technique was median-unbiased, in contrast to for OLS regression, whether or not based mostly on averages over factors at every finish of the collection or simply the primary and final factors.

The explanation for the upwards imply bias when utilizing the distinction technique may be illustrated merely, if errors in y (which on common don’t have any impact on the slope estimate) are ignored. Suppose the true Δx worth is 100, in order that Δy is 100, and that two x samples are topic to errors of respectively +20 and –20. Then the 2 slope estimates might be 100/120 and 100/80, or zero.833 and 1.25, the imply of which is 1.04, in extra of the true slope of 1.

The image stays the identical even when (fractional) errors in x are smaller than these in y. On lowering the error normal deviation for x to 15 whereas growing it to 30 for y, the median and imply slope estimates utilizing  OLS regression have been each zero.84. The median slope estimates utilizing the distinction technique have been once more unbiased whether or not utilizing 1, 10 or 25 information factors firstly and finish, whereas the imply biases remained below zero.01 when utilizing 10 or 25 information level averages and diminished to zero.16 when utilizing single information factors.

Actually, a second’s thought reveals that the slope estimate from 2-point OLS regression have to be unbiased. Since each variables are affected by error, if OLS regression offers rise to a low bias within the slope estimate when x is regressed on y, it should additionally give rise to a low bias within the slope estimate when y is regressed on x. If the slope of the true relationship between y and x is m, that between x and y is 1/m. It follows that if regressing x on y offers a biased low slope estimate, taking the reciprocal of that slope estimate will present an estimate of the slope of the true relationship between y and x that’s biased excessive. Nevertheless, when there are 2 information factors the OLS slope estimate from regressing y on x and that from regressing x on y and taking its reciprocal are equivalent (for the reason that match line will undergo the two information factors in each instances). If the y-against-x and x-against-y OLS regression slope estimates have been biased low that would not be so.

As for a way and why errors within the x (explanatory) variable trigger the slope estimate in OLS regression to be biased in the direction of zero (supplied there are greater than two information factors), however errors within the y (dependent) variable don’t, the best way I take a look at it’s this. For simplicity, I take centered (zero-mean) x and y values.  The OLS slope estimate is then Σxy / Σxx, that’s to say the weighted sum of the y information values divided by the weighted sum of  the x information values, the weights being the x information values.  An error that strikes a measured x worth farther from the imply of zero not solely reduces the slope y/x for that information level, but in addition will increase the burden given to that information level when forming the OLS slope estimate. Therefore such factors are given extra affect when figuring out the slope estimate. Then again, an error in x that strikes the measured worth nearer to zero imply x worth, growing the y/x slope for that information level, reduces the burden given to that information level, in order that it’s much less influential in figuring out the slope estimate. The web result’s a bias in the direction of a smaller slope estimate. Nevertheless, for a two-point regression, this impact doesn’t happen, as a result of regardless of the indicators of the errors affecting the x-values of the 2 factors, each x-values will at all times be equidistant from their imply, and so each information factors may have equal affect on the slope estimate whether or not they enhance or lower the x-value. In consequence, the median slope estimate might be unbiased on this case.  Regardless of the variety of information factors, errors within the y information factors won’t have an effect on the weights given to these information factors when forming the OLS slope estimate, and errors within the y-data values will on common cancel out when forming the OLS slope estimate Σxy / Σxx.

So why is the proof in Gregory et al. AppendixD.1, supposedly exhibiting that OLS regression with 2 information factors produces a low bias within the slope estimate when there are errors within the explanatory (x) information factors, invalid? The reply is easy. The Appendix D.1 proof depends on the proof of low bias within the slope estimate in Appendix D.Three, which is expressed to use to OLS regression with any variety of information factors. But when one works via the equations in Appendix D.Three, one finds that within the case of solely 2 information factors no low bias arises – the anticipated worth of the OLS slope estimate equals the true slope.

It’s a little miserable that after a few years of being criticised for his or her insufficiently good understanding of statistics and lack of shut engagement with the statistical group, the local weather science group seems nonetheless to not have solved this situation.

Nicholas Lewis ……………………………………………….. 18 October 2019

[i] Gregory, J.M., Andrews, T., Ceppi, P., Mauritsen, T. and Webb, M.J., 2019. How precisely can the local weather sensitivity to CO₂ be estimated from historic local weather change?. Local weather Dynamics.

[ii] Gregory JM, Stouffer RJ, Raper SCB, Stott PA, Rayner NA (2002) An observationally based mostly estimate of the local weather sensitivity. J Clim 15:3117–3121.

[iii] Otto A, Otto FEL, Boucher O, Church J, Hegerl G, Forster PM, Gillett NP, Gregory J, Johnson GC, Knutti R, Lewis N, Lohmann U, Marotzke J, Myhre G, Shindell D, Stevens B, Allen MR (2013) Power funds constraints on local weather response. Nature Geosci 6:415–416

[iv] Lewis, N. and Curry, J.A., 2015. The implications for local weather sensitivity of AR5 forcing and warmth uptake estimates. Local weather Dynamics, 45(Three-Four), pp.1009-1023.

[v] Lewis, N. and Curry, J., 2018. The affect of current forcing and ocean warmth uptake information on estimates of local weather sensitivity. Journal of Local weather, 31(15), pp.6051-6071.

[vi] In order that, for instance, the median estimate for the reciprocal of a parameter is the reciprocal of the median estimate for the parameter. This isn’t usually true for the imply estimate. This situation is especially related right here since local weather sensitivity is reciprocally associated to local weather suggestions.

[vii] There was an underlying development in T over the historic interval, and taking it to be linear implies that, within the absence of noise, linear slope estimated by regression and by the distinction technique could be equivalent.

[viii] Correcting the small variety of unfavorable slope estimates arising when the x distinction was unfavorable however the y distinction was constructive to a constructive worth (see, e.g., Otto et al. 2013). Earlier than that correction the median slope estimate had a 1% low bias. The constructive worth chosen (right here absolutely the worth of the unfavorable slope estimate concerned) has no impact of the median slope estimate supplied it exceeds the median worth of the remaining slope estimates, however does materially have an effect on the imply slope estimate.

Like this:

Like Loading…