New Scandal Erupts over NOAA Climate Data

Daily Tech ^ | August 7, 2007 1:07 PM | Michael Asher

[Ernest_at_the_Beach]

Weather station data hidden from public; scientists allege government cover-up

Roger Pielke (right) claims NCDC removed weather station data to dodge public scrutiny. (Source: University of Colorado)

***************************************************************

The theory of global warming began to explain one simple set of factsm-- surface temperature monitoring stations have shown a roughly one degree rise over the past century. But just where does these temperature readings come from? Most are reported by volunteer stations, usually no more than a thermometer inside a small wooden hut or below a roof overhang. In the US, 1,221 such stations exist, all administered by the National Climatic Data Center, a branch of the NOAA.

Two months ago, I reported on an effort to validate this network. A volunteer group headed by meteorologist Anthony Watts had found serious problems. Not only did sites fail to meet the NCDC's  requirements, but encroaching development had put many in ridiculously unsuitable locations -- on hot black asphalt, next to trash burn barrels, beside heat exhaust vents, even attached to hot chimneys and above outdoor grills.

Soon thereafter, a Seattle radio station interviewed the head of the NCDC, Dr. Thomas Peterson, informed him of the effort and quizzed him about the problems. Three days later, the NCDC removed all website access to station site locations, citing "privacy concerns." Without this data (which had been public for years), the validation effort was blocked. No more stations could be located.

Scientists were quick to respond. Climatologist Roger Pielke from the University of Colorado called the act a "coverup" and said it was designed to prevent public scrutiny. More shockingly, he revealed that researchers had been for years pressuring the government to validate the network themselves, and that the NCDC had begun to do j so, but cancelled the project and refused to make the data public, presumably to avoid this sort of scandal.  Joined by Watts and others, Pielke called upon the government to recant.

The resulting furor forced the NCDC to again made site locations public. But so far, they've failed to address to root of the problem, which is the wholly unsatisfactory locations of many of their recording sites, loations which make the resulting data unreliable, and compromise a dataset upon which much of US energy and environmental policy is based.

[JasonC]

I've done some analysis on the revised series. My conclusions, FWIW -

To a first approximation it is white noise.

There is a modest sign of 3 year periodicity in the power spectrum, cold warm warm, but nothing too significant there.

There are definite long periods revealed by moving averages. From the series start in 1880 to WW I, the series is stationary. From WW I to the end of Korea, it is increasing. From the end of Korea to the end of Vietnam, is it decreasing. From the end of Vietnam to now, it is increasing. I am being deliberately vague about the timings because only long periods are correct for moving averages.

The up and down movements in the long periods are of the same magnitude and duration. The overall series goes from low to high because it contains 2 ups and 1 down. A regression line or other fit (I used cubic and sin based fits as well as lines, all give broadly similar slopes) shows a net rate of increase of about 1 degree C in 200 years. If you measure peak to peak or trough to trough, there is no trend.

Plotting the histogram of departures from the mean, normalized to standard deviations, gives a roughly symmetrical distribution with most of the observations well concentrated between +2 and -2 sigma, with no trend. Adjusting for a linear trend instead, or for a sin fit or polynomial fit trend, in each case results in a skewed distribution with far more observations between 0 and -1 standard deviations (of the detrended series) than of positive ones. It also gives more +3 sigma events and a longer tail on the up side (because it classifies early warm period 1930s events as larger departures from the still-low trend).

If this is a noise signal it is therefore more likely stationary than trending. If it supposedly does have a trend, that trend (over cycles) is modest, with an expectation for the next century of an additional increase of 0.5C. And in that case, it is not pure noise about a mean temperature, but skewed with cooler cases more common, balanced by high outliers.

I note that the scale of increase seen in the second part of the series, when higher CO2 is observed, is roughly the same magnitude as can be accounted for by the Stefan-Boltzmann law with no amplifying term or "climate sensitivity". If one looks only at the last uptrend and ignores the rest of the data, one might detect a climate sensitivity around 2-3, but certainly nothing larger. This implies future temperature increases from additional CO2 greenhouse are bounded at about 1-1.5C. And it requires "believing" the current uptrend while ignoring the previous downtrend period, or the previous uptrend period that cannot have been caused by CO2, since CO2 hadn't risen yet.

The natural variations before CO2 can have been operating are equal in magnitude to anything it can physically cause, and random in sign. There is no reason to expect that they have ceased, or that later movement is all attributable to CO2 when none of the early moves (in both directions) were. Therefore, there is no reason to expect anything beyond the direct power terms CO2 can provide directly, which means under 1C changes ahead, maximum.

It is useful to actually quantify the total extra power reradiated by the warmer earth, if this series were taken as representative of the whole earth and the data is reliable (UHI effects ignored for the sake of argument etc), during the previous "up". And likewise to quantify the total energy shortfall from the higher peak to peak level, involved in the lower series from early 50s to mid 70s.

The peak to trough power level varies by about 1% (SB law, 0.7 degrees, 291C base), looking at a long moving average not the year to year noise. We can appoximate the integral of the power as a triangle, with a base 60 years long (roughly 1916 to 1976, for the up and the down combined) and the peak that high.

Let the operating power be somewhere between 250 and 500 watts, then the 1% difference is 2.5 to 5 watts at the max, half that as an average, and operating for that length of time. Then the total excess energy for the "pulse" is 1.2 to 2.5 times 10 to the 23rd, Joules. That is about 20000 exa-joules, or 44 times current world energy consumption. Which was of course significantly lower in the past, but then the period of the pulse is 60 years. Same scale in other words.

Similarly we can measure the shortfall of the cooling period (the "V" made from the Korea peak to the present peak with a 70s trough in between). The baseline in years is about 10% shorter, that is all, so the total energy is about 10% below the previous figure but the same order of magnitude.

This suggests another simple "null" hypothesis, beyond the null SB effect of known increases in CO2 - as humans release energy for their own purposes, the earth has to warm very slightly to glow slightly hotter for a period on the order of a few decades, to reradiate the released energy into space. Keep releasing stored energy at the same rate and you will maintain the marginally higher temperature - there is a human power input term. If so, we should expect to see a modest uptick in the mean temperature as the power "switches on". This is an adequate explanation of the observed regresssion line.

Still does not explain the full fluctuation in the moving average about that average, however. Solar or natural weather driven variations are the likely causes, but certainly not something that starts around 1950 and pushes one way the whole time, as CO2 does.

[ClaireSolt]

My inconvenient truth is that vast strides towards alternative energy and improved technology have occurred during the Bush regime, not under Democrats.

[JasonC]

Correction - 2x10^23 joules is of course 200,000 exajoules (I said 20000, a mistake), and current human energy output is 450 exajoules per year, so the pulse is 7.4 times present human energy consumption (annual rate for both). So the human energy release bit can only be a tenth or less of the excess power (with past energy use lower etc). If the earth glows 0.35C hotter for 60 years, average, then it (naturally) releases an order of magnitude more energy than humanity does today, above the 1880 to WW I baseline.

[ken21]

le kewl!

[Mr Rogers]

Weather stations at airports are there to provide data for pilots about to take off or land. They are NOT meant to provide the average reading across a valley, etc.

Tucson used to hit 102-103 in the summer. Now it hits 110-115...unless you go a few miles out of the city, where it is (drum roll, please)...102-103 on a hot summer day.

Global warming is all about statistics, and how they are interpreted.

[JasonC]

That is true, and improved biofuels are especially promising as an oil replacement (though they make CO2 just like oil does) in terms of the technology coming on line and to be expected in the next 10 years or so. But alternates other than ethanol still only contribute a fraction of 1% of human energy use. Hydropower is more important, and nuclear vastly more important. Coal is not scarce, is cheap, and is more important than all of them. Coal's supposed black mark is its CO2 footprint, and that appears to be largely based on unfounded fears. If we don't care about CO2 footprints in the near term, coal and ethanol look like easy replacements for expensive oil.

[Robert A Cook PE]

Well, there’s “only” 1200 of these local weather stations: Seems like it’s only a matter of a few days for each to be looked at by the local NOAA office - get a picture of the site, and release the REAL locations and environment.

Sure - They don’t want to “lose” the history available by moving a previous site to a new neutral site - but IF ANY has been grossly affected by new construction - or removal - of a nearby building or office or highway or parking lot - then ALL of their precious “historical: data is bunk anyway.

Urban heat islands are SUPPOSED to have been factored out of the data for years - but how “accurately” have the “factors” used to cancel the heat island affect been?

[samtheman]

I don’t know if it’s “puppet masters” but yes, you’re right, the short-sighted, narrow-minded, Rove-oriented, “organization-is-the-message”, “new tone” advisers would be dead set against President Bush making any bold statement on this, or any, issue.

[sourcery]

Very interesting!

[crazyshrink]

“What do we call this?”

How about GWGate.

Global Warming...not bush, but would appease the dums.

[Tolerance Sucks Rocks]

Realclimate's take on the Y2K change:

1934 and all that

Filed under:

• Instrumental Record
• Climate Science

— gavin @ 5:33 PM

Another week, another ado over nothing.

Last Saturday, Steve McIntyre wrote an email to NASA GISS pointing out that for some North American stations in the GISTEMP analysis, there was an odd jump in going from 1999 to 2000. On Monday, the people who work on the temperature analysis (not me), looked into it and found that this coincided with the switch between two sources of US temperature data. There had been a faulty assumption that these two sources matched, but that turned out not to be the case. There were in fact a number of small offsets (of both sign) between the same stations in the two different data sets. The obvious fix was to make an adjustment based on a period of overlap so that these offsets disappear.

This was duly done by Tuesday, an email thanking McIntyre was sent and the data analysis (which had been due in any case for the processing of the July numbers) was updated accordingly along with an acknowledgment to McIntyre and update of the methodology.

The net effect of the change was to reduce mean US anomalies by about 0.15ºC for the years 2000-2006. There were some very minor knock on effects in earlier years due to the GISTEMP adjustments for rural vs. urban trends. In the global or hemispheric mean, the differences were imperceptible (since the US is only a small fraction of the global area).

There were however some very minor re-arrangements in the various rankings (see data). Specifically, where 1998 (1.24ºC anomaly compared to 1951-1980) had previously just beaten out 1934 (1.23ºC) for the top US year, it now just misses: 1934 1.25ºC vs. 1998 1.23ºC. None of these differences are statistically significant. Indeed in the 2001 paper describing the GISTEMP methodology (which was prior to this particularly error being introduced), it says:

The U.S. annual (January-December) mean temperature is slightly warmer in 1934 than in 1998 in the GISS analysis (Plate 6). This contrasts with the USHCN data, which has 1998 as the warmest year in the century. In both cases the difference between 1934 and 1998 mean temperatures is a few hundredths of a degree. The main reason that 1998 is relatively cooler in the GISS analysis is its larger adjustment for urban warming. In comparing temperatures of years separated by 60 or 70 years the uncertainties in various adjustments (urban warming, station history adjustments, etc.) lead to an uncertainty of at least 0.1°C. Thus it is not possible to declare a record U.S. temperature with confidence until a result is obtained that exceeds the temperature of 1934 by more than 0.1°C.

More importantly for climate purposes, the longer term US averages have not changed rank. 2002-2006 (at 0.66ºC) is still warmer than 1930-1934 (0.63ºC - the largest value in the early part of the century) (though both are below 1998-2002 at 0.79ºC). (The previous version - up to 2005 - can be seen here).

In the global mean, 2005 remains the warmest (as in the NCDC analysis). CRU has 1998 as the warmest year but there are differences in methodology, particularly concerning the Arctic (extrapolated in GISTEMP, not included in CRU) which is a big part of recent global warmth. No recent IPCC statements or conclusions are affected in the slightest.

Sum total of this change? A couple of hundredths of degrees in the US rankings and no change in anything that could be considered climatically important (specifically long term trends).

However, there is clearly a latent and deeply felt wish in some sectors for the whole problem of global warming to be reduced to a statistical quirk or a mistake. This led to some truly death-defying leaping to conclusions when this issue hit the blogosphere. One of the worst examples (but there are others) was the 'Opinionator' at the New York Times (oh dear). He managed to confuse the global means with the continental US numbers, he made up a story about McIntyre having 'always puzzled about some gaps' (what?) , declared the the error had 'played havoc' with the numbers, and quoted another blogger saying that the 'astounding' numbers had been 'silently released'. None of these statements are true. Among other incorrect stories going around are that the mistake was due to a Y2K bug or that this had something to do with photographing weather stations. Again, simply false.

But hey, maybe the Arctic will get the memo.

[dennisw]

good info!!

[dennisw]

Steve McIntyre website servers melted down. He had so much traffic after Limbaugh and others broadcast his genius level sleuthing. He has new server now

[Ernest_at_the_Beach]

Thanks.....

Ultra Sonic 007 see post # 131...

[cogitator]

Thanks for saving me the trouble.

[VOA]

bump

[SamAdams76]

Logan Airport in Boston is a classic example of that which you describe. The readings at Logan Airport are an absolute joke and yet these are the official readings that make up Boston's climatological data.

I don't know how many times these past winters that Logan would be reporting 40 degrees and rain while the rest of Boston would be 31 degrees and snowing. In one ludicrous example I can remember a few years back, Boston proper got socked with a foot of snow and yet Logan Airport reported something like 4.3 inches of slush.

But if anybody knows the geography of Logan Airport, they would know that nobody lives out there but the seagulls. It is basically landfill, covered with concrete and surrounded by water on three sides so the temperatures are unduly influenced by the surrounding salt water and all the concrete. About the stupidest place you could put a weather station in the Boston area.

[SunkenCiv]

:’)

[SunkenCiv]

Thanks E. They’ve got more than one problem.

http://www.freerepublic.com/focus/news/1879848/posts?page=53#53