Roast Turkey Since 1995

This posting is a little bit different for me. First off, I’ve made a batch of graphs. That I’ve done so at all is a bit “new” for me. But beyond that, I’ve made the graphs before even starting the article. It was the data that got me looking, but the graphs that made the case. And what is that case?

That it’s a bad idea to play with the instrumentation.

First, the motivation. In looking at the GISS anomaly maps, Turkey is always roasting. There was something about that that just seemed wrong. It ought to have been a more random event; who was Hot Hot Hot and who was not.

So I took a look at the dT/dt report and and noticed that there was a clear “kink” in the data. It was running along flat, then about 1991 – 1995 it popped up and then continued running fairly flat with some rise. Odd, that ‘step function’. CO2 does not do a step function. Airports don’t do a step function. UHI does not do a step function…

( For folks new to this: dT/yr is the “average of the changes of temperature, month now vs the same month that last had valid data, for each year”. An anomaly process similar to First Differences. Then dT is the running total of those changes, or the total change, the “Delta Temperature” to date.)

You will probably want to click on the graphs to get a bigger version to look at.

Notice that the dT line is the “cumulative” dT/yr. Notice also that right around 1990 the dT/yr goes very small in range. The volatility just leaves it. So we see that dT is running along at a lower value, and dT/yr is dead flat for a couple of years, THEN about 1994 dT/yr pops up and dT starts a run for the moon. So what happened between 1990 and 1994?

The Great Dying Of Thermometers took out a lot of thermometers. Further, a bunch of old “Duplicate Number” flags end (that I’ve been calling “modification flags”, but I’m swapping over to the NCDC name) and some new ones begin. Turkey had a fairly large number of thermometers, but that number plunges. Still, it’s a bit large for really seeing exactly what happens thermometer by thermometer. We’ll see a clearer example down below in Mauritius. But for now, we need to slice a little Turkey…

Looking at the temperature data directly, there were a lot of overlapping adds and drops. Duplicate Number flag 0 or 1 that carried forward and overlapped with Duplicate Number flag 2 or even flag 3. So their is a bit of ‘feathering’ or ‘blending’ of the dropping, adding, and changing. I believe that is why when you look at the dT/yr right about 1989 – 1993 it goes to very small volatility, even though a large number of thermometers leave the record (or perhaps, because of it). Eventually we are left with a lot of new Duplicate Number flag thermometers and very few of the older consistent record.

What is a “Duplicate Number” flag? It is an indication that SOMETHING about that data record is different and that it needs different handling than the prior record. It could be a new thermometer at that location, or it could be a new Time Of Observation (changed procedures) or it might be simply that the thermometers and the TOBS are the same, but some “post processing” is different. A different QA method tossing out “outliers” or a different “TOBS adjustment” or just about anything. Basically, it means “Something that matters changed”. I’d say so…

Now look again at the dT/yr line. Notice anything? Run your eye along the peaks at about the 1.3 C level just above the 1 C line. Notice the consistent peaks? Now run your eye along the -1 C line. Notice the consistent peaks starting before 1970? Now run your eye along at about -0.5 C. Notice how much more area is in the down spikes before 1990 when compared with after? Something started to slightly clip the lower excursions in the ’80s, but that 1990 change substantially eliminates them and strongly dampens or blunts any more than about -0.75 C and reduces the number significantly.

Don’t know what it was, but it happens at the same time the Duplicate Number flags change. And We’ve seen this in Canada (as the Smith Effect at Fort Smith) and I’ve seen it in dozens of other cases around the planet. The “warmth” is from clipping of the cold spikes, and it has onset with the Duplicate Number flag change.

The Ugliest Graph You Will Ever Love

In this graph, instead of the average dT/year we see the individual months dT for that year. So there are 12 lines in this “Spaghetti graph” for dT(per month)/yr. Maybe we ought to call it a ‘hair graph’ ;-) and that makes it look useless at this compressed scale. But click on it and enlarge it. Now the monthly extremes are much more visible. The “spikes” of monthly temperature changes are “spikier”. The “clipping” is fairly clearly seen. The start of the newer Duplicate Number flags dampens the really deep downward spikes (as they average with the older Duplicate Number flags) but in 1990 when the old Duplicate Number flags are dropped and only the new carry forward, the “cold spikes” look to me to be lifted to where what in the past was a -6 is now the few -4 about 2001 and the prior -4 spikes are now running about -2.5 C.

There is what looks like a little muting of the “top spikes” but not nearly so much.

Again, why is completely unknown. One would need to look up what those Duplicate Number flags indicate changed at that location to figure out ‘what changed’. But that it changed and the fact that it changed in a step function is pretty clear…

So, Are The Trends of the Two Segments Different?

You Bet!

A Cleaner Case – Mauritius

Is there a simpler place? Somewhere with fewer thermometers and a clearer point in time when things ‘cut over’? Yes, several. This is just one of a half dozen I found in looking at about a dozen randomly chosen places. The scale here is much shallower than for Turkey, since these are Islands in a warm ocean. But even here we can see a “muting’ of the down spikes of dT/yr such that they don’t reach the -0.5 C line and rarely get through the -0.25 C line after the change of Duplicate Number flags.

If we look at the two segments, before and after the 1990 era Duplicate Number flag change, do we find a difference? Oh yeah. Again we do see the top peaks slightly muted, but the bottom going peaks are much more muted and the effect of that “peak clipping” of the dT/yr, and muting the depth and quantity of the bottoms; is a dramatic change in the dT line.

I’ll spare you the “Hair Graph” of the monthly data… Unless requested…

And it is all done with ‘anomalies’ so it must be right ;-)

(For folks unaware of it, perhaps a long time after this is written, there has been a bit of a “tiff” about the fact that I don’t use anomalies for all my investigations. It’s been asserted that if you use anomalies things come out right and if you don’t they come out wrong. Well, I’m from a school of thought that says you use ALL the tools you have and see if they all agree. It’s more of a “forensics” mind set. I’ve done computer forensics, and one of the things you learn is that it is much much harder to ‘cook the books’ so they come out right under ALL available tools… So always “Come to your opponent out of the Sun.” and “Take nothing that he offers to you.” Sun Tzu. And the best way to assure you have found ‘where the sun is’ in a forensics exam is to come at the data from ALL sides. And never do things in the usual way or the way the other side suggests they ought to be done; if possible, approach from exactly the other direction…)

Conclusion

Now, I have no idea what Duplicate Number flag 3 means. It could be the change from Liquid In Glass to Electronic, or it could be a different QA procedure, or a move from “Stevenson Screen by the Palm Grove” to “near the hangar”, or who knows what. But what I can say is that:

1) It’s a Giant “Dig Here”.

2) It is not CO2.

3) It is directly correlated with the bulk of the change of trend to warming in Mauritius.

4) That warming comes, as have most of the cases I’ve seen, as a Step Function or knee, though there can be some ‘blending’ if different records overlap.

5) Dropping thermometers matters (directly, or as here, indirectly via unmasking Duplicate Number flag 3).

6) The specifics of the Instruments, Modifications, and the Siting Matters. A Lot.

7) It’s not the “kept” vs “tossed” that matters it’s the “kept” vs “replacement” at the mod flag level.

What the Input Data Looks Like – Sample Duplicate Number Flags

OK, I can’t make a new posting full of graphs without having at least some tables… don’t want to have withdrawal shock ;-) so here are some small tables showing some of the “Duplicate Number flag” changes in Mauritius. These are mostly for documentation of what to look for in the data for folks who want to look at the data themselves; those not interested in the input data will want to skip this part.

The first 3 digits are the Country Code “129” for Mauritius. Then 5 + 3 for station and sub-station ID “61986” + “000”. Then the Duplicate Number flag. “0” for this first thermometer record at that place. Then the year (1986 in this first line). Then 12 monthly temps in 1/10 C so that first one “273” is really 27.3 C.

... 1296198600001986 273 280 274 269 256 240 235 233-9999 250 258 273 1296198600001987 279 278 289 284 267 249 245 234 238 247 257 265 1296198600001988 279 282 287 274 258 241 235 232 235 248 257 273 1296198600001989 277 280 281 270 254 239 229 226 234 243 251 269 1296198600001990 271 275 273 271 257 243 233 231 242 246 255 272 1296198600001991 277 280 283 273 261 246 234-9999 239-9999 258-9999

We see it ends in 1991 with the last 6 months having 3 missing data flags of -9999.

Same Station ID, but we get a “1” Duplicate Number flag (that actually starts in 1954).

... 1296198600011984 273 278 277 266 247 234 222 225 235 241 255 265 1296198600011985 270 272 276 268 255 243 233 230 233-9999 251 271 1296198600011986 273 280 274 269 256 240 235 233 237 250 258 273 1296198600011987 279 278 289 284 267 249 245 234 238 247 257 265 1296198600011988 279 282 287 274 258 241 235 232 235 248 257 273 1296198600011989 277 280 281 270 254 239 229 226 234 243 251 269 1296198600011990 271 275 273 271 257 243-9999 231 242 246 255 272

And ends in 1990. We also have a very short lived Duplicate Number flag 2 that looks like someone ran a 10 year experiment. This is the whole thing:

1296198600021971 275 270 274 267 255 241 232 224 225 237 259 271 1296198600021972 276 269 277 271 260 255 238 239 239 245 265 278 1296198600021973 282 282 284 277 267 247 239 230 237 244 254 263 1296198600021974 270 271 272 265 252 240 230 229 227 239 251 272 1296198600021975 279 280 276 275 266 247 239 229 236 239 261 266 1296198600021976 273 277 277 273 266 252 239 239 235 245 258 273 1296198600021977 279 281 285 278 266 252 242 239 237 246 258 269 1296198600021978 276 280 269 272 261 245 234 234 238 246 257 267 1296198600021979 279 275 276 274 265 247 238 236 238 248 260 279 1296198600021980 276 275 277 277 258 249 240 232 234 247 258 266

Then we get to the “Duplicate Number flag 3” that carries forward as the other die. It starts in 1987 for a bit of blending and “feathering” in with the other records, then, as they die off in 1990+, it is what gives us the temperatures today.

1296198600031987 279 278 289 284 267 249 245 234 238 247 257 265 1296198600031988 279 282 287 274 258 241 235 232 235 248 257 273 1296198600031989 277 280 281 270 254 239 229 226 234 243 251 269 1296198600031990 271 275 273 271 257 243 233 231 242 246 255 272 1296198600031991 277 280 283 273 261 246 234-9999 239-9999 258 284 1296198600031992 284 280-9999 267 254 242 232 229 234 243 255 255 1296198600031993 276 286 280 275 267 248 232 231 232 242 252 266 1296198600031994 277 281-9999 277 262 249 241 235 236 248 252 268 1296198600031995 276 284 279 275 257 248 239 234 239-9999 251-9999 1296198600031996 278 280 283 269 255 245 231 231 232 237 253 277 1296198600031997 276 281 280 274 264 246 236 238 241 247 272 283 1296198600031998 298 291 290 278 265 253 240 236 239 245 254 265 1296198600031999 271 278 280 275 265 250 233 233 236 244 257 267 1296198600032000 275 278 277 270 264 242 235-9999 235 249 262 274 1296198600032001 284 290 285 281 268 243 240 243 247 252 262 277 1296198600032002 284 285 287 276 264 247 244 235 242 252 266 282 1296198600032003 293 284 286 285 273 250 242-9999-9999 254 265 276 1296198600032004 279 290 287 279 260 245 242 240 246 252 265 277 1296198600032005 291 273 288 279 264 251 240 233 237 245 259 273 1296198600032006 282 280 281 277 265 253 241 238 241 248 265 276 1296198600032007 293 282 276 273 265 247 241 239 241 244 260 271 1296198600032008 279 274 273 271 259 243 233 240 246 251 267 277 1296198600032009 287 285 289 284 273 257 241 238 242 255 270-9999