The surface temperature record really is incredibly robust...

Discussion in 'Environment & Conservation' started by caerbannog, Feb 4, 2012.

  1. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    OK folks, check *this* out.


    Decided to play around with the GHCN raw temperature data a bit more. Tried another interesting little experiment to test the robustness of the global temperature record. I set up a run where only 45 stations (all rural) were used to compute global-average temperature results.

    Here's the approach I used:

    1) I divided up the Earth into large latitude/longitude grid-cells (longitude dimensions adjusted to keep grid-cell areas approximately constant as you move north/south from the Equator).

    2) For each grid-cell, I selected the rural station with the longest temperature record (i.e. contained the greatest number of years of temperature data).

    3) A total of 45 rural stations were selected for processing (GoogleEarth-checked to make sure that no urban stations were accidently included). But since not all station records contain data for every year, the actual number of stations reporting data for any given year was often considerably smaller. For example, prior to 1900, only about a dozen of the selected stations reported any data. During the 20th-Century, anywhere from 25 to 44 of the selected stations reported data for any given year.

    So I ended up using a very sparse array of temperature stations, scattered roughly evenly around the world. I didn't cherry-pick any stations (just selected the one station in each grid-cell that reported the most data).

    Here is a plot of my results; the official NASA/GISS global land-temperature results (copied/pasted directly from the NASA/GISS web-site) are also plotted as a comparison.

    [​IMG]

    I want to emphasize that this is the output of the very first run based on this procedure -- I didn't do any cherry-picking of the "best out of 20 runs" or anything like that.

    Note that even with just this very small set of rural stations, my global-average temperature results track NASA's pretty closely for most of the plotted time period. There's a good bit of divergence before 1885, but my results for the years 1880-1885 were based on data from only 12 stations!

    And remember -- the GHCN global-temperature network consists of *several thousand* stations. So these latest results were computed from on the order of *one percent* of the GHCN station network. I also want to emphasize that these latest results (like all the results I posted earlier in this thread) are based on *raw* (i.e. not "adjusted", "homogenized", whatever) data.

    The algorithm that produced the above results is a very straightforward averaging procedure, something that someone who has had a couple of semesters of introductory computer programming at your local community college could code up. This is not rocket-science!

    The global temperature network really is incredibly robust -- no matter how you "slice and dice" the data, the global-warming signal just jumps right out at you. And that defines robustness -- all kinds of different approaches, using different averaging procedures, different selections of temperature stations, etc., produce very similar results. There really are lots of ways to "skin this global-temperature cat"!
     
  2. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    Ignore this bit:
    This is a near-duplicate of a post I put up on another message-board -- forgot to edit the bit about "earlier results in this thread" out.
     
  3. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    For the lurkers who have looked that this thread, here are several take-home lessons here.

    1) The global surface temperature network is incredibly robust. You can get results consistent with the officially published (NASA/NOAA/CRU/etc) global-warming results even when you use as few as 1 percent of the surface temperature stations. And you can do this with *raw* (i.e. not adjusted or homogenized) data.

    2) It is incredibly easy (at least for those who some programming experience) to "roll your own" experiments to prove that the UHI effect is virtually nonexistent on a global scale, that the "dropped stations" claims pushed by deniers are completely wrong, and that data adjustment/homogenization steps are completely unnecessary. Run *raw* data through a simple anomaly-averaging procedure and you get almost the exact same results that the "pros" get.

    3) All of the data, documentation, and software tools needed to do this are available on line for free; they are only a few mouse-clicks away. Even the cheapest laptop has far more than enough computing horsepower needed to crunch the temperature data.

    4) Doing all of the above is something that an experienced programmer/analyst could take on as a "weekend project".

    5) And perhaps the most important: The deniers who have been pushing claims about UHI, data "manipulation", "dropped stations" , etc. for *years* have done none of this straightforward analysis to test their claims -- this in spite of the fact that all of the above can easily be done in a matter of *days*.
     
  4. Not Amused

    Not Amused New Member

    Joined:
    Jul 23, 2011
    Messages:
    2,175
    Likes Received:
    19
    Trophy Points:
    0
    You are solving the wrong problem.

    How many people die as temperature continue to climb? Less than the number that die when oil and coal are shut off, with no viable alternative to grow and transport food.

    With a viable alternative, you don't need all the scary predictions. The alternative will be used, prices will drop as volume increases. Coal and oil will be more expensive, and fall out of favor.

    An alternative - not 10, sort of alternatives. So far, biofuel from algae looks good. Squeeze out the oil, then burn the rest in existing coal fired plants. Crack the oil into diesel, gasoline and propane, which drops into existing infrastructure.
     
  5. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    No, you are posting to the wrong thread. Now, let me remind you of the thread title, "The surface temperature record is incredibly robust". Now, if you'd like to address the content of my posts here, or ask some serious questions about how I generated my results, please feel free to do so.

    But if you want change the subject to something completely unrelated to the topic of this thread (i.e. the global temperature record and how to analyze it), then go start another thread.

    There's this really cool blue button at the top of the message board that you can use to start your very own thread. If you can't post something relevant to this thread here, I'd like to suggest that you give that cool blue button a whirl.
     
  6. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    (Copy/pasted from the "backup" forum4politics discussion-board)

    Well, maybe not quite *amazing*, but still pretty cool.

    I updated my code to crunch GHCN V3 data and to generate simple XML station latitude/longitude files that GoogleEarth can read.

    I fired off a couple of new processing runs, dumped out the station lat/long info into XML format, read all the station info into GoogleEarth and created a couple of pretty pix of the station locations to go along with the latest processing results.

    For this particular run, I divided up the globe into 20degx20deg grid-cells (at the Equator, adjusted appropriately as you go N/S from the Equator to keep the grid-cell areas approximately constant) and selected the single rural station with the longest temperature record for each grid cell.

    The selection process picked a total of 85 rural stations. However, due to varying station record lengths, data gaps, etc., the average number of stations that reported data for any given month/year was only about 50.

    Here is a plot of the "85 rural stations" results plotted against the official NASA/GISS "meteorological stations" index (copy/pasted directly from the NASA/GISS web-site).

    [​IMG]

    Of course, the results produced with data from a few dozen stations won't look exactly like results produced from data from a few *thousand* stations, but the results here really are quite amazingly similar.

    (Oh, yes -- and I used *raw* - not "homogenized" -- temperature data. Rural stations only. There were no manipulations, adjustments or anything like that applied to the data -- just ran the raw data through a straightforward averaging procedure.)

    Now, let's look at GoogleEarth imagery of the station locations on the globe.

    First, here is a view of the stations used by NASA/GISS:
    [​IMG]


    And here is a view of the stations I used to produce my "85 rural stations" results:
    [​IMG]

    This is a powerful illustration of the robustness of the global temperature network. You can quite literally throw out 98 percent of the stations and still get global-average temperature results that line up quite nicely with NASA's.

    The surface station network is so robust that the global-warming signal pops out no matter how you "slice and dice" the data (as long as you do the math right, that is).

    Now mind you, there was absolutely no "cherry picking" involved in getting these results -- the procedure used to select the stations is completely automatic -- just iterate through the stations in each grid cell and pick the single station with the longest data record. These results (as well as results posted earlier in this thread) are quite literally what popped out on my first processing runs. Absolutely no "trial and error" fiddling was done to get the "best" results.
     
  7. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    Per a request over on the forum4politics board, I finally got around to uploading the relevant KML files into the "Google cloud".

    One file shows locations of all stations that I could compute valid 1951-1990 baseline averages for. This is nearly identical to the set of stations used by NASA to compute its "meteorological stations" index.

    The other KML file contains locations of the stations that I used to compute my "Sparse Rural 20x20" results.

    You will be able to tell which is which from the file names (and file sizes, as well).

    Both can be read into Google Earth to display the station locations on the globe.

    https://docs.google.com/open?id=0B0pXYsr8qYS6YTJ5Q21XRHl6OTQ
    https://docs.google.com/open?id=0B0pXYsr8qYS6QlFyR2pKaGprMTA

    A quick note -- the metadata latitudes/longitudes are of limited precision (2 decimal digits), so station locations on the globe are "rounded to the nearest mile" in the worst case. This is plenty of precision for gridding/averaging, but not quite good enough to pinpoint exact station locations on the Earth's surface.
     
  8. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    Here's another demonstration of the robustness of the GHCN global temperature network.

    How many stations do you need to process to get the long-term global-warming signal to emerge from the noise? Darned few, as it turns out. And when I say *few*, I mean just that: a *few*.

    Below is what happens when you "push things to the limit" in terms of reducing the number of stations processed.

    The plot below shows the official NASA/GISS "meteorological stations" (aka NASA/GHCN) results vs. my own results computed from 4, 9, 21, 33, and 68 rural GHCN stations, respectively. Those numbers reflect the total number of stations selected. Note: since not all stations reported data for every year/month, the actual average number of stations that reported data for any given year was even smaller than the above numbers.

    Remember that the NASA/GISS "meteorological stations" index is computed from something like *four thousand* stations. The fact that you can get a long-term global-warming signal from just a few dozen stations that looks like the global-warming signal that you get when you process *thousands* of stations is a testament to the robustness of the GHCN global temperature network. Of course, the results are "noisier" when you process a very small number of stations (as you should expect), but the long-term global-warming trend is still *very* apparent.


    [​IMG]

    Once again, my results were computed from *raw* (i.e. not "homogenized") temperature data via a straightforward anomaly-averaging procedure that I described here earlier.

    How did I choose the stations? I used the following (very simple) selection procedure.

    1) Parcel the Earth's surface into very large latitude/longitude grid-cells of approximately equal area.

    2) Search each grid-cell for the rural station with the longest temperature record. Use that single station from each grid-cell.

    3) That's it!

    I ran the above procedure with the following grid-cell latitude/longitude dimensions (at the Equator, with adjustments as needed to keep grid-cell areas approximately equal as you move N/S away from the Equator).

    1) 90 deg latitude x 180 deg longitude.
    2) 90 deg latitude x 90 deg longitude.
    3) 45 deg latitude x 45 deg longitude.
    4) 22.5 deg latitude x 45 deg longitude.
    5) 22.5 deg latitude x 22.5 deg longitude.

    Note: these are the dimensions of grid-cells that bordered the Equator. The longitude dimensions for grid cells to the north/south were adjusted to keep grid-cell areas as nearly constant as possible. That is the reason that the number of stations selected did not exactly double when the nominal (bordering the Equator) grid-cell areas were halved.

    The results above (like all the other results I've posted here) were *not* "cherry-picked" in any way. I did not do any trial/error processing to find the "optimum" sets of stations. Just took the results of the "first tries" that implemented the above selection procedure, and plotted 'em up.

    Folks, if you know what you are doing, you will find that it's virtually impossible to get global-warming results that contradict NASA's -- no matter what globally-distributed subset of temperature stations you select.

    If the GHCN station data were really contaminated with local UHI/whatever effects, then different subsets of stations would give you very different long-term temperature trend results. But that is absolutely not what happens.

    In fact, the global-warming signal begins to emerge when you process as few as 4 stations, and strengthens very rapidly as you increase the #stations processed.

    Given that the raw station data comes from the NWS offices of something like 180 independent nations, a conspiracy that ensures that the same "fake" global-warming signal would appear virtually no matter what globally-distributed subset of stations you process is something that could exist only in the minds of the most delusional tinfoil-hatters.
     
  9. Bowerbird

    Bowerbird Well-Known Member

    Joined:
    May 13, 2009
    Messages:
    92,653
    Likes Received:
    74,094
    Trophy Points:
    113
    Gender:
    Female
    Nice work mate - but it seems that the denialists are not listening (surprise surprise)
     
  10. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0
    Yep -- I've pretty much given up any hope of trying to connect the denialists (here or elsewhere) to anything remotely resembling reality. So when I do post stuff, it's intended for lurker eyeballs.

    The major lessons that I hope lurkers take away from this thread are:

    1) The global temperature record really is quite robust -- it's amazingly easy to confirm the published NASA results, even when you use only a tiny fraction of the available data. Nothing fancy is required -- a very straightforward anomaly-averaging procedure applied to raw temperature data will do the trick. You don't need "homogenize" or "adjust" the raw data in any way. Someone who is reasonably "up to speed" with respect to computer programming can bang out some pretty decent "first cut" results in just a few days in his/her spare time (coding everything up from scratch).

    2) The deniers who continue to attack the global temperature results published by NASA/NOAA/CRU/etc. really are completely delusional and/or incompetent. They don't deserve even a minute of serious consideration.
     
  11. EdR

    EdR New Member

    Joined:
    May 12, 2012
    Messages:
    55
    Likes Received:
    4
    Trophy Points:
    0
    That looks like very good work C.

    A glance at the data plot indicates to me that the recent trend is a curve, not linear. It would be interesting to see someone really capable in mathematics (that excludes me) could normalize the data and continue the plot for 50 to 100 years.

    The error inherent in that would be the assumption that the trend will continue at the present pace, an assumption that only has theoretical validity.
     
  12. caerbannog

    caerbannog Banned

    Joined:
    Jun 17, 2011
    Messages:
    190
    Likes Received:
    7
    Trophy Points:
    0

    Just following up to clarify a few things.

    My little exercise isn't intended to prove that CO2 causes warming, or to predict what future global-average temperatures will be.

    It is intended to demonstrate that the favorite denier claims about the global temperature record (i.e. warming is due to temperature "adjustments", warming is due to UHI, warming is due to the "dropped stations effect", etc) are completely wrong, and that it is quite easy to demonstrate that they are wrong.

    The mid-20th-century cooling-trend in the temperature record has been attributed to "dirty coal" aerosol pollution. For a few decades, increased aerosol emissions from the expanded use of "dirty coal" masked the warming effects of increasing levels of atmospheric CO2. As the major industrial nations cleaned up their coal plants (reducing aerosol emissions), the masking effect went away, allowing CO2-forced warming to resume (remember that aerosols wash out of the atmosphere very quickly while CO2 accumulates).

    The globe will almost certainly continue to warm over the next few decades; we know this because of basic physics linking CO2 to warming that has been worked out over the past century. And what really "sealed the deal" linking CO2 to global warming wasn't climate research -- it was military research originally performed to develop heat-seeking missile technology. Don't dismiss that as "only theoretical validity" -- we are talking about a solid foundation of physics developed over more than a century.

    Watch this short (less than 6 minutes) video for more:
    [video=youtube;uHhLcoPT9KM]http://www.youtube.com/watch?v=uHhLcoPT9KM[/video]​
     
  13. Elmer Fudd

    Elmer Fudd New Member

    Joined:
    Jun 29, 2010
    Messages:
    823
    Likes Received:
    11
    Trophy Points:
    0
    My little exercise isn't intended to prove that CO2 causes warming, or to predict what future global-average temperatures will be.

    It is intended to demonstrate that the favorite denier claims about the global temperature record (i.e. warming is due to temperature "adjustments", warming is due to UHI, warming is due to the "dropped stations effect", etc) are completely wrong, and that it is quite easy to demonstrate that they are wrong.

    I have never made that claim. My claim is:

    1) the tiny temperature increase of the last century is insignificant against natural variations and in now way prove the A in AWG
    2) all you have proven is that the climate varies....whoopy - do....:chew:
     
  14. Bowerbird

    Bowerbird Well-Known Member

    Joined:
    May 13, 2009
    Messages:
    92,653
    Likes Received:
    74,094
    Trophy Points:
    113
    Gender:
    Female
    No, the claims you have made and what we are calling you on are

    http://www.politicalforum.com/envir...6748-agw-campaign-issue-5.html#post1061223937

    http://www.politicalforum.com/envir...6748-agw-campaign-issue-5.html#post1061223937


    Now do you want to defend that statement on this thread? I am sure CB would be delighted to assist you
     
  15. Elmer Fudd

    Elmer Fudd New Member

    Joined:
    Jun 29, 2010
    Messages:
    823
    Likes Received:
    11
    Trophy Points:
    0
    BB I honestly cannot follow sometimes what you are asking or driving at. I think this thread has gone beyond you. I shall wait for Caer..the thread author, ...no offense....



    I made the statement that any highschooler could make a mathematical model that would mirror the past temp record.

    You challenged me to do it

    I documented the simple steps you take in excel to do just that...which most high schoolers would understand

    now you want me to "defend" instructions on how to use excel????

    your wacky......welcome to "Ignore"
     
  16. Bowerbird

    Bowerbird Well-Known Member

    Joined:
    May 13, 2009
    Messages:
    92,653
    Likes Received:
    74,094
    Trophy Points:
    113
    Gender:
    Female
    Actually Poor Debater has already wiped the floor with your assertion on the other thread
     
  17. Elmer Fudd

    Elmer Fudd New Member

    Joined:
    Jun 29, 2010
    Messages:
    823
    Likes Received:
    11
    Trophy Points:
    0
    see additional comments above
     
  18. Bowerbird

    Bowerbird Well-Known Member

    Joined:
    May 13, 2009
    Messages:
    92,653
    Likes Received:
    74,094
    Trophy Points:
    113
    Gender:
    Female

    Basically I wanted you to actually do the exercise for yourself because I suspect you got this from a different website and have merely taken their word for it. As for me doing it - why should I? I am not the one disputing the way the data is reviewed

    Poor Debater DID follow your instructions though and here is the result


    http://www.politicalforum.com/envir...6748-agw-campaign-issue-6.html#post1061233642
     
  19. waltky

    waltky Well-Known Member

    Joined:
    Jan 26, 2009
    Messages:
    30,071
    Likes Received:
    1,204
    Trophy Points:
    113
    Gender:
    Male
    Is so hot out there Uncle Ferd tellin' Granny to wait till almost dark when its cooler to mow the lawn...

    US Record Heat Wave to Continue This Week
    July 02, 2012 - Weather forecasters say a record heat wave baking the eastern United States will continue at least through this week.
     

Share This Page