Use access key #2 to skip to page content.

Meet Harry the Programmer

Recs

45

November 30, 2009 – Comments (30)

When this hits Main Street, Cap and Trade is dead. 

A lot of the talk on ClimateGate thus far has centered on the emails and the revelations that scientists are, in fact, human.  Some smart CAPS players know that there is a lot more to it.  Currently spreading across the Net is a text file found in the hacked files called HARRY_READ_ME.  It is a collection of the programming notes of a man named Harry, diligently attempting to debug the source code and data sets for the CRU's climate modeling program.  In simple terms, this collection of programs accept scientist inputs of data readings, collects them in databases, runs them through a few subroutines for verification, and spits out the numbers that form the basis of Anthropogenic Global Warming climate models.  It is a very important set of programs - the CRU TS2.1/3.0 datasets, 2006-2009.

What follows are actual programming code and notes.  Interspersed in italics and [brackets] are comments from myself and other bloggers that have done a remarkable job of deconstructing the code. A reference section will be posted at the end of the analysis. 

I apologize to all the programmers here for the crappy formatting of the code that I have pasted. That's on me, not Harry :)  I've tried to reproduce the indentations but they don't "keep" when I go to submit the post.

The (soon to be infamous) HARRY_READ_ME.txt file and some of the files in the /cru-code/ directory is most likely authored by Ian "Harry" Harris (not confirmed.) He's the poor sod who seems to have been landed with the responsibility of reworking the code after the departure of Tim Mitchell and/or Mark New who apparently wrote much of it (not confirmed.)

Beginner Programming Mistakes

[See if you can figure out the beginner programming mistake that is baffling Harry in this routine.]

17. Inserted debug statements into anomdtb.f90, discovered that a sum-of-squared variable is becoming very, very negative! Key output from the debug statements:

OpEn= 16.00, OpTotSq= 4142182.00, OpTot= 7126.00

DataA val = 93, OpTotSq= 8649.00

DataA val = 172, OpTotSq= 38233.00

DataA val = 950, OpTotSq= 940733.00

DataA val = 797, OpTotSq= 1575942.00

DataA val = 293, OpTotSq= 1661791.00

DataA val = 83, OpTotSq= 1668680.00

DataA val = 860, OpTotSq= 2408280.00

DataA val = 222, OpTotSq= 2457564.00

DataA val = 452, OpTotSq= 2661868.00

DataA val = 561, OpTotSq= 2976589.00

DataA val = 49920, OpTotSq=-1799984256.00

DataA val = 547, OpTotSq=-1799684992.00

DataA val = 672, OpTotSq=-1799233408.00

DataA val = 710, OpTotSq=-1798729344.00

DataA val = 211, OpTotSq=-1798684800.00

DataA val = 403, OpTotSq=-1798522368.00

OpEn= 16.00, OpTotSq=-1798522368.00, OpTot=56946.00

forrtl: error (75): floating point exception

IOT trap (core dumped)

..so the data value is unbfeasibly large, but why does the sum-of-squares parameter OpTotSq go negative?!!

Probable answer: the high value is pushing beyond the single-precision default for Fortran reals?

Missing Data

Here, the expected 1990-2003 period is MISSING - so the correlations aren't so hot! Yet the WMO codes and station names /locations are identical (or close). What the hell is supposed to happen here? Oh yeah - there is no 'supposed', I can make it up. So I have :-)

[This is why you never put large scale programming projects on the shoulders of one individual. Have they never heard of SCRUM?]

Questionable Programming Selection

[When I was a developer, in addition to the concepts of version control and frequent archiving, one thing my evil commercially oriented supervisors insisted on were "code reviews". This is the hated point where your manager and/or some other experienced developer goes through your code and critiques it in terms of clarity and quality.

As a general rule code reviews teach you a lot. And in places where you have a choice of potential language one of the big questions in a code review is often "why develop this in X?"

The first thing I note is that a lot of the stuff is written in Fortran (of different vintages), and much of the rest in IDL. When you look at the code you see that a lot of it is not doing complex calculations but rather is doing text processing. Neither fortran nor IDL are the tools I would use for text processing - perl, awk, sed (all traditional unix tools available on the platforms the code runs on as far as I can tell) are all better at this. Indeed awk is used in a few spots making one wonder why it is not used elsewhere. The use of fortran means you get weird (and potentially non-portable) things like this in loadmikeh.f90 (program written to load Mike Hulme RefGrids ):]

call system ('wc -l ' // LoadFile // ' > trashme-loadcts.txt') ! get number of lines
open (3,file='trashme-loadcts.txt',status="old",access="sequential",form="formatted",action="read")
read (3,"(i8)"), NLine
close (3)
call system ('rm trashme-loadcts.txt')

[For those that don't do programming this is counting the number of lines in the file. It is doing so by getting an external (unix) utility to do the counting and making it store that in a temporary file (trashme-loadcts.txt) and then reading this file to learn how many lines there are before deleting the file. This is then used as test for whether the file is finished or not in the subsequent line-reading loop.]

XLine=0
do
read (2,"(i7,49x,2i4)"), FileBox,Beg,End
XExe=mod(FileBox,NExe) ; XWye=nint(real(FileBox-XExe)/NExe)+1
XBox=RefGrid(XExe,XWye)

do XEntry=Beg,End
XYear=XEntry-YearAD(1)+1
read (2,"(4x,12i5)"), (LineData(XMonth),XMonth=1,NMonth)
Data(XYear,1:12,XBox) = LineData(1:12)
end do

XLine=XLine+End-Beg+2
if (XLine.GE.NLine) exit
end do

[There are a bunchaton of related no-nos here to do with bounds checking and trusting of input - including the interesting point that this temporary file is the same for all users meaning that if, by some mischance, two people were running loadmikeh at the same time on different files in the same place there is the possibility that one gets the wrong file length (or crashes because the file has been deleted before it could read it). Now I'm fairly sure that this process is basically single user and that the input files are going to be correct when this is run but, as Harry discovers, sometimes it isn't clear what the right input file is:]

Had a hunt and found an identically-named temperature database file which did include normals lines at the start of every station. How handy - naming wo different files with exactly the same name and relying on their location o differentiate! Aaarrgghh!! Re-ran anomdtb:

Major Database Problems

[If the underlying database structure is complete junk, even simple input routines become nightmares.] 

You have failed a match despite the WMO codes matching.

This must be resolved!! Please choose one:

1. Match them after all.

2. Leave the existing station alone, and discard the update.

3. Give existing station a false code, and make the update the new WMO station.

Enter 1,2 or 3:

You can't imagine what this has cost me - to actually allow the operator to assign false WMO codes!! But what else is there in such situations? Especially when dealing with a 'Master' database of dubious provenance (which, er, they all are and always will be).

False codes will be obtained by multiplying the legitimate code (5 digits) by 100, then adding 1 at a time until a number is found with no matches in the database. THIS IS NOT PERFECT but as there is no central repository for WMO codes - especially made-up ones - we'll have to chance duplicating one that's present in one of the other databases. In any case, anyone comparing WMO codes between databases - something I've studiously avoided doing except for tmin/tmax where I had to - will be treating the false codes with suspicion anyway. Hopefully.

Of course, option 3 cannot be offered for CLIMAT bulletins, there being no metadata with which to form a new station.

This still meant an awful lot of encounters with naughty Master stations, when really I suspect nobody else gives a hoot about. So with a somewhat cynical shrug, I added the nuclear option - to match every WMO possible, and turn the rest into new stations (er, CLIMAT excepted). In other words, what CRU usually do. It will allow bad databases to pass unnoticed, and good databases to become bad, but I really don't think people care enough to fix 'em, and it's the main reason the project is nearly a year late.

And there are STILL WMO code problems!!!

[....and Harry has had enough. I do really feel sorry for the guy. He's simply in over his head.]

OH F*CK THIS!!! It's Sunday evening, I've worked all weekend, and just when I thought it was done and just when I thought it was done I'm hitting yet another problem that's based on the hopeless state of our databases. There is no uniformed state of integrity, it's just a catalog of issues that continues to grow as they're found.

It's Not My Fault My Predecessor Is An Idiot

[Have you ever heard the story about the monkeys and the banana in the cage? It definitely applies here.]

#####

Welcome! This is the MERGEGRIDS program.

I will create decadal and full gridded files

from the output files of (eg) glo2abs.for.

Enter a gridfile with YYYY for year and MM for month: tmxabs/tmx.MM.YYYY.glo.abs

Enter Start Year: 1901

Enter Start Month: 01

Enter End Year: 2006

Enter End Month: 12

Please enter a sample OUTPUT filename, replacing

start year with SSSS and end year with EEEE: cru_ts_3_00.SSSS.EEEE.tmx.dat

Writing cru_ts_3_00.1901.1910.tmx.dat

(etc)

This took longer than hoped.. running out of disk space again. This is why Tim didn't save more of the intermediate products - which would have made my detective work easier. The ridiculous process he adopted - and which we have dutifully followed - creates hundreds of intermediate files at every stage, none of which are automatically zipped/unzipped. Crazy. I've filled a 100gb disk!

So, anyway, back on Earth I wrote wmocmp.for, a program to - you guessed it - compare WMO codes from a given set of databases.

Stupid Shell Tricks

[It is worth noting that the "wc -l" shell trick is also repeated in the idl files as well - e.g. in cru-code/idl/pro/quick_interp_tdm2.pro where its used even less efficiently in a pipe:]

"cat " + ptsfile + " | wc -l"

[This would almost merit an entry at thedailywtf.com but for the fact that is is far from the worst 'feature' of this particular program.]

Tim, You Are An Idiot

[Ok, maybe not all of Harry's predecessors are idiots and it's not everyone else's fault, but Tim definitely made Harry's life miserable.]

AGREED APPROACH for cloud (5 Oct 06).

For 1901 to 1995 - stay with published data. No clear way to replicate process as undocumented.

For 1996 to 2002:
1. convert sun database to pseudo-cloud using the f77 programs;
2. anomalise wrt 96-00 with anomdtb.f;
3. grid using quick_interp_tdm.pro (which will use 6190 norms);
4. calculate (mean9600 - mean6190) for monthly grids, using the
published cru_ts_2.0 cloud data;
5. add to gridded data from step 3.

This should approximate the correction needed.

On we go.. firstly, examined the spc database.. seems to be in % x10.
Looked at published data.. cloud is in % x10, too.
First problem: there is no program to convert sun percentage to cloud percentage. I can do sun percentage to cloud oktas or sun hours to cloud percentage! So what the hell did Tim do?!! As I keep asking.

[I don't know Harry. What the hell did Tim do?!]

Then - comparing the two candidate spc databases:

spc.0312221624.dtb
spc.94-00.0312221624.dtb

I find that they are broadly similar, except the normals lines (which both start with '6190') are very different. I was expecting that maybe the latter contained 94-00 normals, what I wasn't expecting was that het are in % x10 not %! Unbelievable - even here the conventions have not been followed. It's botch after botch after botch. Modified the onversion program to process either kind of normals line.

[The same file also gets a mention in Part 31 (at least I think its the same one - either that or there are two programs that do the same 'trick'):]

; map all points with influence radii - that is with decay distance [IDL variable is decay]
; this is done in the virtual Z device, and essentially finds all points on a 2.5 deg grid that
; fall outside station influence

dummymax=max(dummygrid(*,*,(im-1)))

while dummymax gt -9999 do begin
if keyword_set(test) eq 0 then begin
set_plot,'Z' ; set plot window to "virtual"
erase,255 ; clear current plot buffer, set backgroudn to white
device,set_res=[144,72]
endif else begin
initx
set_plot,'x'
window,0,xsize=144,ysize=72
endelse

lim=glimit(/all)
nel=n_elements(pts1(*,0))-1
map_set,limit=lim,/noborder,/isotro,/advance,xmargin=[0,0],ymargin=[0,0],pos=[0,0,1,1]
a=findgen(33)*!pi*2/32
[etc.]

[What this is doing is finding out whether stations may influence each other (i.e. how close they are). It's doing this not by means of basic trig functions but by creating a virtual graphic and drawing circles on it and seeing if they overlap! There are a couple of problems here. First, it seems that sometimes, as the next few lines report, IDL doesn't like drawing virtual circles and throws an error.]

for i=0.0,nel do begin
x=cos(a)*(xkm/110.0)*(1.0/cos(!pi*pts1(i,0)/180.0))+pts1(i,1)
x=x<179.9 & x=x>(-179.9)
y=sin(a)*(xkm/110.0)+pts1(i,0)
y=y>(-89.9) & y=y<89.9
catch,error_value ; avoids a bug in IDL that throws out an occasional
; plot error in virtual window
if error_value ne 0 then begin
error_value=0
i=i+1
goto,skip_poly
endif

polyfill,x,y,color=160
skip_poly:
endfor

[In that case we just skate over the point and (presumably) therefore assume it has no influence - which is, um, very reassuring for people trying to reproduce the algorithm in a more rational manner.

However that's not the only problem because as our hero also reports some inconsistencies:]

..well that was, erhhh.. 'interesting'. The IDL gridding program calculates whether or not a station contributes to a cell, using.. graphics. Yes, it plots the station sphere of influence then checks for the colour white in the output. So there is no guarantee that the station number files, which are produced *independently* by anomdtb, will reflect what actually happened!!

[Fortunately our hero is able to fix this (although it isn't at all clear to me how the new process is integrated with the old one)]

Well I've just spent 24 hours trying to get Great Circle Distance calculations working in Fortran,
with precisely no success. I've tried the simple method (as used in Tim O's geodist.pro, and the
more complex and accurate method found elsewhere (wiki and other places). Neither give me results
that are anything near reality. FFS.

Worked out an algorithm from scratch. It seems to give better answers than the others, so we'll go
with that.

[And really at this point I think I am going to nominate this program to the thedailywtf.com and move on.]

Does Any Of This Really Matter

[Not according to Harry:]

Gotta love the system! Like this is ever going to be a blind bit of use.

=====================================================================

Ok, the fun is over. There is a great deal more, but we have space and time limitations, so let's stop.

So what to make of all this?  And what you are you doing right now?  Why aren't you checking the source code yourself to verify everything I posted here?  Let's get to it Fools! :)

GISS and NCDC Agree

In their defense the CRU note that their temperature series mostly agrees with the ones from NCDC and GISS as if this excuses the abysmal quality of the processing. This is indeed true but given that there is significant shared source data and apparent cooperation between the teams this is not precisely reassuring. I haven't paid any attention in this analysis to the mathematical calculations being done since that's not what I can readily comprehend but it seems quite likely that each system applies similar data "cleansing" and "homogenizing" because the synthetics (see above) are going to be similar. Hence the confirmation bias problem as results that don't match up with the other two will tend to be scrutinized (and possibly adjusted) while the ones that do are left alone.

Since the adjustment mechanism, indeed the entire process, is opaque the claim that black box 1 produces are result similar to black box 2 is not convincing.

Can the AGW Science Be Salvaged

Absolutely. Two words: OPEN SOURCING

In my opinion the entire project needs to be rewritten, and part of the rewrite should include using an actual database (SQL of some variety) to store the intermediate data. In fact the more I read through the HARRY_READ_ME doc (and glance at the code) the more I realize that open sourcing this, as suggested by Eric Raymond,  would get huge benefits. By open sourcing it, the calculation and work flow would have to become more clear and we would be able to identify questionable parts. Open sourcing would also lead to the imposition of basic project management things like archiving and version control which seem to have been sorely lacking and it would allow a large group of people to think about the data structures and the process flow so that it can be made less subject to error.

Furthermore there are enormous problems in terms of station IDs and identifying whether stations have moved, been renamed and so on. And there are problems with data quality for stations (record missing, mistranscribed etc etc) where a distributed team would almost sertainy do better in terms of verifying the data (and potentially adding more raw data when it is found).

Even if the end result of the open source version were identical to the current one, opening this stuff up to public scrutiny and allowing people to contribute patches would go a long way towards improvng the quality of the output and go even further towards improving the credibility of the output which right now is low amongst anyone who's actually taken a proper look at the process.

What Happens When The Public Finds Out About HARRY_READ_ME

The same thing that happened to Scientology when the Internet Gods decided enough was enough.  In the modern age, momentum turns quickly.  Cap and Trade is officially dead.  Whatever happens at Copenhagen will be moot.  The public outcry to investigate the total irresponsibility, incompetence, and arrogance of the AGW scientific establish will blow the doors on Air Force One as soon as President Obama lands upon his return from Europe.

David in Qatar

References:

1.  L'Ombre de l'Olivier

2.  What's Up With That?

3.  Climate Audit

4.  Devils Kitchen

5.  Neutral Net Writer

6.  Asimov

Final Notes:

If you can't see how badly this destroys the credibility of the IPCC and the AGW community, you are married to your ideology.  I don't have any beef with you, just please don't take the tremendous amount of time that so many great bloggers have spent breaking down this code and throw it back in my face as some twisted act of "right wing" propaganda.

As for me, my background is IT as well.  I've been in the field for 7 years (I'm a rookie compared to most.)  I currently work in a mixed Unix/Windows environment and I have enough experience to know a crappy piece of work when I see one.  

But I'm by no means the final authority nor is the verdict in.  Let's hear from some of the more experienced programmers in the Fool community.  Let's see what we find when the rest of the scientific community divulges all of their codes and data sets.  If it all looks like this, what will be the response?  What will be the defense of the AGW believers?  How will they rationalize it?

Unfortunately, I have to run.  I'm on vacation and I have a flight to catch to Chicago.  I'll be in Chicago on Tuesday night (staying in the Wrigleyville area.)  Any locals want to get in touch and have a beer, I might be able to do that.  You can email me here.

David in Qatar

30 Comments – Post Your Own

#1) On November 30, 2009 at 1:17 PM, DaretothREdux (49.28) wrote:

David,

This should be your most recommended blog ever.

Dare

Report this comment
#2) On November 30, 2009 at 2:30 PM, nzsvz9 (< 20) wrote:

Finally,

perl gets mentioned with global warming. I just KNEW that Larry Wall was to blame somehow.

As a computer programmer for over 25 years, this stuff is gold. Programmers don't lie ... Project Manager do.

Known as code reviewer nzsvz9

Report this comment
#3) On November 30, 2009 at 4:20 PM, DaretothREdux (49.28) wrote:

1. Ask a Question

2. Do Background Research

3. Construct a Hypothesis

4. Test Your Hypothesis by Doing an Experiment

5. Analyze Your Data and Draw a Conclusion

6. Communicate Your Results

Somebody forgot the last step.... Kinda hard when you destroy your results.

Dare

Report this comment
#4) On November 30, 2009 at 5:38 PM, Eudemonic (68.68) wrote:

I vividly recall from my collegiate physics labs of getting results that seemed to completely trash the formula under scruitny. My lab partners and I even made jokes as if it were a cliche and everybody in the lab from time to time uttered similiar comments when skewed results befell them too.."When the facts don't fit the theory..dispose of the facts." I certainly appreciate the scrutiny of the scientific approach and also saw how tempting it was to dispose of observations that skewed the data.

For more insight on scrum

Report this comment
#5) On November 30, 2009 at 8:26 PM, dudemonkey (40.73) wrote:

+1 Rec for the spirit of skepticism, which is very valuable.  My guess is that these emails and leaked documents may wind up exposing some tendencies to take the scientific results and make them more extreme, the way Al Gore will do.  But I think in the end we're going to see that the science is basically right and that humans do have a measurable impact on the climate, it just might not be as big of an impact as some of the politicians lead us to believe.  It's pretty hard to deny symptoms like the polar icecaps retreating farther than ever before, or glaciers that have been solid for centuries starting to melt.  You can see those, so it's not something that a bunch of activists can just make up data to show.  I might be wrong, I'm not a scientist, but a lot of the climatologists seem to agree.  However, most of the financial industry operates as if EMT is true, so who knows if the whole idea of AGW is just a product of how we teach it?

I guess I'm just saying that this is a really complicated issue, and I'm suspicious of anyone who is 100% sure of their position unless they've spent 30 or so years doing the science. 

Report this comment
#6) On November 30, 2009 at 11:33 PM, ChrisGraley (30.25) wrote:

dudemonkey I'd feel strong about a guy that has studied for a month, if he could back it up with a provable explanation. The problem is that most people are getting the explanation that they are too stupid to understand and agreeing with it.

The biggest problem is the data is presented in a way to make sure that it unprovable andmore and more info is coming out to show that it's by design.

Report this comment
#7) On December 01, 2009 at 2:18 AM, tonylogan1 (28.22) wrote:

If this code somewhere in there revealed a master plan to grind up people for food for the aristocracy, I am confident neither the media or common man would make any sense or fuss of it.

Oh well, at least it has enraged a few CAPsters...

Report this comment
#8) On December 01, 2009 at 7:59 AM, dudemonkey (40.73) wrote:

The biggest problem is the data is presented in a way to make sure that it unprovable andmore and more info is coming out to show that it's by design

The biggest problem is the people trying to turn a scientific issue into a political one, but ignoring the actual science. The actual science is out there, but universally ignored in these threads.

Report this comment
#9) On December 01, 2009 at 8:23 AM, dudemonkey (40.73) wrote:

http://online.wsj.com/article/SB10001424052748703939404574567423917025400.html?mod=loomia&loomia_si=t0:a16:g2:r1:c0.428565:b29133396

 

That link goes to an article that shows an example of a good way to question the science.  Questioning the science by looking at computer programmer's burned-out comments and code or by harping on out-of-context emails just makes the accuser look like they have an axe to grind, have already made up their mind, and it really just invalidates any attempt at actual discussion.

Report this comment
#10) On December 01, 2009 at 11:23 AM, GADawg (< 20) wrote:

dudemonkey,

The ManBearPig believers are the ones who have politicized the issue. Because ManBearPig is (probably is?, might be?) real, we need to spend billions, or trillions of dollars in an attempt to do what the scientists guess might avert catastrophe, assuming they're not wrong in the first place.

Trying to kill the messenger who has exposed the ManBearPig believers for the black box-using, hand wavers that they are is increasingly pathetic. Release the data that hasn't been destroyed, open the models to outside scrutiny, purge the participants of the cover-up from your ranks, and see where the chips fall.

Report this comment
#11) On December 01, 2009 at 11:29 AM, AvianFlu (34.92) wrote:

It is despicable when science is corrupted by political ideology. Rec +1

Report this comment
#12) On December 01, 2009 at 2:58 PM, nzsvz9 (< 20) wrote:

All,

The "visual evidence" of ice caps melting and glaciers is such an emotional ploy - since you can use year over year pictures to "prove" global warming. It does not prove man-made global warming!

This retreating has been going on since the last ice age. Do you know that glaciers over a mile thick once covered a large part of North America and Europe (See Wikipedia: Last Glacial Maximum) and have been mostly retreating for 20,000 years?

Yes. 20,000 years.

The truth is that the climate is always changing, and that's normal. Otherwise I'd be under a mile of ice right now. Is it snowing outside?

Known as climate denier nzsvz9

Report this comment
#13) On December 01, 2009 at 5:30 PM, lucas1985 (< 20) wrote:

@nzsvz9,
"The truth is that the climate is always changing, and that's normal."
And the fact that forest fires occur naturally is enough proof to discard claims about the existence of arsonists.
If you don't see the idiocy in your argument I can't be of help.

@dudemonkey,
"Questioning the science by looking at computer programmer's burned-out comments and code or by harping on out-of-context emails just makes the accuser look like they have an axe to grind, have already made up their mind, and it really just invalidates any attempt at actual discussion."
Exactly.

"http://online.wsj.com/article/SB10001424052748703939404574567423917025400.html?mod=loomia&loomia_si=t0:a16:g2:r1:c0.428565:b29133396
That link goes to an article that shows an example of a good way to question the science."

Lindzen's arguments are reasonably good until he gets to the topic of climate sensitivity. In that topic, he's horribly wrong:
- There's overwhelming agreement that the climate sensitivity to a doubling of CO2 is in the range of 2 to 4.5°C with a best estimate of about 3°C, and is very unlikely to be less than 1.5°C.
- This estimate of climate sensitivity does not come exclusively from the output of climate models. Observations and evidence from Earth's climate history also agree with this estimate.
- "Skeptics" have failed to produce significant evidence supporting a climate sensitivity of less than 1.5°C for each doubling of pCO2.


The Financial Times has an excellent article where top climate scientists (including Lindzen) share their opinions.
Tim Lambert, a computer scientist, debunks all the hoopla surrounding the coding practices at CRU.

Report this comment
#14) On December 01, 2009 at 6:43 PM, whereaminow (42.76) wrote:

LOL@ Tim Lambert & lucas1985

We didn't say Harry is fudging the numbers. We said the program is sloppy, incompetent, and that Harry is in over his head in trying to fix it.

You didn't even read our analysis. You just went out and found the first article you could that defended your ideology.

Nice try,

Glad to see Phil Jones and Michael Mann are under investigation.  It gets better and better.

David in Qatar

Report this comment
#15) On December 01, 2009 at 7:51 PM, NOTvuffett (< 20) wrote:

whereaminow,

I think I worked at a place after Tim, lol.  

Even people without programming experience should realize you can't implement a good computer model without good code.  Even that is not enough, the model has to be good.  And even that is basically worthless without good data to put into it.

 

Report this comment
#16) On December 02, 2009 at 11:30 AM, GADawg (< 20) wrote:

NOTvuffett,

I think you're confusing real-world programming with ManBearPig programming. A good model is one that supports ManBearPig. Good data is also that which has been adjusted to support ManBearPig.

As lucas1985 said, if you can't see that, I can't help you.

:)

Report this comment
#17) On December 02, 2009 at 1:36 PM, nzsvz9 (< 20) wrote:

lucas1985,

Please be polite and don't call me an idiot. Personal attacks do not support your position or belief. I much prefer a discussion to name calling.

First, you appear to have a logical flaw in your position on arsonists and fires. To say forest fires occurs naturally proves there are no arsonists is not equivalent to climate change is normal. I made a one-part statement, and did not link cause and effect as you did with your two-part statement. So your point does not make sense to me.

And why has there been warming since the last glacial maximum 20,000 years ago? You did not address this.

And, if the CRU has been fudging numbers (according to their own emails and programming notes - thanks whereaminow) why should we implement any public policy changes if the leaders and their science has been so discredited?

During the early history of the Earth, there is much evidence (See Evolution of Earth's atmosphere from Wikipedia) to show that the composition of the atmosphere changed significantly over time. Once there was almost NO oxygen, now it's about 20%. So what is the "normal" amount of oxygen in our atmosphere? What is the normal amount of CO2?

I will repeat my claim, that the climate is always changing, and that that is normal. Any attempt to call one set of conditions over another normal is to put one's preference ahead of reality.

Lastly, if as you calim, man-made global warming is happening, then please change the climate along Hudson's Bay, as I would like to set up a Sandal's style resort there that I can drive to. Also, please make it sunny and 85 degrees all year long. Also, please make it rain only on Tuesday nights (guest turnover time). And have the sun set to the North instead of the west so we can get prettier sunsets. [BTW: this section is satirical]

Man-Made global warming is a fraud, come hell and high water.

Known as objectivist nzsvz9

Report this comment
#18) On December 02, 2009 at 6:53 PM, lucas1985 (< 20) wrote:

@nzsvz9,
"Please be polite and don't call me an idiot. Personal attacks do not support your position or belief. I much prefer a discussion to name calling."
I apologize if you felt offended by my remarks but the illogical and fact-free arguments of denialists really bring me despair.

"First, you appear to have a logical flaw in your position on arsonists and fires. To say forest fires occurs naturally proves there are no arsonists is not equivalent to climate change is normal. I made a one-part statement, and did not link cause and effect as you did with your two-part statement. So your point does not make sense to me."
Let's see if we can sort out this logical conundrum. Forest fires do exist, are mostly natural in origin and there's nothing abnormal in that. In fact, certain forest fires are even beneficial to the underlying ecosystem. That said, arsonists exist and they also cause forest fires which are generally harmful to wildlife.
So, your point that the climate is always changing and that it's a natural and normal phenomenon, while factually accurate, is misleading. Your statement ignores the following aspects:
- the possible existence of an anthropogenic forcing.
- The magnitude and velocity of the change.
- the capability of adaptation of human and natural ecosystems.
Concluding: your statement is factually accurate but misses context and the bigger picture and it's nothing more than an attempt to score brownie points.

"And why has there been warming since the last glacial maximum 20,000 years ago? You did not address this."
Orbital forcing augmented by the CO2 feedback and subsequent feedbacks (e.g., ice-albedo feedback) explain the transition from a glacial epoch to an interglacial one. Satisfied?

"And, if the CRU has been fudging numbers (according to their own emails and programming notes - thanks whereaminow) why should we implement any public policy changes if the leaders and their science has been so discredited?"
- There's no evidence of data manipulation. What we have is attempts to evade FOIA requests (some of them justified since part of the data is property of national weather services and can't be shared freely) and some bad data archiving and coding practices.
- HadCRU isn't the only reconstruction of global average temperatures. There are others which use freely accessible data and share their code and documentation.
- AGW/ACC theory is underpinned by hundreds of lines of independent evidence. The instrumental record is only one of them.

"During the early history of the Earth, there is much evidence (See Evolution of Earth's atmosphere from Wikipedia) to show that the composition of the atmosphere changed significantly over time. Once there was almost NO oxygen, now it's about 20%. So what is the "normal" amount of oxygen in our atmosphere? What is the normal amount of CO2?"
Current CO2 levels are higher than anytime in the past 800,000 years and probably the last 20 million years. In other words, it's probable than Hominids never saw CO2 levels this high in their entire history. We've been in "icehouse Earth" conditions since the start of the Antarctic glaciation. The last episode of "greenhouse Earth" was the Paleocene–Eocene Thermal Maximum and it wasn't pretty. Agriculture and civilization were developed in a period of exceptionally stable climate, the Holocene.

"Any attempt to call one set of conditions over another normal is to put one's preference ahead of reality."
I prefer the stable climate of the Holocene which allowed the development of modern civilization and shaped the patterns of settlement and economic activity. Can mankind afford to lose key breadbasket regions? How big will be the costs of adaptation to a changing climate? Will you welcome climate migrants? What will be the consequences of climate-induced water scarcity to nuclear-armed countries with billions of hungry mouths?

"Lastly, if as you calim, man-made global warming is happening, then please change the climate along Hudson's Bay, as I would like to set up a Sandal's style resort there that I can drive to. Also, please make it sunny and 85 degrees all year long. Also, please make it rain only on Tuesday nights (guest turnover time). And have the sun set to the North instead of the west so we can get prettier sunsets. [BTW: this section is satirical]"
I'm glad you say this is satirical. Confusing global trends with regional impacts and the chaotic nature of weather is a cause of concern.




Report this comment
#19) On December 02, 2009 at 11:19 PM, tonylogan1 (28.22) wrote:

for those not familar, lucas1985 is a 100% waste of time.

hope I saved someone some typing somewhere.

p.s. My long awaited "ignore" feature is still not active.

Report this comment
#20) On December 02, 2009 at 11:40 PM, lucas1985 (< 20) wrote:

"for those not familar, lucas1985 is a 100% waste of time. "
If you don't like facts, that's your problem not mine.

Report this comment
#21) On December 02, 2009 at 11:58 PM, DaretothREdux (49.28) wrote:

lucas1985,

History is also supposed to be a "collection of facts," but put into the right order they can tell whatever story the author wants. You complain about David cherry-picking his stories to promote his agenda....yet when someone points out that's exactly what you are doing....

Pot have you met the kettle?

Dare

Report this comment
#22) On December 03, 2009 at 12:37 AM, lucas1985 (< 20) wrote:

@DaretothREdux,
"You complain about David cherry-picking his stories to promote his agenda....yet when someone points out that's exactly what you are doing..."
Feel free to point out my examples of cherry-picking facts, distorting them or fabricating them. For example I'll ask you (or anybody else) to tell me which facts I've ignored and what are the consequences of bringing them on the table. Would these "missing" facts undermine ACC theory or tell us to do more research before enacting policy?

Report this comment
#23) On December 03, 2009 at 9:25 AM, whereaminow (42.76) wrote:

"Yep, environmentalism is a big weapon for Progressives (I'm one.)" - lucas1985

Scumbag. 

You've been proven wrong. Go crawl into your hole.

David in Qatar

Report this comment
#24) On December 03, 2009 at 12:21 PM, nzsvz9 (< 20) wrote:

All,

As I review the chart "Holocene Temperature Variation" it would appear that any "trendline" starting from 8,000 Years Before Present to today, would show the gobal average temperature (by the average of methods in that chart from wikipedia) represents a trend of global cooling.

Starting at 8,000 years ago is subjective - I chose it to advance my point - as is starting 100 years ago (i.e. exaggerating the hockey stick - "hide the decline" so to speak) as would any timeframe except roughly 4.5 billion years ago.

To say agriculture and civilisation sprang forth during this time frame thereby makes the climate in the same itme frame optiomal is a leap of association - i.e. a personal belief. The Dinosaurs flourished during the 225-65 MYA period - and it was MUCH warmer then. They were the most successful radiant top-level species ever to inhabit the land. And the Cambrian Explosion (the most prolific radiant of life in the history of the earth) occurred at a time even warmer than that. The argument that this current climate range is best for life, is unsubstantiated - it is a belief, and is probably wrong.

So you've chosen the label of "Progressive". Well, have a nice day. I'm not using any weapon here to advance an agenda - I am however very opposed to someone else doing the same to me - which affects my family.

If you want to be really Progressive, then "conserve" energy by turning off all your energy transforming devices (energy is never consumed, it is only transformed) sell your home and posessions, donate all the proceeds to the government or outlet of choice, and cease to exhale CO2 - to do your part to save the planet. Leave us alone - if you truly believe in your cause then do yourself first what you would force us to do by laws and coercion. Lead the way by example - don't tread on me.

Known as the compassionate nzsvz9

Report this comment
#25) On December 03, 2009 at 12:23 PM, lucas1985 (< 20) wrote:

@whereaminow,
""Yep, environmentalism is a big weapon for Progressives (I'm one.)" - lucas1985"
Feel free to distort my words to fit your preconceived notions.
I said:
"Yep, AGW is a big weapon for environmentalists and progressives (I'm one)"
AGW/ACC is a fact, like the failure of the Soviet Union. The later is a political weapon for conservatives, libertarians, free-market advocates and lunatics like you. The former is a political weapon for people who acknowledge the existence of market failures, the importance of natural capital and sustainability.

"Scumbag."
Illiterate, innumerate, ignorant, fundamentalist, bigot, denialist, fraudster. See? I can insult too.

"You've been proven wrong."
Wishful thinking is no substitute for evidence and logic. Keep trying.

"Go crawl into your hole."
Yep, bury yourself in the hot sand. That's the best you can offer to the world.

Report this comment
#26) On December 03, 2009 at 1:28 PM, lucas1985 (< 20) wrote:

@wnzsvz9,
"As I review the chart "Holocene Temperature Variation" it would appear that any "trendline" starting from 8,000 Years Before Present to today, would show the gobal average temperature (by the average of methods in that chart from wikipedia) represents a trend of global cooling.
Starting at 8,000 years ago is subjective - I chose it to advance my point - as is starting 100 years ago (i.e. exaggerating the hockey stick - "hide the decline" so to speak) as would any timeframe except roughly 4.5 billion years ago."

- I selected the most recent 8,000 years to show the stability of the Holocene and because the Holocene is the relevant climate period to discuss. Civilization appeared here, not earlier.
- If you see the right-end of the chart you'll note the sharp warming trend which shows the end of the last glacial epoch. Temperature variations between glacial and interglacial epochs are about 4-7 ºC. So, if the global average temperature dropped 5 ºC we'd be in a new ice age.
- As you note, after temperatures reached a maximum - called the Holocene climatic optimum - they started a long-term cooling trend (consistent with orbital forcing) interrupted by few warm periods (e.g., the Medieval Warm Period) and now reversed by anthropogenic forcing.

"To say agriculture and civilisation sprang forth during this time frame thereby makes the climate in the same itme frame optiomal is a leap of association - i.e. a personal belief. The Dinosaurs flourished during the 225-65 MYA period - and it was MUCH warmer then. They were the most successful radiant top-level species ever to inhabit the land. And the Cambrian Explosion (the most prolific radiant of life in the history of the earth) occurred at a time even warmer than that. The argument that this current climate range is best for life, is unsubstantiated - it is a belief, and is probably wrong."
- Please tell me how the climate of the dinosaurs is relevant to the current climate which has:
* A different range of species (e.g., angiosperms)
* A different distribution of land masses.
* Higher solar output.
* 7 billion of human beings wanting food, shelter, clothes and a pleasant life.
* A complex, global economy.
- I didn't say that the current climate is the best for life. I said that the current climate is the one in which mankind flourished and the one in which the overwhelming majority of species evolved (I already told you that we're in icehouse Earth conditions since the Antarctic glaciation which happened 35 million of years ago and that current CO2 levels are the highest in at least 15 million years). Sure, climate change is an evolutionary pressure and unfit species should become extinct but how many species will adapt and evolve to abrupt and rapid climate change when they're already dealing with other pressures (pollution, habitat loss, hunting, ecosystem degradation, etc)? How many food staples will flourish in a changed climate?



"If you want to be really Progressive, then "conserve" energy by turning off all your energy transforming devices (energy is never consumed, it is only transformed) sell your home and posessions, donate all the proceeds to the government or outlet of choice, and cease to exhale CO2 - to do your part to save the planet. Leave us alone - if you truly believe in your cause then do yourself first what you would force us to do by laws and coercion. Lead the way by example - don't tread on me."
- I have no car and don't intend to have one.
- I use public transport and/or a bike to move myself around (living in a small city helps)
- I grow my own vegetables.
- I have a solar panel for hot water.
- My house is well insulated.
- I have the most energy efficient appliances available on the market.
- I reuse or recycle as long as it's possible.
- The sun dries my clothes.
- My city has a sewage treatment plant where the treated water is used to irrigate a planted forest.
- My electricity is generated by NG power-plants, hydro power-plants and a nuclear power-plant.
- My CO2 output (a byproduct of metabolism) is neatly captured by natural processes.

Report this comment
#27) On December 04, 2009 at 11:24 AM, nzsvz9 (< 20) wrote:

Nice pictures.

The "Holocene Climatic Optimum" is only so-named because someone arbitrarily named it so which is subjective. It could easily have been named "Holocene Climatic Warmer Period" instead which would be much more objective. The common definition of optimum here is being obscured with the mathematical definition of optimum, but many people would not see the distinction, and would easily be misled.

Do you see the bias at all?

Known as 'n", the simple derivative of nzsvz9.

Report this comment
#28) On December 05, 2009 at 12:12 AM, lucas1985 (< 20) wrote:

@nzsvz9,
"The "Holocene Climatic Optimum" is only so-named because someone arbitrarily named it so which is subjective. It could easily have been named "Holocene Climatic Warmer Period" instead which would be much more objective. The common definition of optimum here is being obscured with the mathematical definition of optimum, but many people would not see the distinction, and would easily be misled.
Do you see the bias at all?"

So, from the whole content of my response and my careful answering of your concerns, you pick this nit?
From the Wikipedia link:
"The Holocene Climate Optimum was a warm period during roughly the interval 9,000 to 5,000 years B.P.. This event has also been known by many other names, including: Hypsithermal, Altithermal, Climatic Optimum, Holocene Optimum, Holocene Thermal Maximum, and Holocene Megathermal."
Do you see bias in using the adjective "optimum" to name the warmest period of an interglacial?
Do you see that using the adjective "warmer" is also subject to bias because the warming wasn't global?
"Northwestern Europe experienced warming, while there was cooling in the south"
"In the far southern hemisphere (e.g. New Zealand and Antarctica), the warmest period during the Holocene appears to have been roughly 8,000 to 10,500 years ago, immediately following the end of the last ice age. By 6,000 years ago, the time normally associated with the Holocene Climatic Optimum in the Northern Hemisphere, these regions had reached temperatures similar to those existing in the modern era, and did not participate in the temperature changes of the North. However, some authors have used the term "Holocene Climatic Optimum" to describe this earlier southern warm period as well."

Is this your best critique?

Report this comment
#29) On December 14, 2009 at 8:56 AM, nzsvz9 (< 20) wrote:

I chose a simple bias in the representations to make a point - that the preferred naming "Holocen Optimum" is not objective. And you criticize me instead of addressing the point.

David's article on the hockey stick is here and it illustrates bigger points from significant data problems with AGW.

Known as insult target nzsvz9

Report this comment
#30) On December 16, 2009 at 2:16 PM, lucas1985 (< 20) wrote:

@nzsvz9,
"I chose a simple bias in the representations to make a point - that the preferred naming "Holocen Optimum" is not objective. And you criticize me instead of addressing the point."
I didn't criticize you. I pointed out that "warmer" isn't a more objective name than "optimum" because:
- The warming wasn't global.
- The warming didn't occur in sync.
- Optimum refers to the state of the biosphere in some parts of the globe when this epoch was first discovered.
"Current desert regions of Central Asia were extensively forested due to higher rainfall, and the warm temperate forest belts in China and Japan were extended northwards.
West African sediments additionally record the "African Humid Period", an interval between 16,000 and 6,000 years ago when Africa was much wetter due to a strengthening of the African monsoon by changes in summer radiation resulting from long-term variations in the Earth's orbit around the sun. During this period, the "Green Sahara" was dotted with numerous lakes containing typical African lake crocodile and hippopotamus fauna."
- Wikipedia.


"David's article on the hockey stick is here and it illustrates bigger points from significant data problems with AGW."

I'll respond to that article soon. In the meantime you should know that:
- The hockey stick isn't central evidence of AGW. You could falsify it entirely and the scientific picture of AGW wouldn't change.
- The "divergence problem" affects a small number of tree-ring chronologies in the Northern hemisphere and it's extensively documented in the scientific literature.
- The "divergence problem" says nothing about the quality or significance of other tree-ring chronologies and their use in paleoclimatic reconstructions.
- There's ongoing research to resolve this issue.

Report this comment

Featured Broker Partners


Advertisement