Friday, April 23, 2010

IPCC consensus unravels from within

Judy Curry's Q&A about the CRU Oxburgh investigation findings and her repeated assertion of IPCC corruption is well worth a read: an inconvenient provocateur

Combine that with Pielke snrs blog posts about fundamental issues and we see that the IPCC consensus is unraveled not just in the eyes of ranting sceptics but from within, from peer reviewed published climate scientists

Where does that leave us? Back to where science is treated as a contested method or methods of investigation and not as a source of authoritative pronouncement.


Mark Miller said...

Re. your last statement. It's good of you to point that out, because a lot of people misunderstand this, not least because of the way science is typically taught.

I agree with Curry that the process does matter. I thought the contrast with Mann's comments was good. To suggest that the process doesn't matter because the data is correct misunderstands something real important about science. As I'm sure you know, part of science is sharing methodologies and results. This is called peer review, though the term "peer review" in the field of climate science I think has been abused. It's commonly used in the context of getting published. Each publisher does its own peer review of work before they publish it, and if they think it's up to snuff that's what they do. It's my understanding that this is what's commonly called "peer reviewed" when the term is used in the AGW debate. From what I understand though that sort of thing is done by about 5 reviewers per publication. My understanding is the purpose of scientific journals is to bring forward different models so that others, hopefully hundreds or thousands, can check them against observations. That's the real peer review. You cited Howard Feynman earlier, I think in the 1970s, saying that even back then it was common not to try to replicate experiments. Maybe that was just in the field of physics. I have heard of instances where replication still goes on. My point is if that process of sharing information and analysis is broken, then it's likely that the models are not going to be very good.

I found this video of retired climatologist Tim Ball back around the time when the CRU e-mails scandal broke. I think he does a good job of explaining the process problems within climate science. He echoes some of Curry's sentiments. The interview was done a few months before the e-mail disclosure.

Back when I was debating with a couple warmists about the issue, one of them finally convinced me to take a look at what was then the latest IPCC report, produced in 2009. I didn't look at the whole thing. It's quite a lot to wade through. I didn't have enough patience for that, but I tried to cut to the chase, getting to the meat of the matter. I found the whole thing frustrating. There was a significant amount of information on various computer climate models, which I found to be worthless, because there was no discussion that I could see about how the models were derived, what the process was in creating them, what was discovered in the process of making them, etc. This is very important, because if the models were derived from knowledge of the temperature record (and I had reason to suspect that they were from other accounts by scientists who know the field), then they're not even worth looking at. I found a couple scientific papers cited which were based on actual observations, but I found that one citation was not current. The scientist had written a later paper, which I found on the internet, reanalyzing his results which were cited in the IPCC report. In the reassessment he said that he was mistaken about the severity of the warming caused by CO2, that it was less severe than he thought. This reanalysis was not included in the IPCC report, only his earlier work which was more alarmist. Overall, it was a disappointing experience. I didn't feel like I derived anything of value by looking at the report. It was apparent to me that while there was a lot of technical content, the whole effort was focused on advocacy, not a dispassionate look at what scientists had learned about the climate.

Anyway, it's good to see Curry taking a critical look at the field. I remember you citing her earlier, calling for more openness among climate scientists.

Mark Miller said...

I meant to add a clarifying comment to the video I referenced with Tim Ball. He mentions the data being "adjusted", that when the temperature records are published, it's not the raw data. My understanding is that climatologists acknowledge that when taking temperature readings from what are called "surface stations" (temperature sensors installed in various locations around the Earth) that the readings don't necessarily reflect what they're trying to measure. There can be interfering factors like being inside a city where there's asphalt and other radiative materials that can interfere with the "natural" temperature that they're after. So they will take the raw data stream from those sensors and try to estimate what the actual temperature would be if those artificial structures weren't there, and put an adjustment on the raw data, and that's what gets published. The thing is some of the adjustment methodologies I had heard about sounded simplistic, like a guess. In my research I didn't hear anyone say that there had been a study done to see how much the artificial materials were adding or detracting from the natural ambient temperature, and then using such studies to then estimate the adjustment.

In any case, what I've also heard scientists say is that this regime of applying adjustments to the data has opened the door for some to adjust the data in such a way that it agrees with their own preconceived notions of what the data should say. This is the reason why Ball says that sharing the methodology for "how you came up with that number" is so important, because the adjustment needs to be scrutinized.

Bill Kerr said...

hi mark,

wrt the peer review process the Ross McKitrick account of his experiences in trying to correct an important false statement about the temperature data in the IPCC report is pretty interesting. Roger Pielke jnr put it up on his blog:
"Fabrication" or "Lie" in the IPCC AR4 WGI. I read the 16 page pdf that Roger linked to and it seems pretty damning to me of the IPCC process, it's a fairly amazing read, even surreal as McKitrick says.

Also, Pielke snr has a guest post from david schultz about attempts to improve the peer review process in response to an earlier post by Pielke snr documenting problems with the peer review process

Schultz said that Pielke’s suggestion to relax guidelines won’t work because the original authors are not good at improving papers that are flawed. He also has a crack at Nature and Science for their "self-inflated sense of prestige". Clearly, he is challenging established and powerful interests.

I followed the links to Schultz’s electronic journal and downloaded a pdf of a paper written by Schultz about how to write good journal articles. The reviewers comments are not anonymous and are included at the end of the article. It's standard policy for this journal:
"EJSSM encourages active participation and debate of our published manuscripts. Reviewers do not maintain their anonymity. Edited versions of the reviews of the manuscript and authors’ responses to substantive comments are published along with the accepted manuscript. Readers are encouraged to post their comments on published manuscripts online.
- "

This more open peer review process may be the way to go. Schultz’s electronic journal has been around for 4 years. It’s based on open access journal software. We need an improved peer review process. It would be desirable if it used the modern technology available to provide more access and feedback as Schultz suggests. I see this issue of improving science to public communication as an important one and have been doing some reading about the nature of science and it's interaction with politics and policy making, in part inspired by Pielke jnr's blog