Last month, Judith Curry had an important essay at Physics Today that deserves more attention than it has received.
Curry argues that unlike the industry-funded climate skeptic movement of the past, contemporary debate is driven by a new generation of blog-based "climate auditors" who merge their own professional expertise with online communication strategies to demand a greater level of transparency in climate science. Here's how Curry describes the movement:
So who are the climate auditors? They are technically educated people, mostly outside of academia. Several individuals have developed substantial expertise in aspects of climate science, although they mainly audit rather than produce original scientific research. They tend to be watchdogs rather than deniers; many of them classify themselves as "lukewarmers". They are independent of oil industry influence. They have found a collective voice in the blogosphere and their posts are often picked up by the mainstream media. They are demanding greater accountability and transparency of climate research and assessment reports.
In this sense, the blogosphere has reduced the information asymmetry of climate change communication.
In the past, scientists could expect strong deference and trust in the peer-review process and in the decisions of expert committee reports such as those from the IPCC. There were of course dismissive challenges from figures such as James Inhofe, but apart from the conservative base and elected officials, these objections to peer-reviewed science were never really taken seriously.
But in recent years, as advocates have argued that the peer-reviewed science is the principle reason to pass policies such as cap and trade legislation, climate auditors have responded by asking for a "second-level" of review, one that they would like to make open, accessible, and participatory to non-scientists via the Web.
Let's switch issues and the ideological lens for a moment and consider the following comparison, which I think sheds light on the nature of trust and the need for transparency when research findings on any issue are argued as the major reason for policy action, especially in the new era of Web-based communication and discussion.
Conservative economist John Lott's 1998 book More Guns, Less Crime elaborated on his past journal articles, concluding through statistical analysis that more permissive hand gun laws reduced crime rates in counties and states that had adopted such laws. Conservatives and gun rights advocates argued that Lott's empirical research was reason to repeal gun laws across the country, since the result would be a decrease in crime rates.
In reaction, contending experts ranging from sociologists to law professors argued that before any policy was adopted based on Lott's research, that they should be able to examine Lott's raw data and the decisions that he made in conducting his analysis. [For more see, this article at Mother Jones by Chris Mooney.]
[UPDATE: In response to Lott's reply and comment at the end of this post providing further details on the case and the timeline involved, I have updated this post. I also encourage readers to read the transcript of Lott's interview with Mooney for the Mother Jones article.]
Even though the body of work in climate science rests on far stronger evidence and research than Lott's conclusions about gun laws, similar factors are shaping the objections and the effectiveness of the new generation of climate auditors. Any time a policy question is simplistically reduced down to a question of "following" or "heeding" the science--and especially when the research findings are hyped or exaggerated by certain parties as a way to compel action--that science will come under intense scrutiny.
And in today's blogosphere, information technology has reduced if not eliminated the historic asymmetry of science communication, enabling highly motivated individuals with relevant areas of expertise or professional training to lobby for additional transparency in how scientists have reached their conclusions.
So what is a way forward across this new leveled playing field of assertions about climate science? Judith Curry provides some suggestions in her essay, and she's right that it takes far more than tailoring communication to audiences or re-framing action in terms of benefits to the public. Along with these strategies, new relationships and forms of interaction and dialogue are needed. Curry, for one, has taken a lead by sharing her views and expertise across multiple Web outlets ranging from blogs written by conservative-leaning climate auditors to science organization Web sites such as Physics Today.
The "science audit" movement is not going away, in fact, it is only likely to grow, expanding beyond climate change. Demands for a second-level of inclusive and participatory review of research in areas ranging from nanotechnology to biomedical research to vaccine safety are likely to be at the very center of future science debates.
With this reality, for science organizations to fight the science audit movement is a mistake and will only promote additional distrust and conflict. Instead, new public engagement mechanisms are needed that serve to increase transparency and that include members of the public in the examination of scientific data and findings.
Here are additional thoughts from Curry to end her essay, the full text of which should be read:
Rebuilding trust with the public on the subject of climate research starts with Ralph Cicerone's statement "Two aspects need urgent attention: the general practice of science and the personal behaviors of scientists." Much has been written about the need for greater transparency, reforms to peer review, etc. and I am hopeful that the relevant institutions will respond appropriately. Investigations of misconduct are being conducted at the University of East Anglia and at Penn State. Here I would like to bring up some broader issues that will require substantial reflection by the institutions and also by individual scientists.
Climate research and its institutions have not yet adapted to its high policy relevance. How scientists can most effectively and appropriately engage with the policy process is a topic that has not been adequately discussed, and climate researchers are poorly informed in this regard. The result has been reflexive support for policies proposed by the United Nations Framework Convention on Climate Change (UNFCC) such as carbon cap and trade by many climate researchers that are involved in the public debate, which they believe follows logically from the findings of the IPCC.
The policy advocacy by this group of climate scientists has played a role in the political polarization of this issue. The interface between science and policy is a muddy issue, but it is very important that scientists have guidance in navigating the potential pitfalls. Improving this situation could help defuse the hostile environment that scientists involved in the public debate have to deal with, and would also help restore the public trust of climate scientists.
- Log in to post comments
"unlike the industry-funded climate skeptic movement of the past"
as compared to the massive amount of government funds supplied to the proponents of AGW?
somehow you make it seem that the industry was showering gobs of money upon climate skeptics which is not the case. Government money ie our taxes went to fund research that supported only one point of view. That is not how I expect my tax monies to be spent. If the government is going to distribute money to researchers then that money needs to be distributed equitably amongst skeptics and proponents.
Ah, yes. The mysterious "computer crash" that caused the loss of data from a survey that was never conducted, using students whose names are lost in the dustbin of history.
I wonder if Mary Rosh will be making an appearance.
For âPosted by: maxwell | April 1, 2010 4:37 PMâ
What you want would have been possible if Ronald Reagan and supporters had not set about putting their greed above all else with the votes of spoiled Baby Boomers. My GBRWE predictions have been the tombstones and other markers of ~1 to 2.5 million dead and dying from predicted, violent global warming catastrophes. (This excludes the far greater number who are horribly damaged from cradle to grave exposures to global warming chemicals, including myself.) The regular people that you now want to involve are currently voting for science with their feet. However, their deaths and destruction has been increasing in those areas dominated by extremist media outlets, like LABI Limbaughs, who fail to warn and encourage listeners to ignore local officials using public infrastructure.
For me, the important lesson from the story is this one:
Make backups, not love.
Interesting post. But I wonder if "new public engagement mechanisms" that increase transparency would make it easier for some groups to spin and distort the science. While the auditors may not be funded by the oil-industry, I suspect many of them ideologically oppose regulation.
Perhaps in addition to more engagement there also needs to be more attention to the idea that science doesn't compel any specific policy outcome. Scientists routinely say this, but I'm not sure how much we really believe it. Maybe if everyone truly internalized this idea people would feel more comfortable debating the science without the fear that it's either strengthening or undermining climate policy. Thoughts?
Praj
This piece makes inaccurate claims about my work. The data in my original paper with David Mustard was released even before the paper was published. Our paper was published in the January 1997 of the Journal of Legal Studies. We provided our data to critics such as Dan Black and Dan Nagin as well as Jens Ludwig in August 1996. In fact, the Brady Campaign put on a panel at the National Press Club on December 9, 1996 where I debated them on their analysis of our data.
The first edition of More Guns, Less Crime was published in 1998. All the data for all the regressions and all the tables and figures in the book was again released to others even prior to the book being published. Most of that research involved the same data from 1977 to 1992 that was in my original paper with Mustard, but the additional data in the book that extended to 1994 was also released.
Ironically, I had a computer crash in July 1997 where I lost the data that I had already provided to the critics to whom we had already provided the data. David Mustard and myself asked these individuals to get back a copy of our data, but they declined to do so. I know that David asked them several times. Because we realized that we had an obligation to make our data available to others, David and I spent more than several months putting the data together again. (David spent significantly more time than I did putting that data back together again.)
Since then the data used in More Guns, Less Crime has been provided to over 200 hundred people at universities around the world. Dozens of papers have been published using the data. The paper by Black and Nagin was published in the Journal of Legal Studies in January 1998 (which was four months BEFORE More Guns, Less Crime was published). Given the lead-time in refereeing and publications, it should be fairly obvious that they had submitted their paper a year earlier in late 1996. How could they have submitted their paper for publication in the Journal of Legal Studies before my paper with Mustard was published if we hadn't shared the data?
So your story is backwards. Our data was given out immediately. When the data was lost in a hard disk crash, our critics refused to give us a copy back of our own data. David and I redid the data a second time. It was not something that we had to do (especially given the many months of work involved), but it was the right thing to do. It would have been better if our critics had simply given us back a copy of our own data.
It is easy enough for you to check out when the Black and Nagin paper was published. Another critical paper was published by Jens Ludwig in May 1998 in the International Review of Law and Economics. An entire issue of the Journal of Law and Economics in 2001 published papers on gun control, most of those papers using the data in my book.
Hi John,
I have updated the post in response to your comment. Thank you for the details on the case.
Best,
Matt
Ahh ... what a relaxing respite from scientific rigor and accountability.
I have audited many of the climate auditors. It is a one-way street. They donât care for my âsuggestionsâ that include pointing out current CMEs and sunspots in response to their rehashing of global cooling through reduced solar activity. American Thinker hosed their entire archival section deleting some of my correct earthquake predictions based on their global warming. Do not surrender and give in to these âclimate auditorsâ ideology in place of the scientific method. Hold them to minimum standards of predicting anything within their sphere of claimed expertise starting with 1 second into the future. Otherwise, we deserve to let the GOP/T-Party control every aspect of our lives making a new Dark Ages. From the Rwanda of education, âI am ... total credibility ... Do not read anything written by any "scientist" who tells you there is man-made ... global warmingâ (âThe Limbaugh Peace Initiative: Amnesty for the Duped Warmersâ; rushlimbaugh.com, 2/15/10).
Robert Rhodes aka Ozonator
Hello, Matt: Nice post--I join you in your enthusiasm for Curry's proposals. But maybe it can be refined by your own "auditors"!
I agree with Praj (here and on his own blog), that we'd be better off not only thinking about scientific process in a more expansive way, but thinking about the policy process more expansively, too.
I want to focus on the science side, however. You write: "information technology has reduced if not eliminated the historic asymmetry of science communication." I think this goes a little to far. I suspect that climate scientists are likely to continue to resist what Curry aptly calls "climate auditors"--and for good reasons. The web has reduced information asymmetry, but not to zero.
Auditors may have access to raw data as good or even better than scientists (e.g., local weather stations), and access to specific scientific publications (if they can cross the paywalls). What most of them are going to lack (almost by definition) is 10+ years of apprenticeship within current climate science: they won't have the comprehensive, "cultural" knowledge of the history and current state of the field that climate scientists possess.
On the one hand, that means that participation by auditors is likely to benefit the science. The auditors won't share the sense of proportion/implicit judgments of importance/... that the "culture" of climate science enforces. So they can provide what Ravetz has called an "extended peer community" that forces scientists to address evidence, reasoning, concerns outside their ordinary worldview.
On the other hand, it also means that the scientists are going to resist strongly if an auditor claims to know climate science or be (as good as) a climate scientist. Auditors are not *doing* climate science.
Maybe we just need more words to describe the different ways people can participate in the science-making process: like "auditors," "extended peer community," or Collin & Evans' different fields/kinds of experts (*experts*--not scientists). The metaphor wouldn't be "level playing field," but maybe "different positions on the football team."
Real science is just to darn hard. We are on the slimy slope to polling graphs on Fox News.
Scientific Predictions 0, Year end bonus from the Skeptic Tank priceless
âFollowing up on the excellent initiative of Dr. Judith Curry (see Judithâs post and my response), I would like to see what I can do to rebuild the justifiably lost trust in climate science. I want to bring some clarity to terms which are used all the time but which donât seem to have an agreed upon meaning. In the process, I want to detail my own beliefs about the climate and how it works. ... Question 4. Is the globe warming? This is a trick question. It is a perfect example of a frequently asked question which is totally meaningless. ... Question 13. Is the current peer-review system inadequate, and if so, how can it be improved? There are a number of problems with the current peer-review system ... engineers should be invited to review papers as well. Many theories would benefit from practical experience. Finally, âcitizen scientistsâ such as myself should not be excluded from the process. ... Finally, I would like to invite Dr. Judith Curry in particular, and any other interested scientists, to publicly answer these same questions here on Watts Up With Thatâ (âTrust and Mistrustâ, Guest Post by Willis Eschenbach; Anthony â20â Watts; wattsupwiththat.com, 3/31/10).
This is an interesting question. I expect this sort of "auditing" to happen in my own field (freshwater ecology) in Australia given the apparently increasing frequency and intensity of droughts we are seeing and expected to see in the future.
From what I have seen in the climate change debate the auditing is getting to the point of reversing the asymmetry of information flow. While the peer review process has its flaws, it's no trivial matter to publish a journal paper. It's a slow, almoust dour, process. Yet what I have seen from the auditing on the blogosphere is that a peer reviewed article can be dismissed with little more than innuendo or erroneous use of statistics. This becomes particularly disappointing and frustrating when such dubious or erroneous claims get mainstream coverage here in Australia at the expense of scientific work receiving coverage.
I'm certainly not against the auditing itself. Critical analysis of current understanding will only advance science. But I would very much like to see the self appointed auditors publish their work in the peer review literature and subject themselves to that level of scrutiny. Sadly I don't anticipate this happening.
So in all I'm not sure what the solution is because as I said above, I do think this will continue. Perhaps I'm a bit naive but I think it could be an excellent reminder to scientists that we cannot go beyond our data. If we restrict ourselves to claims that are wholly supported by our data, then we have little to fear in auditing.
Demands for a second-level of inclusive and participatory review of research in areas ranging from nanotechnology to biomedical research to vaccine safety are likely to be at the very center of future science debates
So what?
These people are still ignorant laymen and their demands are still beneath the dignity of the real experts in the field. I have not one iota of respect for any of the "science audit" bloggers, none of whom having ever earned it or seeming likely to ever do so in the future. The peer-review process is more important than them. It is not to be corrupted or dumbed-down or uselessly aped in order to placate them. What do the anti- or questioning-but-basically-anti vaccine columns at Huffington Post "contribute" to actual vaccine safety and policy discussions? Absolutely nothing constructive or good. They'll keep demanding their extra layers of self-constructed accountability, and I'll keep saying "so what?"
Attention is the vector by which the disease of their ignorance spreads.
TTT said,
'These people are still ignorant laymen and their demands are still beneath the dignity of the real experts in the field.'
I had more then one research mentor or adviser tell me that the expert is the one who has made the most mistakes.
I think that 'auditors' or 'citizen scientists' would be rather important to serve this purpose. In order to find the 'right way' to do something, one must go through several, if not more, wrong ways of doing before one can pinpoint the most important parameters in the system of interest. That is the nature of research. By having interested layman attempting to study these rather important scientific issues might make the going a bit easier because more paths to the 'right' answer are being explored at once.
I find amusing the idea that science should somehow shut its doors to those individuals who are interested in science because they are not 'experts' in what ever field is being discussed. I am not a climate scientist, but this lack of 'expertise' in the field does not mean I do not have an understanding that allows me to see issues in a research program of any kind. Criticism is good in most situations and when I'm wrong about something, I'm wrong about it. I don't think I'm wasting anyone's time being wrong 9 out of 10 times if the one is useful. Again, that's research, right?
If the political issue of climate change is the most pressing issue to some individuals, then I would see absolutely no reason to believe why such individuals wouldn't want every single interested person working on it in one way or another. It seems rather illogical to me.
Talk about some entirely wrong-headed comments: Ms. Goodwin argues that outsiders lack the âcultural knowledgeâ of the field and the 10+ years of training. With all due respects to Ms. Goodwin, that is a load of balderdash. Many of we outsiders are experts who for our own reasons chose to leave the academic cocoon to practice in the private sector. We, too, share the 10+ years of education and have practical experience that comes with doing science for a living. As for the âcultural knowledgeâ that apparently includes an apparently primitive approach to computer programming and statistics amongst the self-taught practitioners.The first thing you learn in the private sector is when you don't know how to do something you don't fake it, you find an expert to do the job for you, while academics are forced to make do with their own resources. If I want to confirm my statistics I will hire a professional statistician over a professor who dabbles in statistics. Similarly, if I want to develop a program to harmonize my data I will hire a programmer who will produce a fully documented piece of software that produces reproducible results. One of the surprising revelations of the release of the emails from CRU was how primitive the software and statistics used in this critical research really was.
As for TTT, well he/she clearly has to get out more. The âauditorsâ he/she maligns are not teenagers in their bedrooms but highly trained professionals who are not part of the peer-review process because the peer-review process ignores our existence because we are not part of âinstitutional scienceâ. Many of us are experts in our field but will never be asked to peer review work because we are not associated with a university. I read articles in my field with institutional authors detailing the operations of systems over a couple hundred hours. We, meanwhile, have tens of thousands of hours of operation on the same systems (since many of these systems were invented by private sector scientists) and yet are never asked to comment because we are in the private sector.
Ultimately the drive for outside auditors is driven by the fact that the issue is multidisciplinary and much of the expertise in our society is from outside the purely academic sphere. While academic scientists were satisfied with simply producing data and not being policy advocates the rest of us in the private sector were satisfied to leave you to your own knitting. Now that you are trying to create policy that will affect our daily lives, those of us outside the world of institutional science with the knowledge and training want to see the data and make up our own minds.
at the Klimaforum09 last year in a public forum given by leading climate scientists I called for Democratic Carbon. Open access to the information for the entire carbon cycle. Food, Ecology, Hydrocarbons (oil etc) and said that having open climate science (and some embarrassed scientists) would be well worth catching the greater carbon criminals in the oil and fossil fuel industries, agribusiness etc...
relying on some cranky bloggers is not enough, we need systems change, and a change in power asymmetry, not just information asymertry
http://gaiapermaculture.com/projects/climatewar/blog/2009/11/30/democra…