In the wake of ClimateGate: findings of the misconduct inquiry against Michael Mann.

Remember "ClimateGate", that well-publicized storm of controversy that erupted when numerous email messages from the Climate Research Unit (CRU) webserver at the University of East Anglia were stolen by hackers and widely distributed? One of the events set in motion by ClimateGate was a formal inquiry concerning allegations of research conduct against Dr. Michael E. Mann, a professor in the Department of Meteorology at The Pennsylvania State University.

The report (PDF) from that inquiry has been released, so we're going to have a look at it here.

This report contains a lot of discussion of how the committee pursuing the inquiry was constituted, and of which university policies govern how the committee is constituted, and of how membership of the committee was updated when members left the university for other positions, etc. I'm going to gloss over those details, but they're all there in the ten page report if you're interested in that kind of thing.

My focus here will be on what set the inquiry in motion to begin with, on the specific allegations they considered against Dr. Mann, on how the committee gathered information relevant to the allegations, and on the findings and decisions at which they arrived. Let me state up front that committee decided that one allegation merited further consideration in an "investigation" (which is the stage of the process that follows upon an "inquiry"), and that to my eye, that investigation may end up having broader implications for the practice of science in academia.

But let's start at the beginning. From the inquiry report:

Beginning on and about November 22, 2009, The Pennsylvania State University began to receive numerous communications (emails, phone calls and letters) accusing Dr. Michael E. Mann of having engaged in acts that included manipulating data, destroying records and colluding to hamper the progress of scientific discourse around the issue of anthropogenic global warming from approximately 1998. These accusations were based on perceptions of the content of the widely reported theft of emails from a server at the Climatic Research Unit of the University of East Anglia in Great Britain.

It sounds an awful lot like these "numerous communications" came not from other climate scientists but from members of the general public. (The report notes that the sources of the communications included federal and state politicians, University alumni, and people with no connection to Penn State.)

There is nothing prima facie wrong with members of the general public communicating their concerns. There may be questions about how well grounded these concerns are in facts, but that's the sort of thing an inquiry committee can investigate. And, to the extent that a university like Penn State places importance on the public's trust, responding to these complaints was clearly something the university saw as being in its best interests.

The complaints raised in these communications, if true, could potentially amount to research misconduct, so the wheels were put in motion to mount an inquiry:

Under The Pennsylvania State University's policy, Research Administration Policy No. 10, (hereafter referred to as RA-10), Research Misconduct is defined as:

1. fabrication, falsification, plagiarism or other practices that seriously deviate from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities;
2. callous disregard for requirements that ensure the protection of researchers, human participants, or the public; or for ensuring the welfare of laboratory animals;
3. failure to disclose significant financial and business interest as defined by Penn State Policy RA20, Individual Conflict of Interest;
4. failure to comply with other applicable legal requirements governing research or other scholarly activities.

RA-10 further provides that "research misconduct does not include disputes regarding honest error or honest differences in interpretations or judgments of data, and is not intended to resolve bona fide scientific disagreement or debate."

However, from the sounds of things, most of the complaints the university received about Dr. Mann in the wake of ClimateGate were not couched in the official language of the policy on research misconduct. (This is no surprise. Not too many people from outside academia wallow in official policy language.) Thus, to address the substance of the complaints in the context of academic research and the relevant university policies governing it, the members of the inquiry committee had to do some work to extract the allegations the numerous communications, taken together, were making against Dr. Mann:

At the time of initiation of the inquiry, and in the ensuing days during the inquiry, no formal allegations accusing Dr. Mann of research misconduct were submitted to any University official. As a result, the emails and other communications were reviewed by Dr. Pell and from these she synthesized the following four formal allegations. To be clear, these were not allegations that Dr. Pell put forth, or leveled against Dr. Mann, but rather were her best effort to reduce to allegation form the many different accusations that were received from parties outside of the University. The four synthesized allegations were as follows:

1. Did you engage in, or participate in, directly or indirectly, any actions with the intent to suppress or falsify data?
2. Did you engage in, or participate in, directly or indirectly, any actions with the intent to delete, conceal or otherwise destroy emails, information and/or data, related to AR4, as suggested by Phil Jones?
3. Did you engage in, or participate in, directly or indirectly, any misuse of privileged or confidential information available to you in your capacity as an academic scholar?
4. Did you engage in, or participate in, directly or indirectly, any actions that seriously deviated from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities?

If I were on the inquiry committee, I'd be inclined to point out that these are actually questions rather than allegations -- allegations, I'd think, would be the statements that assume an affirmative answer corresponding to each of these questions. As I was not on the inquiry committee, its members were spared my nit-pick.

You'll note that each of these allegations falls squarely under point (1) of RA-10 (quoted above). Also, you'll note that allegation #4, the question about deviating "from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities," is a broader question than the other three.

What sort of information did the inquiry committee consider as they examined these allegations? They started with a set of more than 1000 emails "purloined from a server at the University of East Anglia". (Side note: I love the inquiry report's consistent use of the term "purloined" to describe these emails.) From these, they identified all the messages of which Dr. Mann was a sender or a recipient (or even a participant at some point in the "chain" of the email discussion), plus all the messages that mentioned Dr Mann or his research or publications (even the ones that he neither sent nor received).

From among these 377 emails, the inquiry committee focused on 47 emails that were deemed relevant. On December 17, 2009, the inquiry committee (Pell, Scaroni, Yekel), Dr. Brune [Head of the Department of Meteorology, whose role was to provide consultation to the rest of the members of the inquiry, but not to make decisions at the inquiry phase] and Dr. Foley [Vice President for Research and Dean of the Graduate School] met to review the emails, discuss the RA-10 inquiry process and go over what their respective activities would be. It was agreed that these individuals would meet again in early January and that they would use the time until that meeting to review the relevant information, including the above mentioned e-mails, journal articles, OP-ED columns, newspaper and magazine articles, the National Academy of Sciences report entitled "Surface Temperature Reconstructions for the Last 2,000 Years," ISBN: 0-309-66144-7 and various blogs on the internet.

Please take a moment here to absorb the description of the sources of relevant information (in addition to the ClimateGate emails) the inquiry committee decided to dig into over the December break: journal articles, magazine and newspaper articles, op-ed pieces, and various blogs on the internet.

I think this has to be empirical data in support of the claim that even people who do not blog are now paying attention to blogs. I suppose this also means that bloggers may want to write with the understanding that what they post may be used in a university inquiry at some future date.

Anyway, the members of the inquiry committee drew on these sources to compile a document that included specific questions in addition to the four formal allegations. They used this document to interview Dr. Mann, asking follow up questions in response to his answers to the questions in the document:

On January 12, 2010, the inquiry committee (Foley, Yekel, Scaroni) and Dr. Brune met with Dr. Mann to interview him. Dr. Mann was asked to address the four allegations leveled against him and to provide answers to the fifteen additional questions that the committee had compiled. In an interview lasting nearly two hours, Dr. Mann addressed each of the questions and follow up questions. A recording was made of the meeting, and this recording was transcribed. The committee members asked occasional follow-up questions. Throughout the interview, Dr. Mann answered each question carefully:

• He explained the content and meaning of the emails about which we inquired;
• He explained that he had never falsified any data, nor had he had ever manipulated data to serve a given predetermined outcome;
• He explained that he never used inappropriate influence in reviewing papers by other scientists who disagreed with the conclusions of his science;
• He explained that he never deleted emails at the behest of any other scientist, specifically including Dr. Phil Jones, and that he never withheld data with the intention of obstructing science; and
• He explained that he never engaged in activities or behaviors that were inconsistent with accepted academic practices.

I imagine at this point that someone might raise the objection that it doesn't matter what Dr. Mann says, since clearly a scientist who has falsified or suppressed data, or destroyed data or communications, or misused privileged information is going to lie about having done so. Remember, however, that the inquiry committee was not only relying on Dr. Mann's testimony. They were also drawing on the purloined CRU emails. This means that if Dr. Mann's claims seemed to be in conflict with the evidence of the email messages, it would undermine his testimony.

Indeed, on the second formal allegation, the inquiry committee sought more than Dr. Mann's testimony:

On January 15, 2010, and on behalf of the inquiry committee, Dr. Foley conveyed via email an additional request of Dr. Mann, who was asked to produce all emails related to the fourth IPCC report ("AR4"), the same emails that Dr. Phil Jones had suggested that he delete.

On January 18, 2010, Dr. Mann provided a zip-archive of these emails and an explanation of their content. In addition, Dr. Mann provided a ten page supplemental written response to the matters discussed during his interview.

The fact that Dr. Mann produced the emails that it was alleged he had improperly deleted looks like convincing proof that he did not delete them.

With the testimony from Dr. Mann (both from his interview with the inquiry committee and from the supplementary written response he provide) and the information from other sources (including the purloined emails) in hand, the inquiry committee sat down to deliberate. After their deliberation, they issued a finding and a decision for each of the four formal allegations against Dr. Mann:

Allegation 1: Did you engage in, or participate in, directly or indirectly, any actions with the intent to suppress or falsify data?

Finding 1. After careful consideration of all the evidence and relevant materials, the inquiry committee finding is that there exists no credible evidence that Dr. Mann had or has ever engaged in, or participated in, directly or indirectly, any actions with an intent to suppress or to falsify data. While a perception has been created in the weeks after the CRU emails were made public that Dr. Mann has engaged in the suppression or falsification of data, there is no credible evidence that he ever did so, and certainly not while at Penn State. In fact to the contrary, in instances that have been focused upon by some as indicating falsification of data, for example in the use of a "trick" to manipulate the data, this is explained as a discussion among Dr. Jones and others including Dr. Mann about how best to put together a graph for a World Meteorological Organization (WMO) report. They were not falsifying data; they were trying to construct an understandable graph for those who were not experts in the field. The so-called "trick" was nothing more than a statistical method used to bring two or more different kinds of data sets together in a legitimate fashion by a technique that has been reviewed by a broad array of peers in the field.

Decision 1. As there is no substance to this allegation, there is no basis for further examination of this allegation in the context of an investigation in the second phase of RA-10.

None of the evidence the inquiry committee considered was anything like a smoking gun to support the charge or suppression or falsification of data.

Allegation 2: Did you engage in, or participate in, directly or indirectly, any actions with the intent to delete, conceal or otherwise destroy emails, information and/or data, related to AR4, as suggested by Phil Jones?

Finding 2. After careful consideration of all the evidence and relevant materials, the inquiry committee finding is that there exists no credible evidence that Dr. Mann had ever engaged in, or participated in, directly or indirectly, any actions with intent to delete, conceal or otherwise destroy emails, information and/or data related to AR4, as suggested by Dr. Phil Jones. Dr. Mann has stated that he did not delete emails in response to Dr. Jones' request. Further, Dr. Mann produced upon request a full archive of his emails in and around the time of the preparation of AR4. The archive contained e-mails related to AR4.

Decision 2. As there is no substance to this allegation, there is no basis for further examination of this allegation in the context of an investigation in the second phase of RA-10.

Dr. Mann produced the allegedly deleted emails, strongly suggesting (by their existence) that they had not been deleted, either properly or improperly.

Allegation 3: Did you engage in, or participate in, directly or indirectly, any misuse of privileged or confidential information available to you in your capacity as an academic scholar?

Finding 3. After careful consideration of all the evidence and relevant materials, the inquiry committee finding is that there exists no credible evidence that Dr. Mann had ever engaged in, or participated in, directly or indirectly, any misuse of privileged or confidential information available to him in his capacity as an academic scholar. In media reports and blogs about Dr. Mann and other paleoclimatologists, those who are named in the CRU email files are purported to have been engaged in conspiratorial discussions indicative of a misuse of privileged or confidential information. Although it is not clear where the exact accusation lies in this with respect to Dr. Mann, it is inferred that the emails prove the case. Those who have formed this view feel that, in their capacity as reviewers, Dr. Mann and his colleagues had early access to manuscripts from other authors with whom they disagreed, and that they could somehow act on those to reject them for publication. Actually, when one does due diligence on this matter, and asks about what papers were involved, one finds that enormous confusion has been caused by interpretations of the emails and their content. In some cases, the discussion and related debate centered on papers that were about to emerge which members of the purported conspiracy had written, but which were simply under embargo. In other cases, the discussion and related debate centered on papers that have emerged in otherwise notable scientific journals, which they deemed to have been published with a lower standard of scholarly and scientific scrutiny. The committee found no research misconduct in this. Science often involves different groups who have very different points of view, arguing for the intellectual dominance of their viewpoint, so that that viewpoint becomes the canonical one. We point to Kuhn as an authority on how science is done, before it is accepted as "settled."

Decision 3. As there is no substance to this allegation, there is no basis for further examination of this allegation in the context of an investigation in the second phase of RA-10.

The findings on the third allegation are interesting, as they point to the fact that scientists in a field may have access to information well before that information is public (say, because it is in a paper which is under embargo but has been distributed to them). Discussing these papers does not mean that the scientists discussing them have any control at all over whether the papers are accepted for publication by reputable journals. (Indeed, "under embargo" suggests that the papers have already been accepted for publication but have not yet been published.)

Others of the papers involved in the complaints the university received were apparently already published. This means that even if the purloined CRU emails included discussions about how these papers should not have been accepted for publication (because the people discussing them in these emails didn't think the quality of the data they reported, or the scientific argumentation they presented, was high enough), the discussants could not have suppressed the publication of these papers without being in possession of a time machine.

In other words, the content of the emails revealed disagreements (about results and standards for evaluating reports of scientific findings) within the scientific community, but they did not reveal that Dr. Mann acted to suppress results or reports of scientific findings with which he disagreed.

For those keeping score, that's three allegations which the inquiry committee determined to be without substance.

Finally, we get to the fourth allegation:

Allegation 4. Did you engage in, or participate in, directly or indirectly, any actions that seriously deviated from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities?

Finding 4. After careful consideration of all the evidence and relevant materials, the inquiry committee could not make a definitive finding whether there exists any evidence to substantiate that Dr. Mann did engage in, or participate in, directly or indirectly, any actions that deviated from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities. It is the case that there has been a public outcry from some quarters that Dr. Mann and his colleagues did deviate from what some observers claim to be standard academic practice. All disciplines and scientific fields work within broad bounds of "accepted scientific" practice that apply to all researchers. However, within different disciplines of science there are additional elements of accepted practice that may be specific to those disciplines and therefore are different from those of other disciplines and fields. ...

Policy RA-10 speaks not just of research misconduct but also of research conduct and is explicit regarding the responsibility that we have as scientists to maintain the public trust. The preamble is as follows:

"Public trust in the integrity and ethical behavior of scholars is essential if research and other scholarly activities are to play their proper role in the University and in society. The maintenance of high ethical standards is a central and critical responsibility of faculty and administrators of academic institutions. Policy AD-47 sets forth statements of general standards of professional ethics within the academic community."

Furthermore, the preamble speaks to the high ethical expectations that Penn State has for its faculty and administrators. These expectations are embodied in another document, Policy AD-47 General Standards of Professional Ethics. The purpose of AD-47 is stated as follows:

"To set forth statements of general standards of professional ethics to serve as a reminder of the variety of obligations assumed by all members of the academic community."

Here, the inquiry committee is pointing out that researchers at the university has a duty not to commit fabrication, falsification, or plagiarism, but also a positive duty to behave in such a way that they maintain the public's trust. The inquiry committee goes on to highlight specific sections of policy AD-47 that speak to cultivating intellectual honesty, being scrupulous in presentation of one's data (and careful not to read those data as being more robust than they really are), showing due respect for their colleagues in the community of scholars even when they disagree with their findings or judgments, and being clear in their communications with the public about when they are speaking in their capacity as researchers and when they are speaking as private citizens.

The inquiry report continues:

It is clear to those who have followed the media and blogs over the last two months that there are two distinct and deeply polarized points of view that have emerged on this matter. One side views the emails as evidence of a clear cut violation of the public trust and seeks severe penalties for Dr. Mann and his colleagues. The other side sees these as nothing more than the private discussions of scientists engaged in a hotly debated topic of enormous social impact.

In other words, we're not just looking at scientific conduct here. Rather, we're looking at scientific conduct in an area about which the public cares a lot.

What this means is that the public here is paying rather more attention to how climate scientists are interacting with each other, and to the question of whether these interactions are compatible with the objective, knowledge-building project science is supposed to be.

This public interest is not a completely new development:

The allegation inquires about whether Dr. Mann seriously deviated from accepted practices within the academic community for proposing, conducting, or reporting research or other scholarly activities. In 2006, similar questions were asked about Dr. Mann and these questions motivated the National Academy of Sciences to undertake an in depth investigation of his research. The committee that wrote the report on surface temperature reconstructions found that Dr. Mann's science did fall well within the bounds of accepted practice. What has changed since that time is that private emails have come to our attention and that of the public at large, and these give us a glimpse into the behind the scenes workings of Dr. Mann and many of his colleagues in the conduct of their science.

In other words, the purloined emails introduce new data relevant to the question of whether Dr. Mann's research activities and interactions with other scientists -- both those with whose conclusions he agrees and those with whose conclusions he does not agree -- are consistent with or deviate from accepted scientific practices.

Evaluating the data gleaned from the emails, in turns, raises the question of what the community of scholars and the community of research scientists agree counts as accepted scientific practices.

Decision 4. Given that information emerged in the form of the emails purloined from CRU in November 2009, which have raised questions in the public's mind about Dr. Mann's conduct of his research activity, given that this may be undermining confidence in his findings as a scientist, and given that it may be undermining public trust in science in general and climate science specifically, the inquiry committee believes an investigatory committee of faculty peers from diverse fields should be constituted under RA-10 to further consider this allegation.

In sum, the overriding sentiment of this committee, which is composed of University administrators, is that allegation #4 revolves around the question of accepted faculty conduct surrounding scientific discourse and thus merits a review by a committee of faculty scientists. Only with such a review will the academic community and other interested parties likely feel that Penn State has discharged it responsibility on this matter.

What this means is that the investigation of allegation #4 that will follow upon this inquiry will necessarily take up the broad issue of what counts as accepted scientific practices. This discussion, and the findings of the investigation committee that may flow from it, may have far reaching consequences for how the public understands what good scientific work looks like, and for how scientists themselves understand what good scientific work looks like.

Regardless of the specific findings of that investigation committee with respect to the fourth allegation against Dr. Mann, this could be big.

* * * * *

For the record, here's Dr. Mann's statement about the inquiry report:

"I am very pleased that, after a thorough review, the independent Penn State committee found no evidence to support any of the allegations against me.

Three of the four allegations have been dismissed completely. Even though no evidence to substantiate the fourth allegation was found, the University administrators thought it best to convene a separate committee of distinguished scientists to resolve any remaining questions about academic procedures.

This is very much the vindication I expected since I am confident I have done nothing wrong.

I fully support the additional inquiry which may be the best way to remove any lingering doubts. I intend to cooperate fully in this matter - as I have since the beginning of the process."