The Culture of Caution at NIH and Academia's Role

Yesterday's article by Gina Kolata about cancer research mistakes a symptom--caution due to a perceived lack of funding--for the disease, which is the symbiosis between academia and the NIH. Don't get me wrong, a lot of research should involve academics. But the priorities of NIH have become overwhelmed by the priorities of academia. I think Kolata's dead on target with this part of her article (italics mine):

One major impediment, scientists agree, is the grant system itself. It has become a sort of jobs program, a way to keep research laboratories going year after year with the understanding that the focus will be on small projects unlikely to take significant steps toward curing cancer.

"These grants are not silly, but they are only likely to produce incremental progress," said Dr. Robert C. Young, chancellor at Fox Chase Cancer Center in Philadelphia and chairman of the Board of Scientific Advisors, an independent group that makes recommendations to the cancer institute.

The institute's reviewers choose such projects because, with too little money to finance most proposals, they are timid about taking chances on ones that might not succeed. The problem, Dr. Young and others say, is that projects that could make a major difference in cancer prevention and treatment are all too often crowded out because they are too uncertain.

I'm not going to comment further on the specifics of cancer research--it's not what I do. But I've observed the phenomenon elsewhere in NIH--which is why I think it's a far more general phenomenon. Instead, I will focus on genomics and antibiotic resistance, which are two areas in which I am an expert of sorts.

What I'll propose (and I'm putting this up front because it's a long proposal) is that this dysfunctional system is derived from the emphasis NIH and other funding agencies place on R01 grants (or R01-'like' grants if NIH isn't the funding agency). By R01 grants, I mean grants that are budgeted at less than $500,000 per year--and usually less--and awarded to one main PI, although there may be a few other Co-PIs. To fix this, NIH (and other funding agencies) should be funding larger collaborative projects--the dreaded 'Big Science'--and downplaying R01s. (Note: I will refer to NIH and R01s as shorthand, even though the same types of grants are available at other funding agencies--I'm trying to avoid lots of alphabet soup). The R01 grants should function more as demonstration or proof-of-principle grants.

So onto an antibiotic resistance example. A while ago, I wrote about some excellent work by ScienceBlogling Tara Smith, and that was written up in The NY Times by Nicholas Kristof. Basically, Tara established that a clone of MRSA, ST398, which spread throughout Europe's farming system, and is now entering Dutch hospitals, and has established itself as a commensal in the human population, is found at really high levels in two Iowa farm systems (covering ~20 facilities). Now, under our current system, the next step would be that Tara applies for a grant--and Tara's very smart so she would probably get it--to look at more farming systems over a longer period of time (An aside: Having been a weird hybrid of federally funded researcher/policy wonk, I have some experience in what typically gets funded in this field). This is about what an R01-like grant with several years of funding could accomplish. And if Tara does go down this route, I wish her best of luck--she certainly has earned it.

But that's not ideally how the question should be addressed (and I've actually testified in front of federal committees about this--I'm not just ranting). To address the question of the introgression of resistance genes and resistant organisms from the farm to the clinic, you would need to do work on a scale that simply can not be funded by the R01 mechanism. You would need to collect tens of thousands (if not more) of isolates from farms, abattoirs, retail meat, fecal lagoons, farm workers, human commensal isolates (both in farming communities and distant from pig farms), pigs, and clinical isolates. These isolates would need to be systematically stored, screened for resistance, and genotyped (including sequencing, plasmid profiling, virulence factor typing (PCR and hybridization). There should also be a genomic component for a subset of the strains, both gene array and genomic sequencing. Then, of course, you need the informatics to store all of these data. And this is a multi-year project, and probably should be expanded to a couple of other critical organisms.

This is expensive. Really expensive. That means if this fails, it's very risky.

The culture at NIH, both in terms of outside reviewers (most of whom are drawn from academia) and program officials, has been really risk averse**. That's only partly due to a lack of funding (after all, unless a huge percentage of proposals were to be funded, someone's going to get shafted). From a program officer's perspective, spending lots of money in on a single program is very risky--I've seen large projects fail and it's not pretty. At the same time, that officer or institute will come under fire for starving other lines of research. As long as there's no pressure from those outside the research community, why rock the boat? Simply turn the difficult policy decisions over to the scientific reviewers and keep funding small grants.

From the perspective of academia, more resources devoted to larger projects means fewer smaller grants--and grants are how untenured professors get tenure, and tenured professors become full professors. Sure, if academics are involved with large project grants, they can swing some funding (sometimes a lot of funding), but it typically won't be 'their' funding, and that also matters for tenure decisions. Publications will have many, many authors, and, again, they can't be claimed as 'theirs.' Now, in defense of this system, it is very good for training students. It is very good at coming up with novel research ideas. It's a good system for academia, but, if you want to turn basic research into translational research (never mind interventions), it's not designed to do this. At some point, some of these ideas have to be 'embiggened': we need to give them the resources needed.

An example of this is a project some colleagues are doing that attempts to find novel molecules ('natural products') that bacteria produce, and turn them into medically useful compounds (e.g., antibiotics, tumor fighting drugs, etc.). The project starts with sequencing a bunch of bacteria, and, even though genome prices are dropping, it's still expensive--much larger than an R01 grant. Then the genes of interest have to be identified and cloned into the appropriate bacteria (you don't want to use E. coli for this). After this, the compounds have to be isolated (no small feat), slightly tweaked in thousands small ways, and screened in a high-throughput fashion against a variety of targets (e.g., bacteria, tumors, and so on).

As is the case in the theoretical example above, this is expensive. It also will take some time. But most importantly, it requires institutional commitment to see all of the steps through. You can't simply publish a paper identifying potentially interesting genes and leave it at that--you then have to do the next steps. And, even after all of this, it could still be a phenomenal bust. But if anyone has a better idea for discovering new classes of bioactive compounds, I'm all ears. It's worth the risk. And if you take enough risks, there will be some spectacular successes, along with some catastrophic failures. But, right now, looking at where we are in the area of antibiotic resistance anyway, the bacteria will evolve faster than our incrementalism--bacteria don't care about the professional concerns of NIH officials or academics.

So I'm staking my marker--here's what I think needs to happen:

1. Funding needs to be shifted from R01s to larger project grants that provide the resources to do the science the 'right' way. Even if some large projects are basic research, we need to switch away from producing lots of little pilot studies that only hint at answers.
2. R01s need to be viewed as pilot studies--if, after five years the results are promising, then provide adequate--that is, lotsa--funding. Yes, this will be harder on the career prospects for some faculty--that competitive renewal just got harder since competitive renewals will be larger and therefore fewer (not to mention, you'll probably have to round up a bunch of other collaborators). But the primary goal should be moving the science forward, not providing tenure milestones for universities.
3. Program officers need to move policy directions back towards the NIH and away from advisory committees. Obviously, they shouldn't be micromanaging, but, too often, the direction that programs take is determined almost exclusively by grant review boards.

Anyway, this won't be the last I'll discuss this, so I'll stop here, even though there are a lot of other ideas bouncing around in my head.

Added after writing: David Gorski has a different take:

Blogger Mike the Mad Biologist has asked on two occasions, first in response to the NEWSWEEK article and then in response to this NYT article: Are these critics mistaking the symptom for the disease? As Mike points out, the problem goes beyond funding levels. Rather, it is incentives. It is where the money goes and what is funded. On that score, the NIH is profoundly schizophrenic these days. The sorts of promising high risk proposals do not in general come from large, multi-institutional, collaborative groups. There are too many interests, and such groups tend to have too much at stake to take many risks. The very sort of researcher who will propose the risky projects that all these reformers [want] is the small, independent researcher funded by an R01.

Needless to say, I think that 'big science' can be very transformative, although I agree that, right now, the debate is focused on anecdotes, not a systematic review of data. Might have to write a post about this....

*Two additional points. First, I'm focusing on the genetic epidemiology and population genetics--adding clinical/veterinary components would increase the cost further. Second, even if you disagree with my particular sampling scheme (this is a blog post, not a white paper), the point is that the project would to be big to draw any general conclusions.

**That's partially why all of the grief Francis Collins received when he was proposed as NIH chief was such bullshit: Collins knows how to get risk averse program officers to take risks.