Peer review problems: just in the Nature of things

The Nature blog, The Great Beyond, has an interesting although not surprising report of accusations on BBC that a cabal of researchers has been impeding publication of important stem cell research to help themselves or help their friends:

Truly innovative stem cell research is being suppressed by a clique of peer reviewers for high profile journals, several researchers claimed today.

They told the BBC that the problem lies with those responsible for producing the reviews of research that journals such as Nature use to decide whether to publish the work.

Two scientists told the BBC they believe that in some cases reviewers are submitting negative comments or demanding additional and unnecessary experiments to delay publication and allow their friends to publish first. (Daniel Cressey, Daniel Cressey, The Great Beyond)

It's interesting because Nature's blog is in essence calling out Nature, the journal, although it's not the journal's sole responsibility. As a journal editor myself, it's not easy to police behavior of this type, assuming it's true. And that's the "not surprising" part. I don't know if it's true or not but if it were, I wouldn't be surprised. Getting published in a high profile journal like Nature is good for one's career in a way that publishing the very same paper in an excellent but lower profile specialty journal isn't.

For its part, Nature and similar Big Time journals like Science and The New England Journal put a premium on publishing results first. All three journals published so-so papers on swine flu because they were the first papers on swine flu. In some instances the papers added little to what we knew already. It was a race for high visibility and press coverage. These journals are too often interested in newsworthiness than science worthiness. They have active media operations and tantalize journalists and reporters with embargoed papers, making them think that because they are embargoed they are getting some kind of hot science news and with it visibility. The more a journal is mentioned in the news the more scientists want to publish there first, thus setting up the kind of dynamic at issue here.

Nature, Science and The New England Journal are among the most important scientific journals in the world. I subscribe to the first two and read the third at work. But they are also businesses. Sometimes business works with science and sometimes against it. This appears to be a case where the business side has indirectly worked against the science side. The indirectness, though, makes the effect more insidious and pervasive than just the stem cell story.

Remember you heard it here first. Well, not exactly. You heard it on Nature's blog, first. Just goes to show you.

More like this

It is clear that if you want to get a so-so paper published in a top tier journal, the best way to do it is to write about a breaking medical news event and get there first. We saw this with avian influenza and SARS and now it's being repeated with swine flu. The Scientist had a story yesterday…
[This post was originally published at webeasties.wordpress.com] Considering the forum, you can probably guess my answer, but it seems the editors at Nature agree... sort of: Institutions need to recognize and to encourage such outreach explicitly -- not just as a matter of routine, but…
Over the last few decades, there has been a veritable explosion in the quantity of scientific journals and published papers. It's a veritable avalanche. Some of the reason for this is simply the increase in the number of scientific researchers that has occurred over the last few decades. Another…
Vacation time! While Orac is gone recharging his circuits and contemplating the linguistic tricks of limericks and jokes or the glory of black holes, he's rerunning some old stuff from his original Blogspot blog. This particular post first appeared on January 16, 2006. Enjoy! During my usual weekly…

Correct Nature blog link here

By Uncle Glenny (not verified) on 03 Feb 2010 #permalink

They are the "top" journals for many values of "top" of course. But I have to say, in my practical work I more often than not end up not citing the Nature/Science paper but other, often later papers by the same group published in more specialized journals. Perhaps it's the relative lack of pressure or simply the larger available space, but those papers tend to simply be better written and more comprehensive.

On the subject of brevity, one scientist (solar physicist) I know mentioned he has to pay a page fee for his published papers. His last paper cost him 11,000 dollars ( Ithink the paper was 6 pages), so they try to keep the papers as short as possible. Not sure this applies to all journals, but you would think with the subscription fees, advertisements and pay per view, that charging the scientists for publication (paid for by government or private grants presumably) is a bit much.

Me thinks a poor patent officer today with a a paper on quantum physics and special theory of relativity would have trouble getting published today, unlike in 1905. Even if he got past the peer review (and their "who is this guy anyways"), he would not be able to afford to get the paper published (unless the page fee is the exception and not the rule).

"The most important scientific journals in the world ... are also businesses."

And as businesses, their editorial decisions may be unduly influenced by the profits to be made from drug companies who support the journals by spending on advertising and article reprints.

Not the best possible system.

The 50 Best Health Blogs

Journals that hide the peer-review process from scrutiny are just asking for these kinds of criticism. (And yes, I know that's all of them.) Transparency is the way forward, although researchers, being ultra-conservative, haven't realized it yet and will probably fight it every step of the way.

I think the day will come when article reviews are signed like book reviews are now.

Zen; It's not all journals. Many of the Open Access journals published by BMC practice Open Review. The journal I edit does as well. Reviewers and authors are known and the reviews are accessible to readers via a separate link on the website. Our experience, going back years, is that this improves the review process, making it more constructive and helpful and producing better papers.

@pft: pay for publish journals remain rare in most fields and often they are low voltage or highly specialized journals.

A somewhat muddled presentation here. Most papers I see as a reviewer for several journals, associate editor for one (spanning public health and social science) are awful. Most of the stuff that gets rejected outright should be, although I sometimes worry that junior investigators get discouraged too easily--many of their papers show signs of insufficent mentoring. When it comes to good papers, matters of preferred methodology do block publication, as does politics. I had a rather innocuous paper spiked because of someone who clearly didn't like the results. In another case, statistical analysis was the sticking point--obviously the same reviewer at 2 different journals, who wanted me to use an analysis that was completely inappropriate to the level of measurement in the data. Fortunately, I knew the editor at journal #2 and made a successful appeal.

In Medicine both the British Medical Journal and the PLoS Medicine practice open peer review.

The authors know who the reviewers are and vice versa.

It changes the dynamic and I personally like it. But I don't know if it's better for science or not.

By antipodean (not verified) on 03 Feb 2010 #permalink

I agree with comrade. And shit it is. Problem is it not only involves the journals, but grant reviewing process for the NIH etc. These folks bring home the grant proposals from the committee work that are suppose to be confidential, hand them out to their post docs, grad students, and quickly assign projects to the underwings with the intent of scooping the grant submitters idea or approach. Peer review needs to be revamped.

By BostonERDoc (not verified) on 03 Feb 2010 #permalink

*Feigning shock/surprise*

Echoing Comrade PhysioProf sentiments - everyone involved in research has dealt with this behavior from reviewers/editors high on some kind of power trip! The only way to get around it sadly, as Rich mentioned is knowing someone higher up to appeal to.

Why isn't the integrity of a journal questioned when science and research takes a back seat in favour of other matters (conflict of interest anyone?!) Instead we jump through all these hoops and our motivation is unchanged in seeking to publish in these same journals.

Who is held accountable for delaying publications?

Why are there no measures in place that name and shame reviewers/editors who engage in this behaviour? Certainly, it's hard to prove until after the point (almost identical paper published a month later).

Punishment not enough?

What about rewarding editors/reviewers of journals with a better impact factor/waiving of page fees for submission/invitation to submit, etc.

I'll go back to dreaming...

Alex: Only big journals can afford this. Most journals use their editors and associate editors to distribute to peer reviewers which they select from their editorial boards or rosters of peer reviewers or through authors' suggestions or looking at the references or through pubmed searches. Journals vary in size and editorial support, but most specialized journals depend on volunteer labor. Only the publishers make out and some of them make out like bandits. So a rich journal can do this, most journals don't. They get along with rudimentary editorial support and some help from the publisher.

Thanks for clarifying.

Hmmm...Aren't the journals pretty desperate for reviewers? I say volunteer your services for a while before submission. Then you case for appeal is much likelier to be approved.

@pft: "one scientist (solar physicist) I know mentioned he has to pay a page fee for his published papers. His last paper cost him 11,000 dollars ( Ithink the paper was 6 pages)"

What journal? I know that ApJ, which is high-impact for solar physics, charges $110 per page, $350 for color. Solar Physics, the journal, has no page charges. Who charges $1800 a page?