Donald Roberts, "Scientific Fraud", and DDT

In this piece Roger Bate, Donald Roberts and Richard Tren accuse the UN of "Scientific Fraud against DDT". Their Accusation is based on an Opinion paper by Roberts and Tren published in Research and Reports in Tropical Medicine. So let's look at their paper and see where the "Scientific Fraud" is.

Roberts and Tren's key argument is that reductions in malaria in the Americas were not the result of Global Environmental Facility interventions but were caused by increased use of antimalarial drugs. In their own words:

"However, their successes were not a
result of the interventions we describe as components of the
GEF project. Their successes were mostly a result of wide
distributions of antimalarial drugs to suppress malaria (see
Table 1).
Data in the Table reveal trends of increased numbers of
antimalarial pills distributed per diagnosed case and decreased
numbers of cases. Equally obvious is the decreased numbers
of pills distributed per diagnosed case, and increased
numbers of cases in two countries (Costa Rica and Panama)."

So their argument rests on table 1. Here's table 1.

Country  pills/case  pills/case  % change in  % change
            in 1990     in 2004   pills/case  in cases
Mexico          235        2566         1092     -1307
Belize           21          82          390      -287
Costa Rica      653         100         -653       112
El Salvador      34       22802        67064     -8276
Guatemala        38          54          142      -144
Honduras         30          51          170      -338
Nicaragua       279        1319          473      -519
Panama          202         140         -144      1337

The first thing that leaps out at you is that the table shows reductions of more than 100%, which is impossible. Panama cannot have experienced a decrease of 144% in pills/case. According to the two previous columns in the table there was a decrease from 202 to 140, which is a 31% reduction, not 144%. 202/140 is 144%, but it is not the case that the column contains the ratio of pill/case in 1990 divided by pills/case in 2004 (ie, is just labelled wrongly), because then the number for Guatemala would be 70%, not the 142% shown in the table. The column appears to show the bigger number divided by the smaller. That is, all the percent changes in that column are calculated incorrectly and the increases and decrease were calculated differently.

The next column (% change in cases) also contains impossible reductions and would seem to have been calculated in the same incorrect fashion.

I checked the source for the column "pills/case in 2004" and found that all these numbers were incorrect, being too high by a factor 10. The correct number for Panama, for example, was 13.99, not 140. I wasn't able to check the source for the column "pills/case in 1990", but it seems likely that it is too high by a factor 10 as well. (If that column is correct, Roberts argument fails because then every country increased pill/case, even those that experienced increases in malaria.)

So every number in Table 1 seems to be wrong. This isn't the real problem with Roberts argument, however. Since he is concerned with changes, the errors in the pills/cases ratios cancel out. And his argument is based on an the sign of the changes, not the magnitudes, so getting all the magnitudes wrong does not matter that match.

The problem with his argument is that pill/cases ratio changes if the number of cases changes, even if there is no change in the number of pills used. If, for example, the GEF interventions decreases the number of cases, the pill/cases ratio increases and Roberts and Tren then claim that the increase in this ratio was the cause of the decrease in cases when it is in fact an effect of the decrease in cases.

To correct for this we should be comparing the change in cases with the change in pills. I show this below, with correctly calculated percentages.

Country    % change % change
           in cases in pills

Mexico          -92      -16
Belize          -65       36
Costa Rica       12      -83
El Salvador     -99      710
Guatemala       -31       -1
Honduras        -70      -50
Nicaragua       -81       -9
Panama         1237      828

There isn't any consistent pattern here.

There is another serious problem with Roberts and Tren's analysis -- it purports to examine the effect of the GEF interventions, but these started in 2004 and they compare the situation in 1990 with that in 2004, so their comparison says nothing at all about the effects of the GEF interventions.

I computed the changes in pills and cases from 2004 to 2007 and show them below.

changes from 2004 to 2007
Country    % change % change
           in cases in pills

Mexico          -13      -84
Belize          -20      -90
Costa Rica       -5       -5
El Salvador     -56      -98
Guatemala       -47      467
Honduras        -27      -25
Nicaragua       -80      -99
Panama          -75      -98

Now we have fewer pills associated with fewer cases. Of course we can not conclude that reducing the use of pills reduces the number of cases of malaria. It is likely that causation runs the other way -- that reduced malaria rates meant that fewer pills were used to treat malaria.

To summarize: Roberts and Tren made many errors in their key table. Correcting their errors reverses their results.

You can see a spreadsheet with my calculations here.

More like this

> The column appears to show the bigger number divided by the smaller. That is, all the percent changes in that column are calculated incorrectly and the increases and decrease were calculated differently.

Cue the usual socks, shills, trolls, and bots going

1. 'waah, why are you crucifying someone for having a different opinion' or
2. 'waah, you sound nasty therefore you're wrong' or
3. 'this is a smear piece and Roberts and Tren are absolutely right (but I won't explain why)' or
4. 'give up Tim the world is slowly waking up to the DDT ban hoax prepare for the glorious libertarian revolution' or
5. 'here's some irrelevant linkspam now refute it haha'

ad infinitum...

And of course the socks, shills, trolls, and bots will miraculously ignore the fact that Roberts and Tren were making arithmetic errors that'd make a grade school student cringe.

Someone (Tim?) should write a letter to the editor of Research and Reports in Tropical Medicine suggesting a retraction/correction. It's a brand new journal--their credibility is really on the line here.

Yes, I second the motion to notify the journal editors, offering to write a brief note for possible publication. What an embarrassment for the authors...

Even allowing for their dodgy data, R&Ts arguemnt that there is no difference between the GEF areas and non-GEF, doesn't support their 'don't stop using DDT' stance very well.

Oh, and the weird "no. of pills" metric seems to be part of some kind of anti-ACT campaign.

Again, very well done. And yes, I think a letter to Tropical medicine is in order!

Also, this is a VERY young journal. From their own website:

Research and Reports in Tropical Medicine
(17) Articles
(36,955) Views

That's pretty puny.

The Res Rep Trop Med's Editor-in-Chief comments (hover your cursor over the NB graphic)

This paper represents a comprehensive critical review of claims that insecticide free methods of reducing exposure to mosquitoes represent an effective method for the control of malaria transmission. Activists have been using the reports by the PAHO environmental unit to press for a ban on DDT for malaria control worldwide. This is a very controversial stand and one that is actively opposed by most of the medical entomology community. This critical review debunks the claim of the effectiveness of insecticide free methods and represents an important contribution to the debate.

So good luck on any submission to the E-in-C about this issue if that comment is not just a version of the authors' PR statement about the opinion piece (which I presume was still subject to PR?).

But there is hope: their policy on is

As we are all human we all make mistakes. You may find after your paper has been published that it contains an error. If this happens you need to contact the editorial team immediately, the paper cannot be changed (it will already have been deposited with PubMed) but an addendum to the paper can be published in the journal as soon as possible. If the error is a major one eg, there was a problem with your original study that now invalidates your findings the paper may be withdrawn but this is very rare.

Oops! Don't know what happened there. That should have read:

But there is hope: their policy on ["Errors and letters"] is ...

Glad I didn't have malaria in El Salvador in 2004. 63 pills a day (which is what you'd need to take to reach 22802 in a year) seems a little hard to stomach.

Tristan, the reason why pills/case is high is because they give them to you if you show malaria symptoms rather than waiting until it is confirmed by a blood test.

By Tim Lambert (not verified) on 08 Mar 2011 #permalink

When they say "...number of pills distributed...", how far down the distribution chain have they gone to take the count?

All the way down to the end-user?

(eg, in the music industry, the number of a particular single sold to end-users was not the same as the number of that single sent out to the shops from the record company - and the record company preferred to use the latter measure when boasting about how many copies of something they had sold).

By Vince whirlwind (not verified) on 08 Mar 2011 #permalink

Well that was an acutely embarrassing demolition thanks Tim! Just say "No" to innumeracy, kids.

The other reason for a change in rate (and pill numbers) is the simultaneous roll out of pyrethrin impregnated bed nets and ACT (artemisin combination treatment), largely as part of the Gates' Foundation work, which have had a profound impact on malaria incidence and prevalence in most parts of the world.

There is also some very interesting data from East Africa showing a quite significant fall in the incidence and prevalence of malaria, not accounted for by any treatment or prevention regime. It's postulated that this may be an effect of the education of women or the penetration of mobile phones (or both), we don't really know why yet. What we know is that the decline is independent of the use of DDT, bed nets or ACT.

Either way, Roberts is scientific roadkill.

It is physically impossible for a decrease to be larger than 100%.

Hey Rob, stop trying to politicise this.

By Vince whirlwind (not verified) on 09 Mar 2011 #permalink

But, but, but....GALILEO!!!!!!!!!!

By Mercurius (not verified) on 10 Mar 2011 #permalink

Wait a minute. They're saying that it wasn't the insect control program, but medicine that made the gains?

Are they idiots? Or do they think everyone else is an idiot? It's always medicines that make the gains. You can't beat malaria by killing mosquitoes -- they always, always come back -- but as Fred Soper proved in Brazil in the 1930s (without DDT), if you can knock down the mosquito population, AND THEN can treat enough people and cure them of malaria, when the mosquitoes come roaring back, there will be no pool of disease from which the mosquitoes catch it. Consequently, mosquitoes cannot spread a disease they cannot catch.

Medicine, or other ways of getting the disease out of humans, is always part of the answer. DDT always functioned in conjunction with beefing up health care. Where DDT alone was used, no progress was made.

Their entire premise is silly, and wrong, it appears to me.

Is this the most incompetent arithmetic yet seen in a "paper"? For decreases they divide the "old" figure by the "new" and bung a minus sign in front! But their "% change" is rubbish anyway. If you had any examples where the old and new figures were the same (no change), you'd get "% change" = 100, not zero!

By TrueSceptic (not verified) on 10 Mar 2011 #permalink

Ed @ 18

The message us......yes, but DDT!

that should be 'is'.

Shouldn't they at least know how to calculate percentages before they try to publish?

Maybe they should hire my 14 year old autistic grandson to do their arithmetic.

I clicked on the link at Ed Darrell (#18) and, instead of being transported to the bathtub, I got a webpage promising to find a girl who matches me sexually. NSFW. It was a disappointment.

"I clicked on the link at Ed Darrell (#18) and, instead of being transported to the bathtub, I got a webpage promising to find a girl who matches me sexually. NSFW. It was a disappointment."

Yeah, the link to his name goes to a porn site. Very odd. Especially since the post was spot on and obviously from him.

By Robert Murphy (not verified) on 10 Mar 2011 #permalink

GaryB, if your grandson is unavailable, my 8-year old could fill in with a minimum of preparatory training.

In fact I'm thinking of submitting her last school assignment on "The Environment" to "Research and Reports in Tropical Medicine". Might help the editors improve the reputation of their paper.

By Vince whirlwind (not verified) on 10 Mar 2011 #permalink

This is the smoking gun. I'm actually going to look into submitting this to Smoking Gun in fact.

Anyway, it's a real gift. From now on, I'm going to say "You're trashing Rachel Carson based on crap like this? Really? Really?"

By Marion Delgado (not verified) on 11 Mar 2011 #permalink

I see that Marje Hecht has posted a "rebuttal" to LAmbert's debunking -- but on the wrong thread.

> Lambert goes on to state reductions of more than 100% are impossible. Actually, there is no mathematical reason for not having a positive or negative percent value and you can have a percent value greater than 100. The validity of numerical values is dependent on the reader having a clear understanding of what the numerical values represent. This is true whether you call the value a percent, a proportion, or some other relative unit of measure.

This calls for a "facepalm" emoticon.

-- frank

Perhaps, instead of Tim Lambert writing in to the journal to suggest a correction/retraction, the writing in should be done by GaryB's grandson. Or Vince's son, for that matter...

-- frank

Frank - I'll get my 10-year-old onto it. She loves an argument. (Killer right hook, too).
Her grandfather heads up a maths department, so I'll get him to help me prime her.
Could be fun.

By Vince Whirlwind (not verified) on 12 Mar 2011 #permalink

Tim, still keen on nuclear energy?

3 reactors at Fukushima in trouble and now a second site further North appears to be in trouble.

I think you should start a new thread on this subject.

Where are Flying Binghi and Cohenite's comments demonstrating how wrong your analysis is Tim. C'mon guys show Tim that he is wrong.....c'mon......please......pretty please....

Oh I get it, Tim has CENSORED your comments, obviously, I should've known thats why you haven't turned up. Don't worry, be brave, keep trying.

Thanks Marion, very interesting!