Lancet

Lancet post number 200

tlambert — Mon, 25 Oct 2010 12:21:38 +0000

Lancet post number 200

The WikiLeaks Iraq archive, while incomplete, reveals many more previously unreported violent deaths in the Iraq war -- Iraq Body Count say that the archive reveals 15,000 people shot, blown up, had the heads cut off or killed in some other way that they had not recorded. So Tim Blair, who claimed that the Iraq Body Count was way way too high (and predicted that the coalition would suffer "below 50" casualties) has posted a correction. Ha ha, just kidding. Blair has a post claiming that the WikiLeaks archive, which is, as I have already noted, incomplete, proves that the Lancet study on war-related deaths in Iraq is wrong. This does not follow. Since the WikiLeaks archive is incomplete, the number of deaths recorded is just a lower bound. That's because the archive is incomplete. This is just the latest in Blair's innumerate criticisms of the Lancet study.

We also have Fred Kaplan, who writes:

However, the WikiLeaks documents add further doubts to a controversial report in a 2006 issue of the medical journal the Lancet, claiming that, even that early in the war, 655,000 Iraqi civilians had been killed, most of them by U.S. air and artillery strikes.

In fact, the study attributed 31% of the roughly 600,000 violent deaths to the coalition , and just 13% to air strikes. I guess checking what the study actually found is too much trouble if you are a journalist.

And it's not just Blair and Kaplan. Thers on Glenn Reynolds:

In the world where things like human beings dying and being tortured in the course of senseless wars matters, the recent Wikileaks documents release is accompanied by headlines like "A Grim Portrait of Civilian Deaths in Iraq."

In Glenn Reynold's squalid little world, however, what the documents show is that the war was even more glorious than one he'd always masturbated to, and will be good news for Republicans!

tlambert Mon, 10/25/2010 - 08:21

Tags

LancetIraq

Fred Kaplan

Glenn Reynolds

iraq mortality

Lancet

Tim Blair

One thing that always drove home to me how dishonest and/or ignorant Glenn Reynolds and his ilk are, was how none of them seemed to have ever found any fault with how the war was being conducted except possibly for those who thought we should have killed al-Sadr when we first got there. Why were so many tankers and artillerymen killed fighting as infantry? Why were so many soldiers complaining about their rifles jamming? Who fed the wildly inaccurate Jessica Lynch story to the press and why? These are question separate from whether or not you supported to war, but they'd never ask them in a million years because it would burst their ideological bubble that everything about the U.S. military is perfect and wonderful and anyone who critizes any aspect of it is either a commie-lib or an isolationist paleo-con.

And, of course, Andrew Bolt has to fall into line as well, as I've noted at my blog.

We all agree that the "the archive is incomplete." But is it incomplete by more than a factor of 5?

Unless you think that the US military was lying to itself, this new data should cause you to believe the Lancet less and folks like IBC and Spagat more.

Tim: It might be handy to update your chart of credible estimates, dropping the ORB (now thoroughly debunked) and adding Wikileaks.

David, did you even bother to read the post.
The IBC always was a lower bound rather than an estimate as you were want to claim. The new data is also a lower bound which is much higher than the IBC.
The reasonable response would be "Oh, so the IBC was way to low."

Don't worry, I am not holding my breath.

Hi guys,
Don't you find this argument pointless, how many civilian deaths were too many? How many innocent people need to die before someone says, 'Ooh, thats bad'?

I'm also disgusted how much of the mainstream commentary is concerned with the potential fallout, showing more concern for that than the dead innocents.

When a country is illegally invaded, 1 death is a murder and those responsible should be held accountable, and i don't just mean the soldiers who have taken unlawful action (like those who have already been shown enjoying blowing innocent civilians away in the 'apache incidents).

At least with Wikileaks this is getting some coverage, how central america envies this modern phenomena.

Dicksonator @ 5,

There is a reasonable argument that Saddam was already killing civilians (notably Kurds, also political opponents, etc) in large numbers, not to mention the deaths caused by wars he initiated (yes, I know that he often had US support).

If (counterfactually) there had been only a few thousand civilians killed followed by peace (unlikely, but bear with me) I think there would have been a reasonable argument that the lives saved exceeded the lives lost in the long run. Which is the line that Bush et al tried to run. Studies and accounts of civilian deaths give the lie to this argument, which is why it's important to count them.

Bear in mind this is Tim "as in too innumerate and stupid to be David Kane being innumerate and stupid" Blair.

It takes David Kane to be David Kane being stupid and innumerate, Tim Blair!

I was going to leave a comment on Blair's blog until I read through the other comments and realised the average intelligence level I'd have to be pitching it at.

Talking to 3 year old minds is fine, but it can be hard work dumbing down the message into a format they can understand.

Given the IBC and Wikileaks counts of deaths from specific, recorded incidents the Lancet estimates for total excess deaths subsequent to the CoW invasion must be in the right ballpark - mustn't they?

Frankis, to understand that, you would have to be [smarter than a six year old](http://scienceblogs.com/deltoid/2007/04/pinata_whacking_not_outsourced…).

James, I think a 'bit of an argument' is surely a red herring in this case and hardly relevant. There were and are ways to deal with Sadaam in the case of any such violation.

I actually rather think though, that considering Iraq was under continual patrol and attack ever since GW1, this is rather unlikely unless we turned a blind eye to it for later propaganda use.

However, even this is a terrible reason, as Sadaam gassed most of the Kurds whenever we were his bosom buddies, actually just shortly before he invaded Kuwait where he probably thought we'd ignore the situation since we were his friends. Woe betide anyone who interferes in the sphere of global influence of the US/UK and of course, controlling oil is paramount in this regard.

Also, we seem to be quite happy to let Turkey persecute this Kurdish.

In short, i don't buy that argument.

Dicksonator,

Great posts. You have nailed it directly. The war was illegal according to just about every national and international law - the UN Charter, the Nuremberg Code, and the US Constitution for starters. By that reckoning, the US is responsible for every death that occurred as a result of the invasion. And now Iraq is utterly devastated. One of the region's experts, Nir Rosen, said in 2008 that "Iraq has been destroyed, never to rise again". Pretty much the entire political and social infrastructure of Iraq was blown to smithereens by invaders who were pursuing their own agenda that had nix to do with WMD, promoting freedom or democracy. Yet this myth has repeatedly been rammed down our throats by the corporate and state MSM.

And also note the media's non-coverage of the recent study which showed that cancer cases in Falluja within 2 years of the US bombardment of that city in November of 2004 well exceeded the rate in Hiroshima after the atom bomb was dropped there. Falluja was once a teeming city with 600,000 inhabitants; now it is a burned out shell with the latest census reporting 300,000 residents, meaning that half of the population fled and has not returned or were killed in the US assault.

I find it takes remarkable hubris for anyone to try and downplay the horrific death and destruction inflicted on Iraq, first by the 'sanctions of mass destruction', and then by the illegal invasion. The utter hypocrisy of western foreign policy, part of which Dicksonator has alluded to (e.g. 'worthy' Kurds in Iraq, 'unworthy' Kurds in Turkey) as well as the support of monstrous regimes elsewhere under US control or influence (e.g. Montt, Marcos, Somosza, Pinochet, Mbutu, Suharto) speaks volumes. And those countries who do not play according to the rules set out in the "Washington Consensus" - Venezuela and Bolivia being prime examples - are forever vilified in the media. The fact that outright expansionism, nullification of alternative systems, and subjegation of other country's assets drive western foreign policy should be brazenly obvious by now, even if the body count as a result of such actions is enormous. Yet to many it isn't. We in the rich world prefer to think of ourselves as 'noble victims', and the media cultivate that image well. We trust our politicians, who usually hide their agendas behind a veil of lies and deceit. If one wants to really understand what drives western policies, its easy to do so: read declassified planning documents which are available in most large libraries. They could not be clearer.

> We all agree that the "the archive is incomplete." But is it incomplete by more than a factor of 5?

Since

a) from TFA: "In fact, the study attributed 31% of the roughly 600,000 violent deaths to the coalition , and just 13% to air strikes." Therefore it's 1/0.44 times smaller than that discrepancy. Taking it down to 2.2 times.

b) it is only the figure of UNREPORTED deaths, you have to add that number on to the death toll in the post heading. This brings it down to what?

> There is a reasonable argument that Saddam was already killing civilians (notably Kurds, also political opponents, etc) in large numbers

Why wasn't he tried for these many crimes?

Would that be because he had western aid and backing for these crimes?

They had to look REAL hard to find something both bad enough to warrant execution yet nothing they were involved in.

This is one reason why the trial waited so long.

Iraq was a punishment beating done in public because Iraq was going to do all international trade in Euros, with North Korea, Iran and Venezuela also looking likely to follow suit.

Those four names are the "Axis of Evil" the USA put out.

I never understood Venezuela being in the list until I watched Robert Newmann's "A History of Oil" and found that little fact.

David Kane, of course the military sometimes lies to itself: have you not heard e.g. of Pat Tillman? As you well know Spagat is not exactly the most credible figure.

The bottom line is that reported deaths in a conflict zone like Iraq are a lower bound for actual deaths and for obvious reasons can be very far from being a tight lower bound. That lower bound has just been raised (indeed it looks like IBC's new lower bound will be above their previous "upper bound") and your reaction is to pretend that this reduces confidence in the higher estimate of total excess deaths?

As I have said before, the main objective of the war party in this case was to try and make any estimate of civilian deaths in Iraq during the invasion (as well as because of the sanctions and also in Afghanistan) open to conjecture. The logic behind this is that, without hard data, the death toll is impossible to determine, and might therefore be very low. Finally, discussion of the toll caused by the invasion will melt away and disappear down the memory hole, along with huge range of other western crimes and atrocities over the past 100 years. The strategy was used to downplay the carnage inflicted by U.S. forces when invading the Phillipines in 1901-02 as well as in Nicaragua, Guatemala, Chile, Korea, Viet Nam and Cambodia. But of course, the corporate media will continue to focus laser-like on crimes committed by 'officially designated enemies', irrespective as to when they occurred.

The other theme I see repeatedly raised by those who supported the illegal invasion of Iraq is in presupposing that Britain and the United States possessed the 'moral authority' to do so. This is quite honestly absurd, given the appalling history of foreign (and domestic) policy abuses by both nations. But the very fact that many people think the west possesses any kind of moral authority shows how successful the state propaganda system has been in indoctrinating people. The past 200 years of US foreign policy has, IMO, been characterized to a large extent by senseless butchery and democracy deterred.

>"Iraq was a punishment beating done in public because Iraq was going to do all international trade in Euros, with North Korea, Iran and Venezuela also looking likely to follow suit."

Hang on, if that's the reason, then why not attack Austria, Belgium, Cyprus, Finland, France, Germany, Greece, Ireland, Italy, Luxembourg, Malta, the Netherlands, Portugal, Slovakia, Slovenia and Spain? All countries in the Eurozone carry out their international trade in euros...

MFS, I'd guess it's because of Europe's relative lack of oil wells and the absence of convenient excuses to do so.

>*Hang on, if that's the reason, then why not attack Austria....*

Iraq, Iran and Veneszuela are differentiated from the Euro countries that you identify under the term and the theory of the [Petrodollar](http://www.google.com.au/search?hl=en&safe=off&&sa=X&ei=n3PHTLPPBoy6vQO…).

I guess that leaves North Korea as the odd one out. How fitting!

Maybe this is old news to some here, but if you haven't seen the documentary movie, "No End In Sight" take a look.
The reason for so many civilian deaths is that the U.S. never established civil order in Iraq. No martial law, disbanded the army etc. Only place that was protected was the oil mininstry and the "Green Zone", which is also a good movie, related to the first one.
Starring Matt Damon

Sailrick,

I believe the reason for so many deaths in Iraq is that the U.S., quite frankly, didn't care. It wasn't that the Bush II deliberately set out to kill civilians, but that his neocon regime knew this was a distinct possibility but factored that against the benefits of controlling 'the greatest material prize in history' and a 'source of stupendous strategic power' as a 1950 U.S. State Department described the region. Their main concern with respect to high civilian death tolls was not the effect it would have on public opinion in Iraq, but in the United States. This explains why there have been such mendacious denials of high death tolls as a result of the invasion, with the Lancet and its authors being subjected to continual attacks by the corporate msm.

The trouble is that the U.S. is loathed in most of the Middle East, and most Iraqis who were at the receiving end of the the U.S. club know full well why the U.S. invaded their country. They aren't dumb - even if our own media aims to dumb down our own populations through the mass production of ignorance. The main concern has always been public opinion at home, most importantly with respect to the death toll of our own soldiers, and, secondarily, of civilians. And, as I said above, the concern is that civilian deaths will not be tolerated by public opinion at home, hence why casualties are always downplayed or ignored.

> It wasn't that the Bush II deliberately set out to kill civilians

Plenty of this.

But Amorality is, when coupled with extreme power, worse than explicit evil.

I believe the reason for so many deaths in Iraq is that the U.S., quite frankly, didn't care.

I read a while ago that the US attitude to Iraqis was that they were Untermenchen, i.e. lesser humans, which was a term used by the Nazis.

The [BBC talked to Les Roberts](http://www.bbc.co.uk/news/world-middle-east-11613349) about the implications of the Wikileaks documents, and [Robert Shone replied](http://dissident93.wordpress.com/2010/10/28/bbc-les-roberts/).

Iraq was invaded to terminate the cost of patrolling the no fly zones 24/7/365 which was costing ca 25-30 billion per year. The aircraft for this have a finite service lifetime after which they must be replaced espescially those used in flight operation from aircraft carriers.

Saddam and his family could potentially be in power for quite sometime--like forever. He played the cat-and-mouse game with WMD because he need the Coalition Forces for protection. Otherwse he would be fighting wars with Kurds in the north and Shia in the south. He probably would have
lost these since his army was all but destoyed in the war with the US.

Since the first gulf war never really ended, there is a distinct possibilties that the Iranians might invaded Iraq as the first phase of regional and world conquest. I have heard on radio talk shows that their plan is capture all mideast oil, invade Saudi Arabia and expel the infidels from the holy cities.

They are then ready to start the Battle of Armagedden with Israel. When the battles starts, the 12th or lost iman, who fell down a well in 942 AD, will rise up, take command of the army of Islam, defeat the infidels and go on to establish a world caliphate. Here is the really scary part: They believe that they can not lose the battle with the infidels.

RE: Chemical and Biologial Weapons

No modern military maintains "stock piles" of these weapons. The US began decommioning nerve agents in the early 1970's. The US just flat out lied about this. They also lied about Saddam's nuclear facilities that could make bombs. These were totally destroyed and no attempt was to made repair them

There is the possibily that Saddam could still make nerve agents. These are the most acutely and deadly poisonous man made chemicals. One kg of Sarin or VX has the potential to kill 1 million adults in 30-45 sec and as many as 10 million childern. These nerve agents kill all animals including worms in the soil.

I would not be surprised that the Israelis told the US to get boots on the ground because we really don't want to nuke the Iranians. Even today there is still the possiblilty that the Iranians might invaded Irag. Recent surveys have indicated that Irag's oil reserves might a possible 200 billion barrels.

> Iraq was invaded to terminate the cost of patrolling the no fly zones 24/7/365 which was costing ca 25-30 billion per year.

Alternatives:

1) Stop air patrolling the no-fly zones.

Alternative taken:

1) Spend $1 trillion on a war and kill thousands of servicemen

Iran would only have invaded Iraq because Iraq was secular. The US has now done it for them.

"Iranians might invaded Iraq as the first phase of regional and world conquest. I have heard on radio talk shows that their plan is capture all mideast oil, invade Saudi Arabia and expel the infidels from the holy cities. "

Only the best of sources from our always-reliable Harold Pierce.

---
Wow:
"Spend $1 trillion on a war and kill thousands of servicemen"

Not to mention somewhere between a half million and one million dead Iraqis, and upwards of 1/5 of all Iraqis as either internal or external refugees at peak.

Although I don't agree with #26 and Chris' statement about the humanity in Iraq and how the US percieves it, but the US military don't have the same sort of fall out when foreigners die than when their own people die.

Even if it's only because the bereaved families of USians who die vote in US elections and those of dead Iraqis don't.

This isn't a case of believing the Iraqis non- or sub-human but that their own people are more important to them.

Just because you lift someone up doesn't mean you drag someone down, though this is by far the easiest way to get people to lift themselves up (holding off from godwinning this thread even more).

The soldiers themselves are TRAINED to feel like the opposing forces are not human. Humans aren't very good at killing other people so you have to train them not to think of others as people.

It doesn't always work but it works sometimes.

Which *may* be where Chris got his info from. If so, that isn't a reflection on how the US people think of others but in how effective the training of armed forces is.

[Robert Fisk appeared on the ABC](http://www.abc.net.au/lateline/content/2010/s3051340.htm) this evening.

He is sure to attract the fiskers, given his less-than-subtle statements about US government/military behaviour in Iraq.

ATTN: Lee

I heard this on "Coast to Coast AM" on a number of occasions. I also read recently in Newsweek that a muslim cleric said, "We will not rest until the black flag of Islam flies from the Tower of London."

ATTN: BJ

I am a US citizen. I would like to see all US forces come home. If this happens, the Chinese would walk over all you guys and grab all your minerals, BEER and Wine. And turn OZ into a resort.

We kept the Russians from capturing all of Europe until communism collasped. Did they say "Thank You" Nope. They just spit in our face at every opportunity. They can go to Hell.

It seems that Harold Jnr. believes that a world that bestows 24% of its global resources onto the 5% of its global population that happens to have a controlling interest in global trade and finance, 1000+ of their military bases in 63 different countries, and a permanent state of war for the past 70 years, isn't grateful enough.

Harold Jnr. wonders where we would all be without all that, and doesn't doubt for a millisecond that the answer is 'worse off'.

Being a part of the 5%, Harold Jnr. likes to think that.

>"Harold Jnr. wonders where we would all be without all that, and doesn't doubt for a millisecond that the answer is 'worse off'. Being a part of the 5%, Harold Jnr. likes to think that."

Were it not because I have seen many other equally odd posts, I'd have to say Harold Jnr. is [a poe](http://rationalwiki.org/wiki/Poe%27s_Law).

Most Americans are isolationist we could care less about the rest of the world.

All this mil stuff we have round the world is a hangover from the cold war and is corporate welfare for the millitary-industial complex. However the Iranians are trouble makers and the Chinese are gettin to be a pain in the butt. However they hold about 800 billion of our markers so we actually own them..

Harold,
Hate to break it to you but the US doesn't actually have any troops in Australia (discounting the odd visiting warship). Or in Vietnam, Thailand, Malaysia or Indonesia, all of whom China would have to go through to get to get to us. I believe the last time the Chinese won a war with Vietnam was 1407, but you never know, they may want to have another go.
In a broader sense we benefit from US hegemony in the Pacific. But unless you propose that the US abandon Hawaii and its own access to the middle east, I don't see the US giving up that role any time soon.
PS: Australia is already a resort for the Chinese. What do you think they spend all those US dollars on? Holidays, university educations, and investments here.

[MSF](http://scienceblogs.com/deltoid/2010/10/lancet_post_number_200.php#comm…).

It was probably before your time on Deltoid, but there were several threads here where HPJ waxed lyrical about the countless numbers of t-tests he performed to disprove global warming. No matter how much I tried, I could not make him understand why his approach was bogus.

Strangley, I can't find any of the threads with a search - perhaps someone with a memory for the relevant thread could remind us all so that we can revisit the hilarity...

Can't be bothered to go through everything, but he mentions t-test here

Harold Pierce Jr. writes this utter crap:

*I am a US citizen. I would like to see all US forces come home. If this happens, the Chinese would walk over all you guys and grab all your minerals, BEER and Wine. And turn OZ into a resort.

We kept the Russians from capturing all of Europe until communism collasped. Did they say "Thank You" Nope. They just spit in our face at every opportunity. They can go to Hell*.

This is a comic level book understanding of the world. US forces are stationed in forward operating military bases in over 140 countries - if we include lesser operating bases, the total exceeds 160. And none of this has been done to protect other countries, but to ensure that the US retains control of vital markets and resources. Hence the reason for the existence of 'Africom' and 'Southcom'. All one needs to do is to read declassified planning documents from influential people like George Kennan, Paul Nitze, Henry Kissinger, Zbignieuw Brezonski and others that are available if one looks beyond the end of their noses. They shred Harold's frankly puerile comments and consign them to the asheap where they belong. Read State Department memos as well. Same notation. Read the 'Grand Area Strategy' from the Council on Foreign Relations. Or Brezinski's 'Grand Chessboard'. Or 'Project For a New American Century'. Or Paul Wolfowitz's 'Defense Planning Guidance'. Its all there in black and white. And these have been discussed widely in the literature by academics like Chalmers Johnson, Andrew Bacevich, Greg Grandin and many others.

Nothing more needs to be said. Harold's view of the world is an illusion. Hardly surprising when you've been drip fed the same mendacious propaganda since birth.

It's also interesting that Harold Pierce refers to the Iranians as 'trouble makers' but would never apply the same term to the United States, which has been a global rogue state for some time. When was the last time Iran invaded and occupied a foreign country? Further, I would like to know if he would consider Iran to be a 'trouble maker' after 1953, when the democratically elected government of Mohammed Mossadegh was overthrown in a joint British-US coup and was replaced by the Shah, who ruled the country quite brutally for the next 26 years. Why was Mossadegh overthrown? Because he was a nationalist who dared argue that profits from the country's oil, which largely flowed out of the country to the British based Anglo-Iranian Oil Co., should be used for the benefit of his country's own people.

Of course this theme has been repeated over and over again throughout the course of recent history. The fear of US planners was not the spread of communism but of indigenous nationalism. That countries, particularly in Latin America, will pursue economic systems in which the prime recipients of their resources are aimed at internal development and are not the property of US investors. Greg Grandin describes this in quite graphic detail in 'Empire's Workshop'. And the threat of nationalism is still is a major concern to US planners.

Afterthought BJ: since very little came up in a Google search, I wonder whether he was largely deleted. Mind you, I couldn't find your replies either, which is strange. Perhaps someone else will have better "luck".

[P. Lewis](http://scienceblogs.com/deltoid/2010/10/lancet_post_number_200.php#comm…).

Yep, that was one.

These are a few of the other hilarious thread I found:

but frustratingly I can't find the one where I tried to drum the concept of repeated measures into HPJ's head.

Still, I note that he went quiet on the matter of his one-man basement project. Perhaps he figured it out himself in the end...

> We kept the Russians from capturing all of Europe until communism collasped.

By capturing all of Europe.

Cheers.

The difference was SO visible...

[P. Lewis](http://scienceblogs.com/deltoid/2010/10/lancet_post_number_200.php#comm…).

"Strange" is certainly the word, because I know I needled HPJ several times about the matter of repeated measures, and I couldn't find the posts on first pass either. Perhaps Tim Lambert might have a clue?

[Wow](http://scienceblogs.com/deltoid/2010/10/lancet_post_number_200.php#comm…) re HPH's tale on mid 20th century Europe - they say that history is written by the victors... It is patently apparent that HPJ believes the preponderant version of history that he reads, and which agrees with his ideology.

ATTN: Deltoid Dingo Dogs!

If I didn't come here and pull your tails, you dogs would get bored, lazy and fat!

Jeff@41

The Anglo-Iranian Oil Co. (now BP) duped the naive Americans who were paraniod with respect to the Russians and communism. Mohammed Mossadegh was foolish. The British (and other Europeans) were never nice about colonizing quite a number of countries, raping these counties for cheap raw materials and brutely putting down any uprising of the natives.

What did he expect? Suppose he had said had that a modest royality would be be placed on oil, and the funds would be used for purchasing new arms from British companies as well as goods and services from them for upgrading the country's infrastructure. Once he had built up over time a strong military, he then could have kicked the British out of the country. That is what I would have done.

I also recall that British told the Americans that Russians had visions of incorporating Iran into their empire so they could have access to warm water ports and control the shipment of oil out of the Persian Gulf. Or something like that.

If the Americans had completely withdrawn from Europe, what do you think would have happened?

We Americans are the new kids on the block in a very old neighbood. We have no proper breeding, never went to finishing school, ain't got manners and culture, really don't know much of or care about the ways of the Old World, are idealistic and sometimes are easily fooled. So don't scold us since your ancestors have many centuries of blood on their hands as well as that of the First Nation Peoples in your country.

Jeff@40

So what! All major countries of the Old World have be doing this kind of stuff for centuries.

> We Americans are the new kids on the block in a very old neighbood. We have no proper breeding, never went to finishing school, ain't got manners and culture, really don't know much of or care

You can only speak for yourself.

But the first step on the way to self improvement is to recognise the problem.

You still have to fix it, mind.

> All major countries of the Old World have be doing this kind of stuff for centuries.

The defence of the four-year-old: He did it first!

ATTN: BJ

The method of analysis I used for some Quatsino weather data is from "Climate Change and Global Warming" by Andrew Masterman which is at:

http://www.usefulinfo.co.uk/climate_change_global_warming.php

He analyzed the CET on a month-by-month basis at 30 year intervals. By using t-tests he found that there was no significant difference in monthly mean temperature for each month for ca 300 years except for the fall months and season. I also found a fall seasonal effect in the Quatsino data.

Note after the "/" there should be:

...climate change global warming...

with a "_" between the words.

Anybody know why the url does not display properly?

Go check this out, and stop bugging me.

Wow@48

I speak for most of us down home folks except for the card-carrying members of the Eastern Liberal Establishment and the silk-stocking enviros and limosine liberals in NYC.

"I speak for most of us down home folks except for the card-carrying members of the Eastern Liberal Establishment and the silk-stocking enviros and limosine liberals in NYC."

Like hell you do.

You don't speak for me.

You speak for the douchebags and twits

"I speak for most of us down home folks except for the card-carrying members of the Eastern Liberal Establishment and the silk-stocking enviros and limosine liberals in NYC."

It's always helpful when someone just comes right out and says "Yeah, I'm a troll." Saves time. Thanks.

I suppose that's it for this thread then. No more Iraqi casualty discussion.

"He analyzed the CET on a month-by-month basis at 30 year intervals. By using t-tests he found that there was no significant difference in monthly mean temperature for each month for ca 300 years except for the fall months and season. I also found a fall seasonal effect in the Quatsino data."

IOW, used the least powerful method he could come up with.

>"So don't scold us since your ancestors have many centuries of blood on their hands as well as that of the First Nation Peoples in your country.

Hahahaha!!!

That's so funny! Except your logic is somehow flawed. If I were English I'd say my ancestors remained here in the ole country happily drinking beer and minding their own business.

However YOUR ancestors, Harold, travelled to the new world, raped and pillaged the natives, and remained until your generation to enjoy the spoils. YOU are their currrent representative. Look a bit closer at your hands and you might see the blood of Native Americans murdered by your very own ancestors.

Is it really your hypothesis that Spagat gathered all that data, produced the graph that you show above and then erased the points/lines he didn't like?

-- David Kane

I dunno if that's Tim Lambert's hypothesis precisely, but it approaches mine - that Spagat looked for a plausibly deniable third-party source for a cherrypicked graph that did just exactly that.

Somebody fabricated the graph. I don't know whether or not it was Spagat. But by publishing it he has taken responsibility for the fraudulent graph.

A few updates on this topic:

Patrick Ball, the researcher who was often cited in support of the Lancet 2006 study, has apparently changed his mind, and says:

"I want to be clear that I have no interest in defending the Burnham et al. estimates. The flaws in that study are now well known." (Patrick Ball, 28/4/10)

Les Robertsâs colleague, Francesco Checchi, has stated in an interview with the BBC that he thinks the ORB figure was âimplausibleâ, that the poll had a âmajor weaknessâ and that the Iraq death count is âlikely to be between 200,000 and 500,000â³. (BBC World Service, 27 Aug 2010)

Meanwhile, Gilbert Burnham has been reduced to posting defenses of his study on the message board of a discredited "media criticism" website. I think the scientific literature is probably a better place for it.

I don't have the links to hand for the above just now, sorry, but if anyone is still interested in this stuff, contact me via my blog (click on my name below) and I'll dig them out.

Robert Shone:

> contact me via my blog

That's a bit hard, since you've disabled comments everywhere. By the way, why do you feel the need to impose a blanket ban on comments anyway?

-- frank

The Patrick Ball quote that Robert Shone cites is here---

link

The whole thread is interesting.

The Francisco Checchi quote that Robert Shone cites is near the end of this 8 minute 45 second BBC report, which is worth listening to all the way through. (I wish there was a written transcript, but if so I didn't notice it.)

link

What struck me was the following--

JoshD (who was a participant here for years defending IBC) said the true number could be as much as two or three times the IBC figure, so he's willing to concede a violent death toll possibly as high as 200-300,000. 3 times IBC would have been about what the Lancet 1 paper said (excluding Fallujah). It's also consistent with the paper published in the NEJM.

Checchi's range (200-500,000) overlaps with JoshD's range at the low end and overlaps with the low end of the Lancet 2 paper at the high end. Checchi also said that estimates of "tens of thousands" were almost certainly wrong--those, of course, are the ones often cited in recent NYT reports (except when they say 100,000).

One last comment--Iraq War supporters Tony Blair and Michael O'Hanlon are also in the BBC clip linked above and make predictably repulsive comments. Blair wants everyone to be clear that it was Iraqis killing Iraqis (there's not much to say about which of those Iraqis might have been allied with us and he seems to have missed the thousands of civilians killed by the coalition even according to the IBC statistics).

O'Hanlon says the death toll is between 100 and 150,000 (he knows it couldn't be higher, somehow) and says this is much smaller than the number estimated to have been killed by Saddam in the 80's. So apparently estimates of large numbers of deaths as opposed to IBC-style counts are perfectly acceptable to him if they support his views.

That last point is always what has bothered me about this--you can just about guarantee that the press and pundits will use very high estimates (sometimes pulled out of the air) for the number of people killed by our enemies, but insist on conservative IBC-style counts for deaths that can be attributed directly or indirectly to our actions.

Donald: JoshD (who was a participant here for years defending IBC) said the true number could be as much as two or three times the IBC figure, so he's willing to concede a violent death toll possibly as high as 200-300,000.

I don't recall that he was ever willing to entertain that possibility here. I wonder have his views changed or was it just that comments about IBC riled him, prompting him to push for lowish numbers? If he happens to read this I would be interested to know his current view.

At the end of the day the fact is that the Blairs and Kaplans have won this particular argument. There's no great incentive to conduct new studies. Most history books, insofar as they attempt to quantify Iraqi deaths at all, will contain the low numbers.

Kevin Donoghue:

I don't recall that he was ever willing to entertain that possibility here. I wonder have his views changed...

I don't think IBC's position on this has changed. For example, here's something from a February 2006 presentation they gave: "Assuming even the most pessimistic outturn for violent civilian deaths, our database must include a substantial proportion of all victims, certainly not less than 25%, probably significantly more than half."
http://reports.iraqbodycount.org/d/on_ibc.pdf

I think that was based on their view that ILCS provided the most reliable estimate at the time. Since then IFHS was published, which seems to have confirmed their earlier-stated views. Incidentally, the first Lancet survey (2004) estimated 57,600 violent deaths - roughly three times the number that IBC counted over the same period.

Frank:

That's a bit hard, since you've disabled comments everywhere. By the way, why do you feel the need to impose a blanket ban on comments anyway?

I guess you didn't spot the clearly-labelled "Comments" page, alongside the clearly-labelled "About/Contact" page.

*We Americans are the new kids on the block in a very old neighbood*

Harold, you Americans aren't as 'new' as you think. The history of mass murder at the hand of your country goes back more than 200 years - first at home (check out quotes from the likes of Washington, Jefferson and JQ Adams 'defending' the use of mass murder to extirpate native Americans) to the present day. Basically Ward Churchill summed it up when he said American history can be characterized by senseless butchery and democracy deterred. This in no way excuses mass death inflicted on the poor by European elites, but any notion that the US is 'excused' for many of the horrors it has inflicted on the unfortunates over the past centuries is absurd.

As for Robert Shone, Josh D and other IBC supporters, how 'magnanimous' of them to argue that the death toll in Iraq as a result of the invasion might 'only' be 200,000. Of course this ignores the ongoing suffering inflicted on Iraq, through destruction of the country's civilian infrastructure, ravaged first through the 'sanctions of mass destruction' which certainly left hundreds of thousands dead in their wake, followed by the illegal invasion. And what about the recent study reporting the effects of the American assault on Falluja? What about depleted uranium? Unexploded cluster bombs? Or the non-controversial figure of 2-4 million internally displaced refugees?

What irks me the most is that the people at IBC and their supporters, like Robert Shone, should be batting on the same team as the Lancet authors and Opinion Business Research in highlighting the utter devastation inflicted on Iraq since 1991. But all I see from the IBC crew are attempts to defend lower estimates of the death toll - as horrific as that would be anyway - whilst apparently ignoring the slow and indirect death toll mounting as a result of US and UK policies against Iraq going back 20 years. Where is the rage agains those who are culpable for this destruction? Why does it seem to me that it is directed mostly against three studies whose estimates do not tally with their own?

Jeff Harvey:

As for Robert Shone, Josh D and other IBC supporters, how 'magnanimous' of them to argue that the death toll in Iraq as a result of the invasion might 'only' be 200,000.

Jeff puts the word "only" in quotes, but it's his word, not mine or IBC's. For me, the case against war is that just one death is too high. There's no "only" about it.

Robert,

Good. That's a start. Now we are getting somewhere. But I still have some questions for you:

Why did IBC not wholly condemn George Bush and Tony Blair when they quoted IBC figures in an attempt to 'downplay' the carnage? Both leaders knew that the Lancet studies had been published, and I suppose their advisers and government planners thought that the best way to 'manage the outrage' in PR terms was to (1) cite the much lower IBC figures, and (2) attribute most of the deaths to Iraqis.

I waited with baited breath for a rebuttal from IBC after their figures were cited by the two war criminals in chief, and yet (correct me if I am wrong) I saw and heard nothing.

I also would like to know how many people Robert thinks died as a result of the sanctions that former chief of humanitarian aid for the UN Denis Haliday described as 'genocidal'. His replacement after he resigned, Hans von Sponeck, also later resigned for the same reason. So Robert, combine the effects of (1) the first war, which targeted the civilian infrastructure, (2) the sanctions [1991-2003], and (3) the 2003 invasion, including the ongoing suffering and deaths it has caused, as well as the displacement of millions of people, and how many Iraqi civilians do you think have died and will die because of all 3? My guuess is that it will exceed a million by a long, long way.

Most importantly, why concentrate your ire on the Lancet studies, OBR, and (for those not aware of it here) Media lens? The latter are just exposing western media hypocrisy with respect to the war and other conflicts, where atrocities that result from our policies are downplayed or ignored and those from non-alinged regimes are endlessly analysed and condemned.

Jeff, the anti-IBC smear campaign based on Medialens's catalogue of errors was exposed a long time ago. You don't boost your credibility by reheating it at every opportunity:

http://tinyurl.com/medialens-errors

http://tinyurl.com/medialens-ibc

"I think that was based on their view that ILCS provided the most reliable estimate at the time. Since then IFHS was published, which seems to have confirmed their earlier-stated views. Incidentally, the first Lancet survey (2004) estimated 57,600 violent deaths - roughly three times the number that IBC counted over the same period."

The IFHS estimate is in almost perfect agreement with the first Lancet survey on violent deaths (taking the version where Fallujah is excluded). They both say about 100 violent deaths per day in the first 18 months. So the opposition to Lancet1 by IBC supporters in 2005-2006 now seems misplaced.

"I don't recall that he was ever willing to entertain that possibility here."

I don't either. I think I recall that Sloboda was willing to concede up to 3 or 4 times IBC, but JoshD said he thought that was way too generous. That might have been at medialens. Maybe the IFHS study changed his mind.

"At the end of the day the fact is that the Blairs and Kaplans have won this particular argument. There's no great incentive to conduct new studies. Most history books, insofar as they attempt to quantify Iraqi deaths at all, will contain the low numbers."

Not sure about the history books, but in the American press that's how it is. You've got a choice between "tens of thousands" or 100,000 deaths for the Iraq War. But the BBC report linked above was reasonably balanced, IMO.

Donald Johnson

The IFHS estimate is in almost perfect agreement with the first Lancet survey on violent deaths (taking the version where Fallujah is excluded).

Whereas the second Lancet survey estimates 450,000 more violent deaths than IFHS for the same period surveyed. Did Les Roberts ever retract his assertion that Lancet 1 and Lancet 2 supported each other?

Robert Shone:

> I guess you didn't spot the clearly-labelled "Comments" page, alongside the clearly-labelled "About/Contact" page.

I did, but all I saw on those pages are a contact form and your claim about how receptive you are to other people's comments unlike everyone else who's closed-minded. Given how open-minded you claim to be to others' feedback, it's a bit strange that your blog doesn't show, um, other people's comments.

So, what's with your weird comments 'policy' of banning comments everywhere while crowing about how receptive you are?

"Whereas the second Lancet survey estimates 450,000 more violent deaths than IFHS for the same period surveyed. Did Les Roberts ever retract his assertion that Lancet 1 and Lancet 2 supported each other?"

I'm not going to defend Les Roberts. It seems to me that the more extreme partisans on both sides were pretty good at emphasizing some facts and ignoring or deemphasizing others.

The more partisan Lancet supporters emphasized the fact that the total excess death toll for the first 18 months was in agreement (IIRC), but deemphasized the fact that Lancet2 attributed nearly all the excess deaths to violence, while Lancet1 had it roughly 50/50. Lancet critics praise the IFHS report without ever mentioning its close agreement with Lancet1 and without mentioning that they were harsh critics of that 57,000 violent death figure back in 2005-2006. Are there any Lancet critics, any at all, who will say that they were wrong to be so skeptical of the 100 violent deaths per day figure back when L1 came out?

Frank:

So, what's with your weird comments 'policy' of banning comments everywhere while crowing about how receptive you are?

Banning? I think you're confusing me with the website which you're clearly a disciple of (Medialens), since your falsehood about my not allowing comments originated with them (almost word for word). Here are some recent comments at my blog: http://dissident93.wordpress.com/2010/10/17/quotes-to-cogitate/

"e more partisan Lancet supporters emphasized the fact that the total excess death toll for the first 18 months was in agreement (IIRC), but deemphasized the fact that Lancet2 attributed nearly all the excess deaths to violence"

Actually, that came out a little harsher than I meant. If someone pointed out that the total excess death toll in L1 and L2 was in agreement while conceding that the two studies had a different breakdown for violent and nonviolent deaths and then pointed out that given the error bars, there was rough agreement, as far as I can tell that was the simple truth. But I've never seen an L2 critic say that they were wrong to come down so hard on L1, given that a completely different study supported L1's violent death toll.

Donald Johnson:

So the opposition to Lancet1 by IBC supporters in 2005-2006 now seems misplaced.

As I recall, this "opposition" was not to the L1 study itself (or to its 57,600 violent deaths estimate), but mostly to ill-informed claims surrounding it - eg that it showed IBC was out by a "factor of ten or more", or that IBC was the lowest figure of "eight" studies, etc.

Some of the more extreme anti-IBC campaigners framed the debate in way that made one think IBC was in "opposition" to L1 merely by existing. (They called for IBC to be "shut down").

Also, if you look at IBC's own press release on L1 you'll see no "opposition" to it at all: http://www.iraqbodycount.org/analysis/beyond/lancet100000/

Robert Shone:

I have here

1. a snapshot of your supposed blog post on other people's comments (written by yourself) which has no comments form, and where the comments you choose to publish happen to be from people you like;
2. a snapshot of your "Comments" page where you claim you're extremely open-minded and receptive to criticism, but which contains no comments from other people as well as no comments form; and
3. a snapshot of your blog's front page, which says "comments closed" everywhere.

Should we believe you, or should we believe our lying eyes?

-- frank

Thanks for the off-topic snapshots, Frank. On my comments page there's a contact email and link to contact form - either can be used. Wordpress-comments is deactivated (because it's rubbish), hence the automatic "comments closed" caption. http://dissident93.wordpress.com/comments/

Your attempts to smear me are interesting considering that your falsehood about my not allowing comments originated with Medialens (almost word for word). I recently invited Medialens (email, 18/10/10) to respond to my criticisms - even offered to include their comments in any form of their choosing. But they've chosen not to respond. See also:

http://dissident93.wordpress.com/2010/10/29/medialens-falsehood/

> Thanks for the off-topic snapshots, Frank.

How can it be off-topic when you bring up your strawman concern trolling, Bob (short for Kate?)?

> Your attempts to smear me are interesting

and successful. This is what happens when you're a lying concern troll. You have refused to allow comments.

Of course they've not responded. You're nobody but a blowhard.

> At the end of the day the fact is that the Blairs and Kaplans have won this particular argument. There's no great incentive to conduct new studies.

Because big companies made big money from the war. If, like climate change mitigation, there had been significant potential losses at some time in the near future, you can BET there'd be plenty more studies into the war and the lies spread about it.

> I speak for most of us down home folks

Seems appropriate for you:

> You've got to remember that these are just simple farmers. These are people of the land. The common clay of the new West. You know... morons.

Problem here is that you assume that everyone thinks like you therefore the reprehensible cannot be true, since YOU'RE ALL AMERICANS. And they're the GOOD guys, yeah?

But you're a small clique and you're not the good guys. Just because you can find someone who is worse than you doesn't mean you're not an incarnation of evil.

For anyone interested, I've uploaded a clipping of Science journal's coverage of the Wikileaks material (it's available only by subscription elsewhere). It mentions IBC, Lancet 2006, IFHS, etc.

http://dissident93.files.wordpress.com/2010/11/science-wikileaks.jpg

IBC was subtly denigrating about L1 in their 2005 two year summary--I won't look for the link. And at this blog and medialens, Joshd didn't give an inch to L1. The vitriolic response to L1 was based on the notion that there was no way that the true violent death toll could be several times higher than IBC's. IBC later upped their death toll for the period through Sept 2004 up to 19,000, I think, but when L1 was written it was around 15,000, and people compared that number to L1's 98,000 for all causes and said "no way". Where I'd acknowledge blame on the L1 side was that comment in the paper that most of the casualties came from US air strikes--which was only true if you took the Fallujah cluster as representative. That muddied the waters on all sides, because people would think that the widely cited 98,000 dead figure meant 98,000 dead mainly from US air strikes. But still, I never saw a single L1 critic who said "98,000 dead mostly from US air strikes is unbelievable, but the true violent death toll could in fact be several times greater than IBC's." JoshD spent his time here arguing that there was an unbridgeable gap between L1 and the UNDP report that gave 24,000 dead by March 2004.

Robert Shone:

> Thanks for the off-topic snapshots, Frank.

You asked people to reply to you on your blog, and yet when I popint out that you've disabled comments everywhere on your blog, suddenly it's 'off-topic'?

> Wordpress-comments is deactivated (because it's rubbish),

Umm, do you know that

1. wordpress.com has a facility for keeping comments in moderation, and
2. it also has a facility for automatically allowing comments from previously cleared commenters, and
3. it has blacklists for flagging comments for moderation or the spam bin, and
4. on top of that, its spam detection is actually quite good?

I actually used to host my blog at wordpress.com, and I never had any problem with keeping out spam, or keeping out trolls, or letting through informative comments.

Again, what's with your blog's comments policy of 'let's ban everything with extreme prejudice except on this one page but even then'? That's a pretty weird way to express your supposed open-mindedness to negative feedback, isn't it?

Honestly, have you ever tried customizing the comments policy to anything else? You might be pleasantly surprised.

* * *

Also, you said:

> your falsehood about my not allowing comments originated with them [Medialens] (almost word for word)

And now you're saying,

> considering that your falsehood about my not allowing comments originated with Medialens (almost word for word)

Hmm. Almost word for word. Projection much?

* * *

Wow nails it:

> This is what happens when you're a lying concern troll. You have refused to allow comments.

> Of course they [Medialens]'ve not responded. You're nobody but a blowhard.

Donald Johnson:

The vitriolic response to L1 was based on the notion that there was no way that the true violent death toll could be several times higher than IBC's

But that wasn't a response to L1. It was a response to the misinformed idea that L1 showed IBC was out by a factor of 5-10. Note the way in which the 2005 IBC report (which you describe as "subtly denigrating") correctly describes the violent death count of L1, which was around three times that of IBC's count for the period in question (17,687):

This produced a nation-wide estimate of 98,000 deaths between 19 March 2003 and mid-September 2004, of which 57,600 would have been due to violence and the remainder caused by accidents, infections and chronic diseases (based on the breakdown of causes seen in their reduced sample).

The 2005 IBC report continues:

A highly creditable and often overlooked aspect of the Lancet survey is that it systematically obtained information on post-invasion âexcess deathsâ from all causes, including the everyday deaths that donât make the
news. Such efforts should be joined. http://tinyurl.com/ibc-2005

I can't see anything at all in that report which remotely "denigrates" L1.

Yes, some of the debate among individuals on forums got heated, but you have to ask who started it all, and why. Where was the vitriol prior to Medialens's vitriolic accusations which began in early 2006? In fact there was no vitriol and no IBC-related accusations before Medialens started their malicious and ill-informed campaign.

I saved many of those old Medialens message-board debates to disk (the IBC-bashing ones, which went on for two months before Josh even posted a word there). IBC were accused of being "silent" for months. Silent vitriol? Those Medialens "discussions" make interesting reading. You're a regular participant in many of them, Donald - but I recall you were milder in your criticisms of IBC than the frothing-at-the-mouth Medialens horde were.

ATTN:wow

You've got to remember that these are just simple farmers. These are people of the land. The common clay of the new West. You know... morons.

Typical arrogant elitist city slicker. Modern farmers are not morons. If they were, they would not last long in the farming business.

You know squat about us Americans. As I have said, we don't care what the Old World people say or write about us.

Everybody bad-mouths America and the Americans until they emigrate.

Re: MediaLens, Robert says:

*But they've chosen not to respond*

Probably because they don't think that you are worth the effort. I can see their point.

As expected, you did not answer my last post at all but, as expected, laid into Media Lens. In contrast with your own arguments, I do not think in any way that you, Slobaoda or IBC have undermined the arguments they put forward. *You think* you have, but that is only your opinion. Many others would disagree.

My last point remains: IMHO you and IBC unleashed more of your apparent wrath on Media Lens and the Lancet studies than on the toll inflicted on the civilian population of Iraq by the sanctions and combined effects of two wars. Methinks this is because there were egos at stake, mostly yours and Sloboda's when the Lancet studies briefly toomk attention away from IBC. Like many others, I think David Edwards and David Cromwell at ML did a great job in their critiques of IBC and at the profound silence that ensued when the IBC results were used by the war party and its supporters to downplay the carnage in Iraq.

Like it or not, Robert, Media Lens is gaining in popularity because of its fantastic job in exposing media hypocrisy and selectivity and in the way it downplays western crimes.

Harold,

>"Everybody bad-mouths America and the Americans until they emigrate."

Very true. I find Americans who have had enough of their country and emigrated to be far more likeable, sensible, and easier to get along than those who remain there...

Jeff Harvey:

Probably because they [Medialens] don't think that you are worth the effort.

In fact they've put more effort into responding to me (a mere nobody with a blog) than in responding to the Daily Mail (a newspaper which spews lies to millions daily). And they've put more effort into smearing IBC than in criticising the whole of the rightwing press put together.

The last time they put a lot of effort into responding to me (they wrote a long-winded 'alert' about my blog), George Monbiot wrote this to them:

I have just read the response that Robert Shone makes to your latest Media Alert. I found it rather more persuasive than the alert, but I would say that, wouldn't I? One of the things I learnt from it is that Medialens does not allow him to post messages on this board. So here is another question for the editors: is this true, and if so why? Doubtless you would like to answer this question by issuing another Media Alert or, if you prefer, by taking out a full-page advert in the Times. Alternatively you could just explain it here.

Those who might wish to read what Robert Shone has to say, rather than simply dismissing or vilifying him, can do so here:

http://dissident93.wordpress.com/2008/12/05/full-reply-to-medialens/

With best wishes,

George

Source: http://tinyurl.com/monbiot-ml

They always put effort into responding when they imagine they can win the argument. They even spend a lot of time responding to the likes of Oliver Kamm. But they are unable to respond to (for example) my ZNet article listing their errors on IBC - because they realise that no amount of rhetoric will convince people that they weren't clearly and repeatedly in error. http://tinyurl.com/medialens-errors

Robert,

Wows description of you as a "Concern troll" strikes me as being accurate. You have invested a lot of energy (for some strange reason) in defending IBC and their clearly flawed estimates of civilian deaths in Iraq, responding repeatedly to Medialens, and much less effort in highlighting the fact that the death toll in Iraq has been horrendous and unneccesary, whatever it may be. What about the sanctions of mass destruction? The ongoing suffering? Why is your apparent rage so focused on those estimating higher death tolls than IBC, with its clearly flawed methodology?

You have failed, time and time again, to answer this simple question, which suggests, to me at least, that Wow`s definition of you is quite appropriate. At the same time, I commend the team at Medialens for highlighting what IMO is clear hypocrisy on the part of IBC, who seem more intent on defending their lower estimates of civilian deaths in Iraq from the higher L1, L2 and OBR estimates than on condemning the war party.

Sorry to say this Robert, but you and IBC lost credibility in the eyes of many, with the possible exception of the corporate MSM and those defending the invasion, a long time ago.

Thanks, Jeff. I refer you to the passage I quoted from George Monbiot:

Those who might wish to read what Robert Shone has to say, rather than simply dismissing or vilifying him, can do so here: http://dissident93.wordpress.com/2008/12/05/full-reply-to-medialens/ With best wishes, George

Source: http://tinyurl.com/monbiot-ml

I also refer you to IBC's rationale: http://www.iraqbodycount.org/about/rationale/

> Typical arrogant elitist city slicker

Typical response of an ignorant moron.

You are a minority of Americans. You project your poisonous attitude on others based merely on perception that if they live in the bondooks like you, they MUST think like you (because then your poison is no problem: "everyone does it").

Robert, I'd have to go back and reread the 2005 report, but that's what initially made me somewhat disgusted with IBC--up to that point I'd taken them seriously when they said that their methodology would only pick up a fraction of the deaths , but they seemed excessively attached to the wonders of their methodology, almost unwilling to admit it could be flawed. It left a bad taste in my mouth.

JoshD here never acknowledged that the 57,000 number was reasonable--he tried to show it was inconsistent with the UNDP report. I remember seeing a news report where Sloboda admitted a number as high as 4 times IBC was possible. If IBC and its representatives had consistently stuck to that position, personally I wouldn't have had much of a problem with them.

As for the medialens people, yeah, you often get overheated arguments online. It goes with the territory. But JoshD was partly to blame there. And anyway, that's a side issue, important though it might seem to those of us who get a bit too involved in these online catfights. What was important in the bigger political picture was that people like Kaplan and others thought L1 was totally out of bounds--it certainly wasn't clear to the mainstream press (in the US at least) and rightwingers anywhere that L1's number was in fact a perfectly reasonable estimate, not falsified at all by IBC's count.

And we've got people like Michael O'Hanlon saying as though it were confirmed fact that Saddam caused far more deaths in the 80's than the Iraq War has. That's what happens when estimates (or even guesstimates) of deaths are accepted for the crimes of our enemies, but conservative bodycounts are used for our own crimes.

Donald Johnson:

I'd have to go back and reread the 2005 report, but that's what initially made me somewhat disgusted with IBC

Perhaps you should go back and reread it, because some of your claims about it (eg that it "denigrates" L1) are inconsistent with what it actually says. http://tinyurl.com/ibc-2005

So Robert, when did the opinions of George Monbiot become the 'bottome line' when estimating the civilian death toll in Iraq?

Do not get me wrong, I admire many of George's views, but in this one it is my contention that he is wrong.

I think the views of Patrick Ball sum it up well as described by Medialens in one of their alerts:

*Consider that a study of deaths in Guatemala from 1960 to 1996 by Patrick Ball et al at the University of California, Berkeley (1999) found that numbers of murders reported by the media in fact decreased as violence increased. Ball described the âproblem of relying on the journalistic recordâ in evaluating numbers killed*

Bingo. And, as I have said before, what struck me was that the mainstream media and war party increasingly used IBC figures to estimate civilian deaths in Iraq only aftger the Lancet, OBR et al began suggesting that the toll was a lot higher. At the same time, I saw little evidence at the time of frustration in the IBC camp that their figures were actually being used to *downplay* the carnage by those who were either involved in the invasion (e.g. US and UK governments) or had provided support (the media). Instead, these abuses of IBC figures were, from my pewrspective, greeted with resounding silence from the IBC. Why was this? IMO it stemmed from the fact that they were being given media coverage, irrespective of the motives, and they liked it. Had they vigorously responded by condemining the abuse of their data to support the invasion, I wouyld have been more sympathetic, but as far I can tell they did not.

Bear in mind that IBC came in from the cold only after the first lancet study came out. Until then, IBC had been attacked for estimating the civilian death toll in Iraq by those who supported tyhe invasion. But after Lancet 1 was published, everythign suddenly changed and public relations became the name of the game. In PR terms, the most important aspect has always been to "manange the outrage". I am sure that government planners and advisers, in using perception management techniques, realized that death tolls above some threshold affect public opinion in the west much more than lower rates. At the time, IBC figures seemed high, but L1 and especially L2 figures suggested that the west was cuplable in crimes that were much worse than even the IBC figures suggested. This may explain why I saw IBC shift from from a pariah to a reputable NGO in the space of only a few months.

Another point I would like to make is that I always wonder why there is such debate over the death tolls occurring as a result of western crimes, but that these debates rarely crop up when the victims are those of 'offically designated enemies'. For instance, the number of victims of Saddam's regime has led to a range of estimates, with some politicians and pundits often coming up with astronomically high numbers (e.g. millions) if it suited their agenda. In spite of this, I saw little in the way of denial from the westerm media and governments when these estimates were made; heck, it was the last Labour government that publicly made some of these estimates. But when we are the perpetratoes, the carnage is generally downplayed or ignored using whatever means to do so.

> But when we are the perpetratoes, the carnage is generally downplayed or ignored using whatever means to do so.

German officers were executed for the crime of waging an aggressive war.

When the victors of that are the victors this time, no such trial.

And George still hasn't made up for his travesty with the CRU emails and the unwarranted attack on the CRU (IMO fueled by his desire for seeing the FOIA remain untarnished), so it's not like George is a darling.

However, as you point out, nothing about what George has done makes Kate's refusal to allow comments on his forum any less hypocritical in face of his continued "concern" for the lack of openness on "teh establishment".

I think my memory of the 2005 IBC report has blended with a later one where they did a direct attack on L1 (contrasting it with the UNDP report). But I was disgusted by the 2005 report because it seemed to me they took their numbers too seriously and did not sufficiently emphasize both how low they were likely to be and also how biased the breakdown of deaths was likely to be given their reliance on reporters. For instance, there's a table showing month by month who was killing civilians and except in the first two months and during the two assaults on Fallujah, the coalition is shown as killing less than one civilian per day or around 20 per month, sometimes less, sometimes more. I'm not going to look it up, but according to numbers in the Brookings Institute (and from what I remember elsewhere), the US claimed it was killing hundreds of "insurgents" per month. Now we know from some cases (Haditha) that some civilians reported killed by insurgents were killed by the US and we know that sometimes civilians killed by the US were reported as insurgent (the Wikileaks film some months back, for example) and anyway, all past experience with guerilla wars shows that the occupying armies invariably lie about both who dies and who did the killing--in Vietnam there was. There were estimates that 70-90 percent of the Iraqis in Abu Ghraib were not insurgents. It would have been nice if IBC had strongly emphasized that one of the drawbacks to relying on media reports is that often reporters (especially embedded ones) merely pass on what they are told because they're in no position to do otherwise and in some cases don't wish to (for fear of appearing unpatriotic). Here, by the way, is an article related to this about Vietnam--

link

I largely agree with Jeff's 8:35 post. In fact, just before the latest Wikileaks documents came out, the NYT was actually going back to numbers that were lower than IBC's--the Iraqi government had put out somewhat lower figures and one commonly read "tens of thousands" for the number of Iraqi deaths. IBC did become the go-to place for the US press after L1 came out and I expected them to come out emphasizing the fact that even their maximum number might be several times too low and that the breakdown of who killed whom was also likely to be unreliable (due in part to the fact that it would be difficult to prove that some "insurgent" the US killed was really a civilian).

Donald Johnson:

I think my memory of the 2005 IBC report has blended with a later one where they did a direct attack on L1 (contrasting it with the UNDP report).

A "direct attack" or "subtly denigrating"? Make your mind up. Which report are you talking about, exactly? I don't recall any IBC report making a "direct attack" on L1.

Maybe more than one incident is being recalled, Kate...

For instance, the number of victims of Saddam's regime has led to a range of estimates, with some politicians and pundits often coming up with astronomically high numbers (e.g. millions) if it suited their agenda.

Of course, these estimates always include military victims of the Iran-Iraq war. The IBC leaves out military victims of the US-UK-Australian invasion.

Here's an attack on L1 by IBC. Most of JoshD's own posts pre L2 were attacks on L1 and he's one of the authors of the report I link here.

link

I don't really get what your stand is, Robert. Each time you respond to me you single out some particular point that you think is wrong and ignore the parts of my posts that I think matter, as though all that matters is who was right and who was wrong in the stupid medialens arguments. I think both sides were partly right and partly wrong. I got too heated myself sometimes. Does this mean you agree that I have legitimate points otherwise? I mean, it was obvious to anyone who paid attention in 2004-2006 that L1 was attacked by critics who said that the true death toll was much lower than what L1 claimed, citing IBC among others. IBC itself (or the members) seemed to go back and forth. JoshD in the link above is trying to show (as he did here) that the ILCS study was inconsistent with L1. People here The most important point, it seems to me, is what I've said several times without hearing you agree or disagree--there's fairly good reason to think the true death toll is a few times higher than IBC's number, but one simply doesn't see that conveyed in the American press.

IBC and the Lancet team aren't the important issue here, or they shouldn't be. The important issue is about intellectual and moral consistency--the press should fairly describe the methods used to derive all casualty estimates, and not use conservative bodycount methods for US-caused atrocities and methods that produce larger numbers for enemy atrocities. Which I've said several times now, to the point where even I'm bored with repeating myself.

"People here"

I got distracted and forgot to finish the sentence. People here will be familiar with the attempt in that IBC link to establish a gap between ILCS and the L1 estimate. If I recall correctly, it later turned out that the ILCS estimate didn't include some of the most violent areas in Iraq, because they were too dangerous for the interviewers.

"I recall correctly, it later turned out that the ILCS estimate didn't include some of the most violent areas in Iraq, because they were too dangerous for the interviewers."

I didn't mean to print that--I wanted to have a link for it, but so far my googling for one hasn't succeeded.

Donald Johnson:

Here's an attack on L1 by IBC.

That's not an attack on L1. It contains criticisms of things Les Roberts said outside L1, and some mentions of L1 (in similar terms to the 2005 report, which you now seem to admit wasn't "denigrating" or an "attack" - unless your memory's still playing tricks).

I don't really get what your stand is, Robert.

My "stand" is that I think you should get the facts right.

Robert,

It is you who ought to get his facts straight. Here is John Sloboda speaking in 2006:

*Some critics of the Lancet study have said it's like a drunk throwing a dart at a dartboard. It's going to go somewhere, but who knows if that number is the bulls eye.
Unfortunately many many people have decided to accept that that 98,000 figure is the truth - or the best approximation to the truth that we have*

and

*We've always said our work is an undercount, you can't possibly expect that a media-based analysis will get all the deaths. Our best estimate is that we've got about half the deaths that are out there*

If this isn't an 'attack' on L1, I do not know what is. This was also happening before the Media Lens furore. It seems to me that Sloboda and IBC were annoyed that a study in a top, peer-reviewed journal by experts in the field produced results that greatly exceedded their own and that they were more concerned about the credibility of IBC than of uncovering the true civilian death toll.

The IBC team also took exception to the fact that they were called 'amateurs', which seems surprising to me given that it seems that none of them are trained in the way that Roberts, Burnham and others are. If the shoe fits, wear it.

Most annoyingly, and this is a point you continually ignore, IBC never appeared to respond by the use of their database by the media or the US and UK governments to downplay the death toll in Iraq. If my data was being used to support something that I was strongly against, I would say so. IBC figures became 'accepted' pretty much at face value only after the L1 and especially L2 studies were published. As I have said several times, the western media, most of which were complicit as accessories in supporting the invasion based on lies and disinformation, were anxious to manage public percpetions by downplaying the death toll or by attributing most of the violence to Iraqis. This is when the IBC figures became the rule simply because their estimates were profoundly lower than those from L1, L2 and OBR. I was waiting for IBC to bitterly denounce abuse of their data by the war party but, as far as I can see, it never happened. Correct me if I am wrong. Instead, they spent more time trying (and failing IMO) to defend their figures which it must have been apparent to many anyway seriously underestimated the carnage.

Robert has little in the way of defence against these points. As I have said before, I salute Media Lens for exposing western hypocrisy, as well as that of the IBC. They have done a great job.

Jeff Harvey:

Here is John Sloboda speaking in 2006:

Yes, speaking in an interview. There are no "direct attacks" on L1 in any IBC report.

This was also happening before the Media Lens furore.

Let's see how many examples you can provide to support that falsehood.

Way to avoid the point by closing your eyes, Kate...

Robert,

In the interview he specifically addresses the 98,000 estimate from L1 - and then makes a dumb remark about critics saying the the figure being like throwing a dart at a dart board, which is patently false, if one knows anything about statistics and confidence intervals.

Moreover, where was your indignation over the media smears of Roberts, Burnham and others when they were desperately trying to downplay the death toll? I think IBC gets off lightly by comparison. And for the 'nth' time, please explain to me why you think IBC was silent when Bush, Blair and the corporate media were using their mortality figures to downplay the death toll, at least when juxtaposed with L1, L2 and OBR. No, instead of responding to those abusing their estimates to downplay the civilian toll, IBC were writing lengthy rebuttals of the much higher estimates measured in L1 and L2 and also in refuting criticisms from ML. Methinks their frustration was misplaced, and it was at this time that I began to be much more critical of IBC.

Given that you avoid the most important comments by myself and others here, Robert, its clear that you have nix left to say. No wonder ML refuses to respond to you. They have bigger issues to address.

Jeff Harvey, your posts here typically consist of you giving lectures to people about what they should and shouldn't feel "indignation" or "rage" over. It's very repetitive and usually not very on-topic. Meanwhile you consistently show no regard for facts. And why do you keep referring to the ORB poll as "OBR"? Can you not demonstrate at least a minimum amount of care over these matters of detail while you lecture people on what they must think?

Kate, can you stop with the sockpuppets.

Ta.

> Typical arrogant elitist city slicker

Actually, born in a rural, heavily agricultural town. Raised in another rural agribusiness town.

Dumbass.

Of course it was an attack on L1, Robert. There was a context here. L1 was under heavy attack by defenders of the Iraq War and IBC comes out with a report that tries to establish that L1's number (the 57,000 figure) was probably too high. And JoshD was arguing that case much more forcefully here and other places.

"My "stand" is that I think you should get the facts right."

That's a weirdly hostile and evasive response. I tried to see if we had some beliefs in common, but you won't say, for some reason.

Donald Johnson:

Of course it was an attack on L1, Robert.

Your assertions owe more to myth than to fact. I've searched every reference in it to L1 - there are no "attacks" on L1. The closest thing I could find to a so-called "attack" was the following:

The ILCS survey is superior to the Lancetâs on sample size, geographical distribution of samples, and number of deaths recorded.

So, IBC express a preference for ILCS, but they certainly don't "attack" or dismiss L1. In fact they go out of their way to emphasise that L1's findings are important and shouldn't be dismissed (eg see abstract and page 6 - they also did this in their 2004 press release and 2005 report). As for the "57,000 figure", here are the two references to it (excluding footnotes & appendix):

But data from the Lancet study itself shows that only a third of 57,600 violent deaths were due to criminal activity, leaving 38,400 combat-related violent deaths. A later re-analysis of Lancet data by the Small Arms Survey placed this figure at 39,000.

[...]

Of the 98,000 Lancet-estimated deaths applicable to the entire country outside Falluja, 57,600 were violent. Forty-three percent of the violent deaths were caused by US forces, 67 percent of them by air strikes.

The first was made in the context of comparing L1 to ILCS; the second in a comparison of US-caused deaths between IBC and L1. Neither of these contexts contains anything remotely resembling an "attack" on L1.

There was a context here. L1 was under heavy attack by defenders of the Iraq War...

You can invoke "context" all you like, but it doesn't change what's in the report.

You said earlier that your memory had "blended" two different reports - that this explained why your assertion about the 2005 IBC report was incorrect. I conclude from the above that your memory has blended fact and myth regarding the 2006 IBC report. Mostly myth, I think.

Or perhaps, like the misinformed at Medialens, you saw some of the criticisms of misleading statements made by Les Roberts about L1, from outside L1, and formed the incorrect impression that these were "attacks" on L1.

As Josh Dougherty commented in a Deltoid thread which discussed the above IBC report at length:

You say that Lancet reports five times as many deaths as IBC. This is not true. Roberts' own comparisons (excluding deaths from accidents or disease, which are the comparisons used in our own paper) show that the real difference is closer to three times. More generally, IBC is not "taking on the Lancet study" but various misinterpretations of it. http://scienceblogs.com/deltoid/2006/04/ibc_takes_on_the_lancet_study.p…

SteveK,

Go jump in the lake.

My posts were aimed at highlighting the apparent hypocrisy of IBC, which appeared to be much more critical of L1 and L2 but gave the war party a free pass when their own figures were (ab)used by the said war party to downplay the civilian toll in Iraq. And, if you understood even the basics of the PR industry going back to the time of Edward Bernays (ever heard of him?), then you'd understand that the main aim of PR has always been to "manage the outrage". Propaganda with respect to the initial war, which targeted the civilian infratructure of Iraq, the subsequent sanctions, which left more than 500,000 dead in their wake, and lastly the Iraq invasion and destruction of the country, was always aimed at western public opinion, and not anyone else, certainly not public opinion in Iraq.

As for ORB/OBR mistake, here are my humblest apologies for making such a profoundly important error.

> "My "stand" is that I think you should get the facts right."

> That's a weirdly hostile and evasive response.

Also rather hypocritical from someone who continues to avoid the facts of the death toll and IBC's under-reporting bias.

One would hope that someone demanding others get the facts right would first get their facts right or say they don't know the facts.

A new and comprehensive debunking of IBC estimates of the civilian toll in Iraq has been posted on the latest Media Lens alert. Its a great read.

http://www.medialens.org/alerts/

More on surveys of violent war deaths

tlambert — Tue, 29 Dec 2009 03:44:12 +0000

Tags

Do you have any thoughts on the recent Burke et al. paper Warming increases the risk of civil war in Africa. It claims to have found a small but robust correlation between war-deaths recorded for sub-Saharan Africa in the PRIO dataset and country-scale temperature increases in the period 1981-2002 and says that 'this historical response to temperature suggests a roughly 54% increase in armed conflict incidence by 2030, or an additional 393,000 battle deaths if future wars are as deadly as recent wars.'

I'm no statistician but ...

The period under review neatly straddles the end of the Cold War, a continent-wide 'external forcing' that the study totally ignores. Here's an annual tally of African conflicts listed in the PRIO database. Is there a clear post-Cold War jump? Perhaps not. But I bet the apparent jump is more 'robust' than any correlation with temperature.

I haven't had time to read both papers yet, but this looks really interesting. There are some thoughtful comments accompanying [another Gelman article at Statistical Modeling, Causal Inference, and Social Science](http://www.stat.columbia.edu/~cook/movabletype/archives/2009/12/update_…).

The PRIO battle death data looks fascinating. The Obermeyer paper says that the PRIO "compiles and reconciles
passive reports on violent deaths related to war globally
from 1900 onward," but that doesn't appear to be at all an accurate description, based on the [PRIO data's documentation](http://www.prio.no/sptrans/1555324504/PRIObd3.0_documentation.pdf).

"The regression that used . . . " maybe s/b "The regression that they used . . . "?

Thanks for keeping this awful stuff in the public eye.

A recent paper from Spagat, Johnson, et al (on conflict deaths in Iraq, Afhganistan & Columbia) made the cover story of Nature last week:

http://www.nature.com/nature/journal/v462/n7275/covers/index.html
http://dissident93.files.wordpress.com/2009/12/gourley_nature_cover.jpg

Knock on wood, but in Big Lebowski terms, it seems our Donnie on this issue has stopped wandering in and saying "I am the Walrus."

However, it also looks like the registration system at pharyngula is causing the insane to spill out all over scienceblogs, the way the crazed homeless went out after Reagan closed the mental health facilities in California.

Tim

There is indeed evidence that people misremember past deaths. [See](http://www.ingentaconnect.com/content/apl/spub/2003/00000031/A062s062/a…)

Like you, I found it odd that individuals might misremember the death of siblings. But remember that the WHO surveys have recall periods going back 40 odd years. (The SMART protocol states flatly that recall periods should not exceed ONE year because of concerns about recall bias.) Remember too that many of the individuals who were asked about the violent deaths of siblings may well have been children when the wars were being waged. It wouldn't be so surprising if they misremembered the death of an infant brother or sister.

I should note though that the recall bias issue is peripheral to our critique of Obermeyer et al.

On population health surveys and excess death estimate more generally, the Human Security Report Project has a study due out on January 20th which examines the use of surveys to examine all excess war deathsââthose from disease and malnutrition as well as violence. It makes the case that, while such surveys are a vitally important source of data in war-affected poor countries, major challenges arise when they are used to estimate excess deaths.

The challenges lie not so much with surveys themselves, but with how to figure out the baseline mortality rate. Here the need is for pre-war trend data, not just a point estimate. The counterfactual 'what would have happened had there been no war' has to be taken into account in estimating excess deaths, but this requires understanding not just the mortality rate in the immediate pre-war period but the trend in mortality over several years.

Peacetime mortality rates are declining in almost all poor countries where most wars are waged today and if this isn't taken into account in estimating excess mortality, the death toll will be underestimated.

But, as our report points out, the practical challenges of "measuring from the slope" of a declining (or increasing) mortality trend line are almost insurmountable.

Census data can sometimes be used to estimate excess deaths from all causes, but incident reporting of the sort undertaken by PRIO and Uppsala can't be. There is no way of determining from reported deaths whether those who succumb to disease and malnutrition in wartime wouldn't have died anyway had there been no war.

Andrew

You folks sure like to think about who died... why did they die... should the memories of survivors count? Right through the New Year, you guys keep thinking for the unwashed, bloody sheep... Help us please with the wounded numbers in all these and more conflicts, would you please? The wounded divided by the dead. In most wars the ratio runs from 3-5 for each person killed. Where can I find these numbers please? In Iraq? In Gaza? In all areas of the world I have watched as the number of wounded has shot up over the years. In "Intifada"(to shake off), 1 & 2; this ratio got to well over twenty to one. To me this says that people are being wounded for sport by soldiers. Is this a war crime if proven to be true? It is not because of improved medical care. In most cases this only applies to troops in the area of operations. Not civilians. So please, do not bring up this as a red-herring. Also why don't the type of wounds count in your reports. How may shot in the hand? Arm? Head? Center of mass? Leg? You all surly see how important this data will be, we all can... Where can I get that info? Thanks to all.

Gelman writes:

We last heard from Dr. Spagat when he shot down the notorious article in the Lancet by Burnham et al. that estimated post-invasion deaths in Iraq using a sample survey.

Tim: Would you agree with Gelman's assessment? If not, you should explain where he goes wrong. I, of course, agree with Gelman that Burnham's article was "notorious" and that Spagat succeeded in shooting it down.

Tim's claim that you'll always know of an overestimate because of one obvious cluster seems like a baseless claim, to put it mildly.

Tom

I don't think there any good answers to your question.

Research organisations like the International Peace Research Institute in Oslo and Uppsala University's Conflict data program don't try and collect injury data because they believe the reports are just too unreliable. Getting good data on fatalities is hard enough. Plus what constitutes an "injury" can vary dramatically from place to place. A minor wound that didn't prevent an individual continuing in a combat role might counted as an injury in some contexts -- not in others. But there is little room for ambiguity when someone is killed.

One of the few serious studies we know of notes this:

"The ratio of the number of people wounded to the number killed ranged from 1.9 to 27.8. Two additional articles gave the proportions of people wounded who eventually died in major conflicts since 1940, without giving absolute numbers. Total deaths were never more than 26% of all casualties, a wounded to killed ratio of 2.8."

This is from a lit. review in 1999 by Robin Coupland and David Meddings, both then at UNHCR. See:http://www.ncbi.nlm.nih.gov/pmc/articles/PMC28193/

I think that there is actually quite a lot of evidence that the ratio of injuries to deaths is much higher in modern armed forces like those of the US because excellent first aid facilities prevent soldiers from dying who would certainly have died in earlier periods of warfare.

In most poor country wars people bleed to death from what would otherwise be non-fatal wounds because there are no medics to stop the bleeding, or medevacs to get them quickly to a field hospital. So here the ratio of deaths to injuries is low.

I think quick access to medical services would likely also explain the Palestine case where political violence takes place in urban settingsââas against remote jungles as is the case in many conflicts in the developing world. Civilians get treated as well as fighters here. Moreover Palestinians have health services that are far superior than those in most wars in Africa and elsewhereââand lots of practice in using those services to treat victims of violence.

Andrew

David Kane: "I, of course, agree with Gelman that Burnham's article was "notorious" and that Spagat succeeded in shooting it down."

Here's what appears to me to be the most relevant quote from Gelman at the link: "P.P.S. A reporter called me about this stuff a couple months ago, but I'm embarrassed to say that I offered nothing conclusive, beyond the statement that these studies are hard to do, and for some reason it's often hard to get information from survey organizations about what goes on within primary sampling units."

I feel this shows David Kane to be a person of dubious character and that Andrew Gelman may be prone to loose language on occasion. I've no doubt whatsoever that Kane can take Gelman's comments far further than Gelman has and far further than he would want them taken. That's what people like David Kane do, the devil finding work for those with idle minds etc etc.

Anon, I don't think it's fair to criticize David, when he's quoting Gelman accurately. But I think you're probably right about Gelman, since his "shot down" link goes to a [March 2008 post](http://www.stat.columbia.edu/~cook/movabletype/archives/2008/03/ethical…) in which he admits that he hadn't read Spagat's paper. Presumably he's read it since then, and agrees with it... but if so, he ought to have said something to that effect.

DK: "Would you agree with Gelman's assessment? If not, you should explain where he goes wrong"

I've already explained, in detail, where Spagat goes wrong. One of the posts is even linked in my post above. Gelman has not offered any assessment of the Lancet studies.

Alas, we didn't make it through the movie before the child started speaking.

I recall Les Roberts, Gilbert Burnham and Medialens (who were, IMO, Roberts's inept proxy for attacking Iraq Body Count) touting the false claim (wrt the Obermeyer study) that:

"But a study of 13 war-affected countries presented at a recent Harvard conference found that more than 80 percent of violent deaths in conflicts go unreported by the press and governments."

http://web.mit.edu/humancostiraq/further-reading/burnham-roberts2007.ht…

http://tinyurl.com/yfdh3l2

http://www.medialens.org/forum/viewtopic.php?t=2613

Robert,

Medialens were the "inept proxy" in your frankly inept opinion. I think Medialens did a great job explaining exactly why the mainstream (= corporate) media downplayed studies reporting the horrific toll of the illegal invasion of Iraq; had the Lancet studies been based on body counts as a result of the military activities of "officially designated enemies", and had these had been juxtaposed with those of Iraq Body Count, then I am sure the media would have focused on the larger toll for purely propaganda purposes. The bottom line is that the western media constantly downplay or ignore crimes committed by 'us' and our proxies whilst focusing laser like on crimes committed by official enemies. This is hardly suprising, since the msm is generally owned by commerical elites who clearly see the media as a means of promoting elite explanantions.

The IBC toll of the carnage in Iraq - almost certainly way too low - was only given full atterntion by the msm when the Lancet studies were published. The reason for this should have been obvious; in public relations terms the media were desperate to 'manage the outrage' that may have resulted had it been made clear that the real death toll of the Iraq war was many hundreds of thousands or even more than a million. Suddenly IBC came in from the cold because 30,000 deaths sounded 'acceptable' whereas 600,000 plus did not. What I found most annoying about IBC was that, when Bush and Blair began to downplay the death toll in Iraq using IBC figures, IBCs response to this abuse was apparent silence. They should have spoken up and argued that the war was illegal and that the death toll was horrific and that the war parties should not be using their own figures to defend the war. But I did not see that happen. The war party was allowed to abuse IBC figures to downplay the human toll of the war. The media went along for the ride.

I appreciate the enrormous efforts made by David Edwards and David Cromwell and Medialens to expose the media's cupability as 'vital cogs in the machinery of industrial western killing' as well as the hypocrisy of the IBC. As far as I am concerned, Robert, you should be expending more effort in exploring the real reasons for US military adventures and occupations in Asia and the Persian Gulf and its human toll, instead of attacking those who expose western actions and the actual motivations behind them (e.g. imperial expansion and efforts to prevent the rise of 'near-peers'). For that I salute Medialens, Pepe Escobar, William Engdahl, Paul Street, Tom Engelhardt, Andrew Bacevich, Noam Chomsky, John Pilger, Robert Fisk, and others who are saying it like it is.

Thanks for the lecture, Jeff. I too "salute" Noam Chomsky and Robert Fisk (or rather I greatly admire their work) but, unlike Medialens, they're generally careful with the facts, and they don't conduct smear campaigns.

If you're interested in why I find Medialens inept on this topic (I doubt, somehow, that you are), see my ZNet article which lists their many errors on IBC:

http://www.zmag.org/znet/viewArticle/22309

See also the following piece on their heavy-handed censorship, which also mentions how ZNet recently refused to publish their attempt to smear George Monbiot:

http://dissident93.wordpress.com/2010/01/04/medialens-embarrassing-arch…

Robert,

Methinks you are resorting to the word "smear" too easily here. Medialens were annoyed with IBC for the same reason that I alluded to above. IBC appeared unconcerned that their estimates of deaths in Iraq were being (ab)used by members of the war party to downplay the carnage. Bush was asked what he thought the death toll of the Iraq invasion was (back in December 2005 or 2006 I think it was) and he responded with the 30,000 estimate that appeared to come straight from the pages of IBC. The next day I expected those at IBC to respond angrily, accusing Bush of misinterpreting the intentions of their survey. Did they? Did they hell. Their only response was silence, as if they were pleased to see the msm and western leaders who waged or supported the war citing their work. Their silence spoke volumes in my opinion.

You also appear unconcerned at the much more egregious smear campaign that was launched at Roberts, Burnham, Gilbert et al. after the publication of their Lancet studies. Given the results of these studies (subsequently supported by the Opinion Business Research survey) apparently showed western culpability in mass murder, a finding which contradicts the carefully cultivated image of our basic benevolence, it was hardly surprising that the corporate media did everything it could to marginalize them.

But I digress. As I said above, given the underlying actual causes of the conflicts in Iraq and Af-Pak, I am bemused as to why you expend such efforets at defending IBC. Fiddling while Rome burns is my take on it. As for Medialens, I admire their tireless efforts to expose the msm for what it is: subservient to commerical elites and powerful vested interests. Their two books, "Guardians of Power" and "Newspeak in the 21st Century" are outstanding reads. They have criticized George Monbiot in the past for some of his views, and good for them. I admire Monbiot but the so-called "liberal" media, which is hardly liberal as far as I am concerned, needs to be held to account. We know where the right wing press stands, but as David Edwards and David Cromwell have shown, it is the allegedly liberal press which appears equally beholden to power. Along with the likes of Robert McChesney, Richard Falk, Fairness in Accuracy and Reporting, Democracy Now and other outlets, it is good to see those media sources apparently championing liberal causes to be held to account.

"Apparent silence", "appeared unconcerned", "fiddling while Rome burns" - the way Jeff Harvey frames his "criticisms" of IBC reminds me of Frank Luntz's manual - or perhaps of witch-hunts from earlier times.

Yes, of course, one has to be really "unconcerned" to devote nearly seven years of one's life, as a volunteer, to corroborating and documenting violent deaths. It takes so much more concern and integrity to lecture people on web forums about how they should spend their time, and on what premises they must accept to avoid the accusation of "being silent". Of course, of course.

Robert,

Your rather shallow posts shed a considerable amount of light on the way that you think. All of the gibberish you wrote in post 19 could be equally (if not more so) applied to people like Les Roberts and Gilbert Burhnam who are both professional epidemiologists. Yet I have never detected a scintilla of concern from you over the way these scholars have been treated and dragged through the mud. Have they not dedicated their lives to uncovering the scale of the carnage in Iraq? Where are your sympathies for them? After all, the IBC investigators are not professional epidemiologists but (as far as I can tell) have careers in other fields like the arts.

The IBC team has not, as far as I have seen, responded to the clear abuse of their 'estimates' by defenders of the war party, including those who carried it out (e.g. Bush and Blair) nor of the intent of those in the media who routinely cite IBC estimates as 'facts' whilst downplaying or ignoring other studies that have produced much higher estimates of death. Care to speculate why the IBC team have remained silent? Or why the corporate/state msm often treat IBC estimates as reliable while ignoring or attacking the Lancet findings? I have my own suspicions. In fact, it should be patently obvious. But I have yet to see you discuss this rather important point on your web cite or elsewhere. Why not, Robert?

In my opinion you are full of hot air. Lots of bluster but little substance. And especially little when it concerns the Anglo-American *modus operandi* or *casus belli* with respect to the illusory 'war on terror' or of the consequences of our ongoing military and economic policies in the Middle East and Caucasus region. Its just the same 'attack Medialens for daring to criticize IBC' refrain.

To give just one example of BBC bias, today their is an on-line discussion asking the question:

"Can the UK and US prevent extremism in Yemen?"

Again, all of the usual assumptions are packed into this one question. First of all, it assumes that the intentions of the US and UK governments in the region are purely benign; second, that our governments are anything but extreme in the policies they pursue. In other words, western policies are always packaged by the msm into heart-rending tales of concern for human rights and freedom, irrespective of the actual facts on the ground. Crimes committed by the west are gnerally ignored, or forgotten. The real agendas are seldom, if ever, disclosed. This is the kind of media hypocrisy that Medialens challenges on its web site. And I for one think that they do a fantastic job.

Jeff writes:

Yet I have never detected a scintilla of concern from you over the way these scholars have been treated and dragged through the mud.

That shows only that you haven't looked very far. I once highlighted the contrast in BBC coverage between the Tsunami deaths (which BBC covered daily) and Iraqi deaths (eg the Lancet 2004 estimate, which they headlined on one day, but then forgot about). In fact, Medialens (no less) once used my material (correspondence with the head of BBC news) in one of their alerts. I've also defended Les Roberts in the past from attacks by rightwing pundits. (That was before Les Roberts ran for Congress, before he started attacking IBC via Medialens, and before he said that the shock-and-awe campaign was "very careful").

Not that I should have to defend myself against accusations of "appearing silent" on issues of your choosing. When witchfinder-generals such as yourself start to accuse people of "appearing silent", you might want to take a look at yourself. (Incidentally, I can tell from your ignorant remarks about IBC that you haven't spent much time looking at what they've written. Clue: they have addressed Medialens's baseless accusations in full. But Medialens have "appeared silent" on that, which is possibly why you don't seem to know about it).

Robert,

You seem singularly obsessed over this IBC-Medialens issue. Get a life, pal. There's far more important issues to address in this world. It seems that you have dedicated yourself to the IBC cause. Are they not intelligent enough to take care of themselves?

I know that IBC have responded to Medialens criticisms. I just do not know why they have not expended so much effort to counter msm articles which cite their estimates for the death toll in Iraq as a means of downplaying the results of other studies that produce far higher estimates. Curious, that one. Or perhaps not.

Do you not think you are making a mountain out of a molehill? What about all of the other material medialens covers? Or does this IBC thing stick in your craw for some reason?

Jeff writes:

Do you not think you are making a mountain out of a molehill?

Well, I'm responding to accusations. I guess it depends on whether you consider your own (actually Medialens's) accusations to be mountains or molehills.

shorter Robert Shone:

Comment by Robert Shone blocked. [unkill]â[show comment]

funnier Marion Delgado:

http://fascistoar.blogspot.com/

Ater having read both the OMG paper and the Spagat/Mack/Cooper/Kreutz (SMCK) paper, I'd say that the SMCK paper makes a convincing case that there are significant flaws in OMG. Previously, I didn't understand at all how OMG arrived at their conclusion that war deaths in the most recent period were higher than they were in the 1965-1974 reporting period. After reading SMCK, I understand how OMG came to that conclusion, and I think their method was just plain wrong.

Like Tim, however, I have a few minor misgivings. Pointing to the [SMART Protocol](http://www.smartindicators.org/SMART_Protocol_01-27-05.pdf) as though it were some sort of definitive authority on what should and should not be allowed in mortality surveys strikes me as just plain silly. The introduction to the SMART Protocol states that "This manual is designed to be used in conjunction with the accompanying software. NUTRISURVEY for SMART which is free from www.nutrisurvey.de/ena/ena.html." I don't see why the software manual for a nutritional survey should be regarded as gospel for demographers who are trying to assess war casualties.

In general, yes, longer periods would lead to greater uncertainty, but a one-year limit is arbitrary and often unrealistic. The Protocol notes that the recall period is inevitably a compromise: "If the recall period is too short, a very large number of households will need to be visited and interviewed, which makes the survey unwieldy."

And while I agree that surveillance projects like the Iraq Body Count provide an absolutely essential metric, I'm only in partial agreement with SMCK's statement that these projects "reveal trends - whether things are getting better or worse - and can do so in near real-time." They can reveal trends, but they don't necessarily reveal trends. As SMCK points out in their own report: "Governments may forbid reporting of war deaths - particularly of their own forces. Journalists and other observers are sometimes banned from war zones, as in Sri Lanka in 2009, or may stay away because conditions are simply too dangerous. And in wars with very high daily death tolls - like Iraq in 2005 and 2006 - violent incidents with very small numbers of deaths may go unreported." Those factors can have significant effects on our perception of the overall trends.

Regards,
Bruce

Bruce wrote:

Pointing to the SMART Protocol as though it were some sort of definitive authority on what should and should not be allowed in mortality surveys strikes me as just plain silly. [...] I don't see why the software manual for a nutritional survey should be regarded as gospel for demographers who are trying to assess war casualties.

Well, I agree that it's not definitive gospel but I don't think it's silly to have good guidelines gathered together in an accessible document. Doing field work under crisis conditions is not a walk in the park so almost any advice you get from people who've actually done it helps you avoid egregious errors. I have no particular problem with the SMART protocol if it's viewed in the context of "best practices under way less than best conditions." However, like many manuals of the sort, it's limited in what it can and cannot cover. One of the topics that field manuals just don't have the room to cover is "how rotten will the estimates be if you don't follow best practices to the letter?" Of course, you *want* to follow best practices but often in the field you can't. Thinking that any deviation from best practices vitiates an entire study is the fallacy of the "Aha! I found a critical error" type. You can see lots of evidence of that in this topic. Some errors are far more egregious, and therefore consequential, than others.

Lancet correction published

tlambert — Fri, 06 Mar 2009 09:32:33 +0000

Lancet correction published

As promised, the Lancet has published a correction to the 2nd Lancet study:

Burnham G, Lafta R, Doocy S, Roberts L. Mortality after the 2003 invasion of Iraq: a cross-sectional cluster sample survey. Lancet 2006; 368: 1421--28--The Methods section of this Article (Oct 21, 2006) stated that "Participants were assured that no unique identifiers would be gathered." Upon review, it was determined that a significant number of the surveys contained names of respondents and household inhabitants. This was a lapse in the authors' obligations to protect participants. However, to the authors' knowledge, the completed surveys remained in possession of the research team at all times and there were no known breaches in confidentiality.

tlambert Fri, 03/06/2009 - 04:32

Tags

LancetIraq

iraq mortality

Lancet

Forward slash, Tim.

Surely the ideologues will trumpet this as proof-PROOF!! that there were fewer deaths in Eye-rack than reported, and probably the lack of coverage about the showering the troops with rose petals is related.

Best,

No correction to the (inaccurate) description of the sampling scheme? That seems weak. If you are going to publish a correction, you might as well correct all the mistakes.

David, why not "man up", and go first?

where is your "complete corrections" of all the David Kane errors?

patiently waiting for the link,

sod

I knew Kane would provide us with instant gratification of our need for humor ...

sod,

1) The first step in manning up is to provide people with your real name. I suspect that yours is not "sod."

2) I have certainly made many mistakes the course of my investigations of the Lancet work. But I also believe that I have corrected every single one of them either in various discussion threads or, more completely, at my blog. If you are aware of a single error that I have not corrected, please point it out and I will do so.

3) Why the hate? I realize that there are issues on we might all disagree, but isn't it universally accepted that the sampling description provided in the paper was wrong? (Some background here. The Lancet authors claim, not implausibly that the mistake arose from space constraints and awkward editing. Fair enough. And it is even plausible to believe that such a mistake does not, on its own, merit a correction. But, once you have to send a correction anyway, why not correct the other mistakes?

Kane says: "But, once you have to send a correction anyway, why not correct the other mistakes?"

This probably wouldn't be very feasible David. If they were to really correct all the mistakes there'd hardly be any paper left:
http://personal.rhul.ac.uk/uhte/014/Ethics%20and%20Data%20Integrity_8_0…

See section 3.8.

So, some households inhabitants' names were gathered, and the sampling description was inaccurate, and therefore the Iraq War failure is nothing but a gigantic hoax.

Profit!

Presumably Kane is putting the finishing touches to his Monte Carlo analysis using the stated scheme and the freely available geographical data for Iraq's road systems classified by type.

1) The first step in manning up is to provide people with your real name. I suspect that yours is not "sod."

i am rather glad, that i have the choice. while you are talking to pretty rational and friendly people here, we are dealing with insane lunatics.

you have been calling people "LIAR" without any real reason lately. remember?

2) I have certainly made many mistakes the course of my investigations of the Lancet work. But I also believe that I have corrected every single one of them either in various discussion threads or, more completely, at my blog. If you are aware of a single error that I have not corrected, please point it out and I will do so.

you are not asking The lancet to correct errors anywhere. you are demanding a full correction of all errors in ONE publish.

perhaps i missed that long list of all your errors, published on the Michelle Malkin website?

well, at first we should agree, not to call people liars without any evidence. i think that is pretty much universally accepted. i guess we can then move forward from there..

oh and David, i understand your disappointment.

i guess this press release about a pretty insignificant error was not, what you had in mind, when you wrote "I declare victory!"

Kane,
You declared "fraud" at the very outset, lo those many years ago. Will you now say that, after all of your nitpicking, that there is no fraud to be found? Man up, man...

As for error correction, I'm waiting for Lancet1 critics to freely acknowledge that a violent death rate of 100 per day for the first 18 months turns out to be quite plausible in view of the identical number obtained by the IFHS survey. Way back in 2004-2005, I didn't hear too many L1 critics saying "Yes, the violent death toll could be several times higher than IBC's numbers."

And then we can all agree that no matter what one thinks of L2, the mass media is understating Iraqi mortality when they continue to talk about "tens of thousands" or even "100,000" deaths.

Actually, prevailing has never been the game of the vicious denialists like Kane et al. Not at all; that's why they're barely phoning it in.

No, the only goal, really, once they realized they couldn't cover up the Nuremberg War Crime Tribunal-level nature of the American et al. aggression in Iraq, they determined to do 2 other things; one, as Napoleon counseled, was to cover up the truth, not forever, but "long enough." The other was to so vilify and harrass the researchers that the NEXT group wishing to do a study even remotely like this would think twice or 3 times.

Kane, for instance, knows, I think, that he's spouting incoherent pseudo-mathematical gibberish. His reputation didn't exist and so it's of no value to him, the service he can give to elite power, on the other hand, is going to guarantee him networking support and probably a permanent berth for articles and tv appearances in the future, as well as advancing his personal right-wing imperialist political agenda. I would not be shocked to see him become a kind of Bjorn Lomborg of politicizing and denying American, British, etc. atrocities, or even a Regnery author.

And his name is legion, really. The demonization of "the Lancet study" has been one of the big projects for the imperialist conservatives and the neoconservatives. I can't see how this could avoid having a chilling effect, and if it does, they've won, and won absolutely.

Marion,

You are making good points, but we should not overestimate the significance of the going ons in this matter. It is quite clear that:

1. Kane is not really an ideologue or part of a well-concerted or well-thought out effort - he is in this almost entirely as an attempt at self promotion. How well he managed, time will tell, but I suspect he didn't make much headway.

2. Attempts at the destruction of whoever bears politically inconvenient news is an old game. Burnham et al. must have known what they were getting into and have gone into it after having carried out their second and third round of thinking. Kane or not, this situation will persist.

3. The destruction game is often played at much higher levels than the case was here - see for example the Rathergate incident. The reason the fray here was left to pathetically small fish like Kane is because of the extremely low stakes. The study's findings were written off as absurd by the big guys - Bush, of course, but also the Democrats and the media. There was therefore no need to use the big guns to destroy the authors. Most people have never heard of the study and never will. Most of those few that have heard of it, heard of it as being unreliable, because that was how it was reported about in the media - long before Kane slithered into the field. They have never heard of Kane, main street bias, or any of the other players in this little fly circus we have been witnessing.

Journal of Peace Research publishes badly flawed paper, part 3

tlambert — Thu, 26 Feb 2009 11:59:13 +0000

Journal of Peace Research publishes badly flawed paper, part 3

This post contains some more notes on a reply to the badly flawed "Main Street Bias" paper.

In my previous post I showed that the MSB papers was wrong to claim that it was plausible that the unsampled regions was 10 times as large as the sampled region. In this post I look at their model. Their model is wrong because it assumes that there is no main street bias in the sampled region and because of this they massively overestimate any bias in the Lancet sampling.

Let's start with a correct model of the situation. I've adopted their terminology where possible.

We have a population of size N divided into Ni people inside the survey space (Si) and No people outside the survey space. The death rate for people living in Si is Bi and Bo for people living in So. The overall death rate B is just the weighted average of Bi and Bo:

B = (Ni*Bi + No*Bo)/(Ni + No)

The bias introduced by using Bi instead of B is

R = Bi/B

if we let n = No/Ni and b = Bo/Bi, after a little algebra we find that

R = (1+n) / (1+n*b)

If we plot this function, you will see that to get significant bias
you must have both n significantly bigger than zero and b
significantly different from 1. Even in an extreme case where n and b
are both 2 (ie two-thirds of the houses are missed and the death rate
is twice as high in the unsampled region), the bias factor is only 0.6.

To get the large R=3 bias that the MSB authors propose, they need
implausibly extreme values for both n (10) and b (0.27) -- that is,
the unsampled region is ten times as big and residents suffer just one
quarter the risk of violent death.

To get their implausible value for b, the MSB folks use their own
model. They have parameters, q which is the risk of violent death while
you are in Si divided by the risk for times when you are in So and
parameters fi, the fraction of time residents of Si are in Si and fo
the fraction of time residents of So are in So.

The formula they derive for R is equivalent to saying that

b = (q - (q-1)*fo) / (1 + (q-1)*fi)

To see what's wrong here, look at their argument for a high value of q:

It is likely that the streets that define the samplable region Si are
sufficiently broad and well-paved for military convoys and patrols to
pass, are highly suitable for street-markets and concentrations of
people and are, therefore, prime targets for improvised explosive
devices, car bombs, sniper attacks, abductions, and drive-by
shootings. Given the extent and frequency of such attacks, a value of
q=5 is plausible.

Where do they think that the people at street markets and those forming concentrations of people come from? The people in the unsampled region have to go to markets as well and there is no reason to suppose that they spend less time there than people from the sampled region. This means that attacks on markets and concentrations of people produce no main street bias.

Let's make this concrete with an example they reflects the pattern of violence that the MSB authors think leads to main street bias and has exactly the parameters in their model that the authors claim are plausible.

We have 3,000 people in So and 300 people in Si, so n = 10. We have 30 violent deaths occurring in So and 15 in Si, so q = 5. There is a market in Si where folks from So spend 1/14 of their time and people form Si spend 1/14 of their time in the market and 1/14 in So.
11 of the 15 violent deaths in Si happened in the market. This gives the "plausible" parameters used in the paper and their formula says that R=3.0.

Because the market draws people equally form Si and So, 1 of the deaths at the market was a resident in Si and the other 10 were from So. So residents of So suffered 40 violent deaths and Bo = 40/3000 = 1.3%. Residents of Si suffered 5 violent deaths so Bi = 5/300 = 1.7% and b = Bo/Bi = 0.8. Plugging n=10 and b=0.8 into my formula and we get R= 1.2. So in this example, despite a huge value of n and deaths tending to occur on main streets, the bias was negligible and the MSB model wrongly suggested that the bias was large.

The reason why their model gets it wrong is that it assumes that the risk is the same, on average, everywhere in Si. That means that even though Si residents only spend 1/14 of their time at the market, the model assumes that are exposed to the risk from the market 24 hours a day.

tlambert Thu, 02/26/2009 - 06:59

Tags

Although it is only tangentially related to Timâs post, here is something about the MSB paper which really pisses me off. Since Iâm unlikely to get a more appropriate thread in which to sound off about it, I will do so now.

They never really define the survey space, Si, properly. Sure, itâs clear which households are in Si. But thatâs not enough. We need to know which deaths took place in Si. If a bomb goes off outside the mosque it may kill people from Si, or from So, or both. But did they die in Si or in So? If in Si, why so?

They just donât say what their criteria are for allocating deaths to the two zones. Until they address that, their âmodelâ isnât really a model at all. And it won't do to take the usual line and say that Burnham should supply the answer to this question. Nobody - not even David Kane - contends that Burnham has, or ought to have, information about where deaths took place.

Tim:

1) Thanks for posting your analysis. This is just the sort of way that science ought to make progress.

2) If one of the Johnson et al authors responds, would you be willing to post their response in the main post, so of "The authors reply " option? Given the popularity of Deltoid, I think that they might respond if placed in the main post. I would not recommend that they bother with something in the comment thread, since only we true Lancet afficionados read the comments.

3) In previous discussions, you seemed to imply that there was something wrong with the math. I read this post as you saying: "The math is fine but the parameter values are wrong/misleading/implausible." Is that a fair summary of your view?

nice post Tim. I think there is a "d" written, when you actually are talking about the variable "b". i hope i ll have some more time to look at the details later.

3) In previous discussions, you seemed to imply that there was something wrong with the math. I read this post as you saying: "The math is fine but the parameter values are wrong/misleading/implausible." Is that a fair summary of your view?

David, your attempts to trick people into comments, that you will then repeat all over the internet, are strange at best. childish actually is a better description.

the lack of substance in your comments is starting to become (another) trademark of David Kane.

DK, Tim is saying the model is crap. Specifically, the part of the MSB model that defines what Tim calls b, the ratio of death rates in the unsampled to sampled area. Getting the math right is trivial. Anyone can do algebra. It is the creation of the original equations that is at question.

kane is at it already in this thread:
"In previous discussions, you seemed to imply that there was something wrong with the math"

No, kane. Lambert didn't imply anything of the kind. He said (not 'implied', but declared outright) that the MODEL was wrong. He never said or implied there was anything wrong with the math.

Why do you pull this crap, kane? Rhetorical question - no answer necessary.

Tim wrote:

The overall death rate B is just the weighted average of Bi and Bo:
B = (Ni*Bi + No*Bo)/(Ni + No)

Tim, was this necessary? Everyone but everyone knows that this is how one combines separate rates to get an overall crude mortality ra..... ooops. Hmmm. Well, on second thought perhaps one can't be too careful after all.

sod:

>I think there is a "d" written, when you actually are talking about the variable "b"

Fixed. It used to be "d", but I changed it because when fo=fi=1, we have b=1/q and b is q upside down.

Kevin, regarding your point in #1: I'm not sure that this is a problem.

The location of death is important in that it gives us some insight into the relative danger of the surveyed space, and the unsurveyed space: in other words, it helps us quantify that mysterious "q" variable. In that case, it seems like we only need to know how many people died, and whether or not the location of the attack falls inside or outside the boundaries of the survey space. We don't actually need to know whether the victims are from Si or So, do we?

The whole scheme still seems very arcane, but as far as I can tell, the lack of detail on allocating deaths isn't a problem.

Regards,
Bruce

DK:

>If one of the Johnson et al authors responds, would you be willing to post their response in the main post,

Sure, but we've already had plenty of comments from LancetStudy/Ron/ozzy/etc

>In previous discussions, you seemed to imply that there was something wrong with the math.

No. Their formula is wrong as I demonstrated. I never said that there was a mistake in their maths. Their formula is wrong because their model is wrong.

But [I repeat myself](http://scienceblogs.com/deltoid/2009/02/journal_of_peace_research_publ_…).

Bruce: it seems like we only need to know how many people died, and whether or not the location of the attack falls inside or outside the boundaries of the survey space. (My emphasis)

That's what I'm getting at. The MSB model divides Iraq into two zones: Si and So. Now suppose we are having an argument about q, or fi, or fo. We're agreed about n so that's not a problem. In fact let's suppose, for the hell of, that we agree with Johnson et al. that n=10. We haven't seen Tim's critique of their maps. Clearly the values of the parameters q, fi and fo must depend to a considerable extent on where the mosques, markets, factories, offices and hospitals are. Maybe I argue that for the most part these things must be distributed in the same way the households are, with 91% of them in So, outside the survey space. But you say, no, a lot of these places will be located near main streets which says to you they are in Si. We can't possibly have a sensible discussion, can we? We're not just contending for different values of the parameters. We're using two completely different models. My Si is nothing like your Si. Sure, it has the same households in it, because we're singing off the same hymn-sheet where the main roads are concerned. So we've coloured in every single residential building in Iraq. The MSB paper tells us how to do that much. But our map doesn't say which zone all the other stuff belongs to. So there's a huge gap in our understanding of the model.

When we go to Johnson et al. to see what guidance they give us on this, we find the don't give us any at all. So and Si are not defined. All the paper says is that the surveyable households are in Si and all other households are in So. How do we determine where the other buildings belong? Unless the shootings and bombings take place in residential areas we just don't know which zone to allocate the deaths to.

It isn't just a crappy model. It's so poorly specified that it really doesn't deserve to be called a model at all.

Apologies for dragging in my own pet gripe instead of addressing Tim's post (which I need to think about). But what with the sock-puppets and all, it's hard to keep discussions around here on topic anyway.

A very lucid post, Tim. The MSB value of b = 0.27 really is very hard to believe.

Tim,

You don't seem to understand their model. You write:

The death rate for people living in Si is Bi and Bo for people living in So.

Johnson et al (2008) does not define or use a death rate for everyone living in a particular area. They define death rates for people, whatever the location of their house of residence, when they are in a particular area. The paper makes this as clear as can be.

Probabilities of death for anyone present in Si or So are, respectively, qi and qo , regardless of the location of the households of these individuals.

So, your critique, as interesting as it may be, has nothing to do with the paper they published.

Now, you may feel that you have created a "correct model of the situation." Good for you! Please write it up and submit it for publication. But you have not found a flaw in their model. You have created a model of your own. Nothing wrong with that, of course. Just don't confuse yourself into thinking that "[t]heir formula is wrong." It isn't.

David, you don't seem to have read my post very carefully. Johnson et al don't define a death rate for people living in So. That's why I defined it myself as Bo. They do implicitly use such a rate in their derivation of their formula and the derivation would be clearer if they had defined it and used it explicitly.

And if their formula is correct, how come it gives the wrong answer in the example I gave?

D Kane says

"You have created a model of your own. "

Without weighing in on the actual estimation issues, how so? T Lambert says

"The formula they derive for R is equivalent to saying that

b = (q - (q-1)*fo) / (1 + (q-1)*fi)"

I read the post as T Lambert saying "If you wanted to do the study correctly, you would do ...; however in this study the authors choose to do ... . This is implausible, because ... is thought to be true. Now, here is an example which illustrates why I think this." It's obvious that T Lambert is making a claim concerning the work done in the paper. Surely, you see this?

The amusing thing about David Kane is he is soooooo.....superficial. He posts this

Johnson et al (2008) does not define or use a death rate for everyone living in a particular area. They define death rates for people, whatever the location of their house of residence, when they are in a particular area. The paper makes this as clear as can be.

Probabilities of death for anyone present in Si or So are, respectively, qi and qo , regardless of the location of the households of these individuals.

triumphantly, without realizing that it concedes the previous argument about the sex of victims. Remember for the ladies Kinder, Kueche, Kirche. For the guys Mosques, Markets and Main Streets.

Ok, who goes to the market, who goes to the mosque, who goes to the main street to shop if it is dangerous. Guys.

DK, qi and qo are not death rates. TL extends the previous analysis by calculating a general bias factor given varying death rates between the ni and no populations. The question then is what is that different death rate. This is where the MSB q's and f's come in. The MSB paper is essentially proposing a model for b(q,n). One could make up any sort of model for b, but would still obey the same equation for R(n,b). This is very clear in the description above.

Not to defend the MSB paper, but one could still imagine some sort of street bias--to the extent that death squads kill people in their homes, there might be a bias towards certain types of streets, though I don't know which way that bias would work. That is, would death squads operate more on main streets or in remote side streets or would there be any particular pattern? And where do death squads pick up their victims --homes or streets? And there are cases of collateral damage, where people get killed in their homes because of fighting, though again I don't know which way that possible bias would go.

And I don't know how one would investigate any of this--you could try using IBC data, to the extent that it has info on such things, but then you have unknown forms of reporting bias to worry about (something that one of the sockpuppets dismissed in an earlier thread).

Something which still hasn't been pointed out about these peoples' work, to the best of my knowledge, is that they are not proving the existence of main street bias in the lancet papers. To do this, they would need to run a simulation on a map of an Iraqi city (or some model thereof) with chosen values of Bi, Bo etc. and show that the sampling method used would have led to significantly different values of the parameters of interest.

They haven't done this (to the best of my knowledge). All they are doing is rehashing a theory on effects of sampling bias from another paper and taking as the basis for the bias their unsubstantiated assertion about the differences.

Why this failed to escape the peer reviewers is not exactly a mystery to me, but it is disappointing that such tediously derivative and uninformative work got published.

I see what Lambert is doing. His conclusions about his model are misleading - specifically over how he derives values for Bo and Bi. In his "concrete" example, he claims to use JPR parameter values (in an attempt to show that JPR exagerrates the bias), but he derives "b" from one of his own assumptions - namely that his hypothetical market "draws people equally form Si and So".

In other words, he's reincorporating some of the assumptions from his earlier Platonic market example, whilst removing "f" from the equation (which turns out to be convenient for this misleading exercise).

But in order to credibly derive b (in his model), one must still make realistic assumptions regarding the diffusion of people among zones. So Lambert misrepresents JPR when he asserts that his contrived market example "reflects the pattern of violence that the MSB authors think leads to main street bias". Here is the offending/misleading bit from Lambert's example:

There is a market in Si where folks from So spend 1/14 of their time and people form Si spend 1/14 of their time in the market and 1/14 in So.

What is the problem with Lambert's crucial assumption that "the market draws people equally form Si and So"? It's pretty obvious, so I won't spell it out for you.

Bottom line: Lambert's "b" model is no more than a sleight-of-hand used to bypass the earlier criticism which demolished his hypothetical market assumption.

To summarise Tim Lambert's (mostly inept and misleading) 3-part critique of MSB/JPR:

Part 1 - Lambert claims JPR uses "absurd" parameter values. He suggests an alternative value of 2/15 for fo, which implies that the average Iraqi (including women, children and the elderly) spends only 3 hours out of each 24-hr day in their own home/zone (presumably sleeping), and spends the other 21 hours outside their zone. Since this is clearly ludicrous, Tim has said he's redefining "f", but he hasn't explained how anyone could take his redefinition and arrive at the value of 2/16.

Part 2 - Tim claims, by adding two further "main" streets to a map provided by the MSB authors, that "their map and the unsampled area is 0. In their model, that means n=0 and there is no main street bias." But this is totally misleading. The unsampled area is not 0 in "their map" - it's 0 in Tim's tiny portion of their map (and it would be even tinier if we dismissed Tim's second "main" street, which is half the width of his first main street). Tim hasn't even bothered to state his guess of the value of n for "their map" (as a whole) taking into consideration his one credible additional main street.

Part 3 - Lambert presents an alternative "b" model which he claims demonstrates how the MSB model exaggerates the bias. But it's just another misleading exercise in which he claims to plug "exactly" the MSB parameter-values (or equivalents) into his own model. He does no such thing. He derives Bi and Bo from his own dubious hypothetical market assumptions which "loads" the result in precisely the way he wants.

#20, part 1, first sentence: "2/15" is of course "2/16".

Robert Shone, you can't make an argument that people don't go to markets, so yes, it is likely that people from all areas go to the markets to buy stuff. You can make an argument that if it gets dangerous to go out, people will send the guys in the family and not the kids and women, and they will not linger which makes their exposure smaller, e.g. decreases the so called bias.

Now there is one exception, the people who live at the back of the store IN the market, but markets themselves are off main streets, not on them (look at the Google maps).

What is missing here is the fact that abductions will not happen on main streets as a general rule, that drive bys and abductions are a lot easier on uncrowded streets, and that the actual killings take place in isolated areas. Add to that the fact that vast housing tracts in Iraq were controlled by sectarian militias that could easily grab their victims wherever and whenever they wanted and the so-called MSB vanishes.

As Eli has been saying lo these many threads (sg is getting it), to demonstrate a bias related to proximity to main streets, you have to show that there actually is one with the available data (IBC for example), not merely throw the thought out. Otherwise you are engaged in academic onanism, a pleasant occupation, damaging to your metal and social well-being.

Eli Rabett writes:

Robert Shone, you can't make an argument that people don't go to markets...

That's not the argument I was making. You're starting to think in the right direction, but you don't go far enough. What other factors make the assumptions behind Tim's "concrete example" values for Bi and Bo unrealistic and unreasonable (and unrepresentative of the MSB authors' parameter values, contrary to what Tim claimed)?

If there is such a bias, Robert, its magnitude and presence have to be confirmed before the debate over these equations becomes relevant. I suggest simulations. These haven't been done, even though the authors of this paper are physicists well familiar with such methods. It hasn't been done because it won't prove what they want to prove.

sg writes:

If there is such a bias, Robert, its magnitude and presence have to be confirmed before the debate over these equations becomes relevant.

The presence of bias seems to be accepted by the Lancet authors, who claimed they used (unpublished) procedures "in an effort to reduce the selection bias that more busy streets would have".

And presumably they wouldn't have planned and used such (unpublished) procedures if they didn't think the magnitude of the bias was significant enough to need reducing.

We might have a different situation if the L2 authors' response to the MSB criticism had been: "Well, since no bias has been demonstrated, we didn't include any procedures to reduce it". But that wasn't their response, because they're not stupid.

Given the claim made by the L2 authors, there's a burden of evidence upon them (to disclose the procedures) - regardless of speculations over the value of q in the case of Iraq.

Note also that it's important to distinguish the relative risk from network-proximity to main streets (on the one hand) from the posited distortion present in the results of a survey (on the other). Both have been referred to as "bias", which I think has led to some confused (and confusing) comments.

Unfortunately, Tim's "b" model gets rid of "q" as a separate parameter - with the potential for increasing confusion (in my opinion).

No Robert, there is no responsibility on the part of the original authors to run simulations to confirm the non-existence of a fantastical claim.

Spagat et al claim that there is main street bias in the published selection method of the L2 study. More than that, they claim a general phenomenon of "Main Street Bias" which they present a (not really) new formula to describe the consequences of. It is their responsibility to prove that this bias is present in someone else's study, and its magnitude.

They could do this very easily with a set of simulations on either an existing or a theoretical map, in which they posit a range of values for the ratio of So to Si, and simulate, say, 1000 samples from the sampling method described by the L2 authors. They could even use a range of definitions of "main street". Then they could give a range of values for q, b, etc. This would be a pretty trivial process for most physicists. They don't.

Until they do, they haven't shown that their putative new form of bias exists in this or any other study, nor have they given the conditions under which it could occur.

But it's easier for them and you to claim without proof that it exists, and dispute the precise wording of the paper, than to do the leg-work which would show that under all reasonable definitions of "main street" the bias just doesn't exist. Tim Lambert did the outline of a single simulation in his previous post, and a bunch of mendacious pricks spent 100 comments arguing about whether an obviously main street was or wasn't a main street. Most decent physicists with Spagat's training could probably whack out a simulation in the time it took me to read that odious thread.

This shows the level at which you denialists are operating, i.e. you are lazy (won't do the simulations, would rather do the name-calling) and you are nasty.

For those so interested there is now a GIS map available at geofabrik.de of Iraq, with roads classified along the lines of trunk, primary, secondary, resiential etc.

You can also download for free quantum GIS (QGIS), and from there the fun begins. It has taken me about two days in total to get a half arsed version of Si vs So and I'm not even a geographer, a compsci or an epidemiologist.

SG:

But it's easier for them and you to claim without proof that it exists, and dispute the precise wording of the paper, than to do the leg-work which would show that under all reasonable definitions of "main street" the bias just doesn't exist. Tim Lambert did the outline of a single simulation in his previous post, and a bunch of mendacious pricks spent 100 comments arguing about whether an obviously main street was or wasn't a main street. Most decent physicists with Spagat's training could probably whack out a simulation in the time it took me to read that odious thread.

You are completely correct. This is exactly what I was arguing with LancetStudy before (s)he went off in a huff. There is absolutely no reason to believe that the academic navel gazing of the JPR paper correlates with anything in the real world.

Unless and until the authors of that paper can be bothered to show - through simulation as you say, or callibration or matching patterns of actual deaths as recorded somewhere, or a complete analysis of certain maps or something - that their pretty equations actually correspond to something in the real world, then there's no reason to take what they say seriously.

Although, some cynical part of me can't help think that they have done the simulations and have shown that their bias does not exist... But that's just me being overly suspicious.

As I said before: poor effort.

Regards

FWIW, it is better to use m=1/n= Ni/No, mostly because if Ni-->0 then m=0.

A little off-topic, but I thought this question would fit better in a Lancet/Iraq thread than in the open one. Anyway, does anyone (preferably meaning professional demographers or people capable of making informed guesses on this) think that the estimate of 740,000 Iraqi widows lends support to the L2 estimate of excess deaths by 2006, or are there too many uncertainties? I saw the article (link below) and started making guesses about how many might have been widowed from pre-2003 causes and in the normal course of events, but I don't really know enough--other laypeople have done similar things.

Link

"other laypeople have done similar things."

I mean I've seen at least one other person who argued that this number of Iraqi widows supports a very high Iraqi war death toll, and I thought it might myself, but maybe people with some background in the field would say "No, there are too many uncertainties" or even "No, if anything, it supports a much lower death toll."

Donald wondered:

does anyone [...] think that the estimate of 740,000 Iraqi widows lends support to the L2 estimate of excess deaths by 2006, or are there too many uncertainties?

There are too many uncertainties.

By which I mean, we don't know where that estimate came from, or the ages of the widows, or how long they've been widows, or the number of surviving children and their ages, or a host of other data elements that could be used to evaluate the estimate.

Thanks Robert. I suspected that. Oh well.

some sort of age-standardised widow-rate per 1000 population would be in order. I suspect that's rather difficult to calculate...

Since Tim is clearly away and enjoying himself, Eli would like to point to yet another chapter in that long running soap opera "The Incoherence of Denial", where Spagat steps on his own line and shows that Main Streets were not the place not to be in Iraq, or as we say at Rabett Run, Back Alley Bias.

Yes, Eli is blog whoring, but he thought Tim would have picked this up by now.

nice one Eli.

the only real evidence that Spagat ever brought up to support the existence of his "main-street bias", is a very strange map: (page 10)

http://www.cred.be/SurveyConference2007/documents/Conference2007/spagat…

notice the huge dots marking the explosions and the completely false claim that

Note that incidents of this size almost certainly cover over half of all deaths.

neither he nor Kane ever excused for that blatant error...

Nice one thanks Eli. With his name on a respectable paper now, I wonder whether Spagat will ever acknowledge any fault or embarrassment over his "main street bias" puffery? Surely not soon, probably never, but it would be good to see.

Journal of Peace Research publishes badly flawed paper, part 2

tlambert — Thu, 19 Feb 2009 08:54:31 +0000

Journal of Peace Research publishes badly flawed paper, part 2

This post is some more notes on a reply to the badly flawed "Main Street Bias" paper.

The authors claim that it is plausible that the Lancet paper's sampling scheme could have missed 91% of the houses in Iraq. (That is, their parameter n, the number of households in the unsampled area divided by the the number in the sampled area could plausibly be 10 or more.) The only support they offer for this is a reference to this analysis of Iraqi maps.

To the right is a detail from their map. The red lines are main streets and the yellow are secondary streets. They assert that the blue areas are not samplable using the Lancet scheme, and yes, the blue area covers 90%+ of households. But there are two things wrong with their map.

Look at this larger scale version of their map:

First have a closer look at their yellow road that they say should not be counted as a main street and compare it with the one they count as a main street. (Scroll around on Google Maps if you want to see more of each street.) It's just as wide and has a similar amount of traffic and a similar number of large buildings. Clearly it should have been classified as a main street and it is not even slightly plausible that someone who was trying to get every house in the sample frame would leave it out.

Second, the random house on the secondary street is only the start point for the cluster, which also includes 39 houses neighbouring the start point. This means that you can sample houses on tertiary streets that are a few houses from the secondary street. I added two obvious main streets to the ones they chose (the one across the bottom of the map above and one that it is off the map above), and redrew the blue areas to taken this factor into account.

What's that? You don't see any blue areas in my map? That's because there aren't any. Make just these two corrections to their map and the unsampled area is 0. In their model, that means n=0 and there is no main street bias.

tlambert Thu, 02/19/2009 - 03:54

Tags

LancetIraq

deaths in iraq

Journal of Peace Research

Lancet

Michael Spagat

Neil Johnson

nice work Tim.

they also prefer to chose Baghdad when talking about "mainstreet bias". choose a random village and use the lancet approach, and you will nearly always find nearly everything covered...

Another thing is that you don't tend to find markets and such on wide main streets cause you can't get goods in and out and customers cannot cross easily.

Tim Lambert writes:

They assert that the blue areas are not samplable using the Lancet scheme...

No, they don't assert this. Nobody knows what's samplable using the Lancet scheme, because nobody knows how the Lancet scheme worked in reality (since the details haven't been released).

The MSB authors make it clear that Burnham et al's definition of main street "is too vague to answer these questions definitively". They continue: "For starters, we cannot be sure of what is a major commercial street or avenue. But it is possible to explore some scenarios and we do this below. We encourage everyone to do their own explorations based using (the miraculous) Google Earth."

If that's not clear enough for Tim, they also write the following [my emphasis]:

We highlight the roads that we consider major commercial streets or avenues in red. We highlight the crossroads to the major roads in yellow. The blue shaded area contains residential building that could not be sampled under the authors' stated scheme and accepting our classification of major commercial streets or avenues. If these were indeed the major commercial streets chosen in The Lancet paper, then only a tiny sliver of Kirkuk could have been sampled. A more liberal rendering of major commercial streets or avenues would enable deeper penetration.

One can see more clearly from the full map (rather than from the particular corner of it that Tim has chosen to focus on) why the road marked yellow might be considered a cross street - it cuts straight through the residential area, in contrast to the red-marked roads.

I've no idea whether n=10 in the real study. It seems infinitely more plausible than n=0.

why the road marked yellow might be considered a cross street - it cuts straight through the residential area, in contrast to the red-marked roads.

so they assume that only roads not leading through residential areas are "mainstreets", and then conclude some bias of residents not being sampled?

A more liberal rendering of major commercial streets or avenues would enable deeper penetration.

for some weird reason, they didn t provide too much data, on such a "deeper penetration".

and instead went for n=10 in their number works. just by chance?

Incidentally, if you take Tim Lambert's "yellow road" scenario for producing n=0, and apply it to the whole of Iraq, you get interesting (but not remotely credible) implications for how the Lancet sampling scheme "must" have worked.

i am seriously puzzled by the Tikrit map on the Spagat page.
How did they chose those red "mainstreets"

and why is there so much pink shading around blue roads?

There is relevant information from a BBC intervies

c) most urban clusters spanned 2-3 blocks as we moved in a chain from house to house so that the initial selected street usually did not provide the majority of the 40 households in a cluster and d) people being shot was by far the main mechanism of death, and we believe this usually happened away from home.

especially d

I couldn't make any sense of their Tikrit map. These are the only two maps they offer in support of their n=10 theory.

1) Thanks to Tim for taking the time to do this analysis and start a new thread. This is the way open-minded scientists search for the truth.

2) Tim: Why don't you ask Les Roberts whether your "two corrections" are plausible? After all, surely Roberts knows what the survey teams considered to be "main streets" . . . right?

3) I believe that I have listened to more publicly available presentation by the Lancet authors than almost anyone. I have never heard them give a description that is consistent with a methodology that would lead to a map like the one that Tim describes above. They weren't using Google maps after all! That doesn't mean that Tim's description is false. We just have no idea what they actually did. (Pointers to the contrary are welcome.)

4) I hope this mean that Tim has given up on his "proof by contradiction" that the formula is wrong from here. If so, it would be nice of Tim to confirm that he no longer maintains that there is a mistake in the maths.

DK:

2) I don't need to ask Roberts. He's already answered this.

3) I disagree. I just followed the methodology they described in their paper. Of course they didn't use Google maps, but you can use Google maps to determine what someone at street level would do.

4) No. Their formula is wrong as I demonstrated. I never said that there was a mistake in their maths. Their formula is wrong because their model is wrong. Stay tuned for part 3.

Here is an overview of what we know and what we don't know about the sampling for L2. Summary:

L2 authors have given different (and conflicting) accounts of exactly what the interviewers did and/or were supposed to do. They have made no final statement about what sampling plan was followed. Anyone who claims to "know" what the sampling plan in L2 was is lying.

Tim is correct that his description of L2 sampling could be correct. Alas, he has no real evidence for it since there is no clear, final statement by the authors themselves. How to make progress? Tim should ask Les Roberts to clarify, once and for all, the exact methodology followed by the Iraqi interviewers. For example, did they have maps?

Tim:

We cross-posted. You write: "I just followed the methodology they described in their paper."

But we already know that the methodology described in the paper is wrong. Burnham/Roberts have told us that! Do you disagree?

Can you provide a citation to your claim that Roberts "already answered this?" I am not doubting that you have such a citation, but collecting the various contradictory things that Roberts has said about the sampling plan is a hobby of mine.

I look forward to part 3.

DK, see [here](http://scienceblogs.com/deltoid/2006/11/burnham_and_roberts_reply_to_s…) for Roberts' comments. They added this to the description in the paper.

>"As far as selection of the start houses, in areas where there were residential streets that did not cross the main avenues in the area selected, these were included in the random street selection process, in an effort to reduce the selection bias that more busy streets would have."

Clearly this makes no difference to my result above. Adding streets to the set of main streets will only improve the coverage.

And note that I didn't go out a search for a map where the scheme described in the paper worked, this was the one they offered to show that n=10 was plausible. It isn't.

Well thus far with my map of london centred on where I live, a bit of string, a ruler and a random number generator I'm getting pretty wide dispersal with only officially designated A roads as main streets even. Will hopefully get this process automated so as to do some proper analysis.

Tim:

In the link you provide (thanks), Burnham/Roberts claim that "Sampling for our study was designed to give all households an equal chance of being included." This is a contradiction of the sampling scheme you describe above. For example, consider all the houses that are near the intersections of two (or more) cross streets. Those houses are much more likely to be included in the sample since picking either cross street still gives them a chance. Other houses are near (meaning within a possible 40 house sample) only one cross street. Your picture illustrates that perfectly.

So, either Burnham/Roberts were wrong in their letter to Science or you are wrong to claim that your picture illustrates the sampling plan they undertook. Which is it?

Tim Lambert should read the following comment from Les Roberts (in a 2007 radio interview, via David Kane's blog) as it makes Tim's latest "deep, thorough penetration" scenario questionable:

LR: So they were just not under any circumstances going to use the GPS system, so they did more or less everything the same except they... when they got to a neighborhood and sort of drove around and saw the outline of the neighborhood they sort of counted - they picked off the main streets and then picked one and then picked the side streets of them, or picking one at random, counted all the houses on that street and picked one at random. So they had a slightly different randomization technique.

And then there's this comment from Burnham (MIT, 2007):

Burnham: [they made a list of] "all the residential streets that either crossed it or were in that immediate area."

Huh?

Jody is doing what Spagat and Co should have done.

Eli: Instead of telling Spagat and Co what they should have done, why don't you explain to us what Burnham and Co actually did? You have followed this debate closely. Do you think that this is a true statement by Burnham/Lambert?

Sampling for our study was designed to give all households an equal chance of being included.

If so, then isn't Tim's example wrong?

For example, consider all the houses that are near the intersections of two (or more) cross streets. Those houses are much more likely to be included in the sample since picking either cross street still gives them a chance. Other houses are near (meaning within a possible 40 house sample) only one cross street. Your picture illustrates that perfectly.

David, this is getting stupid (again). you have exactly the same problems if you randomize GPS coordinates (single house will get higher chance of getting picked) or telephone interviews (people with multiple numbers or people only using mobiles or people who travel a lot..).

all of this is "as best as circumstances allow".

i doubt that anyone including Burnham takes

"Sampling for our study was designed to give all households an equal chance of being included. "

to mean EXACTLY the same chance. you do understand, that populations move a lot in Iraq?

David Kane:
>If so, then isn't Tim's example wrong?

No.

I take it that no-one disputes that if you follow the sampling scheme described in the Lancet paper and apply it to the map that the MSB authors chose, you get almost complete coverage, n at most 0.1 rather than n=10 as they claim. Don't you think that this is a serious flaw in their paper? Is that why you are trying to change the subject?

Tim Lambert writes:

I take it that no-one disputes that if you follow the sampling scheme described in the Lancet paper and apply it to the map that the MSB authors chose, you get almost complete coverage.

Tim is confusing the "Lancet sampling scheme" with the "Lambert sampling scheme".

We actually know more about the latter than the former, which is a pretty sad reflection on the Lancet authors.

Or, to spell it out for the slow-learners, Lambert's analysis is disputed by those who realise that since Lambert doesn't know what L2 took to be a main street, his own guess wrt "main" street vs cross street is irrelevant.

His choice (suggesting that nearly every street in Iraq was in the samplable space) is at odds with arguments made by the L2 researchers about limitations due to safety and shortage of time, etc.

The Lancet authors claim "all households" had equal chance of inclusion. The burden of evidence is clearly upon them.

Shone and Kane are claiming perfect ignorance, which in their cases is an acceptable starting point. However, down here on earth it is clear that you should be able to look at any city, impose a condition for what a main street is and look at what the results are as that condition is varied. That is what Jody is doing. It is also clear that Spagat, et al (Hi there et, or is it al) imposed an unreasonable condition a la McIntrye and McKitrick, e.g. they picked a set of "main streets" that yielded the effect they wanted.

Now this may not be an easy problem. For example, when Eli looked at the maps, he saw that one reasonable way to pick main streets was to see where the mosques are (domed structures), the markets (lots of sheds and people) and a few other tells. In the main these were NOT the streets picked out by Spagat, Johnson, et al.

So, let us ask Spagat, Johnson and co. what condition THEY used for picking main streets and see if it is reasonable.

Further, a point which appears to be missed, is that most of the deaths by violence were from gunshot wounds, something not as likely to be concentrated on main streets as bombings might be (depending on your definition of a main street).

Eli Rabett writes:

However, down here on earth it is clear that you should be able to look at any city, impose a condition for what a main street is and look at what the results are as that condition is varied.

Have you asked the Lancet authors which "conditions" they "imposed" to establish whether a street is a "main" street? I thought not. Why don't you try asking them, and then share their response with us.

Suppose, for the sake of argument, that Gilbert Burnham is such a shameless fraudster that he merits a place on Tim Radfordâs list. Would that excuse what Johnson et al. did with that map? In my view it would not. Is anybody prepared to defend them? So far in this thread nobody has even tried. All weâve seen from Robert Shone and David Kane is attempts to change the subject.

Shone, Kane:

Why on earth are you two not responding on target to THIS criticism of THIS paper, which is what THIS thread is about?

Spagat et al claim that n=10 is a reasonable estimate. Use of a value near that - at least much greater than 1 - is necessary to their conclusion. They support their estimate of n=10 by the maps to which Tim links. One of those maps is incomprehensible.

This map, the one Tim analyzes, shows that an estimate of n=10 can only be arrived at by making facially absurd exclusions of major streets from the category of 'main streets,' which in turn excludes obviously secondary streets from the category of "secondary streets."

This is a criticism of Spagat's argument, which is based on an absurd classification of streets. Rather than divert, would the two of you please deal directly with that point, and its consequences to Spagat's argument?

Or you can continue being irrelevant, if you prefer.

Robert Shone doesn't get it. By examining the result of a sequence of choices for main streets you can get a very good idea of how resilient the result is to the specific method of choice without asking anyone what their choice was.

Frankly the question that I have about main street choice is based on experience. Many places have small commercial zones on long streets/roads. This is especially true about small towns and from the maps also appears to be true about Iraqi cities. If this is so, and the length of the street is much longer than the commercial zone, Spagat, Johnson and Cos argument about main streets vanishes.

Lee writes:

Shone, Kane:

Why on earth are you two not responding on target to THIS criticism of THIS paper, which is what THIS thread is about?

Try reading my comments #3, #5, #16, etc - they address Tim's points directly (in fact they address little else). There's nothing in Tim's new post that's not already covered in my blog entry:

http://dissident93.wordpress.com/2009/02/15/pope-of-debunkers/

Slightly related, gives you an idea at least perhaps of the challenge in surveying this kind of thing. Unfortunately behind a paywall:

http://tinyurl.com/adufua

More Lancet Stuff

Robert, I'll admit that I fall into your category of "slow-learners." I have only a high school education, I have no particular math skills, no background in statistics, and no special knowledge of Iraq. I am, however, trying to keep an open mind, and I'm trying to grasp the whether or not the main street bias argument makes sense.

You've repeatedly said that Tim doesn't know what the Lancet team actually did. How is Tim's guess about what the L2 authors did any less valid than the Johnson et al's guess? I think Tim makes a convincing case that the methods Johnson et al are using to determine the extent of the coverage are not plausible.

Do you have a comment on Sod's observation that the criticisms here have focused almost exclusively on Baghdad? And how would main street bias affect the results in smaller towns? Was this what you were referring to in comment #5? I'm sorry if this is tedious for you, but I can't understand what you are saying, and I think I'm probably not alone. What do you think Tim's "yellow road" scenario implies when applied to the whole of Iraq, and what is it about these implications that is implausible?

Earlier, in the previous JPR thread, I asked you to clarify what information the L2 authors could have provided that would allow Johnson et al to come up with a reasonable value for q. (You conceded that the parameter value Johnson et al chose might not be correct.) You didn't reply, but Ozzy/Ron/LancetDebunker/Tell suggested that BBC maps of violent incidents could be used to determine the relative frequency inside and outside of the survey space. Kevin, Sod and I pointed out that this doesn't help at all: Sod noted that the maps show where the killing happened, and not where the people involved lived, and Kevin and I pointed out that virtually everyone agrees that the media is only reporting a fraction of the deaths, and that the pattern and scope of unreported deaths is exactly what we need to determine.

You say in #22 that the burden of evidence is on the Lancet authors to show that all households had an equal chance of inclusion. Why do you believe that they didn't have a (practically) equal chance of inclusion? (As Sod noted in responce to David's comment, yes, there are some minor differences of probability... but that would be true of ANY sampling scheme.) Do you agree that if the Lancet team used "the Lambert sampling scheme," they would have had a good chance of including nearly any household? As far as I can tell, your only reason for arguing that the burden of proof rests with the Lancet authors is that you have a prior assumption that they were either dishonest or incompetent. Maybe they are, but it seems to me that the burden of proof rests with you, particularly when Tim's example shows that, depending on whether or not one sensibly defines a "main street," the scheme is not necessarily subject to "main street bias."

You've admitted that you don't know what "q" should be, and you don't know what "n" is. As Kevin noted in the last thread: if we can plug whatever parameters we want into the model, there doesn't seem to be a whole lot of point to this exercise.

Regards,
Bruce

Bruce did sum up those points much better than i ever could. thanks!

Bruce writes:

You've repeatedly said that Tim doesn't know what the Lancet team actually did. How is Tim's guess about what the L2 authors did any less valid than the Johnson et al's guess?

Tim's "guess" suffers from two problems.

First, he refuses to admit that it is a guess. He acts like he knows (and everyone else should know) exactly what the Lancet sampling plan was. But he doesn't know! (See here for extensive documentation of that claim.)

Second, he refuses to help us figure out what happened. Tim could easily e-mail Les Roberts and seek clarification. (Roberts has replied to Tim's e-mails in the past.)

So, the first step in making progress is for Tim to admit that he is guessing and then to work on turning his guesses into facts. Why won't he?

But, to answer your question, I think that Tim's guesses about what the Lancet team did are not unreasonable. I think that Johnson et al's guesses are not unreasonable. Which ones are better? Tough to say! And, to be honest, not that interesting (to me).

I think Tim makes a convincing case that the methods Johnson et al are using to determine the extent of the coverage are not plausible.

Really? What evidence does Tim cite for this? Start with a specific example. How many main streets are there in a city like the example that Tim creates? One? Five? 50? 500? The number of main streets is, obviously, a crucial input in deciding what the coverage is and, therefore, what n is. But Tim, you, I and everyone else has no idea how many main streets there are in this cluster or in any cluster because the authors refuse to tell us (or anyone else). From my link above:

Seppo Laaksonen, a professor of survey methodology in Helsinki, requested and was denied any information on main streets, even the average number of main streets per cluster (Laaksonen, 2008).

Again, it could be that Tim's guess about n is more accurate than Johnson et al's guess. Or maybe your guess is better still! But no one in the debate has any good evidence for why there guess is good because the Lancet authors refuse to provide any useful information for estimating n (or any other parameter).

Shone, having read this and the previous thread, I find I must bite my tongue, err on the side of politeness from what I think deserves to be said, and simply say "bullpucky."

In 3, you dont respond AT ALL to Lamberts demonstration that the Spagat street selection procedure is absurd. You simply quote the paper's authors to say that other selection procedures might give a different answer. Well, duh! That isn't a response, it is a deflection.

Your 5 is content free, as far as I can see.

In 16, you again fail to respond to any point Lambert is making, instead quoting the L2 authors on selection procedure. How do those quotes address Lambert's point that THIS PAPER's authors use an absurd criteria for distinguishing main streets form secondary streets, and that if one uses more reasonable criteria, their argument fails?

David Kane: Seppo Laaksonen, a professor of survey methodology in Helsinki, requested and was denied any information on main streets, even the average number of main streets per cluster.

The way I read it, Laakonsen wasnât âdeniedâ information :

Burnham told Science, however, that the Johns Hopkins team does not have such detailed information. "Our goal was to reduce any type of risk to the community and the participants," says Burnham. "While we have much of the raw data, we requested that anything designating the interviewers or the location of the neighborhoods visited not be sent to us." Laaksonen responds that he would not have published "any figures for the country" if he didn't have direct access to such raw information from surveyors.

A man canât be said to deny things to people if he doesnât have them to give. Do you have any reason to believe Burnham has the information Laakonsen requires?

Kane, Roberts has said that their selection procedure was designed to attempt to give every house a chance of being selected.

Unless you are going in with the assumption that they are lying and committing fraud - and you clearly are, Kane, and it is IMO despicable behavior - then that adequately answers the question about the numbers of main streets. Their procedure is designed to attempt to give every house an equal chance of selection, and that means that the map Lambert is criticizing in this thread very obviously and clearly does not match the Roberts sampling scheme.

Donoghue at 34:

I suspect you've just hit on the motivation for much of this.

Roberts et al were designing a survey under very edgy ethical conditions. There was a credible very-non-zero risk of death to the surveyors, and to the people being surveyed for answering the questions. He was and is ethically constrained to to do everything possible to minimize the risk to the people being surveyed, and he designed the procedure with that in mind.

Information that was even borderline related to ways to identify surveyed neighborhoods or people was simply not transmitted to him, in order to maintain the confidentiality of people who were potentially at risk of their lives for answering the survey.

Now these guys come along and invent scenarios which allow them to demand that Roberts release information that he by design did not retain, for ethical reasons - and then use his ethically-constrained survey design and retention policy to lambaste his ethics, because he is not releasing the information that he for ethical reasons CAN NOT release and did not retain.

I suspect that this is intentional on the part of Robert's critics. I know that it is despicable behavior.

Bruce Sharp writes:

You say in #22 that the burden of evidence is on the Lancet authors to show that all households had an equal chance of inclusion. Why do you believe that they didn't have a (practically) equal chance of inclusion?

I may be wrong, but your questions suggest to me that you haven't read the MSB paper or its associated explanatory web pages. It's pretty clear, for example, on the above point. The sampling scheme as published in the Lancet journal wouldn't give all households an equal chance of being included (not even a "roughly" equal chance). Even Burnham/Roberts seemed to acknowledge this - eg they stated that they omitted from the published account the procedures which would enable them to "reduce the selection bias that more busy streets would have" (see comment #13 for link).

That comment from Burnham/Roberts makes no sense if they believed that the published sampling procedures by themselves "reduced" the bias from "busy streets".

They've never released the details of these necessary "additional" procedures. They simply made assertions that they somehow reduced this bias to zero (or near zero). Hence the burden of evidence.

Forgive me if I don't answer all your other questions. I've made the point repeatedly that people are free to suggest their own parameter values based on their own assumptions, and that until we hear from the Lancet authors these are speculative as far as the Lancet study is concerned. Based on what I've read about the study (eg see my comment #16) I do think that the MSB example parameter values are plausible - certainly far more plausible than Tim's n=0 suggestion. That's just my opinion based on the limited information available. It may turn out to be wrong. The MSB authors suggested parameter values which they found plausible based on the limited information available. You may disagree with them, but they've committed no sin by stating what they find plausible. And if you don't like their suggested values, they've provided an exploration of the parameter space.

The bottom line is that it's down to the Lancet authors to support their own assertions wrt how they "reduced" the bias to zero (or near zero).

The tail end of these threads tend to get repetitive (and abusive and pointless). I'm not going to repeat myself further - this is my final post in this thread.

A summary of my take on Lambert's feeble attempt to debunk MSB is available at my blog:

http://dissident93.wordpress.com/2009/02/15/pope-of-debunkers/

Not to answer for Shone, who's done a good job on his own, but Bruce Sharp writes:

How is Tim's guess about what the L2 authors did any less valid than the Johnson et al's guess?

It's not really. They are just different guesses. Johnson et al's presentation of this is more valid than Tim's though because they say they do not know which streets or type of streets the L2 authors (field team actually, not the authors) called main streets in practice. Tim claims to be "correcting" their guess by replacing it with his guess. So he is making invalid claims about his guess while Johnson is not.

I think Tim makes a convincing case that the methods Johnson et al are using to determine the extent of the coverage are not plausible.

I don't. The issue here is basically that the wider the definition of main street in practice, the more coverage and the less bias. The narrower the definition, the less coverage and the more bias. Tim guesses a wider definition was used than the guess in Johnson's example. Is this more plausible? I don't think so. To the extent we want to debate these guesses I think Tim's guesses are less plausible in practice than Johnson's for a number of reasons.

There doesn't seem to be a formal definition of what a main street was for L2. So this definition would have to be a post hoc one, simply the outcome of whatever the field team did. At least three factors work against the wide definition hypothesis: the extreme time constraint on the field teams, the fact that they had to discover the streets in question in each neighborhood, and the security issue.

The time burden here is large as they already have to do 40 interviews in a day. In the case of this example, the teams would have had to drive to Kirkuk (from where?), then select the area to work with (how big is the area?), then travel around that area trying to find every "main street" in the area to put in their list. If a very wide definition of main street is used, their job of discovering and enumerating the main streets becomes harder, more time consuming and more dangerous. If a narrow definition is used the job is easier and takes less time, and they can get right on to cramming those 40 interviews into whatever is left of the day. Then there is security. The wider the definition, the more time they have to spend driving or walking around these strange neighborhoods looking for all the main streets while exposed to whatever dangers there might be.

There appears to have been little in the way of any specific criteria or conditions about what had to be called a main street. This was up to them to decide. Did they choose to make their job as difficult and time consuming and dangerous as possible, or did they choose to make it less? Which is more plausible?

Let's bend over backwards for the Lancet study as Tim is doing and assume the field teams chose to make their job as hard as possible and used a wide definition of main street. They would then have to travel around the selected area (again how big is this area?) looking for all the main streets to list them. In the best case scenario, they find them all, but that is only the best case. In every other case they don't discover all these widely defined main streets to begin with. Or, they get a couple of the most obvious ones down on the list and then stop traveling around looking for more main streets that they don't even really know are out there to find. The same problems would apply to enumerating the cross streets.

More plausible, I think, is that in practice the definition would be narrow. They'd grab the most obvious main streets and get to work.

Do you have a comment on Sod's observation that the criticisms here have focused almost exclusively on Baghdad?

Yes. The observation is wrong. The map in question is Kirkuk, for example. There are three maps shown on the MSB page linked above and none are for Baghdad.

Why do you believe that they didn't have a (practically) equal chance of inclusion?

Because the sampling methodology does not establish an equal chance, practically or otherwise. In the best possible scenario it would give a very highly unequal chance of inclusion for each house, but some chance for all. One of Tim's claims is that his guess about the yellow lines would get every house into the sample frame for the piece of the map he chose because of possible spillover onto tertiary streets when progressing through the 40 houses. But even if this were to get every house some chance of selection it would be nothing like an equal chance of selection.

This issue of progressing through the 40 houses also raises another can of worms. It is again a problem of vague sampling protocol and lots of discretion left to the team in the field to do whatever they want. Tim uses the phrase "neighbouring the start house" to describe how the teams progress from the start house to the rest. The Lancet authors have used the words "adjacent" or "nearest". All are vague and allow discretion (and therefore bias) to enter in. For example, the team does it's start house, there's a house on either side that are both about as "near" to the first, they are both "neighboring" or "adjacent". The team looks to one side and see a row of bombed or battered houses. They look to the other side and don't see anything like this. Which way do they go? They could go either way and still be following the (vague) protocol. This allows the team to both follow their instructions and (consciously or unconsciously) seek out high mortality houses.

The way to progress from house to house is wide open. Do they always stay on the same side of the street and travel in a line from one house to the next? What do they do when they hit a street corner? Do they instead go in a kind of concentric circle out from the start house, where maybe the house across the street in front and behind and the two on either side are the next four "nearest" houses? They could conceivably do any of these or something else and still be within the vague method. This problem worsens if the locals are involved (such as neighborhood children used in the survey) who might try to direct the teams to houses they know have deaths to report. Since there's discretion in how teams progress from the start house, how or why would they refuse a local who's telling them to go to those houses over there? They can't tell them it's against the rules to go to them because it's not. Nor would they want to do something that might insult the locals or make some of them start to question the "benign intent" of the survey. They can just choose to progress in the direction of those houses from the start house. In each cluster they'd have 38 chances to employ this discretion (and introduce bias).

As Kevin noted in the last thread: if we can plug whatever parameters we want into the model, there doesn't seem to be a whole lot of point to this exercise.

If you mean trying to determine the extent of this bias in L2, you're probably right that there is not much point in the exercise. The necessary information isn't there and isn't likely to ever be there. So almost any of the parameters could be the right ones. This of course means directly that there is not much point in deriving an estimate of nationwide deaths from L2 data because it hinges on choosing (knowingly or unknowingly) whatever of these parameters we want. The L2 authors (unknowingly) chose to plug in the parameters that gives them zero bias. These parameters make the number come out to 600,000. Another perhaps more plausible guess made (knowingly) by Johnson is the parameters that make the number come out to about 200,000. Since there's no solid case for either parameter set, or others besides, there's not much point in the exercise of making an estimate based on L2 data if the aim is to discover the level of deaths in Iraq. We may as well cut out the middle man and just guess the number, rather than guess the parameters that produce the number.

If you mean trying to explain or explore the potential for main street bias (what Johnson is doing) in a survey such as L2, then being able to plug a lot of different parameters into a model and showing the wide range of these and their outcomes has a useful point that, unlike L2, advances our knowledge about the topic it is addressing.

Raymond says:

"Johnson et al's presentation of this is more valid than Tim's though because they say they do not know which streets or type of streets the L2 authors (field team actually, not the authors) called main streets in practice. Tim claims to be "correcting" their guess by replacing it with his guess. So he is making invalid claims about his guess while Johnson is not."

One more time - BULLPUCKY!!!

Roberts has said that their main street - secondary street selection procedure was designed to give all houses an equal chance of inclusion. Lambert has shown that it is very easy to make a selection of main streets - secondary streets such that all houses have an approximately equal chance of inclusion. Johnson's scheme is at odds with Roberts description - Lambert's is not.

Johnson's scheme is preferable only if you assume that Roberts is intentionally lying and L2 is fraudulent. You are adding to the despicable behavior here, Raymond.

"Roberts has said that their main street - secondary street selection procedure was designed to give all houses an equal chance of inclusion."

All houses having an equal chance of inclusion is a desired outcome. A design intended to achieve this may succeed or not. The design the L2 authors have described can not claim to achieve this. If there is some hidden elements of their design that do achieve this they should let other people know what they are. That they don't makes such statements seem like so much wishful thinking.

"Lambert has shown that it is very easy to make a selection of main streets - secondary streets such that all houses have an approximately equal chance of inclusion."

No he hasn't. He's shown you can make a guess about the definition of main streets in one corner of a particular neighborhood map that might give all those households a highly unequal chance of getting into the sample.

"Johnson's scheme is preferable only if you assume that Roberts is intentionally lying and L2 is fraudulent."

I don't do "faith" Lee. Nor should science. If you want me to believe a sample design achieved equal probability for all households then, like the maths teacher says, show your work. The design they've described doesn't do it.

Dave Kane has a good posting up on his blog right now with a lot of comments from Roberts about the sampling which contradict each other and can't all be true at the same time. You should check it out, and leave your faith for Sunday.

Raymond and David, thanks for taking the time to reply. The first point which you both raised -- that Tim doesn't describe his guess as a guess -- seems like a non-issue to me. I'm not interested in how he describes his argument. I'm interested in his argument. Suppose you said to me, "Hey, I've got the funniest joke ever. This guy walks into a bar..."

I'll either laugh or not laugh based on the joke. The fact that you told me it's the funniest joke ever isn't going to make any difference.

When I look at Tim's example, I'm using a dirt-simple method to evaluate it: I look at the map, and I ask myself, "What would I describe as a 'main street'?" I think I'd pick the same things Tim picked. I think most people would.

Incidentally, although I read the draft of the Johnson paper, I had not (until now) looked at the web pages Tim linked to, on the introductory pages. You are right: The authors do discuss other locations, and not just Baghdad. The comments there regarding smaller towns, however, seem plainly at odds with common sense: "It is not clear to us how the major commercial street sampling would work when there is no major commercial street available." I don't think any reasonable person would use the same criteria for what constitutes a "main street" in both a town of 1,000 people and a city of 1,000,000.

Raymond, in your explanation of why you don't think the teams would have found all the main streets, it seems to me that your three reasons would lead to a bias in the opposite direction from what you suggest. If the teams were worried about time and safety constraints, and needed to find main streets, a broader definition of "main street" would be easier and take less time. ("Do you think this is a main street?" "No, let's just wander around aimlessly for a while until we find a different one that looks 'main-ier.'")

In your reply to Lee, you said that the survey design can't claim to have an equal chance of including all houses. I don't agree: depending on the implementation of the survey, that may or may not have been the case. Again, the outcome depends on whether or not the chosen "main streets" were at all reasonable. I'm not sure why you say the chance of inclusion is highly unequal, but I'm going to guess that this, too, will come back to an unsettled question of implementation.

Although the discussion in this thread has focused primarily on the inclusion (or exclusion) of particular areas, it seems to me that even if this question were settled, we'd still be left with the impossible-to-calculate ratio of violence inside and outside the survey space. As Kevin has previously noted, the L2 scheme was borne of exigency, and it doesn't seem likely that it will be used again. I don't see how Johnson et al is useful, if divorced from L2. Its only value is in convincing people that L2 has no value... and I'm inclined to believe that most of the people who think L2 has no value had reached that conclusion long before they ever heard of main street bias.

Regards,
Bruce

Yes. The observation is wrong. The map in question is Kirkuk, for example. There are three maps shown on the MSB page linked above and none are for Baghdad.

their [main example](http://www.rhul.ac.uk/Economics/Research/conflict-analysis/iraq-mortali…), the one from which they draw their horrible "visual summary" of "mainstreet bias" is Baghdad.

it is obvious. what Spagat is doing, is choosing only "city highways" with few exits/crossroads. this is, how he keeps the covered area small.

again: so does a GPS approach. bigger houses or "lonelier" houses have a higher chance than others.

There appears to have been little in the way of any specific criteria or conditions about what had to be called a main street. This was up to them to decide.

funny. i tend to recognise the "mainstreets", when i enter a town or village. the one you enter the town on, is one. the big ones leaving that road (mostly leading to other other towns..) are.
in very big cities, it gets slightly more complicated, but even there its no rocket science.

For example, the team does it's start house, there's a house on either side that are both about as "near" to the first, they are both "neighboring" or "adjacent". The team looks to one side and see a row of bombed or battered houses. They look to the other side and don't see anything like this. Which way do they go?

"bombed and battered houses" will often be abandoned, leading to a NEGATIVE bias, reducing (polled) mortality!

how much experience do you have with polling? imagine you do a telephone poll. one voice on the line is a nice young female one, the other a barely understandable elderly man. who will you try harder to keep finishing the poll?

The L2 authors (unknowingly) chose to plug in the parameters that gives them zero bias. These parameters make the number come out to 600,000. Another perhaps more plausible guess made (knowingly) by Johnson is the parameters that make the number come out to about 200,000. Since there's no solid case for either parameter set, or others besides, there's not much point in the exercise of making an estimate based on L2 data if the aim is to discover the level of deaths in Iraq.

let me see, whether i got this right: the methodology of the Lancet authors, made with the aim to give an equal chance of cover for houses is "as plausible" as the Spagat guess at what happened, which is based on a choice of streets that contradict the target of the study?

this is insane!

In common parlance, a road is a public right of way wide enough for vehicles. A street is a stretch of road with lots of houses, shops and public buildings on it. Industrial estates have roads, not streets. On the map, the red line is a road, the yellow a street.
It beggars belief that any not half-witted survey team with a quota to meet would start from unpopulated stretches of road.

James Wimberley must be from UK (or UK-educated) given the way he distinguishes between streets and roads, which highlights a major misunderstanding on the part of all non-US people on this thread:

**The L2 scheme was designed by a US team. In the US, the "main street" of a town is the major highway. It does not mean a small street in the UK sense.**

So the main street for a US person is a "city highway" as Sod says. People do not tend to live on the main street. L2 started their survey scheme from "main street" in the sense that they then choose cross streets. So very few properties, if any, get included from the main street.
Likewise, in the JPR paper, the main street can happily be included in So, i.e. outside the survey space. The survey space includes cross streets to main streets, not necessarily the main streets themselves.
So the criticisms of JPR based on misconceptions (i.e. non-US interpretations) of "main street" are not valid. The L2 team included residences on cross streets, not main streets. Likewise JPR is completely consistent with main streets being outside the samplable space.

what part of Kirkuk is that on this [map](http://www.lib.utexas.edu/maps/middle_east_and_asia/karkuk_2003.jpg)?

Bruce:
"The design the L2 authors have described can not claim to achieve this."

Dude, Lambert just showed a map that matches what the L2 authors describe, and that does exactly that.

Lee wrote: "..what the L2 authors describe..."

Until we all know what the "main streets" were for L2, any claims are debatable.
The problem is that the L2 authors did not describe this is in sufficiently clear detail to avoid these repetitive debates.

Personally I don't consider either Tim or JPR "right" or "wrong" in their interpretations of the implementation of the scheme. But I do wish I know something more from L2 to narrow down the discussion.

Side note: In terms of applicability of JPR more generally, I personally find it a useful and original contribution since many areas of medical imaging (my field) come across a very similar situation, where the objects which pick up tags (and hence can be seen) may be a biased sample of the population which are actually reacting, and the precise area of the sample being investigated under the microscope may also be biased.

sod,

I think the east-west road that Tim is taking as his main street is roughly where you see the words NAHIYAT PIRYADI on your 2003 map. I fact I wouldnât be surprised if it was just that map which prompted Sean Gourleyâs choice of colours. But the roads in red on that map seem more like major traffic arteries â the sort of routes you would take if you were passing through Kirkuk, or driving from one side of town to the other.

Incidentally, your 2003 map highlights two things: (1) Gourley made lousy choices for his âmain streetsâ, picking one which is actually at the eastern edge of the town; and (2) Kirkuk is a very big place. My guess is that, in such a large city, the JHU team would have sampled their âmain streetsâ from a map (or maps) of the entire area, narrowing down their target cluster to within a few square miles before they even drove to Kirkuk. Thatâs what Iâd do anyway.

James Wimberley is, I am reliably informed, an Englishman raised in the Channel Islands. But I wouldn't be too quick to assume that "all non-US people on this thread" have got the wrong end of the stick. The Lancet is a UK publication and the authors of the JPR paper are mostly from that side of the pond. Actually, if it comes down to dialects, the crucial question is, what does Riyadh Lafta understand by main street? Given Iraq's history I think it's very likely that his English derives from British sources.

None of which takes anything away from Tim's point: the efforts of Johnson et al. to derive n = 10 from that map should have prompted the editor of the JPR to ask what they were playing at.

Kevin Donoghue, you have it wrong. The authors of the L2 paper are mostly from the US, so it is their definition of "main street" which is under debate, since that is what would have formed the basis for the L2 survey plan. Looking at the JPR maps, it seems to me that JPR proceeded based on a US interpretation of "main street".

The possibility that you raise that Lafta may have interpreted "main street" differently, because of British-biased education (your suggestion, not mine) then makes the whole L2 survey even more questionable. You are therefore suggesting that Lafta is the only one who actually knows what main streets would then have been included!! This would indeed be a serious scientific oversight of the US L2 team, since they would be co-authoring a paper (as lead author etc) based on a study where they do not know exactly what constituted a main street in practice.

You just shot down L2.

JPR team offered a theory of a generic situation of sampling in a population where all objects are not equal in terms of sampling. Very interesting. Whether their theory is general enough, or too specific or just right, depends on what you want to do with it. They then offered some values for a specific case, L2, and invited readers to use their own values. Seems like good science to me....

By the way, if you take the JPR interpretation of "main street" from their maps (which they freely admit are just a guide) then I count about 10 lengths of road which is non-samplable, for every given length of road that is samplable. Assuming a constant housing density per length of road, then n=10. So JPR are not unreasonable.

(Note that counting numbers of roads is not the issue, it is lengths of road if we assume constant housing density on residential roads. If we don't assume that, then we are off onto another debate, and another can of worms for this thread).

'good science....'

Wouldn't that involve doing something like, I don't know, testing the model?

LancetStudy,

Well if you say I have it wrong, thatâs that I suppose. Here is the passage which led me astray â and it still does despite your best efforts to show me the light. I hope youâre not going to ask me where I found it.

G Burnham, as principal investigator, was involved in the study design and ethical approval, took part in the analysis and interpretation of results, and led the writing of the paper. R Lafta managed the field survey in Iraq, participated in the study design and the analysis, interpretation, and preparation of the manuscript. S Doocy managed the study data and was involved in the analysis, interpretation, and the writing of the manuscript. L Roberts instigated the study and assisted with the analysis and interpretation of the data and the writing of the manuscript.

To me, that means that Riyadh Lafta was in the driverâs seat. Iâm not prepared to entertain the notion that my Hiberno-English-biased education (as you would presumably term it) is causing me to misread that paragraph. Iâm really quite sure that what it says to me, is precisely what it says to Richard Horton.

I note that you are shocked, shocked at the suggestion that a bunch of Americans might have got into something where they did not know exactly what they were doing. That makes me wonder where youâve been for the last six years or thereabouts. Personally I give a bit more credence to the JHU team precisely because they had the good sense to give the crucial task to a group of Iraqis whom they deemed to have the required skills. If that invalidates the whole study in your eyes, so be it. Perhaps the guys who are losing their jobs on Wall Street will team up with former employees of the Justice Department to produce the sort of study you feel you can trust.

The attempts to defend this paper have turned grotesque.

Are these guys really arguing that because in the US 'main streets' means 'the main street' (which is absurd), that street which are clearly among the main streets on the map are to be teated as secondary streets?

Becasue I'm from teh US, and that definition of main street is absurd beyond words. I drive up and down the California central valley frequently. When I do, I often stop at small freeway towns (5,000 - 20,000 people) to find a 'taco truck' to get lunch. To find one, I simply get off the freeway and DRIVE AROUND ON FREAKING MAIN STREETS!!! MULTIPLE. PLURAL.

They are easy to recognize, clearly distinct from secondary streets, there are always more than one even in small towns, usually more like 3-5, and that map at th etop fp this thread clearly excludes a main street from its main street category.

And I am from the US.

Lee: How many people live on your taco-rich "main street"? Very few. They are main streets/high streets in UK terminology. Few people live there, they are not in the sample space of L2 or JPR. They are just a starting point for the cross street method which L2 used. But knowing what they are is crucial since they are the starting point.

Jody: On this topic, many US citizens would consider Euston Road a "main street". There are no markets on Euston Road, nor are cars allowed to park. Few people also live there in relative terms compared to rest of London. Instead, the action in terms of markets, cars parked, people walking and hanging around is on streets off of Euston Road. And people who live on streets off of these, sufficiently close, will walk there and tend to spend more time there. People who live way off will not.
Your analysis using London is from the perspective of someone who defines "main street" according to a European definition.

Kevin D: I am afraid you hit the nail on the head, albeit unintentionally, when you raised the possibility that there are "main street" definitions which Lafta might have used which differ from the US L2 team. So how can the US L2 team be so sure that no MSB exists? They cannot. How can they put their names first etc. on a paper where they do not know what the sample space actually was? They should not.
This is clearly why the AAPOR and JH college themselves, are worried enough to start investigations. Your argument validates this action.

Pre-empting somebody's comment that the subject of this thread is JPR's n=10 value, and hence that users should address the n=10 issue, I have above. Here it is again:

If you take the JPR interpretation of "main street" from their maps (which they freely admit are just a guide) then I count about 10 lengths of road which is non-samplable, for every given length of road that is samplable. Assuming a constant housing density per length of road, then n=10. So JPR are not unreasonable. (Note that counting numbers of roads is not the issue, it is lengths of road if we assume constant housing density on residential roads. If we don't assume that, then we are off onto another debate, and another can of worms for this thread).

But the most important outcome of this thread, apart from this point, is the bombshell issue that Kevin Donoghue has raised about Lafta and Burnham/Roberts not being on the same page with regards "main street" definition. Should we have a thread on this?

(P.S. What Lee, Tim, Kevin, Jody guess as constituting a "main street" in the L2 survey sense, is irrelevant. What L2 define as a main street was, I had assumed, the main issue. But now Kevin has raised the important point that Lafta's definition is actually a major issue. What a mess for L2!)

'grotesque...'

Burnham et al central estimate of excess mortality 650 000 (390 000 - 940 000), allowing Johnson et al their guessed at R = 3, central estimate becomes 217 000, not sure what it does to the confidence interval.

Just to put this grotesqueness into context.

Lee, re your comment #46: I did not say that. That was Raymond, in #40.

Regards,
Bruce

LancetStudy: But now Kevin has raised the important point that Lafta's definition is actually a major issue. What a mess for L2!

Actually I just pointed out that Lafta managed the field survey. People who had read the paper closely already knew that.

BUT THEY EMPLOYED AN IRAQI!! IN IRAQ!! WHAT A MESS FOR L2!!

WTF? I mean really, is this what the defenders of Johnson et al. are reduced to?

Jody wrote: 'grotesque...'. Yes, I agree. I think that everyone agrees that any death more than zero is grotesque. I am sure that the JPR team think the same. That is not the issue. The issue is to get to the bottom of whether science is being done in a reasonable way. To the extent that conflict surveys become a scientific method, then this method needs to be examined. If it is useful, great. If it needs to be corrected, then we should know -- and we should try to estimate what these corrections might be. Only then will the method be treated as credible and full credit given to survey teams etc.

So yes, definitely grotesque. The question is: reasonable in terms of scientific standards or not? We are not doing any favours to anybody if it is not. 200,000 does the job in terms of being grotesque -- no need to have an inflated figure, particularly if it then detracts from the main issue of how to prevent and stop wars.

KD wrote: "...is this what the defenders of Johnson et al. are reduced to?..."

As a whole, we readers of this thread are left with the remarkable issue that you helped fuel: Do *all* the L2 team know what main streets were possible candidates, and do they *all* agree on this? If they do not, then where does L2 stand?

I have an experiment that I did with someone, and the results are X. Well actually, I wasn't there when they did the experiment, but the results are definitely still X. Well actually, I don't know line-by-line what exactly they did, but the results are definitely still X. Hmmm???

I hope the investigative teams read these threads. This should help them with some of the issues.

As for JPR (which actually should be the least of your present worries if you are an L2 defender and have followed the above thread):

1. JPR article, and the possibility of MSB, stands strong (stronger, arguably, after these discussions)
2. the JPR maths is correct
3. the JPR estimates are indeed estimates, but not unreasonable ones given the uncertainty all round about L2

JPR does exactly what it says it does in the abstract. It does what it says on the can.

I'm sorry Kevin, but like it or not, you said in post 49:

"..the crucial question is, what does Riyadh Lafta understand by main street?"

You said it. I happen to agree with it, but you said it. You hit the nail on the head about why L2 should be investigated.

Jody wrote: "...Wouldn't that involve doing something like, I don't know, testing the [JPR] model?"

Sure, just tell the JPR team the main streets that were used. Failing that, what were the possible main street candidates? Failing that, what did Lafta understand to be a "main street" to use as a starting point? Failing that, what did US L2 team actually write in words in their design description to Lafta to translate the "main street" selection scheme?
Failing that, how did US L2 team conclude that Lafta's implementation of the "main street" scheme gives R near to 1?

Of course, none of you know the answers to these questions. And yet you are surprised L2 is being investigated? Duh??

I guess Ron missed us.

I will repeat (from 36), becae you guys are confirming it fo rme.:

"Roberts et al were designing a survey under very edgy ethical conditions. There was a credible very-non-zero risk of death to the surveyors, and to the people being surveyed for answering the questions. He was and is ethically constrained to to do everything possible to minimize the risk to the people being surveyed, and he designed the procedure with that in mind.

I suspect that this is intentional on the part of Robert's critics. I know that it is despicable behavior."

the map i posted above is from 2003.

i have serious doubts, that the Lancet team was cruising around towns, searching for "mainstreets", when maps were easily available.

i also have doubts, that they would chose "mainstreets" only, that have "city" on one side only.

wouldn t you expect 4 iraqis to know at least one of the big towns they have to poll?

and the problem (as i said multiple times now, and still is being ignored by the "denialists", the problem is much smaller in smaller towns..)

Lee wrote: "Information that was even borderline related to ways to identify surveyed neighborhoods or people was simply not transmitted to him [Roberts], in order to maintain the confidentiality of people who were potentially at risk of their lives for answering the survey."

Yes, I absolutely agree that safety is paramount. No question.
**However**, that compromise brings us straight back to what I said earlier: **Given this admitted lack of information flow (for safety reasons) between members of the L2 team themselves, how can the US L2 team (i.e. Roberts and Burnham) be so sure that no MSB exists? They cannot. And so how can they put their names (in particular, as first author) on a paper when they do not know what the sample space actually was? They should not.** That is, I would think, one of the reasons (or the reason) behind the investigations.

If safety is such an issue, then be prepared to correct for any biases that might be introduced due to the safety constraint. Don't just claim they are not there because you cannot estimate them.

Sod wrote: "...the problem is much smaller in smaller towns..."

Let's assume the 'problem' Sod talks about is the uncertainty of main streets. In that case, it would be helpful if Sod could quantify his statement. What does 'much smaller' mean in numbers? Does he have a quantitative model for this, other than just words and presumption? In short, does he know what was done in terms of picking main streets? Saying that main streets are obvious when you see them, is hardly a scientific approach.

More generally, can someone offer an answer to the questions I have posed several times: Given the admitted lack of information flow (for safety reasons) between members of the L2 team themselves, how can the US L2 team (i.e. Roberts and Burnham) be so sure that no MSB exists? And so how can they put their names (in particular, as first author) on a paper when they do not know what the sample space actually was?

Silence will be taken as a 'We don't know'.

And as for the rest of Sod's statements above: "i have serious doubts, that the Lancet team was cruising around towns, searching for "mainstreets", when maps were easily available. i also have doubts, that they would chose "mainstreets" only, that have "city" on one side only. Wouldn t you expect 4 iraqis to know at least one of the big towns they have to poll?"

So, was the main street selection criterion through maps?? Or driving around?? Or asking locals for choice of main streets?? Or by choosing a fairly symmetric distribution of city on either side?? Or .... what?? You don't know, nor do I, and nor does anyone reading this thread. And nor (it seems) do the US L2 team. So, again, I ask:
How can the US L2 team (i.e. Roberts and Burnham) be so sure that no main-street-bias exists, if they had no strict consensus with Lafta about what "main streets" were (as per Kevin Donoghue's suggestion)? And so how can they put their names on a scientific research paper based on surveying, when they do not know the sample space -- and they have no way of quantifying possible biases in the sample space?

By suggesting that Roberts and Burnham didn't know exactly what Lafta did with regards the precise main-street selection, let alone the actual choices, you are shooting down L2. Kevin Donoghue started it, and now you all seem to be agreeing!

My final comment, is a direct challenge to Tim Lambert, who seemed to start this thread with the notion of shooting down the JPR 'theory' taken to mean their parameter estimates. But since n, and also the f's and q's, depend on the choice of Si and So, you are actually attacking the choice of Si and So made by JPR when estimating these parameter values. Specifically, you are attacking the JPR suggestion of what might constitute main streets, as confirmed by your focus on the Google map at the start of this thread. You think X are main streets, they think Y are main streets - and because of the differences, different parameter values emerge. No surprise.

Unfortunately for you, since the top of the thread, your fellow L2 supporters have started to offer a range of rather different methods for choosing main streets. From 'it is obvious', to asking Iraqi opinion, to using esoteric British road vs street definitions, to driving around looking for taco trucks! You yourself used Google maps, a method which is of course biased toward saying that a main street is a wide street when viewed from above (irrespective of levels of traffic etc.)

But what all this really comes down to, is "how did Lafta's team choose main streets"? I doubt very much that they looked for taco trucks, so they must have followed some other scheme. But which?

I don't know, you don't know, nor does anyone else reading this thread -- and the great revelation to me, prompted by Kevin Donoghue, is the fact that **nor do Burnham and Roberts know**. So there is indeed no point asking them. I get it now.

But now there is a very, very interesting scientific problem for you. What exactly was the sampling performed in the study that you support? How can you be so sure that no MSB exists in this sampling, without knowing anything about the MS's (i.e. main streets)? And how can you support the quantitative conclusions of a survey-based paper (L2) containing several authors (i.e. Burnham, Roberts) who themselves do not know what the sample space was?

Of course you can say "No I don't believe JPR", but believing and scientifically rejecting are very different as you know as a scientist yourself).

"LancetStudy", please tell us, what is keeping you from doing your own study in Iraq?
none of us is holding you back!

the big picture is this:
to show a problem, Spagat needs to show that the Lancet method causes a bias in households chosen (1), that these households have a significantly higher deathrate (2) and that this lead to false results of the study (3).

he has done none of these.

small village will often only have one mainstreet. it is the one, on which you entered the village. and the 40 households rule allows for more "penetration" beyond the crossstreet, in a place that only consists of 100 households.

So, was the main street selection criterion through maps?? Or driving around?? Or asking locals for choice of main streets?? Or by choosing a fairly symmetric distribution of city on either side?? Or .... what?? You don't know, nor do I, and nor does anyone reading this thread. And nor (it seems) do the US L2 team. So, again, I ask: How can the US L2 team (i.e. Roberts and Burnham) be so sure that no main-street-bias exists, if they had no strict consensus with Lafta about what "main streets" were (as per Kevin Donoghue's suggestion)? And so how can they put their names on a scientific research paper based on surveying, when they do not know the sample space -- and they have no way of quantifying possible biases in the sample space?

a mainstreet is something different in a village than in Baghdad. the strict definition that you and Spagat want, would have made work on the ground more difficult, not eassier. a "functional definition" (need enough mainstreet to achieve reasonable coverage) is fine for me.

all those criterions are fine. (even if we know now, that they didn t ask locals, this would have been a reasonable approach).

i don t know the details about most polls i read. (actually i know more about the lancet one than about many others together) with most polls, you will never know all details. (marketing or political polls keep some parts of their methods secret)

the Lancet results are in very good agreement with basically all other estimates of mortality in Iraq.

this is a huge problem, for all these bias claims.

Spagat has shown ZERO evidence, that supports a "mainstreets" being significantly more dangerous to people living nearby. he has at best done a sloppy (biased might even be a better term..) job, in showing a lack of "coverage".

I had another look at Tim Lambert's map (and the area immediately above it). I think Lambert has some explaining to do:

http://dissident93.files.wordpress.com/2009/02/msb-lambert.jpg

Sod wrote:"all those criterions [for choosing main streets] are fine"

Sure, **if** (and only if) resulting biases are then corrected (e.g. using JPR).

Sod wrote: "a mainstreet is something different in a village than in Baghdad"

Of course. So what is a "main street" in Baghdad? Tikrit? Basra? When we know that, we know Si using cross street selection. When we know Si, we know So. Then JPR can make improved estimates of n, f and q.

**Given the admitted lack of information flow (for safety reasons) between members of the L2 team themselves, how can the US L2 team (i.e. Roberts and Burnham) be so sure that no MSB exists? They cannot. And so how can they put their names (in particular, as first author) on a paper when they do not know what the sample space actually was? They should not. End of story.**

Robert, since it's past midnight in Australia, if you want a quick answer I'll hazard a guess as to Tim's meaning. Your own "junction A" suggestion seems to fit the bill. I take it that "above" refers to the map above Tim's text, since he used the word twice in the same phrase and the first reference is clearly to his map.

Thanks, that's what I thought (the road at the left going upwards through "junction A"). If that's the case, I think Tim's got an uphill struggle demonstrating that it's an "obvious" main street, since it appears to be nothing of the sort.

I said I wasn't going to post any more in the bowels of this thread, so I guess I'll leave it until Tim's sequel, MSB Debunk III: This Time it's Personal.

I've amended the text slightly, as Kevin is probably right about the meaning of Tim's wording, "one that it is off the map above".

New version:
http://dissident93.files.wordpress.com/2009/02/msb-lambert-2.jpg

What Tim has to remember as he liberally defines streets as "main streets" is that he is automatically increasing the samplable space Si such that the survey team would (by definition) have to then be able to cover these distances and enter these neighbourhoods. In the end, if n=0, this means by definition that Si covers all of the governerate. So the survey team would have to be able to go **everywhere**.

Since they did not go everywhere, by their own admission for safety reasons and need for speed, we are back in the same position. What roads were left out? In other words, what is the space So in practice for Lafta, and hence what is Si for Lafta (since it is I-So in set formalism)? Only when we know this will better estimates of Ni, No, fi, fo, qi, qo be possible.

More generally, how different is Lafta's implemented Si from Burnham and Gilbert's imagined Si? They don't know, so how can the US L2 team possibly claim R is near 1 for Lafta's implementation?

Robert, AFAICT from Google Earth it's probably on a par with Oxford Street, London or Parnell Street, Dublin - going by the fact that it remains clearly visible as I zoom out from it. Those two streets are main streets in my dialect at least. Also if the accompanying photgraphs are correctly located (can't be sure of that of course) then it's a bustling street with a mosque and other prominent buildings.

I would suggest that those who have any doubts about this use Google Earth to do a comparison with main streets they know.

Kevin Donoghue writes:

it's probably on a par with Oxford Street, London or Parnell Street, Dublin - going by the fact that it remains clearly visible as I zoom out from it.

Come off it. I know rural lanes that are as visible. It looks half the width of Tim's other "main" street. The presence of a mosque doesn't make it a "main" street - or does it?

Given the admitted lack of information flow (for safety reasons) between members of the L2 team themselves, how can the US L2 team (i.e. Roberts and Burnham) be so sure that no MSB exists? They cannot. And so how can they put their names (in particular, as first author) on a paper when they do not know what the sample space actually was? They should not. End of story.

you are using the term "MSB" as if it was a long standing,we ll known and defined phenomenon. it actually isn t. MSB is an invention by Spagat, and he has brought up ZERO evidence, to support his claim, that people living in those areas have a significantly higher violent deathrate.

Tim: I hope you can appreciate how the above to-and-fro exactly sums up the main-street weakness in L2. Even your own supporters cannot define what a "main street" is after 100+ (total) posts, so imagine what the conversations would have had to be between Lafta and Burnham/Gilbert, then Lafta and his team, in order to pin it down prior to survey-day? Impossible.

Sod writes: "you are using the term "MSB" as if it was a long standing,we ll known and defined phenomenon."

Er, it has existed since October 2006 by my understanding, and has been published.
L2 has existed since October 2006 in published form.
Both are 'long-standing, well known and defined phenomenon'.

Your point is...?

Maybe Sod has raised another crucial new issue here, intentionally or unintentionally: Maybe previous/other studies of conflict mortality implicitly contained MSB as well! I hadn't even looked into that. Oops, we should. Has this cross-street-to-main-street been used before? Please let me know. It would be very, very interesting to go back and look at such cases to see what was concluded.

Remember my point is: Yes this cross-street-to-main-street method could conceivably be used, but only if an estimate of MSB is made. It is such an obvious detail to wonder about, after all, even without the MSB name and prior to JPR article. If it is near 1, fine -- but show how you obtained that estimate based on maps, taco trucks, or whatever... Just saying "R is near 1" isn't convincing (hence hundreds of postings on these sites...)

Sod wrote: "MSB is an invention by Spagat"

Er again.... That is what original research is. New results not previously thought of or published. So yes, I guess Spagat et al. are guilty of 'inventing' the term MSB -- but they are not guilty of inventing the effect. To the extent that R is not **exactly** equal to 1, the effect is generated exclusively by the L2 team's failure to examine the Lafta implementation, and adjust their estimate of mortality for any street-selection bias.

Basic scientific thinking -- even if R then turns out to be near 1, somebody has to account for such a source of potential bias during the final analysis prior to publication. Amazing that L2 apparently didn't think of doing it prior to JPR.

Robert Shone: It looks half the width of Tim's other "main" street.

And Tim's other main street looks about the same width as the one the MSB squad chose, which runs vertically at the right of their map. Actually I think the one Tim concentrated on looks a bit more prominent. As for the one you are contesting, which may or may not be the one Tim has in mind, I think it's going through the old part of the town, with the Kirkuk citadel on one side and that mosque (a pretty big one judging by the dome) on the other. It's not unusual for streets to be narrower in older parts of a city.

I refer you again to the examples I gave: Oxford Street looks a lot narrower than Marylebone Road, just to the north of it. So what? Surely Oxford Street is a main street? And Parnell Street is of course only a fraction of the width of O'Connell Street. But they are both visible relative to their surroundings - just about - when I zoom out to an altitude of 12,000 feet or so on Google Earth.

AFAICT for someone doing a survey it would be an obvious choice of main street, assuming they wanted to give as many households as possible a fair chance of inclusion. If you're assuming anything other than that, then you must think they were either dishonest or sloppy; and if you think that, what do you need Johnson et al. for?

Thanks, Kevin, I get the gist. From 12,000 feet up, you think it has similarities with Oxford Street. That makes it an "obvious" choice as a "main" street.

Your "politeness" overwhelms me, Robert. There was really no need to thank me. Just to observe your virtuosity with scare quotes makes all my efforts worthwhile.

Kevin D: The real issue has moved on from "what is a main street" to "how could Burnham/Roberts dismiss street-bias **without** knowing the streets that were samplable?"

**Let's be crystal clear on this: Since R depends on Si, how could Burnham/Roberts state R=1 effectively without knowing Si?**

**R is defined through Si !!**

In terms of the more general bias question "Is R=1 or not?", it matters little whether the scheme used involved main streets, very curvy streets, streets with the letter "s" in them, or streets with taco trucks -- or whatever. What matters is: Is the scheme that was used by Lafta (whatever that was) open to street-bias? And most importantly, how can Burnham/Gilbert say it isn't (i.e. R=1) without knowing a typical Lafta-class of streets (i.e. without knowing Lafta's Si) or even possible Si's that Lafta's scheme would have employed? Actually, you helped raised this issue yourself.

Remember, unlike a disease which tends to spread over large distances without regard to street design, violence is largely fought by people on streets. So this issue is crucial. Violence may have a systematic predominance in particular street topologies, so there is an important potential source of bias depending on street selection process.

And Sod has now raised the amazing possibility that this L2 technique has been implemented in other conflict studies. (That is very, very interesting and I would like to pursue this).

Kevin: One of the main values of the JPR paper is that is provides a concrete framework for this whole discussion. Parameters, estimates etc.

(Tim: Please take this as a personal and informal side comment, from a colleague. With all due respect, maybe you should revisit the title of your thread. Stating that something is 'badly flawed' sounds like something a lawyer would have a field day with in terms of libel. I am no lawyer, but how do you know lawyers don't read this? Since that whole investigation issue arose, maybe we should all be careful about direct accusations. Questions are fine, and valid, and important. Opinions as well, but more direct things are perhaps more dubious....)

Even your own supporters cannot define what a "main street" is after 100+ (total) posts, so imagine what the conversations would have had to be between Lafta and Burnham/Gilbert, then Lafta and his team, in order to pin it down prior to survey-day? Impossible.

my functional definition from #73 is fine. this is just one of many false claims you make!

i have two answers to this:

1. this is the answer i have given before: the lancet method was chosen, because of an EMERGENCY. with GPS becoming smaller and cheaper and google maps, it most likely wont be repeated.
now ask yourself: why did Spagat chose an example for his THEORETICAL paper, that isn t a typical case and wont be repeated? one that (according to them) has multiple other problems, one among them missing vital data?!?
(my answer: PUBLICITY!!!)

2. on the other hand, you are upon something BIG. while there aren t many casualty studies using this approach, you will find thousands of example in marketing!
before telephone polling became state of the art, people did real polling, on real streets. mainstreets, mostly! according to the Spagat paper, the majority of people polled on a mainstreeet (those are the ones who would be killed by a bomb) are living just around the corner, in a street crossing that mainstreet! for decades (?) marketing decisions were based on a small special subset of people, those living with a mainstreet bias!!!

sorry for the font above. the list seems to be using some auto-function.

Er again.... That is what original research is. New results not previously thought of or published. So yes, I guess Spagat et al. are guilty of 'inventing' the term MSB -- but they are not guilty of inventing the effect. To the extent that R is not exactly equal to 1, the effect is generated exclusively by the L2 team's failure to examine the Lafta implementation, and adjust their estimate of mortality for any street-selection bias.

slow again, for you: MSB has two parts in the Spagat paper:

1. MSB meaning that Lancet concentrated on polling mainstreet areas. the argument that Spagat makes on this point, is at best weak, as Tim and others have demonstrated.

2. MSB meaning that more people LIVING in that area get killed by violence. Spagat offers ZERO evidence to make this point. none. nada. zilch.

your claim that the L2 authors should act on such a weak problem, is weak. at best.

so you claim that diseases don t spread better, in places where many people meet?

according to Spagat, diseases should concentrate on people living along mainstreets.....

Tim: Please take this as a personal and informal side comment, from a colleague. With all due respect, maybe you should revisit the title of your thread. Stating that something is 'badly flawed' sounds like something a lawyer would have a field day with in terms of libel.

Tim is stating facts. he is attacking the paper, not making claims about the authors and fraud.

other have made much harder accusations. against the lancet authors. you must have missed this, by chance....

Lancetstudy says:

>Tim: Please take this as a personal and informal side comment, from a colleague. With all due respect, maybe you should revisit the title of your thread. Stating that something is 'badly flawed' sounds like something a lawyer would have a field day with in terms of libel. I am no lawyer, but how do you know lawyers don't read this?

for those who haven't guessed yet,

LancetStudy = ron = ozzy = Nick = Lancet Debunker = Tell

And they are all sock puppets for one of the authors of the MSB paper.

Robert Shone, the other main street I added was the one that runs through your intersection A, intersecting with your secondary roads 1 and 2.

Tim Lambert writes:

Robert Shone, the other main street I added was the one that runs through your intersection A, intersecting with your secondary roads 1 and 2.

Thanks for confirming it. In that case I think your map loses all credibility, since the best you can say for one of your cross roads (road 1 in my map) is that it joins another cross road (2) at a junction (A), and that another road which you have arbitrarily designated as a "main" street runs through the same junction.

That's very weak, and I'm not surprised that you didn't illustrate this part of your selection scheme in your (arguably misleading) map.

Maps.

So although this certainly doesn't help with q or f0 or fi, looking at maps perhaps helps with n.

I've been playing around with QGIS and OSM, with some files from GEOFABRIK. It's a bit tough for a non-compsci, non epidemiologist but actually not too bad once you get into it. So far I think I can tell you that for Enfield, using the sampling process as in Burnham et al, with main street being strictly motorway, trunk or primary road, i.e about 6 roads only, N seems to be for this map around 2.5 judging by road length samplable.

Anyhow you can pull data directly off OSM, and hence for anywhere you like in the world. You can classify it and extract intersections, coordinates, lengths and such like in QGISS, and you can export files to good old Open Office or some such to do the maths.

Diseases.

Yes this is interesting. Cluster surveys have of course much earlier been used to study not only vaccination coverage, but for example diarrhoea prevalence. As you might imagine inhomogeneity is important in such cases. As you also might imaging there is a reasonable literature on what to do - to account for 'design effect', the proper term for which we have the euphemism 'main street bias'. So you'll note that Roberts et al 2004 specifically did their analysis with methodology taking into account 'design effect'. Perhaps tellingly the seminal paper from Johnson et al doesn't mention 'design effect', or the scientific discussion thereof even once.

Robert, I take it that you are no longer disputing that the big street running along the bottom of my map is a main street. So even if you don't count my second additional main street as a main street, the only change to my map is that the yellow street that clips the upper right corner goes away, resulting in a small blue area around that street and the rest of the map reachable and n at most 0.1.

And I know that you will now make a big song and dance about how this proves that the Lancet methodology is completely unknowable, but whether n = 0 or 0.1 makes no significant difference to the results of the Lancet study.

And I note that no-one is disputing my point that the blue areas should not include the 39 neighbours of the start house. This makes a huge difference and the MSB authors somehow forget to include this in their map.

I've written a new blog piece expanding on the areas which Lambert misleadingly left out of his map:

http://dissident93.wordpress.com/2009/02/22/msb-lambert-update/

If you blow up the Google map, just about every street labeled as secondary by the Sapgatti is as busy or busier than one or more of the streets they label as main. Whatever. Look at the maps

Tim claims:

And I note that no-one is disputing my point that the blue areas should not include the 39 neighbours of the start house.

Untrue! I am disputing it. Recall what L2 itself claimed:

On the residential street, houses were numbered and a start household was randomly selected. From this start household, the team proceeded to the adjacent residence until 40 households were surveyed.

I (and, I think, the MSB folks) interpreted this to mean that households had to be on cross streets to be in the sample. That is, at least, one reading of what "adjacent" means. Do you have proof that your definition of "adjacent" is more correct that Johnson et al's?

No. I have shown conclusively that anyone who claims to know the details of the sampling procedure for L2 is lying.

Now, your guess that the string of houses starting from house X could go down all sorts of side streets might be correct. Similarly, Johnson et al's guess that "adjacent" means on the same street as the starting house might be correct. No one knows.

By the way, although this is somewhat forward of me, I would appreciate it if Tim would start a new thread devoted to just the issue of what we know and don't know about the sampling plan used by L2. It would be helpful to gather some of the collective wisdom of L2 defenders on this point.

Tim Lambert writes:

I take it that you are no longer disputing that the big street running along the bottom of my map is a main street.

I do dispute that it "is" a main street. But only because that's a meaningless statement. We have no universal main-street-o-meters. We have to first agree on what defines a main street, then we can make meaningful statements about whether streets fit that definition. I think the Lancet authors defined main streets as "major commercial streets or avenues", which isn't much help, as the MSB team rightly pointed out.

Perhaps the best definition for the Lancet study is: main streets constitute that set of roads which when selected as "start" roads will guarantee, via the Lancet sampling scheme, n=10. Of course, that doesn't help in their selection, unless one knows in advance which roads will guarantee that result.

n=0, I mean, ;)

"LancetStudy = ron = ozzy = Nick = Lancet Debunker = Tell

And they are all sock puppets for one of the authors of the MSB paper."
---
So at least one of the authors of the MSB paper is both fundamentally dishonest and technically incompetent.

Who would ever have guessed?

Tim Lambert wrote: "..for those who haven't guessed yet,
LancetStudy = ron = ozzy = Nick = Lancet Debunker = Tell
And they are all sock puppets for one of the authors of the MSB paper."

Er, wrong again Tim. You have misled yourself through a superficial analysis of apparent assigned IP addresses -- which, come to think of it, sounds a bit like how you have misled yourself through superficial looks at the MSB paper, and the methodology issues around L2 discussed by some of us on this thread. Hmm, maybe you should improve your detective work by focusing on what L2 actually did (or didn't do), and spend less time worrying about who might be who on this thread, or deflecting your general frustrations onto JPR's honest attempt to interpret L2's uncertain methodology?

(PS My interest in this thread was certainly started by a conversation with one of the JPR authors, no secret that. Want to guess which author it was? Or want to guess which street is a main street in city X? Or want to guess what Lafta and Burnham/Gilbert exchanged in terms of specific instructions about starting points? ...Or maybe just admit your anti-MSB campaign ran out of steam...)

Just to point it out again - the MSB authors are implicitly accusing the Lancet authors of incompetence or fraud. Burnham has said that the sampling scheme was designed to give every house an equal chance of being picked - the scheme that gives the coefficients that the MSB authors use misses that mark by a factor of more than 10. The only way they can make street assignments to get such a scheme is to assume that Burnham was lying, or that the design was an incompetent failure by more than an order of magnitude. MSB authors are accusing Lancet authors of incompetence or fraud, while attempting to not actually use the words.

And now we have LancetSTudy here, trying to explain that it only looks like "LancetStudy = ron = ozzy = Nick = Lancet Debunker = Tell", because Lambert "misled yourself through a superficial analysis of apparent assigned IP addresses." Which just happens, it seems, to be the "apparent" IP address of one of the Lancet authors.

To which I can only say: guffaw!

no need to know everything. but it would keep them from making the lousy choices that Spagat assumes they took.

no. but their lists of assumptions (adjanced means in the crossroad. mainstreet means city highway. the Lancet team chose only few roads and mostly those at the edge of town) basically is the SINGLE REASON, why they got the result that they wanted (n=10).

so you admit that their analysis is based on a list of completely biased and wild assumptions?!?

PS My interest in this thread was certainly started by a conversation with one of the JPR authors, no secret that.

for a person with access to the authors, your level of knowledge, the simplicity of your arguments and your behaviour can only be described as pathetic.

i think it is telling, how Kane and the others are trying to shift this discussion into "we don t know anything about the lancet methodology" direction.

let me sum this topic up:

Tim and others, have shown without doubt, that by even a tinny change to the Spagat picture, you get a completely different result!

what we know or don t know about the sampling procedure, is completely irrelevant to this point. Spagat came to his conclusions, by very specific assumptions, that contradict what the Lancet authors say.

So "LancetStudy" are you really trying to argue that the identical IP addresses, writing style and familiarity with the MSB paper was just some whacky coincidence? I only checked the IP addresses because it seemed a bit odd that six new commenters who all wrote the same would all show up one after the other.

@108
edit the penultimate sentence to read - ".. of one of the MSB authors"

It's amazing what a cacophany of complaint the Lancet Study has thrown up.

From the simple "it's wrong" of George W. to the more sophisticated claims of "MSB", the common thread seems to be that of wanting it not to be true.

Tim Lambert writes:

So even if you don't count my second additional main street as a main street, the only change to my map is that the yellow street that clips the upper right corner goes away, resulting in a small blue area around that street and the rest of the map reachable and n at most 0.1.

Since Tim's usual response would be along the lines of "Shone is wrong, it is a main street", I take the above to be his way of conceding that his second additional main street was not an "obvious" main street.

Unfortunately Tim's n=0.1 claim is just as misleading as his map. If he'd included the area just above his map, it's pretty clear the value for n would be much higher than 0.1. Perhaps if he crops the top inch off his map, he can return to his n=0 assertion?

His map is fundamentally misleading, as I demonstrate here.

His map is fundamentally misleading, as I demonstrate here.

i don t like to break it to you, but your post demonstrated nothing at all.

the road at the bottom of Tim s map is obviously a mainstreet. i take your silence on that road as your admittance that Tim is right!

you make a big fuss about junctions. but (as always) i think this is just symptomatic of you, not having this well thought out. in the absence of roadsigns and names, a road ends when it ends, or when it clearly flows into another one, or when it gets smaller for a significant stretch.

the Spagat map is fatally flawed. they chose city highways as mainstreets. in the map that they use to illustrate their point, they chose streets that are on the edge of town. their choice of streets is the real bias!

Lambert has said that the road he redesignated as a âmainâ street (but which wasnât shown in his own map) was the one which runs upwards through junction A (on my map). Lambert originally described this as an âobviousâ main street, but itâs nothing of the sort (itâs half the width of Lambertâs other designated âmainâ street, for example).

The best that Lambert can claim for one of his secondary (or âcrossâ) roads (road 1 in my map) is that it joins another cross road (2) at junction A, and that the road which he has arbitrarily designated as a âmainâ street runs through the same junction.

In other words, even if a survey team agreed with Lambertâs arbitrary designation of the junction A road as a âmainâ street, itâs debatable whether (using the published Lancet sampling methodology) they would select road 1 as a cross street. Looking at Junction A, it seems equally (or perhaps more) likely that theyâd class road 2 as the secondary street, with road 1 as a tertiary road leading off road 2. But nobody actually knows, since nobody knows how the Lancet sampling scheme worked in reality.

What this ambiguity over classes of road shows, at this level of detail, is that Lambert is misleading his readers when he claims that the MSB map (which he redrew) is âwrongâ. The most he can say is that he has a different subjective designation of roads, which has its own problems in terms of plausibility.

Lambertâs map left out the least plausible part of his selection scheme. For further details, see: http://dissident93.wordpress.com/2009/02/22/msb-lambert-update/

Shone, just repeating your claims and continous links to your own article that actually doesnt address the main point, wont help you.

the road at the bottom is a mainstreet. (it is VERY similar to the other mainstreets that Spagat chose) just including that one road, changes the outcome of the "bias" significantly!

the reason for this is, that this road (in contrast to the ones that Spagat prefers to choose) has many crossroads and runs through the middle of a build up area.

your claim is, that you know the Lancet sampling good enough, to be sure that Spagat got it right down to every single road!

and you are still forgetting that Lancet idea of including enough roads to get a big sampled area...

sod writes:

Shone, just repeating your claims and continous links to your own article that actually doesnt address the main point, wont help you.

Thanks - I take your point about repetition. And you're certainly the expert on that (having repeated yourself about a thousand times in bold emphasis on the gender issue in the last MSB thread).

Still, an important point is sometimes worth repeating a few times: Lambert's map is fundamentally misleading (why? See #116).

nice Shone, you managed to write another posts, without addressing that road at the bottom (making a very big difference).

and talking about gender, this reminds me of your silence on the mechanism, that kills local working age males on the street, where they are outnumbered by non-local males and in their houses (where they are outnumbered by females/kids/elderly). Spagat gives a whole new meaning to the term "precision bombing".

Tim wrote: "So "LancetStudy" are you really trying to argue that the identical IP addresses, writing style and familiarity with the MSB paper was just some whacky coincidence? I only checked the IP addresses because it seemed a bit odd that six new commenters who all wrote the same would all show up one after the other."

Just as R is more than 1, there are more than 1 persons who think your arguments are wrong and that you are driven by non-scientific goals with the L2 defence.

sod writes:

...gives a whole new meaning to the term "precision bombing".

Talking of which, do you know what Les Roberts meant by the following:

Our data suggests that the (March 2003) shock-and-awe campaign was very careful, that a lot of the targets were genuine military targets. So, I think it is correct that in 2006, probably in almost any month, there were more civilians dying than during shock-and-awe. http://tinyurl.com/4yo5uw

Is he really saying that shock-and-awe was "careful", with "genuine" targets?

It sure looks like it.

So LancetStudy, you are not denying that you are Ron/ozzy etc.

This has been another addition of SOCK PUPPET THEATER.

Tune in next we as hear Mary Rosh say " As Herr Doctor Professor Lot complete his lecture, all the women in the lecture hall had an I'll-have-what-she's-having moment."

Tim Lambert wrote: "So LancetStudy, you are not denying that you are Ron/ozzy etc."

Yes I am denying it. Sharing common IP addresses does not mean sharing genes!

This is an interesting insight into you/your thread. More interested in working out who is who, that what is what in L2??

Tim, you've failed to consider that some IP addresses are more common than others. This is obviously because of MSB: Main Subnet Bias.

Tim Lambert seems to have form when it comes to really obsessive policing of identity, and there does seem to be a police mentality behind it - the business of "catching people in the act", which is what Tim's blog is mostly about.

He also has form when it comes to revealing the identities of posters without their consent.

Shone, two more posts, not a word on the road at the bottom of the map, changing the Spagat result significantly..

Is he really saying that shock-and-awe was "careful", with "genuine" targets?

yes. as all so often, i agree with Roberts. the spike of violence after the Samarra bombing in 2006 was more deadly for iraqi civilians, that the original US campaign.

This is an interesting insight into you/your thread. More interested in working out who is who, that what is what in L2??

Tim wrote a pretty good post about the weakness of the "stronger" part of the Spagat "analysis". you failed to address the most of it. pretty weak, for someone with access to the authors.

Tim Lambert seems to have form when it comes to really obsessive policing of identity, and there does seem to be a police mentality behind it - the business of "catching people in the act", which is what Tim's blog is mostly about.

you too, haven t taken a look at the post at the top of this topic, have you?

i have seen "police mentality" and outings on denialist blogs. this is nothing of that sort.

i think it is rather interesting to know, that all those posters share IP and that LS has contacts to the authors.

sod writes:

Shone, two more posts, not a word on the road at the bottom of the map...

I've already addressed it in #3, #104, etc, but if you want the obvious stating:

1. It is what it is - a wide, straight road, busy with traffic, cutting directly through a residential area, but with buildings on the left section of it that appear to be larger than residences.

2. Obviously, you could easily classify it as a "main street", depending on your criteria.

3. There might be reasons for not classifying it as a main street, again depending on your criteria (it might not be "commercial" enough - that being one of the Lancet team's criteria apparently: see my comment #3. It's difficult to tell for certain from Google maps).

4. Assuming we classify it as a "main street", what does that say? That for that one street our corresponding "blue" area would shrink. If you cut out a little section of the map, as TL did, you could even show an area with no blue in it.

5. Does that mean n=0 for the whole of Iraq based on the Lancet sampling scheme? Clearly not. Tim misleads a little here (and when he adds his second "main" street, which he doesn't show, he misleads a lot with regard to his n=0 claim).

6. Tim's map is fundamentally misleading (see #116).

It was hardly Tim's aim to show that "n=0 for the whole of Iraq based on the Lancet sampling scheme." He does not claim that in his post. What he does show is that the map presented by Johnson et al. is wildly misleading. The point he is making is clear from his title: the JPR paper is badly flawed.

What Tim actually says is:

Make just these two corrections to their map and the unsampled area is 0. In their model, that means n=0 and there is no main street bias.

That's totally misleading. The unsampled area is not 0 in "their map" - it's 0 in Tim's tiny portion of their map (and it would be even tinier if we dismissed Tim's second correction).

Robert S et al.

Surely the real question is not whether n=0 or not, but whether a reasonable selection of main streets will lead to a bias that is way outside the confidence levels (i.e. the estimated accumulation of inaccuracy) that the Lancet team calculated?

In this case all this talk of "misleading" because n may or may not be actually zero is a complete non-starter, and pretty much irrelevant.

The point that I see here is that with any reasonable definition of what main streets are (and by the way, someone with any kind of local knowledge would not have any trouble at all identifying a main street to start off from, which renders much of the linguist wrangling pretty much irrelevant too) it's extremely unlikely to come up with a bias anywhere near the size that the Johnson, Spagat, Gourley team come up with.

You and I have discussed this before, and I was one of the early folks who drew up a few tables to analyse the sensitivity of their model to changes in the parameters (in fact I even suggested to you that the authors include such an analysis in their paper - I'm glad they did), and what I found then was that the bias is pretty sensitive to the value of the parameters that are selected.

N doesn't have to be zero, then. All Tim has to show (and has shown to my mind) is that it's pretty small for any reasonable understanding of what is a main street. The sensitivity analysis that I have already done shows that the bias drops off rather rapidly to 1 (i.e. only a small likely bias if any at all). Also notice that as we move out of the large cities to smaller ones, the definition of main streets etc tilts the balance away from MSB even further.

And all of the above is giving the MSB paper a huge benefit of the doubt. In fact I am pretty unconvinced (as I was the last time we spoke about this) that their dynamics accurately capture anything close to the reality of either the pattern of violence or the daily movements of Iraqis, thereby rendering it an interesting but purely academic example.

cheers

Aly writes:

it's extremely unlikely to come up with a bias anywhere near the size that the Johnson, Spagat, Gourley team come up with

Unless you know more about how the Lancet sampling procedures worked in reality, most of your statements, like the one I quote above, are nothing more than unsupported assertion or wishful thinking. We've been through this at great length in the above thread.

You're right that n doesn't have to be zero. However, Tim misleadingly suggests that it is zero for "their map".

Robert

No, my comments are based on the sensitivity analysis of the parameter space (something it seems I thought of before the authors themselves) and Tim's analysis of the map above.

It's pretty easy.

The question is how does one have to twist and turn to come up with a sensational bias, and the answer is becoming increasingly obvious that you have to twist and turn quite a lot.

Again, you say:

"However, Tim misleadingly suggests that it is zero for "their map"."

Again I say this is irrelevant and just a way for you to try and obscure the real question. Even you take Tim's second main street out, the bias still drops to something near 10-20% or well within the confidence interval in the Lancet paper, thus proving nothing at all.

Cheers

Aly wrote: "The real question is ... whether a reasonable selection of main streets will lead to a bias that is way outside the confidence levels (i.e. the estimated accumulation of inaccuracy) that the Lancet team calculated"

No Aly, the **real** question is: What on earth did the Iraq survey team actually do? What did Lafta, or Lafta's team, actually do on each survey day? More correctly...

**What did Lafta's team members do on each survey day to guarantee that there is no street bias? Apparently Burnham and Gilbert do not know (they have admitted they do not have the details) so how can they possibly make any valid statements dismissing street-biases?**

Not only do none of us know, but we are being told that **Gilbert doesn't know, and nor does Burnham**. And that is what all the fuss **should** be about in my humble opinion.

Now, you might (and have already) hazard a guess at what Lafta's team did -- and so might Tim et al. And so might the rest of us, including Gilbert and Burnham.

But here is the problem:

**Did Gilbert and Burnham put their name to a survey paper, as principal authors, where they did not know the exact survey methodology used?**

Whatever the rest of us on this thread are guilty of (e.g overstatements, moments of emotion etc.) none of the rest of us did that. So, therein lies your problem (as an L2 defender) and their problem as L2 authors. This is undoubtedly why they are being investigated, again in my humble opinion. The rest of this MSB topic is simply various academics attempts to make sense of little (and seemingly contradictory) information. Don't like it? Fine. Opinions are healthy, of course when backed up by a quantitative argument.

But how come you don't bother to ask yourself the questions in bold above?? I'll repeat them:

Question 1: **What did Lafta's team members do on each survey day to guarantee that there is no street bias? Apparently Burnham and Gilbert do not know (they have admitted they do not have the details) so how can they possibly make any valid statements dismissing street-biases?**

Question 2: **Did Gilbert and Burnham put their name to a survey paper, as principal authors, where they did not know the exact survey methodology used?**

OK Tim, you are a computer scientist: Give us all a working algorithm that can be implemented in a day, for the survey team, which guarantees R=1 plus/minus 20 percent, *but* also taking into account all 'ifs' and 'buts' related to adding in nearby streets etc. etc. (i.e. must be consistent with everything Burnham/Gilbert claims is there).

Oh, and write it in a way such that when translated from English it does not create problems, inconsistencies, ambiguities in interpretation.
Oh, and make it fit on one page such that it can be memorized.
Oh, and make it very simple so that it can be implemented without potential error.
... and I nearly forgot, make it work for every town, city etc. that is sampled throughout the country.

An interesting homework for you. We're all waiting....

(Of course, even in the infinitessimally remote chance that you come up with one and somehow prove in principle that it works, you have no idea if it is *the* one that was used..... Probably isn't ;-))

[BTW, there is no suggestion here that Lafta and his team did not do the best job they could. I am sure they did, and that they did something truly heroic. The issue is entirely different: If Burnham/Gilbert do not know the scheme used *exactly*, how can they make any claims about street bias being unimportant?]

LancetStudy I followed the scheme as described in paper on the map that the authors presented when arguing that n=10 was plausible and got n=0. Yes there are ambiguities in deciding what happens at complex intersections but they don't make much difference since there are usually multiple ways to get to a particular house. Even Robert Shone can't seem to get past n=0.2 on that map.

No-one is disputing that the L1 scheme using GPS gives a better sample, but this makes little difference. In order to conceal this the MSB people present a misleading map that they don't include in their paper but merely reference.

Tim, now you really are trying to be misleading. You say "I followed the scheme as described in paper on the map that the authors presented when arguing that n=10 was plausible "

Scheme? The MSB authors are *not* presenting a *scheme*, ... that is what L2 should have done. They are the samplers, after all. But they didn't -- so, I am afraid, MSB authors are within their rights to present what they see as plausible.

The JPR-related maps are not meant to be taken literally to the street since **nobody** knows what the actual streets are. They are just plausibility arguments -- and reasonable ones at that in my opinion.

I wonder what Lafta would say, if faced with the Google map? Probably not a lot, since maybe he didn't have a map. We just don't know....

So sorry, my homework assignment for you still stands. Instead of saying what didn't happen, tell us a plausible argument of what did happen. (PS Have you checked the danger-element on these maps, such that scheme needed to be adapted on-site as suggested by Burnham/Gilbert? No, unfair question. Impossible to know)

Again, if you want to argue that n=10 is plausible you need to present a map with n=10. You haven't.

Tim Lambert writes:

Even Robert Shone can't seem to get past n=0.2 on that map.

Misleading again, Tim - tut tut. By "that map", you mean the tiny portion of it which suits your particular needs.

Try working out the value of n for the whole map, keeping your first additional main street, but forgetting the second one (for reasons I gave in #116).

It'll be less than 10, obviously, but much higher than 0.1 or 0.2.

Tim writes: "Again, if you want to argue that n=10 is plausible you need to present a map with n=10. You haven't."

I haven't, since I haven't tried. But the JPR team did.

Since you define main street in particular way, you get your n. They define it in their way, estimate n=10 (which follows pretty much from the figure at the top of this thread by the way). But the big, big question is:

HOW DID LAFTA DEFINE IT TIM?

Get it? How... did... Lafta... define... it?

By the way, Tim, you have not checked that your 'main street scheme' is in principle feasible in terms of coverage for a survey team in the allotted time. To make your scheme plausible, you need to show that the extra Si that is introduced (as compared to the JPR choice) across all the survey areas, will still enable the team to make it round a random sampling of these Si's within the allotted time. Selecting what would be 'nice' for an Si, and then implementing it as a survey team in the allotted time, are two very different things. I think this is something JPR had in mind when thinking about the practicalities of the whole 'main street' scheme of L2 -- and is clearly(?) what US L2 and Lafta must surely have thought through (or not??). Saying that large areas are in Si, and then not actually sampling them, is equivalent of course to a Si which is small and biased.

Anyway, I notice you haven't answered my two questions (maybe because you are doing that homework I set you? Well, please drop it one minute, and just answer this:...):

Question 1: What did Lafta's team members do on each survey day to guarantee that there is no street bias? Apparently Burnham and Gilbert do not know (they have admitted they do not have the details) so how can they possibly make any valid statements dismissing street-biases?

Question 2: Did Gilbert and Burnham put their name to a survey paper, as principal authors, where they did not know the exact survey methodology used?

You can leave Question 1 to later, just answer 'Yes' or 'No' to Question 2. Please. Now....
Lee, Jody, Sod, Kevin D., anyone???

Another quick question for Tim, .....sort of a recap of the same old obvious, obvious question:

Take the map at the top that you pulled from the MSB site. Estimate n given *their* choice of main street. Though not cherry picked for any particular value, n=10 is certainly plausible.

You, as you have said, would have defined main streets differently, and got a smaller n.

What did Lafta do? (a), (b) or some unknown (to us) alternative (c)?

Answer: Not (a) in your opinion. So was it exactly (b)? And your proof that he did is...?. More likely (c), right Tim?

[PS. Thanks Tim, this thread is definitely fun... I was told it would be, and it is....]

Time wrote: "LancetStudy I followed the scheme as described in paper on the map that the authors presented when arguing that n=10 was plausible and got n=0. "

(Just saw this...)

Drivel, Tim. Pure drivel.
Got to hone those experimental skills. Computer science won't do that for you mate...

One more fucking time, LancetStudy, and all your various sockpuppets.

Lafta and the rest of the authors defined main streets and secondary streets such that every house in Iraq had an approximately equal chance of being surveyed.
They said so. That has been quoted to you. I've repeated it several times.

Only if one ignores this statement (or acknowledge it and assume Lancet2 to be either massively incompetent or fraudulent), and only if one ALSO goes out of one's way to exclude obvious main streets from main street category, and ALSO does so in a way that is clearly designed to exclude massive quantities of homes from the survey - only if one does all that, can one get a value for N that has any appreciable impact on the Lancet2 analysis.

It appears that the authors together designed a strategy. Lafta implemented the strategy. Ethical confidentiality makes it unethical to release all elements of the designed strategy. Ethical confidentiality makes it unethical to release any elements of the implementation.

Such ethical constraints are unfortunate reality when one is doing survey work that can bring a risk to the people being surveyed. They also open the authors up to completely unsupported and ludicrous charges such as the ones you dismal f***** are flogging to death here - because you all know that they cant defend themselves, because they are ethically constrained to not release the information you ratf***** are demanding.

It is despicable stuff you are doing - and of a kind with the sock puppetry y'all are engaging.

and to address yet another of LancetStudy (et al)'s scurrilous insinuations-but-not-actual-accusations.

I can get off the freeway and drive into a town of 5,000 people, and in 5 minutes identify and understand the layout of every main street in the town - with main street defined as any street with a reasonable probability of some kind of commercial activity on it. I know I can do it, because I do it routinely when trying to find those taco trucks I mentioned above - which are often parked on obscure but clearly commercially-identified 'main streets.'

I can do the same thing in 15-20 minutes in a town of 25-35k people. I know this because Ive done the experiment.

You on the other hand, LancetStudy (et al) are talking out your ass on this, and using your ass-wind to insinuate fraud or incompetence - and again, it is despicable behavior on your part.

As much fun as this thread is, we now know for a fact that Burnham/Lambert have been lying for more than two years about whether or not individual data was collected. (It was, they said it wasn't.) Given that, how can Tim (or any other Lancet supporter) rely on their description of the sampling plan?

Just curious.

By the way, Lee huffs about "insinuate fraud." We know that there was fraud. Hopkins just told us. The only remaining disagreement is over the extent of the fraud.

The expletives are out, so I am off this thread until someone provides a proper answer to my two questions (see above in this thread).

Until then, I just cannot resist replying to Lee's comment:

(1) "Lafta and the rest of the authors defined main streets and secondary streets such that every house in Iraq had an approximately equal chance of being surveyed."

Oh dear, back to square one: This is a *desired* outcome, not a statement of the actual selection probabilities following their choices of the samplable space.

Personally, I would bet a large amount of money that Lafta and his team did indeed do a great job subject to the constraints. And, if properly corrected for systematic bias, the numbers might be very, very useful. I repeat: A little analysis by the US L2 team along the lines of R estimate (or design whatever...) *could have* made his estimates really valuable. But they didn't. Or if they did, they are not telling how. I wish they would, since I for one do not see how it could be done accurately without knowing the main street selection that he used.

(2) Lee, I actually have something favourable to say about your taco truck scheme. At least it is a scheme. In other words, if we all knew the schedule of the taco trucks, and the day of the survey, we could then simulate what houses would have been sampled in a given realisation of the sampling in a given (portion of a) city. Then we could repeat the simulated sampling, and we could build up a profile of probability of being picked for each house. Not necessarily related to main streets (whatever they are) but at least well-defined, and repeatable.
To the extent that taco tricks move around randomly, we might then (on a good day) even remove bias tied to particular streets. And your scheme would translate into other languages pretty well as well. So well done.

However, it is irrelevant. It is the *actual* L2 scheme that we all need. You know, the one the survey team actually used. Actually Used Every Day In Iraq.....

I just read David Kane's message about the censure and then Tim's other page. Thank you for letting us know, David and Tim.
So now we know that at least that data existed post-survey. I do feel sorry for Lafta, he must feel let down. I don't know anything about the Burnham/Gilbert split, so I have no comment on any of that.

But what this probably means is that in principle some independent, assigned person (overviewed, of course) could reconstruct where the sampling actually occurred. With some funding etc. that would probably be useful. Conflict mortality estimates through epidemiology could be a very powerful scientific method, if properly applied with correct estimates of street-bias etc. It would be a shame not to push it forward properly. In fact, we probably all agree on this.... That way we would all really learn what MSB is or isn't about in practice, and a lot of things we don't even imagine possibly. Maybe MSB gets cancelled with some other effect? Or enhanced? Or essentially zero in certain places and large in others? etc.?
Who would fund it though, who would do it, and do enough people want it to happen. MSB or not MSB...that is the question.

My apologies: I have made the typo Burnham/Gilbert in several places instead of Burnham/Roberts....
Burnham/Gilbert ---> Burnham/Roberts under all my entries

LancetStudy(and various sock puppets) said:
"This is a desired outcome"

Bullshit. "There is no polite way to say it."
They didn't say it was their desired outcome. They said it was the criterion upon which they designed their scheme. Those ar ehugely different statements, and the fact that you keep misrepresenting it is telling. Especially while deflecting nearly every substantive point in my post.

Also LancetStudy et al:
"could reconstruct where the sampling actually occurred. With some funding etc. that would probably be useful." and "It would be a shame not to push it forward properly. In fact, we probably all agree on this."

So, Roberts was involved in a serious ethical mistake, and LancetStudy here proposes to amplify and extend that ethical mistake into an intentional greater violation of ethical constraints.

How -ethical- of him.

Given the admitted lack of information flow (for safety reasons) between members of the L2 team themselves, how can the US L2 team (i.e. Roberts and Burnham) be so sure that no MSB exists?

Surely the answer to this is the simple one that Tim has given:

"Because main street bias only exists in a small number of likely unrepresentative corner cases of sampling schemes, like the one that Spagat, Johnson & Gourley picked - for most definitions of "a main street" it is small or nonexistent".

In the end, if n=0, this means by definition that Si covers all of the governerate. So the survey team would have to be able to go everywhere.

Since they did not go everywhere, by their own admission for safety reasons and need for speed, we are back in the same position

This is an absolute 100% cast-iron howler, which frankly calls "Lancet Study"'s credibility into question on every single other point. Having every street included in the potential sample space (n=0) obviously does not mean that the survey team would need to "go everywhere". The survey team would sample exactly as many clusters, whatever the size of the sample space. If you don't understand this, you really don't know what you're talking about. If "Lancet Study" is actually one of the Spagat et al authors, then I profoundly hope this was a silly mistake born out of lateness and anger, because otherwise it's really embarrassing.

The consistent inability/refusal of the Spagat et al party to distinguish between "people killed on main streets" and "people kiled whose houses are on main streets" when making their arguments about the "topology of violence" is also embarrassing if it's an inability and irritating if it's a refusal.

As much fun as this thread is, we now know for a fact that Burnham/Lambert have been lying for more than two years about whether or not individual data was collected.

ah, David the defender.

But what this probably means is that in principle some independent, assigned person (overviewed, of course) could reconstruct where the sampling actually occurred.

again, for a person with access to the authors of the Spagat paper, your proposals are pretty illiterate.

very good sum up, Lee.

i expect the Spagat supporters to prefer a discussion about the Burnham subject. it will allow them to continue to ignore, how adding a single street completely changes the outcome of the Spagat paper for an area. and as i have said repeatedly and as Lee supported with his "taco example", the problem is much smaller in smaller towns anyway.

LancetStudy

this is pretty much my last post on this, as we have gone round in circles about this for a while. I have certainly discussed this in the past ad nauseum and have no appetite to restart the whole discussion.

You say:

What did Lafta's team members do on each survey day to guarantee that there is no street bias? Apparently
Burnham and Gilbert do not know (they have admitted they do not have the details) so how can they possibly
make any valid statements dismissing street-biases?

With respect I disagree. You are including a number of hidden assumptions in this question and I disagree with almost all of them. A couple of obvious assumptions are:

1. There would be street bias (I presume you mean MSB) be default. I disagree. There might or might not be, and this remains to be shown. As interesting as the paper by Johnson, Spagat et al. is, it is not proof that the thing called MSB exists in Iraq, i.e. there is no evidence that given the actual patterns of violence in Iraq, one has a greater chance of being killed by violence if one lives off a street that is close to a main street.

2. That Lafta et al. did not do what Burnham said they did, perhaps with some variation here and there. Again, I disagree. I think there is no evidence to suggest that Lafta and co. did more or less follow the instructions from Burnham, and unless there is some strong evidence to the contrary I see no reason to question that.

A question (which is essentially a rephrasing of the one I asked Robert Shone, and to which he did not really bother to respond) that one might then ask is:

If Lafta and his team deviated now and then from the methodology, and assuming that some streets were mis-characterised (whatever that might actually mean) as main streets, or not main streets, how likely is it, assuming that MSB actually exists that a large bias was introduced?

I submit that based on the sensitivity analysis it is quite unlikely, even in the best case for the MSB team, that is assuming that their model actually captures something of the reality of the situation in Iraq. By the way, and for complete disclosure, I don't accept that the MSB formula is even close to capturing the reality of the violence in Iraq, but I am assuming it does for the sake of argument.

Cheers

Aly, to address your point no. 1: Burnham/Roberts have specifically stated that they made efforts to "to reduce the selection bias that more busy streets would have." (see #13)

In other words, the "hidden assumption" you refer to (that there is a bias that needs to be reduced) is already accepted by Burnham/Roberts.

"... The survey team would sample exactly as many clusters, whatever the size of the sample space..."

Yes, yes, yes.. of course. Goodness! Not born out of lateness or anger, pure laziness in description. OK, if you want to be pedantic: Yes I agree, but after many realizations they would have found themselves at some stage penetrating deep away from busy streets. Like drawing numbers from a distribution that in principle goes to infinity, they would in practice draw houses tucked away deep in a neighborhood.... if Si is very large.

So when I say "they would have to go everywhere', then it means that after many realizations of their scheme, eventually they would have to penetrate deep to such a house. Hence 'everywhere' as opposed to just mainly next to main streets. Get it?
Based on their scheme of selection, this simply would never happen. Please lets not get into a pedantic debate about epsilon --> 0 in continuous probability distributions as opposed to likelihoods in finite realizations. I don't mind, but it is boring and obvious and I certainly ain't going to type out a whole lecture course for you.

So if Si is large, eventually after many realizations they would probe 'everywhere' (i.e. every type of house location).

So in practical terms of running a survey, do this throughout Iraq chosen clusters and you are out of time (and into areas they seemed to want to avoid for reasons X).

Robert Shone re-iterates a key point: "the "hidden assumption" you refer to (that there is a bias that needs to be reduced) is already accepted by Burnham/Roberts". Excellent point.

So L2 defenders, please quantify what exactly is the 'magic ingredient' that US L2 seem to have added to the algorithm in an ad hoc way?

(PS Tim doing the homework? And what about the responses to my two questions?? SIlence is golden guys......)

"..Roberts was involved in a serious ethical mistake, and LancetStudy here proposes to amplify and extend that ethical mistake into an intentional greater violation of ethical constraints...."

Nonsense.

I am sure that conflict epidemiology could be a quantitative scientific method, if (and only if) street-bias is quantified a priori for the particular street-selection protocol. Simple as that.

...homework...

How is the QGIS analysis goig on your side?

Jody write: "..How is the QGIS analysis goig on your side?
..."

Fine thanks. What about the simple yes/no answers to my questions?

Robert and LS

Let me make my points a little clearer.

1. You are both assuming that the mechanism by which the streets were chosen (as described by Burnham et al.) will lead to a street bias by default that needs to be further considered and reduced.

I disagree. So far I have seen no evidence this is the case and I think Tim's analysis of the maps above shows this pretty conclusively.

2. How large (in the case that I am wrong with point 1 above) is the street bias should it prove to exist and is characterised by the formula in the MSB paper likely to be?

By looking at the sensitivity of the model's parameter space, it seems that it will likely be small and hence not change the confidence intervals of the original study.

So far you have contributed nothing new to what we already could see from the L2 paper.

3. How likely is it that the MSB model accurately captures the dynamics of the actual violence in Iraq and is actually more than a meaningless toy example?

Until I see some attempt by the authors of the MSB paper to show, somehow or another that their model accurately captures some of what we are seeing on the ground, through calibration or analysis of patterns of violence or pretty much anything other than what amounts to accusatory hand waving, whilst cherry-picking parameter values that lead to sensational bias estimates, I consider their model to be an interesting but useless toy. Akin to the legendary "spherical cow" in my undergraduate physics lessons.

I hope this is clearer.

Cheers

Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). Hence Lancet article.

Process Y accounts for MSB, deduces R is near 1, and discards any possible effects of MSB.

What is Y?

Spherical cow models can be useful for highlighting issues. They will never explain where to hang a bell on a cow, the colour of a cow's eyes etc. But they can help give answers to other questions (e.g. rate of heat loss etc.). And at the very least they highlight issues, provide a basis for discussion, and pinpoint some parameters (not their values necessarily, but the embodiment of the parameter's meaning). And I guess some people just don't like those kind of models. Apollo 11 landed on the moon with spherical cow arguments. 747's fly with similar arguments. Etc. etc.

Actually, personally I would have classified JPR as one step beyond mean-field theory (i.e. first level beyond an assumption of uniformity). That's all. No more, and no less. Certainly limited, but not 'flawed'. Just one level beyond the uniformity assumption which (unless Y is a magical algorithm) is what L2 implicitly assumed or hoped.

Robert in #154: "the 'hidden assumption' you refer to (that there is a bias that needs to be reduced) is already accepted by Burnham/Roberts."

I alluded to this much earlier when I pointed out that one of the goals of the study design was to eliminate geographic bias. Jody also pointed it out, noting that surveys must be designed in such a way as to account for "design effect."

It is your assumption, Robert, that surveying on main streets would increase the death toll. As sod and others have pointed out repeatedly, that may or may not be the case. Burnham et al made no such assumptions about how it would bias the results. You are implying that their results are probably inaccurate, because they attempted to design a scheme that would make them accurate. Do you see the problem with that argument?

Bruce wrote: " surveys must be designed in such a way as to account for "design effect.""

Yes. Design is statement about what is planned to happen: W

Then there is what actually happened: X

This is fine *if* there is then a clever algorithm Y on the US L2 side, which can discount ways in which X might differ from W (e.g. possible street bias in X).
So Y must be able to assess it, and quantify any difference approximately, before being able to discard it to produce inference Z.

So what is Y?

Bruce Sharp writes:

You are implying that their results are probably inaccurate, because they attempted to design a scheme that would make them accurate.

No. I'm saying they claimed they used (unpublished) procedures which reduced MSB to zero, and should disclose the details - regardless of my (or any one else's) speculation over the value of q.

Robert, if the selection of main streets was sensible, then the amount of space that cannot be surveyed is neglible, and q -- which you cannot quantify anyway -- doesn't even matter.

As far as I can tell, all the fuss over "MSB" can be reduced to: "The L2 team made poor main street choices, perhaps because they were clueless, or perhaps because they are dishonest. Here is a long paper with some equations where you can insert arbitrary parameters to change the L2 estimates to a number that you'll like better."

Everything else is window dressing.

Based on their scheme of selection, this simply would never happen

No, as Tim has explained. Based on Spagat et al's proposed scheme of selection, this would never happen. Based on other selection schemes very similar to Spagat et al's (like the one Tim proposes), it would happen. In other words, in all except a small number of seemingly unprepresentative cases, it would happen. This is the point of the map examples above. They show that the JHU team would have had to have been very unlucky to have happened upon a scheme that would have excluded large proportions of houses from the sample space.

So if Si is large, eventually after many realizations they would probe 'everywhere' (i.e. every type of house location).

So in practical terms of running a survey, do this throughout Iraq chosen clusters and you are out of time (and into areas they seemed to want to avoid for reasons X).

This is not very comprehensible. The first paragraph is simply the statement (admission?) that if a sensible scheme is devised, all houses in Iraq are in the sample space. "Many realisations" is irrelevant here - there was only one survey carried out.

The second paragraph I don't understand at all. "[D]o this throughout Iraq chosen clusters and you are out of time" just describes the fact that for obvious resource reasons, the JHU team carried out one survey, rather than thousands. "[A]nd into areas they seemed to want to avoid for reasons X" just seems to assert that the JHU team would have intentionally designed a sample scheme so as to avoid houses far from main streets. Which a) they didn't, and b) the idea that houses a long way from main streets were more dangerous to sample than houses next to main streets, is obviously inconsistent with the central proposition of the Spagat et al paper, which is that houses a long way from main streets are less dangerous than houses next to main streets.

Bruce at 165:

You left out a critical part in your summary. I'll add it here in bold:

Based on no evidence whatsoever, we are certain that... "The L2 team made poor main street choices, perhaps because they were clueless, or perhaps because they are dishonest. Here is a long paper with some equations where you can insert arbitrary parameters to change the L2 estimates to a number that you'll like better."

LancetStudy

Spherical cow models can be useful for highlighting issues. [snip] at the very least they highlight issues, provide a basis for discussion, and pinpoint some parameters [snip]

Agreed. They can be useful, but not when they are simply used as Spagat et al. did to cherry pick a particular set of parameters and come up with a sensational bias. In that case they are only useful to piggyback off an actual good and well known study to get extra publishing points for their respective university departments. They are useful to try and talk about what a reasonable set of parameters might mean, and how likely these are. This is not what has been done so far, more's the pity.

I guess some people just don't like those kind of models. Apollo 11 landed on the moon with spherical cow arguments. 747's fly with similar arguments. Etc. etc.

This is rubbish and you know it. To compare the MSB model with the same models that fly aircraft and got Apollo 11 to the moon is fantasy.

Aircraft manufacturers and designers run thousands of simulations and tests and sensitivity analyses (plural) and callibrations and and and and...

This is how they can be sure that their models represent something more than nonsense combined with hot air. This is what separates an actual model from a meaningless toy (MSB).

If the authors of this paper were serious about this (and I hope you are not one) then they need to do some kind of work to show that their model callibrates to reality somehow. There are a hundred different ways they could do this, and they have done precisely nothing except draw random marks on random maps.

Poor effort.

Aly wrote: "...they have done precisely nothing except draw random marks on random maps...."

Oh dear, Aly. Like your friends, I see that your logic is full of the waste that a spherical cow emits......

...OK Aly: I want to be kind to you: We now know that you don't really have much of an idea what a model is. Maybe you have never done any research, never been to graduate school, not even some kind of research project?? Anyway, for your education/information, MSB is typical of a first-approximation model in a problem.

But the good news is that you can redeem yourself, here and now in 10 minutes. Just answer a simple question, which of course you know the answer to (otherwise, how could you be so sure of yourself?). Here it is:

Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). But somehow process Y accounts for MSB, allows the deduction that there is no street bias in X, and hence allows L2 to discard any possible effects of street bias. So what on earth is this marvellous Y data procedure?

Lancet Study ( and your various sock puppets):

You keep asking what "marvellous Y data procedure" ... "allows the deduction that there is no street bias."

That is a deeply disengenious and dishonest question. The fact is that THERE IS NO EVIDENCE THAT THERE IS ANY STREET BIAS!!!!!

You guys are making this shit up.

N does not equal 10. Or anything close to it. As I've aid over and over now - the lancet authors stated as a design criteria (not a desired goal, but a criteria upon which they based their sampling design) that every house in Iraq should, to use your language, be in the sampling frame.

Stop being despicable.

To Tim and friends. I have a new question:

**Data recorded by Lafta is X. It then gets passed through a signal processing box Y by US L2 team, producing result Z (i.e. 650,000 plus/minus...). But somehow, this mysterious process Y allows the deduction that there is no possible street bias in X, by quantifying it as negligible using some unknown quantitative model, and hence allows L2 to ignore even mentioning the possibility of street bias in the Lancet article. So what on earth is this miraculous Y data procedure?**

Oops, it is the same question as always. You never answer it, and yet it addresses what makes L2 valid or not scientifically. So maybe you should quit snooping around IP addresses and address this apparently simple question. Or will the Lancet erratum to do so?

Lee claims there is no evidence for street bias. There is certainly no evidence that there is elephant bias, or taco bias etc... But that is because elephants and tacos were not involved either in the design, survey implementation, or the violence itself, as far as I know.

However, streets were involved in all 3. Jody tells us that the design would have incorporated any STREET bias effect (in some way that she does not explain, or know..), the survey involved people walking on a STREET to houses on one STREET or another, and the violence occurred on a STREET. And the common word is...? Yes, STREET.

So the doubt about street bias is reasonable, and the issue should be addressed by the scientists which dismiss it. Just saying 'no, it doesn't exist' is not a scientific argument.

...possibility of street bias...

So presumably as I'm not an epidemiologist I am incorrect when I read:

'The SE for mortality rates were calculated with robust variance estimation that took into account the correlation between rates of death within the same cluster over time. The log-linear regression model assumed that the variation in mortality rates across clusters is proportional to the average mortality rate; to assess the effect of this assumption we also obtained non-parametric CI's by use of bootstrapping. As an additional sensitivity analysis we assessed the effect of differences across clusters by extending models to allow the baseline mortality rate to vary by cluster'.

As being some statement of the efforts spent trying to discern systematic error?

So presumably the difference between rural, small town and city clusters generated by 'main street bias' is not detectable in this way, and further 'main street bias' leads to a level of intra-cluster correlation that is also undetectable by the above process?

Jody writes: "so presumably as I'm not an epidemiologist I am incorrect when I read:..."

1. Not incorrect when you read it. But incorrect to link that phrase to streets. It talks about sampling on the scale of clusters, not the streets within a cluster.

2. ".. street bias' leads to a level of intra-cluster correlation that is also undetectable by the above process..."

Yes, correct.

**What quantitative analysis did Burnham/Roberts do, either post-survey or pre-survey, to correct for possible street bias during the survey?**

Again....

What quantitative analysis did Burnham/Roberts do, either post-survey or pre-survey, to correct for possible street bias during the survey?

street, **street**, _street_, STREET....

Surely the answer to Lancet Sock's question is:

"The inherent unlikelihood of any such bias existing in the first place".

Now that it's answered, perhaps you could post under your real name please?

dsquared wrote: "...inherent unlikelihood ..."

What???

One again, LancetStudy (and your army of mousy sock puppets) - bullcrap!

"Only if one ignores this statement (or acknowledge it and assume Lancet2 to be either massively incompetent or fraudulent), and only if one ALSO goes out of one's way to exclude obvious main streets from main street category, and ALSO does so in a way that is clearly designed to exclude massive quantities of homes from the survey - only if one does all that, can one get a value for N that has any appreciable impact on the Lancet2 analysis."

See that? Ive posted it several times now in response to your spammed-all-over straw man question.

If N has a value much smaller than 10, then street bias does not alter the Lancet2 conclusions in any appreciable way.
This is true also of several other of their parameters, which have been adressed in other threads.

Lambert's post that started this thread is an answer to your oft-spammed question. Lancet 2's authors statement that a sampling design parameter was that every house in Iraq be in the sample frame, ADDRESSES YOUR F****** QUESTION!!!!!!!!!!!!

What???

Are you slow or something, LancetSock? As has been shown above, main street bias only exists in a small number of seemingly cherry-picked and for the most part obviously incorrect schemes for identifying "main streets", and even then only in large towns. As Tim showed above, even in the best-case example chosen by Spagat et al. to illustrate their point, it only existed thanks to a really quite perverse decision to fail to identify a particular road as a main street. From this I conclude that the vast majority of road-identification schemes which a competent researcher on the ground would choose, would not lead to material street bias. Hence, the fact that it is very unlikely that the sampling space was distorted by main street bias, is good grounds for assuming that the sample is not distorted by main street bias.

Now could you use your real name please?

You guys get personal very quickly don't you. So be it...

Since Lee likes cross-postings, here is mine:
Lee writes: "If N has a value much smaller than 10, then street bias does not alter the Lancet2 conclusions in any appreciable way..."

Putting Tim's incorrect analysis to the side for one moment, the question is: Why didn't Burnham/Roberts do a street-bias analysis? Where did they do it? Do they reach the same conclusions? If so, how did they manage to do that prior to JPR, if they didn't have the JPR formula? Did they derive the JPR formula and plug in numbers? If not, what formula? What are the assumptions? What numbers did they plug in? Did they use a map? Which map? Which streets did they pick as main streets? The streets that Lafta generally picked?

Questions, questions.... 'irrelevant' questions? No.

Lee, my friend, invite dquared to grab a taco from one of your main-street-defining-taco-trucks, and read the following out loud, together, very, very slowly while munching...

The survey involved people walking on a STREET to houses on one STREET or another, and the violence occurred on a STREET. So how, where and when did Burnham/Roberts manage to quantify and discard any street-bias effect?

... and that concludes my side of the proceedings.

**Outcomes**

1. possibility of street bias is reasonable, and given the lack of details from L2 and apparent communication issues within L2, sounding more likely to me every day

2. JPR model reasonable. A good first-approximation model.

3. JPR estimates are plausible, reasonable even. Above all, their invite to readers (in the JPR abstract) to suggest their own values is honourable and honest

4. L2 does not contain a quantitative consideration of street-bias in the paper

5. The opponents of JPR, actually use the JPR model framework to discuss street bias through its parameters. A classic proof of the usefulness of a model!

6. Tim's title is incorrect, ... and may be seen as slanderous

You guys get personal very quickly don't you

Sock puppets aren't people. If you want to get treated like a person, be a person.

Why didn't Burnham/Roberts do a street-bias analysis?

Because there's no such thing as a "street-bias analysis". There is no such thing, because, absent cherry-picked cases like the one in the badly flawed JPR paper, the "contiguous houses down secondary street" method samples the space effectively.

In answer to the six new silly assertions which have sprung up as a result of my answering one silly question:

1. No; it has been established that only rare and cherry-picked street choices would result in measurable truncation of the sample space, which is in itself a long leap from "Bias", as nobody has established that any such censoring was informative.

2. No, clearly a bad approximation, as it presents only a single set of parameters which are not robust. As a model of "bias", even worse because its discussion of whether the hypothetical censoring was informative is nugatory.

3. No, the suggested values are unreasonable (as demonstrated above). Since nearly all peturbations of the street identification code give parameter values close to zero, it is neither honourable nor useful to ignore this.

4. Nor does it give consideration of a million and one other almost certainly fictitious phenomena. Not the Lancet's job to do Spagat et al's work for them in inventing the hitherto unknown "Main street bias" phenomenon, which remains wholly speculative.

5. Clearly silly - a model whose only use is to prove that the issues it raises are not important, is not a useful model.

6. No it isn't, and (how it pains me to mention this), if it is true that "LancetStudy" is a pseudonym for one of the authors, they should perhaps look closer to home; Michael Spagat has considerable past history of making slanderous statements about researchers with whom he disagrees, and attracted an open letter of complaint from Human Rights Watch Colombia (IIRC) for doing so.

Sock puppet,
Try for just one second to think like a scientist. Suppose JUST for the sake of argument that people who lived on main streets did have a higher death rate. What Tim has shown is that it is inconceivable that all (or even a significant majority) of the clusters would be on the main streets. There is simply no way to rationally choose main streets without getting many of the clusters off the main streets.

These clusters would have very different rates of death than the clusters on the main streets and this would show up in the analysis and as #174 point out
âGAME OVER MANâ. Since this did not happen, there is no MSB. The end.

elspi wrote: "..The end.."

Au contraire, it is just the beginning...

oh god

LancetStudy :"So how, where and when did Burnham/Roberts manage to quantify and discard any street-bias effect?"
They didn't. They didn't have to - BECAUSE THEY FUCKING DESIGNED IT OUT UP FRONT!!!!!!!!!

Get it yet?

The taco truck example was an answer to your dishonest insinuation that the sampling teams didn't have enough time to identify main streets. Misrepresent much?

They also didn't quantify and discard possible time of day effects, possible scary interviewer effects, possible 'nice interviewer I want to please' effects, possible biased question effects, and on to a multitude of possible effects. They design to minimize all of these - including skewed sampling effects - as in EVERY FUCKING SURVEY EVER TAKEN!!!!

Stop being despicable, LancetSTudy.

Neat..

Invent a sort of incomplete data question where you cant even know how incomplete the data are, and demand the set is complete before believing the result.

Presumably even if we had the full set of household addresses lets say from a true random sample there would still be some clustering, and you could pull some post hoc 'ah but look they cluster near this or that, there's bias'

Anyhow I'm quite interested in how 'main street bias' works without creating increased intra over inter cluster correlation given that the clusters were in different places in different sized towns etc.

they also did ignore all positive effects on the polling process on mainstreets (Taco truck next to cluster: TTntC-effect, reducing massively the time needed for lunch breaks)

neither did they quantify effects that give a too low deathrate (wiped out or fled households) down to the third digit behind the comma!

Interesting, of course, that the potentially quite serious problem of "Didn't Visit Anbar Because It Was Too Dangerous Bias" in the UN IFHS survey has attracted so much less attention.

LancetStudy.

This may appear to be a nit-pick, but to me it speaks volumes...

At #173 you refer to Jody as 'she'. Given the number of times in this thread that you have referenced Jody I am surprised that you remain ignorant of said person's identity.

Google Jody's name, add 'Dr' and 'London' to the string, and find out why your oversight is rather glaring.

If I was extensively debating with someone who hadn't done even a cursory amount of research on basic background such as this, I would be very sceptical of this person's capacity to construct a cogent argument.

Of course, that's just my bias.

nteresting, of course, that the potentially quite serious problem of "Didn't Visit Anbar Because It Was Too Dangerous Bias" in the UN IFHS survey has attracted so much less attention.

i think i have an explanation for this: as they did not go there, they did not have to travel on roads, so there can not be any street bias.

on a more serious note, the IFHS study replaced the numbers for those dangerous regions with numbers calculated from the IBC project. and the IBC take their numbers from newspaper reports.

the serious bias in this approach is, that the number of reports is certainly linked to the number of journalists in Iraq.

and this number gets reduced [during times of high violence](http://www.nytimes.com/2004/10/02/international/middleeast/02media.html)

and even more with the [recent withdrawal of journalists](http://www.brookings.edu/opinions/2008/1015_iraq_media_nessen.aspx)

so let us check some numbers:

he New York Times, Los Angeles Times, and Washington Postâwhich all continue to staff small Baghdad bureausâpublished a total of 858 front-page stories from Iraq in 2003, but only 138 in the first nine months of this year.

ooops, this gives us a JWB (journalist withdrawal bias) of 6.2!!!

dsquared: a model whose only use is to prove that the issues it raises are not important, is not a useful model.

True, if you are looking at the matter from a scientific perspective. But from a troll's point of view, if you can annoy people and get them arguing and lavishing attention on you - even if the attention mostly takes the form of scorn - that's success.

In the previous thread on this subject a professor of demography wrote a lengthy comment on Burnham et al. and where it may have gone wrong; he didn't assume that it did of course. There was practically no response (not that I suppose the author particularly wanted responses). A sock puppet, whose sock-puppetry was obvious from the similar quality of writing and reasoning even before Tim confirmed the shared IP address, got umpteen responses.

If a troll was a scientist, the MSB paper is just the sort of paper a troll would want to write. Whether or not this particular troll is one of the authors, it's easy to see why s/he admires their work.

i think the use of the Lancet iraq study as the main example of their model is extremely dubious.

if you had a good theoretical model, would you use an example from a paper, that you consider to be completely flawed in many aspects? one which you consider to consists of faked, incomplete and unavailable data? a paper that, according to you, does not accurately describe its methodology? one that forces you to guess all parameters, with no way to check them?

it just doesn t make any sense, unless you consider other motives. (mainly publicity)

A good test of the model would be to apply it to something like traffic fatalities, or pedestrians injured by cars. We'd expect more accidents on busy streets. If we applied the L2 sampling scheme, would our results match the actual accident rate, as reflected in official statistics? And if they didn't match, would applying the formulas from the MSB paper give us a correction factor?

My guess is that we'd find that the L2 sampling scheme would result in pretty accurate numbers to begin with, and the utility of an MSB algorithm to adjust those numbers would be negligible.

Well LancetStudy

Touchy aren't you? I guess you are one of the MSB authors after all... what a shame.

I'll leave you to your spherical cow to do what comes naturally. Oh yes, and to speculate about whether or not I have a graduate degree...

cheers, and goodbye

Reaction to the AAPOR press release

tlambert — Thu, 05 Feb 2009 01:57:34 +0000

Reaction to the AAPOR press release

I asked Mary Losch (chair of AAPOR's Standards Committee) to comment on my previous post

I have read your entry and would note that the links you provided did
not supply the questionnaire items but rather a simple template (as
noted in the heading). The Johns Hopkins report provides only
superficial information about methods and significantly more detail
would be needed to determine the scientific integrity of those methods
-- hence our formal request to Dr. Burnham. The Hopkins website
refers to data release but, in fact, no data were provided in response
to our formal requests. Included in our request were full sampling
information, full protocols regarding household selection, and full
case dispositions -- Dr. Burnham explicitly refused to provide that
information for review.

We do not provide public reports of the investigations but if there
are other specific questions that I could answer, I would be happy to
try to do so.

It is more than a little ironic that AAPOR has censured Burnham for not fully disclosing the data behind his study, while themselves failing to fully disclose the basis for their serious charge.

In response to a question from Mark Blumenthal she added:

we requested the survey instrument (including consent information) and it
was not provided. The template did not appear to be much beyond an outline and
certainly was not the instrument in its entirety.

I don't think that Losch or the AAPOR have disclosed, specifically, what is missing from the information that Burnham has provided. I stand by my earlier statement that their press release was misleading. I checked their website and they've now put their press release up, but provided no supporting information. They've also linked to a paper by David Marker published in their journal which implicitly contradicts their press release. Marker clearly felt that Burnham had published enough information for him to evaluate their mthodology, writing:

Burnham et al. attempt to estimate the number of excess Iraqi war dead throughout the country using fairly standard survey methodology, for which they are to be commended. We have examined four specific methodological factors: coverage errors, correct probabilities of selection, migration, and training and control of interviewers. Coverage provided by the first stage of sample selection (the sample of administrative units) appears to have been complete (other than the exclusion of area considered as too violent to allow household interviews as well as the exclusion of the three sampled clusters for the reasons the authors indicated). The coverage at the second stage appears less than complete. In addition to the exclusion of some deaths mentioned earlier (short term household members), there may have been systematic exclusion of some types of households or housing units. The extraordinarily high response rates reported suggest this as a possibility. It should be noted that for the two governorates where data were not collected, they used an underestimate of no excess deaths, choosing to err on the side of understating mortality in these unknown cases. This level of coverage represents a major improvement over other reported estimates.

Tim Parsons responded to my query on behalf on the Johns Hopkins Bloomberg School of Public Health:

The level of civilian mortality in Iraq is a controversial subject. Questions have been raised regarding the findings and methodology of the 2006 Iraq mortality study conducted by Dr. Gilbert Burnham and published in The Lancet. The Johns Hopkins Bloomberg School of Public Health takes any allegation of scientific or professional misconduct very seriously. It believes that the correct forum for discussing the reported findings of the Lancet study and the general methodology that led to those findings is in the regular exchange of views in the scientific literature. The Bloomberg School of Public Health has undertaken a review of the study to determine if any violation of the School's rules or guidelines for the conduct of research occurred in the conduct of the study. That review is nearing completion and the School is unable to discuss the results at this time.

The American Association for Public Opinion Researchers (AAPOR) chose to criticize Dr. Burnham for failure to fully cooperate with the organization's review of his 2006 study. The Johns Hopkins Bloomberg School of Public Health is not a member of the organization and does not know what procedures or standards were followed in reaching the decision regarding this study and therefore is not in a position to comment on the decision.

I guess we'll have to wait for the Johns Hopkins report.

Megan McArdle adds to the mountain of her errors on the Lancet studies with this:

I happened to be writing my story just as the World Health Organization study that was highly critical of Burnham, et. al. was released. Les Roberts, who had become the public face of the team, was making frankly lunatic claims on the radio that the two studies basically agreed, even though the introduction to the WHO study specifically said that their results made it very unlikely that Burnham et. al. had been correct. This claim was so unusual that when I asked neutral conflict epidemiologists, they patiently explained that I couldn't possibly have heard Roberts correctly, because no one with half a brain would ever have said that.

McArdle does not seem to have understood what Roberts was saying was in agreement: the excess deaths in the Lancet study (about 650,000) and in the IFHS study (about 400,000).

tlambert Wed, 02/04/2009 - 20:57

Tags

David Marker, in the study that Tim Lambert links to, also says the following:

"A few years ago, 35 leading survey researchers issued a consensus statement on how to minimize interviewer falsification of data (AAPOR 2003). This statement has been endorsed by the American Association for Public Opinion Research and the Survey Research Methods Section of the American Statistical Association. They listed eight factors that could affect falsification rates. Inadequate supervision, poor quality control and off-site isolation of interviewers were three of those factors that are present in this [Lancet] study. The remaining five factors (training on falsification, interviewer motivation, inadequate compensation, piece-rate compensation, and excessive workload) are harder to assess in this situation due to the limited information available on these topics."

I want to know if AAPOR has substantiated the claims that 6 million Jews were killed in the Holocaust. Is that accurate? And, were the methods use to deduce that number scientific? Is it even OK to question those figures? Is it true that in Germany it is illegal to question those numbers? And, if so, does that qualify as having explicitly refused to provide that information for review? Also. Is AAPOR as aggressive and critical in their research of public opinion when it is pro-western? Or is that allowed to slide because it fits their public opinion agenda?

> We do not provide public reports of the investigations

So AAPOR thinks it has the right to demand Burnham to disclose his data and methods, where Burnham is not a member of AAPOR, and where the right to demand information is based on a secret investigation initiated by a secret complaint.

so far, we have:
40 interviews per day per interviewing team for months on end, with full informed consent; at 20 mins per interview, do the math !

>98% returns; described as "extraordinary" by many professionals

they won't release the original data, survey forms, etc

a widespread array of academics severely criticising the methodology

An independent professional body censures the researchers for failing to release data and methods

But despite this rather surprising concatenation of interesting events, Tim stays behind the study all the way.

the longer you keep this up, the more embarrassed you are going to be.

yours
per

Let's not forget one of the important claims made by the Lancet authors:

"The sites were selected entirely at random, so all households had an equal chance of being included." (Burnham et al, 2006)

Remarkably, researchers are still waiting for an account of the procedures used in the attempt to accomplish this result. It goes to the heart of the study. Perhaps this is what Losch had in mind when talking about the "protocols regarding household selection" which Burnham has apparently failed to provide?

AAPOR:

> We do not provide public reports of the investigations

Hmm, let's repeat that:

> We do not provide public reports of the investigations

and again:

> We do not provide public reports of the investigations

and yet again:

> We do not provide public reports of the investigations

Hahahahah........

AAPOR's richard Kulka

"When researchers draw important conclusions and make public statements and arguments based on survey research data, then subsequently refuse to answer even basic questions about how their research was conducted, this violates the fundamental standards of scienceâ¦."

this is a fundamental standard of science. Perhaps bi and Tim could address this fundamental standard, or explain their own alternative standard. Tim will no doubt be at pains to point out that the authors have not even released their questionnaire, or their informed consent forms...

How do they determine the difference between "civilian" and non-uniformed combatant? How do they screen for disingenuous answers? Like claims the deceased were civilians when in fact they were not?
I don't think any survey can be trusted without complete background checks on each of the deceased. IOWs it's unknowable.

Sorry per, by "the authors" above are you referring to the authors of the Lancet study or of the AAPOR press release?

per:

> "When researchers draw important conclusions and make public statements and arguments based on survey research data, then subsequently refuse to answer even basic questions about how their research was conducted, this violates the fundamental standards of science...."

Sorry, but these are nothing but insinuations with no evidence. Or, to put it another way, with as much evidence as Saddam Hussein's WMD.

* * *

Who the heck are the AAPOR anyway, and why are they on the Internets tubes? Hmm... a whois reveals their snail-mail address to be shared with goamp.com. The goamp.com home page reads thus:

> Applied Measurement Professionals, Inc. (AMP) provides certification organizations, government agencies, professional associations and private industry with innovative assessment and management solutions. With more than 25 years of experience and over 100 clients representing a wide range of professional occupations, AMP is dedicated to providing our clients with "Technology That Works and People Who Care."

> AMP is a private stock corporation located in the Kansas City metro area. AMP was incorporated in the state of Kansas in 1982 as a wholly owned subsidiary of the National Board for Respiratory Care (NBRC). In 1996, AMP acquired Logic Extension Resources (LXR), which now serves as the companyâs LXR Division.

Hmm... hmm... hmm...

To confuse matters further, they also list another snail-mail address which is shared with nbrc.org.

Hmm... hmm... hmm... hmm...

Wildlifer, If you think this distinction is applicable, shouldn't it be up to the aggressors to count the number and status (civilian or combatant) of the dead as a result of their invasion? Why don't you think the US or UK showed any interest in carrying out a detailed survey of the death toll in Iraq as a result of their combined aggression? Or, for that matter, of any wars waged? And is such a distinction between 'civilian' and 'combatant' deaths valid anyway? I am sure that many ordinary Iraqi civilians became combatants as a result of the fact that they don't necessarily like being occupied by a foreign aggressor, particularly one that has murdered their loved ones. Given that the invasion violated just about every standard of international law (as well as the United States constitution), it was illegal anyway and thus so-called combatant deaths are also a crime.

But back to my original point: it should be obvious why the US and UK did not carry out a census of the death toll as a result of the invasion. First, they didn't care. Second, it the actual total was accurately known, at least to within a few thousand, it would fuirther demolish the notion that the US and UK are great respectors of freedom and human rights. But as long as the actual total is very vague, then any estimate generated from any source is open to doubt. This enables the western corporate media to downplay the idea that the invasion resulted in utter carnage, and maintains the myth of the 'basic benevolence' of the west.

The major aim here of defenders of unilateral US aggression is to sow doubt as to the actual death toll in Iraq. As long as there is doubt then it is possible, in public realtions terms, to 'manage the outrage'. Actually, there are similarities with respect to the campaign waged by certain sectors and individuals to donwplay the theory of anthropogenic climate change. The sceptics know that they will never win the scientific argument, but they don't have to: so long as they can sow enough doubt amongst policy makers and the public, then nothing will change. If the true death toll as a result of the war in Iraq was known, then I am sure that there would be repercussions. So the governments, along with much of the corporate mainstream media, do everything they can to obfuscate the truth. They routinely attack scholars whose work suggests that the death toll was enormous, because, as I said above, it demolishes much of the myth that our governments pursue humanitarian agendas. They have agendas all right, but they are vastly different.

per,

Please note from Mark Blumenthal's blog post on AAPOR's censure of Bernham:

The AAPOR censure does not involve Burnham's methodology and renders no opinion on the substantive conclusions of the Lancet study. Instead, it focuses entirely on disclosure, or rather on Burham's failure to disclose "essential facts about his research."

> An independent professional body

Given what I found above, I think I can now ask the good old question again:

Independent... of whom?

Tim writes "I don't that Losch or the AAPOR have disclosed, specifically, what is missing from the information that Burnham has provided."

1) I think you want "don't know that" instead of "don't that".

2) I can help with this! AAPOR want Burnham to disclose the actual survey used. He has not done so. (The National Journal published some sort of survey template but a) It was never clear whether this applied to the 2004 or 2006 survey and b) No Lancet author has ever confirmed that this was the survey used in 2006. Just because The National Journal thinks that this is the survey does not mean that it actually was.

Although I (and other researchers) would able to see some of the household-level data, I (and, I think, all other researchers) were never allowed to view the actual survey (either in English or in Arabic).

AAPOR standards require that survey questions be made public. Burnham et al have not done so.

The AAPOR, the AMP, and the NBRC also share snail-mail addresses with another group, the National Association of EMS Physicians.

This "independent professional body" sure has a ton of 'dependencies', eh...

Bi, with respect, this continual paranoid rambling is making this "science" blog read more like the Medialens message board.

In other words, the AAPOR has no standing in this, and never did. However, the Johns Hopkins report is going to be the strongest possible source, as they have authority and expertise in this situation.
I look forward to reading the report when it comes out, almost as much as the reactions to it.

Robert Shone:

> Bi, with respect, this continual paranoid rambling is making this "science" blog read more like the Medialens message board.

Just the facts, sir. It's always interesting when an "independent" non-profit organization doesn't disclose that it's run from the same address as a for-profit company and a ton of other "independent" organizations. I'm sure one can come up with a perfectly good explanation for this, but the interesting thing is that nobody's even bothered to give one.

Something nice and chunky from the AMP, the company that's situated at the same snail-mail address as the AAPOR.

Coincidence? I report, you connect the dots.

AAPOR standards require that survey questions be made public.

is it typical for english papers to publish arabic questions?
what sense would it make?

AAPOR want Burnham to disclose the actual survey used. He has not done so. (The National Journal published some sort of survey template but a) It was never clear whether this applied to the 2004 or 2006 survey and b) No Lancet author has ever confirmed that this was the survey used in 2006. Just because The National Journal thinks that this is the survey does not mean that it actually was.

is it typical to question whether questions published were really used?

and why do you guys question the questionaire? do you have ANY reason for this?

Bi, that link returns nothing for me.

"the Johns Hopkins report is going to be the strongest..."

this is very simple. Scientists release data, subject to practicalities. It doesn't take Johns Hopkins' imprimatur to validate this very simple principle.

What possible reason can there be for refusing to release the questionnaire and informed consent forms ? It takes two minutes to scan to pdf and put on the web- so what possible reason is there not to ?

Anonymising data and putting it out for reanalysis is something that happens on a regular basis for lots of epidemiology studies. So why not here ?

There are already several reasons for wanting to examine this data. 98% returns- which Tim quotes as "extraordinarily high response rates ". Undertaking 40 questionnaires per day, with full informed consent, in the fast time of 15 minutes each, gives a ten-hour day, and no time for lunch, toilet breaks, or moving from house-to-house. Examining the data might enable a greater certainty that this data is credible. And then again, it might make it quite understandable why they don't want to release the data.

per

Anonymising data and putting it out for reanalysis is something that happens on a regular basis for lots of epidemiology studies. So why not here ?

NO! typically a poll will state what sort of data will be published. you wont give away more data, because you are NOT allowed to do so!

is this your level of "expertise" on this subject?

data from "anonymised" questionairs or even cluster data is highly problematic, as it might be pretty simple to figure backwards!

this is very simple. Scientists release data, subject to practicalities. It doesn't take Johns Hopkins' imprimatur to validate this very simple principle.

this is very simple. scientists share data, if they want to share it. being friendly and cooperative yourself helps. being hostile mostly doesn t.

it takes about two minutes, to write down WHY OH WHY you want to have the questionaire? what part of it do you have doubts about?
how good is your arabic anyway?

the response rate is not that unusual. if you had ever taken a look at the questionaire, you wouldn t think that it takes 15 minutes to fill the form.
releasing more data had absolutely ZERO effect on the people attacking its credibility.
http://tinyurl.com/bhsr8p

> so far, we have: 40 interviews per day per interviewing team for months on end, with full informed consent; at 20 mins per interview, do the math !

With two teams, 47 clusters, and one cluster per team per day, the data collection would be over in less than 4 weeks.

Well, if I didn't, I was in good company, because several prominent conflict epidemiologists flatly told me that this was not a valid interpretation of the results, including one of the few researchers who had access to both datasets, Olivier Degomme. They weren't trying to get Les Roberts; they simply thought that I must have misunderstood. I worked with Roberts' remarks in front of me, and very clearly asked whether the excess deaths being close to the lower bound of the violent deaths meant that they substantially agreed. They said no, that wasn't the way it worked. As, of course, did the authors of the WHO paper.

Cliff notes version of this:

AMP is a propaganda machine and AAPOR somehow shares an address with them.

And we have these "I am become death; destroyer of irony meters." quotes from AAPOR:

"Dr. Burnham provided only partial information and explicitly refused to provide complete information about the basic elements of his research"

"We do not provide public reports of the investigations"

How can the credibility of the Lancet denialists get any lower?

Any day now, I expect to see them on TV flogging ânatural male enhancementâ.

total deaths between the lancet and the WHO are in good agreement, as Tim wrote above.

if you disagree with that statement, you disagree with reality.

McArdle does not seem to have understood what Roberts was saying was in agreement: the excess deaths in the Lancet study (about 650,000) and in the IFHS study (about 400,000).

I'm struggling to understand this claim, so please help me out.

Per the UN/IFHS study found at (http://www.who.int/mediacentre/news/releases/2008/pr02/en/index.html), 151,000 Iraqis were estimated to have died between 2003 and 2006. That isn't the 400,000 mentioned. Further, in the UN's FAQ section concerning their study the specifically address the differences between the Lancet study and the UN/IFHS study:

Q: Which figure is more reliable: the new survey or the 2006 household survey?A: The 2006 household survey shows a very different trend than that seen in the new survey and the Iraq Body Count, with increasing numbers of deaths per day, rising from 231 during 2003-2004 to 491 during 2004-2005 and 925 during 2005-2006. The biggest difference between the 2006 household survey and the other two sources is in the figure for the third year. Most of those deaths occur in six high mortality governorates outside of Baghdad, while in the Iraq Body Count and the new survey, most deaths occur in Baghdad.
The difference between 925 and 126 violent deaths per day is very large. To reach 925, the Iraq Family Health Survey would have to have missed more than 80% of deaths detected in the smaller survey. This is highly unlikely given the much larger number of clusters and households visited in the new survey.

151,000 Iraqis were estimated to have died between 2003 and 2006. That isn't the 400,000 mentioned.

you are confusing violent and total excess death cases.

the questions over household deaths are on page 16 of the questionaire. (do you spot the trouble with this?)

http://tinyurl.com/dgloll

My god, bi--IJI, there's also the Association of Polysomnographic Technologists, Citizen CPR Foundation, and the Oak Park Homes Association! You've uncovered a major conspiracy all right; all over our USA we find these "businesses" and "organizations" clustered together in "buildings". Somne call them "office buildings", but we know better.

Watch the skies!

Megan:
>I worked with Roberts' remarks in front of me, and very clearly asked whether the excess deaths being close to the lower bound of the violent deaths meant that they substantially agreed.

Which is not what Roberts said. The surveys disagreed about the number of violent deaths, but the number of excess deaths from the IFHS was inside the confidence interval for excess deaths from Lancet 2.

per and others:
Scientists share data at times, subject to constraints and other issues. I have access to some data under a written agreement that states what I am going to do with it, that I will share the results of any findings, and not release to other parties. I am involved in several research projects that have involved trying to contact various researchers for information beyond what has been published. Sometimes we get it, often we don't. I wish we always did, but tough - I can't compel anyone. So tough - discriminate between science and propagand.

"NO! typically a poll will state what sort of data will be published. you wont give away more data, because you are NOT allowed to do so!"

in fact, you do not substantiate that statement at all. You do not (cannot ?) point to any undertaking that the authors gave to the participants, so you cannot show that the authors are bound.

Yet another disappearing bit of evidence. Strangely, many other epidemiology groups manage to organise independent review and assessment of their confidential data. What is so "magic" about this study, that they cannot ?

"the response rate is not that unusual."

I merely guess that you are not a professional epidemiologist or pollster ? The issue is that many professionals seem to think that it is extraordinary (see Tim's quote above) and e.g. this post from 2006

Tim mentions talks about the "number of excess deaths from the IFHS" Of course, IFHS did not publish an excess deaths number. This is just the sort of stuff that Roberts is guilty of.

Now, it is possible for you to make a bunch of assumptions that the IFHS authors did not make and derive an excess death number from the raw IFHS data. That is a fun exercise. But that has little to do with whether the IFHS paper --- that actual words that they published in the NEJM --- agrees or disagrees with L2.

IFHS published an estimate and confidence interval for violent deaths. L2 published an estimate and confidence interval for violent deaths. Those two confidence intervals do not overlap. Indeed, the L2 estimate is four times larger. Someone is wrong.

I would understand the secrecy (to some degree) if we were talking about research that might lead to a profitable patent or was subject to some kind of national security consideration of privacy concerns. Is there anyone who would care to explain the reason for secrecy in the Lancet/Iraq study (for the data that has been requested by its peers and other critics?)

If the survey authors release the raw data it may lead to the identification of individuals who participated in the survey. No one needs to know this. If someone does not like the survey's methodology (which has been published) they are free to commission their own.

QrazyQat:

> My god, bi--IJI, there's also the Association of Polysomnographic Technologists, Citizen CPR Foundation, and the Oak Park Homes Association! You've uncovered a major conspiracy all right; all over our USA we find these "businesses" and "organizations" clustered together in "buildings". Somne call them "office buildings", but we know better.

> Watch the skies!

Well, given that the AAPOR and AMP,

1. in addition to being clustered together in one building (in Lenexa, KS), also
2. share the address of another building (in Olathe, KS), and
3. the administrative contact for the AAPOR site also goes to an AMP e-mail,

I think the relationship between AAPOR and AMP goes beyond "independence".

Extraordinary proof requires extraordinary evidence, and it does seem something extraordinary is going on here (even if I don't quite know what it is).

Bud:

> Bi, that link returns nothing for me.

Sorry... it seems the archival took a while. Do try again.

Ah, AMP is a lobbying firm.

[Robin Meija](http://www.motherjones.com/commentary/columns/2008/11/witness-after-mat…):

>More than two years later, the Iraq study remains mired in controversy. But other recent findings suggest that Roberts and Burnham were on the right track. In the summer of 2006, the World Health Organization conducted a large family health survey along with Iraq's Ministry of Health, interviewing about five times as many people as Roberts and Burnham had, and in a more distributed fashion. In August, Mohamed Ali, a who statistician, reported his preliminary results to colleagues at a Denver statistics conference: Nearly 397,000 Iraqis had died because of the war as of July 2006.

>That number falls at the low end of Roberts and Burnham's confidence interval, which ranges from roughly 393,000 to 943,000. But while epidemiologists and statisticians are still pondering questions raised by differences between the two surveys, there's no longer much doubt among them that Iraq's civilian casualties number in the hundreds of thousands.

I stand corrected! And, ironically enough, I actually organized the panel at which Dr. Ali presented that result. (I don't recall the exact number but I have no reason to dispute Robin's reporting.)

We will see what happens when IFHS publishes their estimate. The main feedback that I saw Dr. Ali get about those numbers concerned the underestimation (because of recall bias) of the pre-war mortality. But, of course, Ali and his team are experts, so I am sure that whatever estimate they come up with will be reasonable.

But, anyway, none of that makes the L2 and IFHS numbers more comparable because the central (almost only) critique of L2 has always revolved around their violent death total. No one disputes that their non-violent excess death estimate is plausible.

And, you can't just try Roberts trick of claiming that the violent deaths in IFHS were mischaracterized as non-violent in IFHS because the gender balance of the deaths is radically different.

And we still have the issue of the different sorts of adjustments made by IFHS and not made by L2.

But, Tim is right that, given Dr. Ali's presentation, it is fair to say that IFHS presents a draft estimate of excess deaths which is within (barely!) the 95% confidence interval of the L2 estimate for total excess deaths.

This may be off topic but has anyone worked out how many Iraqi soldiers were killed in the Iraq wars, or is it a matter of male+in uniform+Iraqi = don't matter?

Even if it worked out to one soldier per ton of bombs it would surely add up to well over 100,000.

That's a lot of heartbroken mothers.

David, the IFHS and Lancet surveys had different methodologies. It is absurd to argue that adjustments made to the IFHS survey should be made to Lancet numbers. Especially when you find the resulting number implausible.

[david kane] "none of that makes the L2 and IFHS numbers more comparable because the central (almost only) critique of L2 has always revolved around their violent death total. No one disputes that their non-violent excess death estimate is plausible."

May I submit that the discrepancy in 'violent deaths' may lie with the identity of the pollsters and the interviewed Iraqi's reaction to that? Given that Iraq was in a grip of a vicious sectarian war, it could well be that the respondents modified their answers (possibly for all three studies) in order to please whoever they thought the pollsters were.

For example, the IFHS survey was carried out partly by the Iraqi Health Ministry, which was run by Sadrists at the time - who were also heavily involved in the sectarian fighting. If a Sunni family were interviewed, and their son, for example, had died violently, they might want to hide that fact lest they be suspected of being involved in the sectarian fighting and set themselves up as targets. (It is notable that there were areas deemed "too dangerous" for these pollsters to visit in Baghdad, and very likely these were hardcore Sunni areas where the pollsters as Shiites risked being killed themselves.) One supposes that the opposite might have occurred in the case of Burnham et al's study, too.

As for the study authors not disclosing more detailed information on who specifically was interviewed, it is understandable for the same reasons: those polled would run the risk of becoming sectarian targets. Imagine how difficult it would be for further pollsters in Iraq if it became known that the details of previous interviewees were released. At the moment, for example, Iraqi translators are living in terror that their details will be released to the Iraqi authorities by the US military, for this very same reason.

I think it is hard to imagine the kind of fear that pervades these communities in Iraq, and this will certainly have some sort of effect on surveys.

âWhich is not what Roberts said. The surveys disagreed about the number of violent deaths, but the number of excess deaths from the IFHS was inside the confidence interval for excess deaths from Lancet 2.â

If I may, they disagreed âviolentlyâ about the number of violent deaths, Tim. Letâs be frank here. Similarly, L2 and L1 are highly contradictory in terms of violent death. This notion that the IFHS and the Lancet studies are somehow corroborative, so long as their respective violent death components are ignored, is ridiculous.

These studies were undertaken because there was an armed conflict going on in Iraq. It is widely understood that people die in armed conflicts either directly from violence (bullets and bombs) or indirectly from the collateral effects of the violence on society. Any survey purporting to accurately detect the excess death toll from a conflict must be able to accurately detect the excess death toll of the two subgroups. Obviously, some latitude should be granted for accuracy beyond these two largest, most basic subgroups of death, but there are absolutely no grounds for giving a death study a pass on accuracy for the two main subgroups. The differences between the mechanism of death in each major subgroup (violent and non-violent) are too marked to allow any greying of the two whatsoever. Studies that are in sharp disagreement over the disposition of deaths from natural causes and accidents and those caused by violence are highly contradictory of one another, and call into serious question the accuracy of one or both of the studies.

To put this another way, studies cannot be said to corroborate one another if said corroboration requires the swapping in and out between studies of tens of thousands of deaths, even hundreds of thousands of deaths, between violent and non-violent causes. No one knows the precise number of excess deaths that have occurred in Iraq since March 2003, but there is a precise number. Every one of the victims had an identity, and every one died from a specific cause. Hundreds or thousands of deaths extrapolated from a car bomb in Karbala from one study cannot be balanced off in some form of corroborative equation against hundreds or thousands of extrapolated excess heart attack deaths in Nineveh in another study. In effect, thatâs what youâre doing, Tim. The same person cannot die twice, from two different causes, and in two different parts of the country.

Mike H, You shoot yourself in the foot when you casually remark, "These studies were undertaken because there was an armed conflict going on in Iraq".

Where to begin dismantling this simplistic remark? First of all, there wasn't merely an 'armed conflict' going on; the truth is that the 'conflict' was hugely disproportionate in terms of technology. The second point is that the 'conflict' should be termed 'aggression', because that is what the 'conflict' was: aggression on the part of the US and its proxies. I am amazed how those who somehow defend the invasion always resort to Orwellian termsd like 'operations' and 'conflicts'. The 'conflict' was an act of aggression that violated international law. The result. whatever end of the scale it is measured by, was carnage. Why not call it what it was, and stop decorating it up?

Second, as I said in an earlier post, of course there is a precise number of deaths caused by the war, either directly or indirectly, but why didn't the invaders show any interest in carrying out their own detailed survey? As I said before (a) they didn't really care about the death toll, because they were pursuing an altogether different political agenda, and (b) a detailed number would cast a pretty horrible light on the outcome of the aggression. As long as the figure remains in ambiguity, any figure, particlarly those that project almost genocidal totals, can be dismissed as heresay.

How many people perished in Iraq as a result of the sanctions of mass destruction between 1991 and 2003 that resembled a medievel siege? Two senior UN officals called the sanctions 'genocide masquerading as policy' but the corporate media ignored them almost completely. One of them, Hans von Sponeck, recently wrote and published a book on the effects of the santions on Iraqi civilians, and it was ignored, as to be expected. Again, the body count under the sanctions is exceedingly unclear (one million? Half a million? 250,000?), hence it can also be summarily dismissed by the west.

Finally, I think the studies were undertaken not because there was a 'conflict' (ugh, that word again) going on in Iraq, but because the invasion was highly controversial, illegal under international law and the US contitution, and because a number of critics had argued beforehand that there could be an enormous humanitarian cost. Whatever the critics of the OMB and Lancet surverys say, the critics were correct on all counts. Iraq society was utterly devastated by the war and the death toll was horrific. Moreover, all of the arguments used by Bush, Blair and co. to justify the invasion were found to based on a series of epic lies. What is there left to defend?

Shorter Mark H: We don't know, therefore we know.

[mike h] "To put this another way, studies cannot be said to corroborate one another if said corroboration requires the swapping in and out between studies of tens of thousands of deaths, even hundreds of thousands of deaths, between violent and non-violent causes."

Did the IFHS (NEJM) study ask for death certificates to corroborate its data? The Gilbert et al / Lancet study did.

you are confusing violent and total excess death cases.

Then the author is comparing apples to oranges.

The Lancet study attempted to determine excess violent deaths per this release:

We estimate that as of July, 2006, there have been 654,965 (392,979â942,636) excess Iraqi deaths as a consequence of the war, which corresponds to 2Â·5% of the
population in the study area. Of post-invasion deaths, 601,027 (426,369â793,663) were due to violence, the most common cause being gunfire.

To compare the two studies and claim that the results are similar is simple dishonesty.

Shorter (Mis)Information:

I'll now purposely ignore Burnham et al.'s figure for total eaths.

I'll then make a bogus comparison between IFHS's figure for total deaths and Burnham et al.'s figure for violent deaths.

Then I'll point out that my bogus comparison is, well, bogus.

Profit!

nice catch bi :)

to quote Tim s original post above: (last paragraph)

McArdle does not seem to have understood what Roberts was saying was in agreement: the excess deaths in the Lancet study (about 650,000) and in the IFHS study (about 400,000).

The differences between the mechanism of death in each major subgroup (violent and non-violent) are too marked to allow any greying of the two whatsoever. Studies that are in sharp disagreement over the disposition of deaths from natural causes and accidents and those caused by violence are highly contradictory of one another, and call into serious question the accuracy of one or both of the studies.

this is FALSE and RIGHT at the same time.

basically comparing subsets (like violent dead) is problematic, because you lose NUMBERS. the best comparison is between the biggest numbers you got.
imagine two groups are polling people for a rare event (accidential death). their result for totals is pretty close. but their results for a subset (people killed by lightning) is wide apart.

on the other hand, there IS an easy explanation for the difference:
one study did a very short interview about ONLY death cases. the other is a twenty pages interview, with death on page 16. one team of interviewers were independent doctors. the other was done by members of the iraqi health ministry, a group RESPONSIBLE for a significant amount of the death cases, they were polling...

just by chance, one of the two groups confirmed their death numbers by looking at death certificates....

sod: You think that the Lancet interviewers were "independent doctors?" That's not true. They worked for an Iraqi hospital run by the Iraqi government. That, obviously, does not make them bad people, but their pay checks were written by the Iraqi state, just like those for the IFHS interviewers.

David, i see a difference. don t you?

HOUSEHOLD QUESTIONNAIRE
HOUSEHOLD INFORMATION PANEL
We are from the Ministry of Health and the Central Organization of Statistics & Information Technology. We are conducting a
survey on the health of families, women and children. I want to talk with you about this subject because we believe that family
health is the base of community health. This survey will take some of your time.

this is how they introduced themselves. might sound threatening to a Sunni, actually....

Excellent point. How did the Lancet interviewers introduce themselves?

....

Oh, that's right! You don't know. The Lancet team has never released the script that the interviewers were supposed to follow. This is example #334 of how the IFHS conducts itself and in professional manner while the Lancet authors have not.

Obviously, the fact that the IFHS interviewers introduced themselves in such-and-such a way might create problems. But it is impossible to evaluate the extent the of possible bias in comparison with the Lancet papers unless we get some transparency about what the Lancet interviewers said.

Just to get this clear in my mind: how exactly would you be able to work out the extent of bias if you knew the Lancet team's manner of introduction?

bi -- IJI, you seem to have dropped your claim of their nefarious connection to the National Association of EMS Physicians. So when you said:

I'm sure one can come up with a perfectly good explanation for this, but the interesting thing is that nobody's even bothered to give one.

I pointed out that a perfectly sane explanation is that businesses and associations often group together in office buildings. Sane people think of this possibility instead of jumping immediately to paranoid claims of conspiracy due to the sharing of addresses, and note that to get to the bottom of this oh so scary conspiracy, one might think of going to the site of AAPOR, AMP, or NBRC, where they apparently forgot they're supposed to be some clandestine operation and point out in numerous places that Applied Measurement Professionals is AAPORâs management firm, and that both are a wholly-owned subsidiaries of the NBRC. My god, an organization with differtent holdings! That's scary!

Even more scary connections are revealed when you look at their site (don't they realise they should be hiding this stuff or their conspiracy will fail?). Why they're sponsored by the American Association for Respiratory Care, the American College of Chest Physicians, the American Society of Anesthesiologists, and the American Thoracic Society. Scary stuff kids!

Robert Shone is right when he points out that your posts on this, uncovering through diligent whois detective work what could be found by going to the site of the organization and reading what they say about themselves, makes you look like a loon; given those posts and their tone I'm not sure I agree with what I read as his connotation that this would be an inaccurate characterization.

Jeff, I'm not interested in engaging in a semantics exercise in order to satisfy your desire for a polemical exchange. Further, I'm not interested in having yet another long winded discussion over the case for war, let alone the sanctions period of 1991 - 2003. If you have something specific to my comment about the statistical comparisons of the various studies, then I'll be happy to respond in detail.

Gaz writes:

This may be off topic but has anyone worked out how many Iraqi soldiers were killed in the Iraq wars, or is it a matter of male+in uniform+Iraqi = don't matter?

Even if it worked out to one soldier per ton of bombs it would surely add up to well over 100,000.

That's a lot of heartbroken mothers.

I assume you're not saying here that this many people were killed because of American bombing? Because there hasn't been a lot of American bombing since the very first phase of the war. Most of the bomb fatalities have been from suicide bombers and IEDs.

To clarify my position -- I have no doubt that hundreds of thousands of Iraqis have died due to the war, and the figure may well be 600,000. I haven't examined the details and can't give an informed opinion. I just want to point out that a high casualty total does not ipso facto mean we should not have invaded, though it might well mean we should have done a better job of it. A lot of people died in World War II, but that was still a war better fought than unfought. And again, I'm not endorsing the Bush administration or their motives, though I'm sure somebody here will accuse me of being a neocon.

Shorter (Mis)Information:

How clever! An insult against my chosen screen name. I'm quite certain something even more devestating is percolating inside that head of yours.

I'll then make a bogus comparison between IFHS's figure for total deaths and Burnham et al.'s figure for violent deaths.

Wait a second here. The IFHS specifically warns against using their study for any extrapolation other than excess violent deaths. Per the study:

The non-violent mortality rate increased by about 60%, from 3.07 deaths per 1000 people per year before the invasion to 4.92 deaths per 1000 people per year in the post-invasion period. This was not further addressed in this analysis, which focused on mortality due to violent deaths. Further analysis would be needed to calculate an estimate of the number of such deaths and to assess how large the mortality increase due to non-violent causes is, after taking into account that reporting of deaths longer ago is less complete.

Are you seriously going to accuse me of dishonesty when I simply mistook the poster's reliance on a fictional figure for a figure that the study's authors actually expressed confidence in? Such sarcasm is usually indicative of an individual who is usually immune to challenges to their authority. You're either extremely arrogant or incredibly insecure. Maybe both.

Futher, using the numbers utilized by the UN/IFHS study, I'm still a little confused as to how you arrived at the 400,000 number. Per their figures (3.07/1000 pre-2003, 4.92 post-2003, 28m population over a period of 38 months), the difference is ~160,000. Added in to their estimate of ~150,000 violent deaths leaves us with ~310,000 excess deaths. I'm being sincere when I ask for guidance on where my calculations are off and how the figure of ~400,000 is determined.

Jeff Harvey, predictably, writes:

that is what the 'conflict' was: aggression on the part of the US and its proxies.

Al Qaeda wasn't being aggressive? You do that Al Qaeda in Iraq A) was responsible for a huge number of deaths, and B) was largely made up of non-Iraqis who came in to fight the US? For that matter, were the Sunnis being aggressive when they massacred Shi'ites, or vice versa? How about the Kurds? Was Iran being aggressive when it shipped arms to people fighting the Americans? Or sent in troops, as some people allege it has done?

Portraying this as a simple war of Iraqi good guys against American/UK bad guys is not just wrong, it's grossly ignorant.

Jeff Harvey, still channeling Noam Chomsky, writes:

How many people perished in Iraq as a result of the sanctions of mass destruction between 1991 and 2003 that resembled a medievel siege?

Do you think it would help if Saddam had used the oil-for-food money and spent it on food instead of arms and building programs for the military?

The money was there. It could have been used to feed people. But Saddam was kind of like Kim Jong Il -- he always took care of the soldiers first, and didn't really care what happened to anybody else. But I forget -- in Jeff's world, only the west can be a bad guy.

Al Qaeda wasn't being aggressive?

Not in Iraq.

A) was responsible for a huge number of deaths, and
B) was largely made up of non-Iraqis who came in to fight the US?

The fact that Al Q made an appearance in Iraq post-invasion supports a claim that we invaded due to Al Q aggression pre-invasion?

Interesting.

Portraying this as a simple war of Iraqi good guys against American/UK bad guys is not just wrong, it's grossly ignorant.

Jeff did no such thing. I thought Christians were supposed to be honest.

Did the IFHS (NEJM) study ask for death certificates to corroborate its data? The Gilbert et al / Lancet study did.

Bruno, regardless, we still have stated figures for excess violent death from these studies. I haven't seen any compelling evidence that the IFHS violent toll is an undercount because respondents could have been intimidated into characterizing violent deaths as non-violent

basically comparing subsets (like violent dead) is problematic, because you lose NUMBERS. the best comparison is between the biggest numbers you got. imagine two groups are polling people for a rare event (accidential death). their result for totals is pretty close. but their results for a subset (people killed by lightning) is wide apart.

The problem with your analogy, Sod, is that your subset (lightning) is exceedingly rare. The major subsets of death, violent and non-violent, are not. Every death fits into one or the other, without exception. Unlike death subsets further down the chain, there should be few errors recorded when determing whether a reported death is violent or non-violent, and any such errors would likely be intentional on the part of the survey respondent. I don't believe intentional misleading was a significant practice by those surveyed.

Shorter Mark H: We don't know, therefore we know.

Bi--IJI, Implicit in your sarcasm is the assertion that you and the other defenders of the Lancet studies do âknow.â If L2 and the IFHS corroborate each other, then what have you come up with in terms of an actual excess violent death toll? Did you simply split the difference between the two, or have you employed some other formula for resolving the huge disparity between the two studies?

The truth is, we really donât âknowâ with much accuracy, and the Lancet studies are particularly less than helpful in that regard. Iâll give you an example to illustrate that.

In December 2004, when the first Lancet study was being debated here, I contacted Les Roberts and asked for a breakdown of the bombing deaths and the excess violent deaths not attributed to the coalition. Included in the information Roberts provided me was the location of every excess violent death the study recorded, except for the 3 shooting deaths attributed to coalition soldiers:

http://scienceblogs.com/deltoid/2004/12/lancet11.php

âYes, all 12 non-coalition violent deaths happened outside of Falluja. (1 Kut, 1 Thiqar, 1 Karbala, 7 Baghdad, 1 Diala, 1 Missan, Note Baghdad is about 3-7 times greater in population than these other Governorates so the rates are not so different)

Bombing deaths:

Thiqar
M5, M2, F22 (one family)
Thiqar (different village)
M27

Missan
1mo. & 6mo. in same households (often there are multiple sons with wives under the same roof --- interviewer did not record the gender of the infant)

Falluja
10 girls<12 years, 13 boys<12, M14, 25 adult males, 3 adult women (adult defined as 15--59). â

As you can see from Robertsâ reply, the violent death toll was heavily weighted to the southern Shiite governorates and Baghdad, with more than half the violent deaths in L1 occurring (Falluja cluster excluded) in the south. It is generally understood from reporting on the ground that the southern Shiite governorates have seen much less violence than Baghdad and the mixed Sunni/Shiite governorates in the north-center of the country (Anbar, Diyala, Ninevah, and Salahaddin). L1 does not reflect that reality at all. If you look at Robertsâ figures, we know the location of death for 18 of the violent deaths recorded in the L1 survey, and only 1 of them occurred in Diyala, Ninevah or Salahaddin. We donât know where the last 3 violent deaths (from coalition gunfire) occurred, however it seems highly unlikely that all 3 of these deaths happened in these 3 provinces. Even if, for the sake of argument, we allow that all 3 of these shootings occurred in these 3 high violence governorates, weâre still left with an overall extrapolation of violent deaths in these 3 provinces of less than 12,000.

Then along comes L2, and it provides a very different geographical distribution of violent death from its predecessor. I donât have access to the data from L2, but I have corresponded with someone who does, and he has given me some of the numbers he extrapolated for violent death from governorate to governorate. For the same 18 month period covered by L1, L2 extrapolated 32,000 - 34,000 violent deaths in Diyala alone. Ninevah, 18,000 violent deaths and 36,000 for Salahaddin, for a total in these three governorates of 86,000 violent deaths. As I pointed out earlier, the violent death toll from L1 for these governorates canât be any higher than 12,000, and may well be much less, as low as 3,000. My source for the L2 data advised that the extrapolated violent death toll for Diyala alone, to the end of the L2 survey period in July 2006, was in excess of 100,000.

However, that isnât anywhere near what the media reported in Oct 2006, when Agence France Presse ran this wire service story about the carnage in Diyala:

http://findarticles.com/p/articles/mi_kmafp/is_200610/ai_n16933422

Iraq province loses 9,000 to sectarian killing

Sat Oct 21, 8:38 AM ET

âThe bloodsoaked Iraqi province of Diyala has seen 9,000 of its citizens killed and at least 31,500 forced to flee since the fall of Saddam Hussein, its police chief has said.

Defending the scope of an aggressive new security operation, Major General Ghassan Adnan al-Bawi told lawmakers Saturday that his force was dealing with a massive campaign of sectarian cleansing in the killing fields north of Baghdad.

In the three years and seven months since the US invasion, Sunni and Shiite death squads in Diyala have been battling it out for one of Iraq's most fertile and religiously mixed areas.â

The link provided does not allow for the complete article to be read without registration. This article was widely available when I saved it in 2006, however now, not so much. I have the entire text if anyone wants to read it.

9,000 dead in a province of 1.4 million people is still a very ugly number, and that number went up considerably in Diyala in 2007 and 2008, when many were calling it the most violent place in Iraq. Still, it is completely out of whack with L2âs extrapolation.

Problem with the second link:

http://findarticles.com/p/articles/mikmafp/is200610/ai_n16933422

Third try;

Diyala

In August, Mohamed Ali, a who statistician, reported his preliminary results to colleagues at a Denver statistics conference: Nearly 397,000 Iraqis had died because of the war as of July 2006.

the 400000 number has been confirmed by the WHO, as written by Tim above.

Every death fits into one or the other, without exception. Unlike death subsets further down the chain, there should be few errors recorded when determing whether a reported death is violent or non-violent, and any such errors would likely be intentional on the part of the survey respondent.

would you know, whether a person in your household has died over the last 4 years?
on the other hand, could you have another idea about the cause of death, than even the doctor? or could you be tempted to give a different explanation for the death, when asked about it? (because of shame for example?)

the 400000 number has been confirmed by the WHO, as written by Tim above.

I'm sorry, but that is a reference to an article in Mother Jones, not a reference to anything written in the study. Shall I repost the study's wording?

QrazyQat:

> one might think of going to the site of AAPOR, AMP, or NBRC, where they apparently forgot they're supposed to be some clandestine operation and point out in numerous places that Applied Measurement Professionals is AAPORâs management firm, and that both are a wholly-owned subsidiaries of the NBRC.

OK, OK, I take back those claims of mine.

" on the other hand, could you have another idea about the cause of death, than even the doctor? "

did some work on traffic accidents (in the US) a few years back. official cause of death listed in the government mortality records was pretty much useless, as a large proportion of the folks in a severe car crash, then hospitalized, then dead without ever leaving hospital were listed as cardiac failure. that may well have been the proximate cause of death, but most laypersons would consider that an indirect cause secondary to having your thorax crushed. (the effect is even more pronounced with death by illness rather than trauma)

Barton (#59) - "I assume you're not saying here that this many people were killed because of American bombing? Because there hasn't been a lot of American bombing since the very first phase of the war. Most of the bomb fatalities have been from suicide bombers and IEDs."

I know the surveys being debated here do not concern deaths during the "very first phase".

The point I'm making is really that there's an awful lot of focus on civilian deaths, women-and-children deaths, and deaths during the occupation rather than the war proper.

I just don't accept the implicit value judgment in all this that if a 19-year old girl gets blown up by a roadside bomb it's a tragedy but if her little brother was conscripted into the army then vaporised by a 1000 lb bomb or buried alive in the sand by a bulldozer tank it's just, hey, c'est le guerre, baby.

Is someone killed during the "very first phase of the war" somehow less dead than someone killed six months later? How much less is the life of a conscript (Google: Saddam amputation conscription) worth compared to that of a civilian? Was the "Highway to Hell" in 1991 not that much of a big deal, morally speaking? Is there a formula or a lookup table or something?

I know it's important to find out what's going on now so we can assess the success of the occupation of Iraq but surely the judgment must include the price paid in terms of lives lost in the "very first phase".

When it comes down to it, the idea that the lives of soldiers, especailly enemy soldiers, are worth less than other people's lives is what makes war possible. I try not to get sucked in by it.

Sigh.

I have more chance of understanding GCMs than this weirdass moral calculus.

I know this is off-topic. I just think there's something generally wrong with the topic.

I have more chance of understanding GCMs than this weirdass moral calculus.
I know this is off-topic. I just think there's something generally wrong with the topic.

In a way it's not off-topic. What you're seeing here is very similar to what you see at climate audit. Claims of fraud and incompetence on the part of individual researchers in order to spread FUD about facts that certain people find politically inexpedient. CA et al want to cast doubt on the notion that scientific claims regarding AGW are true. Kane et al want to cast doubt on claims that the Iraq war has cost the lives of a very large number of civilians.

dhagaza: "Kane et al want to cast doubt on claims that the Iraq war has cost the lives of a very large number of civilians."

My point is that even if they fail to do that they've at least succeeded is drawing attention away from the countless thousands of young men in uniform who've been slaughtered.

As a handy test, whenever you see the term "civilians", substitute the term "white folks" and see how it sounds.

So, doing that with your comment: Kane et al want to cast doubt on claims that the Iraq war has cost the lives of a very large number of white folks.

Sounds pretty bad doesn't it, because we know white folks and non-white folks should be valued equally.

I just happen to think those young men in various armies and non-white folks should be valued equally.

I know it's not fashionable, but there you go.

dhogaza writes: "Kane et al want to cast doubt on claims that the Iraq war has cost the lives of a very large number of civilians."

Oh, ye of little faith! I (and every single one of the academic Lancet critics I know) believe that the Iraq War has caused the death "a very large number of civilians." Happy?

We just think that the data underlying the Lancet studies is wrong.

My point is that even if they fail to do that they've at least succeeded is drawing attention away from the countless thousands of young men in uniform who've been slaughtered

Well, no more than about 10,000 Iraqi military personnel are thought to have been killed during the invasion. Even if you only accept lower figures than Lancet II you're looking at most something like a 5% as many uniformed soldiers killed vs. post invasion deaths. The real figure is probably considerably lower.

If you're talking about military and police killed during our occupation, it's a similar "around 10,000" figure - including recruits who've not completed training or even sign-up.

The focus on the much larger group of deaths rather than your "white men" seems justified.

dhogaza, certainly you could justify the focus on civilian deaths on the basis that there have been so many more of them then military deaths but I firmly believe it would be the same if it was the other way around.
I watched in 2003 as everyone mourned the accidentally bombed civilians but even the trendiest bleeding heart liberal wouldn't bat an eyelid when a bunch of troops was blown to smithereens.
(A digressive point - sure "only" 10,000 or 15,000 or so were killed according to most estimates but the Coalition of the Willing was prepared to kill many times that number to achieve their goal. The relatively low number was the result of good luck or good planning, but it was not guaranteed by any means.)
Anyway, my point is that soldiers just don't seem to matter that much to people - it's as if there's a tacit consensus that a war where only soldiers were killed would be just fine. This doesn't just apply to the Iraq war or just to wars the USA is involved in.
I wouldn't want anyone to stop counting the dead civlians on my account though.

I (and every single one of the academic Lancet critics I know) believe that the Iraq War has caused the death "a very large number of civilians." Happy?

not yet. do you believe in the IFHS study of 400000 excess deaths since the invasion? (and seriously more up till now!!)

why don t you spread your confidence in that number via the right wing blogosphere?

that would make me seriously happy!

would you know, whether a person in your household has died over the last 4 years? on the other hand, could you have another idea about the cause of death, than even the doctor? or could you be tempted to give a different explanation for the death, when asked about it? (because of shame for example?)

One might mistake the stroke death of a family member for a heart attack death, but I wouldn't expect a respondent to have any difficulty deciding whether a member of the household died from a stroke, or was blown to pieces by a car bomb.

As I mentioned earlier, the temptation for a survey respondent to deliberately falsify cause of death when asked does not strike me as a widespread occurrence. In any case, there seems to be some serious cherry-picking as to the frequency of this alleged practice whenever this discussion arises, depending on which survey is being discussed.

dhogaza writes:

Al Qaeda wasn't being aggressive?
Not in Iraq.

dhogaza, there is an organization called "Al Qaeda in Iraq" which has been in Iraq since about 2003 and has caused a great deal of death and destruction. Google for the full phrase.

dhogaza continues:

The fact that Al Q made an appearance in Iraq post-invasion supports a claim that we invaded due to Al Q aggression pre-invasion?
Interesting.

You are commenting on a claim I never made. Read what I wrote again.

Portraying this as a simple war of Iraqi good guys against American/UK bad guys is not just wrong, it's grossly ignorant.
Jeff did no such thing. I thought Christians were supposed to be honest.

It looked that way to me. I thought thoughtful posters like you were supposed to not jump to conclusions about the honesty of others without making some effort, however feeble, to check that honesty out. Do you really think I was deliberately misrepresenting Jeff, assuming that's what I did? What the hell is wrong with you?

Gaz, a consensus has developed over the past couple of centuries, at least in the west, that soldiers should try to avoid killing civilians if possible. Yes, it's sad when a soldier dies, too. But the conclusion from "all deaths matter the same amount in all ways" must be either "let's go to total pacifism" or "let's kill whoever we need to to achieve our goals." Since most people disagree with both of these, the distinction between military and civilian deaths will probably continue.

dhogaza, there is an organization called "Al Qaeda in Iraq" which has been in Iraq since about 2003 and has caused a great deal of death and destruction. Google for the full phrase.

We invaded in 2003, and telegraphed that invasion for about six months previous.

Again, you can't use a reaction by AlQ to our aggression to claim that AlQ aggression was justification for our attack.

AlQ wasn't active in Iraq until we invaded.

I googled. First hit:

"Al-Qaeda in Iraq (AQI) is a group playing an active role in the Iraqi insurgency."

Insurgency. AGAINST OUR OCCUPATION.

Mike, bombing continued over Iraq at a rate that would cause multiple fatalities, besides which the US and Brits used air support at an even higher rate. Eli doubts deaths from air strikes with machine guns would be differentiated.

Eli:

Sorry, I'm not sure what you're trying to establish with your post. Are you addressing the issue of violent death/non-violent death subsets, or the violent death disparity between L1 and L2 in the 3 Sunni-populated governates?

Wildlifer, If you think this distinction is applicable, shouldn't it be up to the aggressors to count the number and status (civilian or combatant) of the dead as a result of their invasion?

Not really.

Why don't you think the US or UK showed any interest in carrying out a detailed survey of the death toll in Iraq as a result of their combined aggression? Or, for that matter, of any wars waged? And is such a distinction between 'civilian' and 'combatant' deaths valid anyway? I am sure that many ordinary Iraqi civilians became combatants as a result of the fact that they don't necessarily like being occupied by a foreign aggressor, particularly one that has murdered their loved ones. Given that the invasion violated just about every standard of international law (as well as the United States constitution), it was illegal anyway and thus so-called combatant deaths are also a crime.

No it didn't. We were in a cease-fire with Saddam dating back to '91â one which he continuously violated during the Clinton administration. That was enough to take him out. I didn't need the excuses of shrub's administration.

But back to my original point: it should be obvious why the US and UK did not carry out a census of the death toll as a result of the invasion. First, they didn't care.

Exactly. It doesn't matter.

Second, it the actual total was accurately known, at least to within a few thousand, it would further demolish the notion that the US and UK are great respecters of freedom and human rights. But as long as the actual total is very vague, then any estimate generated from any source is open to doubt. This enables the western corporate media to downplay the idea that the invasion resulted in utter carnage, and maintains the myth of the 'basic benevolence' of the west.

War is hell. I don't need the media to tell me shit and people get blown up in a war.

The major aim here of defenders of unilateral US aggression is to sow doubt as to the actual death toll in Iraq. As long as there is doubt then it is possible, in public relations terms, to 'manage the outrage'.

It was war. What does it matter if it was 10,000 or 10,000,000?

Actually, there are similarities with respect to the campaign waged by certain sectors and individuals to downplay the theory of anthropogenic climate change. The skeptics know that they will never win the scientific argument, but they don't have to: so long as they can sow enough doubt amongst policy makers and the public, then nothing will change. If the true death toll as a result of the war in Iraq was known, then I am sure that there would be repercussions. So the governments, along with much of the corporate mainstream media, do everything they can to obfuscate the truth. They routinely attack scholars whose work suggests that the death toll was enormous, because, as I said above, it demolishes much of the myth that our governments pursue humanitarian agendas. They have agendas all right, but they are vastly different.

It's a stupid argument to have and a lesson I thought we learned in Vietnam. Body counts don't mean anything.
But to expand on my original point. We know Iraqis will lie through their teeth (Baghdad Bob). We can't verify how many were or were not actual regular army troops who died, and are being posthumously âdischargedâ. Nor do we know how many deaths are attributable to sectarian violence (quite a bit it seems). But I do not believe at anytime our forces intentionally targeted civilians.

(I fixed your typos. You're welcome.)

Sorry Mike, it was Barton who was speculating that bombing from aircraft had stopped.

OTOH, might one inquire of David Kane if he was the one who complained to the AAPOR?

He has indicated otherwise, Eli:

And, for those who care, I am not a member of AAPOR and did not initiate the complaint.

It was war. What does it matter if it was 10,000 or 10,000,000?

I love it when people pooh-pooh the deaths of large numbers of other people.

War's hell! What does it matter of 6,000,000 Jews were killed by the Germans!

Such a moral person, this wildlifer.

I love it when people pooh-pooh the deaths of large numbers of other people.

War's hell! What does it matter of 6,000,000 Jews were killed by the Germans!

I love it when people think everything, no matter how unrelated, is the equivalent to Hitler.
I wasn't "pooh-poohing" anything. I'm just not shocked that people actually die in a war.

Such a moral person, this wildlifer.

Thank you.

No problem, Eli.

While I've got your attention, what are your thoughts with regard to my comment #65, and the huge regional discrepancies in violent deaths between L1 and L2?

No it didn't. We were in a cease-fire with Saddam dating back to '91â one which he continuously violated during the Clinton administration. That was enough to take him out. I didn't need the excuses of shrub's administration.

Take him out, and it wouldn't matter if we killed 10,000,000 innocent Iraqis in order to do it.

I love it when people think everything, no matter how unrelated, is the equivalent to Hitler

From a moral point of view, "unrelated" is an empty claim.

Take him out, and it wouldn't matter if we killed 10,000,000 innocent Iraqis in order to do it.

And saved exponentially more current and future lives in doing so.

From a moral point of view, "unrelated" is an empty claim.

Sorry, I must have missed the part where we marched them into gas chambers. You have a cite?

OTOH, might one inquire of David Kane if he was the one who complained to the AAPOR?

in the first post in the first topic of this subject, he said that he neither is a member, nor did make the complain.

funny enough, i believe him. even though he didn t make public all his memberships and neither confirmed his claim with a public oath.

dhogaza writes:

Again, you can't use a reaction by AlQ to our aggression to claim that AlQ aggression was justification for our attack.

Again, I never claimed that it did. Again, you've jumped to a conclusion without taking the time to read what you were critiquing.

Barton (#82), could I respectfully ask you to re-examine the logic of this statement:

"Yes, it's sad when a soldier dies, too. But the conclusion from 'all deaths matter the same amount in all ways' must be either 'let's go to total pacifism' or 'let's kill whoever we need to to achieve our goals.'"

That just doesn't follow from what I've said (even allowing that I'd claimed all deaths mattered equally, which of course I didn't). It's a very un-Barton-like argument, I must say.

Wars result in death and injury. That cost must be weighed up in deciding whether - and how - to fight a war. It is simply not logical to extend that argument to either absolute pacifism on one hand or militarism unfettered by concern about killing on the other.

My argument is simply that the cost in terms of dead soldiers, in particular the dead soldiers on the other side in lop-sided contests like the Iraq wars, is not given enough weight, and often no any weight at all, these days.

The idea that the lives of soldiers, particularly enemy soldiers, don't matter much seems to enters the war-making decision as an axiom.

Well, it's not an axiom, it's a value-judgment even if it is part of a consensus.

Let's put it this way.

Say, you think the invasion of Iraq was justified for some reason - WMD's, getting rid of a tyrant, restoring the ecology of the marshes, whatever.

Now let's say we knew in advance that a million Iraqi civilians would die as a result.

Would it be worth it in order to achieve the objective?

If not, where would the cut-off point be? 100,000? 20,000? 5,000?

Implict in all the discussions about of surveys of civilian deaths is the idea that at some point the price may not have been worth paying (or, more correctly, not worth forcing the Iraqi people to pay).

Now, let's ask this question.

How many Iraqi military deaths would have been worth it?

A million? 500,000? 10,000? 1,000?

Where is the cut-off point?

Is it different from the cut-off point for acceptable civilian deaths? If so, why?

How is the decision made? By whom? On what basis?

By the way, Barton, I disagree with this statement from you: "Gaz, a consensus has developed over the past couple of centuries, at least in the west, that soldiers should try to avoid killing civilians if possible."

Well, maybe that would apply for the past 50 years, but if you go back a couple of centuries you'd have to include World War 2.

And saved exponentially more current and future lives in doing so.

You've got some sort of evidence to back up this claim? Start with *current* lives, please, since you've used the 'and" word. I assume the exponent involved is greater than 0 or 1?

Details, please.

Oh, and while Godwin's law is cool, I can't help but point out that nazi germany justified slaughtering jews with a similar argument, i.e. that eventual jewish domination of the world, if not checked, would lead to Horrifying Consequences For The Master Race and other christians.

And, BPL ...

Jeff Harvey, predictably, writes:
that is what the 'conflict' was: aggression on the part of the US and its proxies.

Followed by your response:

Al Qaeda wasn't being aggressive?

This suggests that aggression on the part of the US was triggered by, justified by, or somehow related to aggression by AlQ.

If you meant this to be a meaningless, pointless deflection from the issue, my apologies for misunderstanding your point. Not that I think such a meaningless, pointless deflection is any more effective at countering Jeff's point.

You've got some sort of evidence to back up this claim? Start with current lives, please, since you've used the 'and" word. I assume the exponent involved is greater than 0 or 1?

Details, please.

Are you entirely ignorant of Saddam's history?

Oh, and while Godwin's law is cool, I can't help but point out that nazi germany justified slaughtering jews with a similar argument, i.e. that eventual jewish domination of the world, if not checked, would lead to Horrifying Consequences For The Master Race and other christians.

You sound just like the right-wing reactionaries, and just as revisionist.

dhogaza writes:

Jeff Harvey, predictably, writes: that is what the 'conflict' was: aggression on the part of the US and its proxies.
Followed by your response:
Al Qaeda wasn't being aggressive?
This suggests that aggression on the part of the US was triggered by, justified by, or somehow related to aggression by AlQ.

No, it doesn't suggest any such thing. It's a complete and total non sequitur.

Let me try and spell this out so even you can understand it.

I understood Jeff to be saying, "The only source of damage in Iraq was American/UK aggression."

I was saying, "No, other parties also caused a lot of damage."

Do you get it yet?

If you meant this to be a meaningless, pointless deflection from the issue, my apologies for misunderstanding your point. Not that I think such a meaningless, pointless deflection is any more effective at countering Jeff's point.

See above.

I understood Jeff to be saying, "The only source of damage in Iraq was American/UK aggression."

Yes, I get it, and I owe you an apology.

I was crediting you with reading comprehension skills.

AAPOR alleges Gilbert Burnham violated AAPOR's code

tlambert — Wed, 04 Feb 2009 10:13:18 +0000

AAPOR alleges Gilbert Burnham violated AAPOR's code

The American Association for Public Opinion Research (AAPOR) has put out a press release alleging that Gilbert Burnham (who is not a member of the AAPOR) violated the AAPOR's code of ethics. What did he do? Their press release states:

Mary E. Losch, chair of AAPOR's Standards Committee, noted that AAPOR's investigation of Burnham began in March 2008, after receiving a complaint from a member. According to Losch, "AAPOR formally requested on more than one occasion from Dr. Burnham some basic information about his survey including, for example, the wording of the questions he used, instructions and explanations that were provided to respondents, and a summary of the outcomes for all households selected as potential participants in the survey. Dr. Burnham provided only partial information and explicitly refused to provide complete information about the basic elements of his research."

That seems to be more than a little misleading. Burnham has released the data from the study. This report goes into a fair bit of detail on how the survey was conducted. And here is the survey instrument which includes the "wording of the questions he used".

The AAPOR press release fails to specifically state what information was not provided. Nor does there seem to be any sort of report available from the AAPOR web site. I've emailed them asking for this information, but so far have received no reply.

Update: Reaction to AAPOR press release.

tlambert Wed, 02/04/2009 - 05:13

Tags

AAPOR's investigation of Burnham began in March 2008, after receiving a complaint from a member.

complain from a member? sloppy research? false informations?

i am holding my breath...

here is a blog post in which Spagat is comparing the methods used to that of the AAPOR:

http://magicstatistics.com/2008/02/12/lancet-study-serious-ethical-laps…

ad in this post, David Kane makes the false claims about tem holding vital data back:

http://magicstatistics.com/2008/01/04/in-the-spotlight-again-lancet-stu…

"The authors refuse to provide anyone with the underlying data," said David Kane,

take your pick...

Alleges? really now.

http://abcnews.go.com/PollingUnit/story?id=6799754&page=1

Tim Lambert writes:
And here is the survey instrument which includes the "wording of the questions he used"

Well, that's the "Iraq Mortality Survey Template" posted on the National Journal site. But is it the questionnaire that was actually used? Nobody seems to know, as Burnham and Roberts (so far, to my knowledge) have declined to confirm or deny it. Other researchers, eg Fritz Scheuren, have requested copies of the questionnaire, but have apparently been refused by the Lancet authors.

Naturally, this is front-paged at RedState who gloss it as:

[T]he author of that âstudyâ [i.e. Burnham] has been officially censured by his professional peers for not meeting either scientific or professional standards in that âworkâ

Love those scare quotes on "study" and "work". And of course they fail to note that epidemiologist Burnham is not even a member of AAOPR (what does an mortality survey have to do with opinion polls anyway?). They follow up with this gem:

This is getting to be drearily predictable - political propaganda is given a spray-paint-coating of scientific imitation, and then is âmarketedâ as being science. Itâs good that someones with better expectations of professional responsibility had the guts to ask for more details, and then reacted appropriately when the basic standards of âscienceâ were clearly being violated.

As an aside, the same situation holds with much (most?) of the so-called âscienceâ associated with âglobal warmingâ (or whatever itâs being called this week). A great deal of professional censure is required there as wellâ¦.

Tim writes: "Burnham has released the data from the study." This is untrue for two reasons. First, Burnham has never released any data to Spagat or to any of his co-authors. Second, although Burnham released some of the data to me and other selected researchers, he has not released anywhere near enough for an outsider to judge the quality of the work.

As always, none of the critics are asking for information that would allow one to identify a particular respondent or interviewer. But, among other things, we would like to know which teams conducted which interviewers. We don't need the names of anyone involved. But if Team A had results that differed significantly from Team B, then concerns would be raised. Note that Roberts promised to release this data to Fritz Scheuren more than a year ago at the Joint Statistical Meeting in Salt Lake City. Roberts reneged.

Until Burnham et al have released the data to all critics and have made enough data available to allow others to judge their work, it is incorrect to claim that "Burnham has released the data from the study." At best, Tim could truthfully claim that Burham has released some of the data to selected outsiders.

And, for those who care, I am not a member of AAPOR and did not initiate the complaint.

Well, here's what they ahve to say on that:

we feel the time is now right to make the data set available to academic and other scientific groups whom we judge have the technical capacity to objectively analyze the data.

We won't freely distribute the data.

Conditions for the Release of Data from the 2006 Iraq Mortality Study

These data will be released on request to recognized academic institutions or scientific groups with biostatistical and epidemiological analytic capacity.

1. The data will be provided to organizations or groups without publicly stated views that would cause doubt about their objectivity in analyzing the data.

2. The data will remain the property of Johns Hopkins Bloomberg School of Public Health, and will be provided only on condition that the datasets are not shared with others.

What #1 means is "we won't give the information to anyone who might disagree with us."

#2 means "we won't let the people we give it to give it to anyone who might disagree with us."

That's not the behavior of scientific researchers, it's the behavior of political whores masquerading as scientific researchers.

If the data honestly supported their conclusions, they'd be willing to let those who disagree with them see it. Their behavior is almost perfectly aligned with what you would expect from people engaging in fraud.

Greg Q:

> > 1. The data will be provided to organizations or groups without publicly stated views that would cause doubt about their objectivity in analyzing the data. [...]

> What #1 means is "we won't give the information to anyone who might disagree with us."

Are you saying that "non-objectivity" equals "objectivity"?

Tim and the AAPOR appear to agree on this, at least; not releasing the information upon which you base your conclusions is a problem.

This in a country (the United States) where lying about Iraq and Iraqi casualties is a cottage industry.

They cannot censure him. They can say they don't agree with his methodology. They can refuse him admission if he applies. But they have no standing.

Hence, they're wingnut assholes, regardless of any other merits they may possess.

I, too, am concerned about the way this case was handled. If the researcher isn't a member of the association, may not even be familiar with the association, or may think of them as irrelvant and inconsequential, why should he/she release details??? A peer-reviewed journal (?) already found his methods to be sufficiently comprehensive!

Perhaps he chose not to aqueisce to what appeared to be intimidation tactics. Now look, his reputation, possibly career, is ruined. Maybe his research wasn't sound, maybe it was. This should have gone through "The Lancet." Without knowing all of the details, this seems like a very irresponsible reaction from AAPOR.

> Hence, they're wingnut assholes,

Now *there's* a mental image!

Cool. Can we get AAPOR to investigate Inhofe's list?

> Perhaps he chose not to aqueisce to what appeared to be intimidation tactics.

Well, we may never know, since the AAPOR hasn't released the exact words they wrote to Burnham, and the exact responses they received. Now, this isn't strictly part of the AAPOR code of ethics, but really, it'll be good for them to release these details for the sake of openness.

I just did a search on the AAPOR site for the terms: "epidemiology" Zero hits.
"age structure" Zero hits.
"mortality" Zero hits
"opinion" 84 hits.

Burnham was not doing a public opinion survey - he was doing an epidemiological survey, which comes with its own different set of ethical constraints. He was not a member of this organization, nor signatory to its code of ethics - properly so, as he was doing epidemiology - nor subject to its enforcement apparatus. The organization has no authority over Burnham, Burnam refused to cooperate with an absurd investiqation by an organization which has no business investigating him, and then he was censured by it.

Witch,hunt, indeed.

Hey, has anyone noticed that Johnson, Spagat, Gourley has finally got his made up fudge factor paper puplished in the esteemed:

Journal of Peace Research 45:5, 653, 2008 'Bias in Epidemiological Studies of Conflict Mortality'

The 'main street bias' paper in fact received The Journal of Peace Research Article of the Year Award.
http://dissident93.wordpress.com/2008/12/15/journal-of-peace-research-a…

Perhaps its critics at Deltoid could pool their informed, scientific comments ("bogus", "fudged", "pulled out of ass", etc) into a paper which they could get peer-reviewed and published?

Or perhaps they could listen instead to genuine authorities such as Jon Pedersen, who wrote (in an email to me, 4/12/06): "I very much agree with the MSB-team that there is some main stream bias, and that this is certainly an important problem for many surveys - not only the Iraq Lancet one... The MSB people have come up with some intriguing analysis of these issues".

That's very embarrassing for the Journal of Peace Research.

Of course. And probably shameful.

Does anybody know whether the published version of the MSB paper was an improvement on the drafts previously discussed here? Iâm not going to fork out 20 dollars to find out.

Journal of Peace Research? Tim L's comment sums it up.

Robert, you should read my comment on the thread above this. I'd like to know why Bush, Blair and co. didn't invest the time and money into doing a survey themselves, given that many punditys estimated that the invasion would lead to massive death totals. Why don't all governments do this? The answer is two fold: first, they don't really care how many die when they are promoting an alternative agenda. Second, if the accurate death toll were known, even if it was "only" 200,000, then this would appear horrific enough. But so long as the actual totoal remains as clear as mud, then the concern can be dismissed. In other words, without 100% unequivocol proof the problem does not exist. I've had to debate all kinds of climate sceptics: those who downplay biodiversity loss, those who dismiss acid rain, climate change etc. And they use employ the same strategy as those who defended the US invasion of Iraq: without concrete, iron-clad evidence of a process, then it just does not exist. So why would the US and UK governments be stupid enough to fund a survey that eventually shows the death toll in Iraq to be (for argument's sake) 200,000 plus or minus 5,000? Such a total could not be so easily sanitized, so they ignore it, and any studies that suggest totals to be exceptionally high are summarily dismissed.

Tim writes "That's very embarrassing for the Journal of Peace Research."

Well, opinions differ on that. But let's make some progress. Instead of merely asserting that this is "embarrassing," why not participate in a round-table on the topic? Deltoid would be the perfect location for that debate. I (and perhaps some of the authors) would be willing to participate. Tim (and other critics) could write something. The authors (and their supporters) could write a response. And so on. That's the way that science should work. To merely assert that something is "embarrassing," without being willing to participate in a debate on the topic, is not productive.

Here is the editorial board of the JPR. Is there any reason for us to think that they are less able to judge the quality of Spagat et al's work than Tim Lambert is? Perhaps! But only if Tim is willing to discuss the topic in detail.

David Kane: Tim (and other critics) could write something. The authors (and their supporters) could write a response.

Tim did write something about the MSB paper - in fact he devoted several threads to it. You responded, as did others, some of whom thought the paper wasn't as bad as Tim made out. AFAIK none of the authors are banned from commenting.

You want a forum for discussion? Here it is.

David, see my [previous post on the main street bias paper](http://scienceblogs.com/deltoid/2006/12/main_street_bias_paper.php).

I don't claim to be an expert in mortality surveys, but the authors of the paper and the editors of the journal dosn't seem to be, either.

Most of the criticism from Tim Lambert and others was directed at one set of parameter values which was presented only as an illustrative example by the msb authors (who later added an exploration of the parameter space).

In other words the criticism missed the point that the actual bias could be determined only as a result of disclosure by the Lancet authors on basics such as sampling procedures and main streets selected as starting points, etc.

So this brings us back, in a way, to the AAPOR thing. The Lancet authors still haven't disclosed the basic level of information which is obviously necessary to assess how their claim of giving all households an equal chance of selection holds up.

If you're extrapolating from 300 actual violent deaths to 601,000 estimated violent deaths, based on this claimed sample-randomness, then it would seem pretty important that the sampling scheme could be assessed in some way. Currently it can't be, because nobody outside the Lancet team knows what that sampling scheme entailed.

Robert Shone [offen as anna plurabella] leaps on anything he can to sledge the Lancet studies [and medialens].

mostly from his hideyhole at mediahell using sockpuppets.

he's not to be trusted an inch

I'm used to David Edwards/Cromwell (editors of Medialens) stalking me (under the pseudonym "Woofles") at mediahell.org, but I hope they'll use their real names on a respectable "science" blog.

Tim: Wasn't that discussion about a draft of the paper? I believe that the published version is different. So, why not re-open the discussion in a new thread? Again, I am not suggesting more back and forth deep in a comment thread (as fun as that is!) but a proper round table, similar to what Crooked Timber occasionally hosts, would be worthwhile. If you aren't willing to back up claims like "very embarrassing," then you shouldn't make them.

I don't claim to be an expert in mortality surveys, but the authors of the paper and the editors of the journal don't seem to be, either.

How do you know what the editors of the journal are experts in? Do you doubt their academic credentials? Do you deny that the paper was peer-reviewed by qualified reviewers? Again, it is one thing to argue with the paper or the authors, but to impugn the editors when (AFAIK) you have not even read the published article seems a bit much. Or is anyone who disagrees with you automatically suspect?

David Kane: I believe that the published version [of the MSB paper] is different.

You believe it's different? Do I take it then that you aren't willing to pay the 20 bucks for it either, then?

I certainly hope it is different for the sake of the journal's reputation, but I seem to recall Michael Spagat claiming that the paper was going to be published without substantial changes.

A reader has kindly sent me a copy of the published paper. My criticism stands.

bi -- IJI:

No, I'm saying they're lying sacks of poor quality fertilizer.

The People who wrote that study are left-wing zealots. Their ability to judge the "objectivity" of anyone else is nil. Given their complete lack of objectivity, their insistence that they wont give the information to anyone else who "lacks objectivity" is just another fraud.

Real scientists routinely give their data to people who want to prove them wrong, or who want to use that data for their own agendas (i.e. a lab that is competing with yours, on the same research subjects). The only reason for refusing to give data, used in a published paper, to anyone and everyone who wants to examine it is that your "research" was a fraud, and releasing the data will show it. See, for example, Michael Bellesiles.

There are no ifs, ands, maybes, or buts. If you want to do science, then you have to be willing to give the data that underlies your published research to anyone who wants to see it, especially those who want to see it so they can prove you wrong.

The refusal to do so is tantamount to a signed admission of fraud.

What would your response have been to John Lott saying "Tim Lambert disagrees with me, and wants to prove me wrong. Therefore he's not objective, and I will not give him any of the data that underlies my claims"? How would that be even the slightest bit different from what the Johns Hopkins people are doing?

did John Lott share some data with Tim? just curious...

Greg Q- Could you wipe the spittle from your keyboard please?

The refusal to do so is tantamount to a signed admission of fraud.

Ethics guidelines established for such research only exist to encourage fraud, in other words.

Tim writes: "A reader has kindly sent me a copy of the published paper. My criticism stands."

You realize, of course, that much of your criticism can no longer stand precisely because we know a lot more about the actual sampling now than we did when you wrote your criticism. (Or, rather, we now know for a fact that much of what the Lancet said they did, they did not, in fact, do.) Tim wrote then:

n, the size of the unsampled population over the size of the sampled population. The Lancet authors say that this number is 0, but Johnson et al speculate that it might be 10. This is utterly ridiculous. They expect us to believe that Riyadh Lafta, while trying to make sure that all households could be sampled, came up with a scheme that excluded 91% of households and was so incompetent that he didn't notice how completely hopeless the scheme was. To support their n=10 speculation they show that if you pick a very small number of main streets you can get n=10, but no-one was trying to sample from all households would pick such a small set. If you use n=0.5 (saying that they missed a huge chunk of Iraq) and use their other three numbers, you get a bias of just 30%.

There are so many problems with this criticism that it is hard to know where to start. Do you really stand by it? We now know that many of the streets were not included in the sampling frame. Do you deny this?

I've said it before and I'll say it again. Since David Kane keeps asking the same question thereâs no reason why he shouldnât get the same answer. Really David, if you now claim to know "a lot more about the actual sampling" can you tell us just what it is you know that you didn't know the last time you brought this up, or the time before that? That's not a rhetorical question - I haven't heard anything new about that since Burnham gave his talk at MIT (which you seemed quite happy with at the time). Did I miss some revalation?

Sorry David, are you seriously defending their n=10 number? Can you provide evidence that they missed 91% of Iraq?

Kevin: There has been a big change in our knowledge of the sampling between Tim's post of December 1, 2006 and today. Perhaps the simplest way to summarize it is for you to tell us what you think the sampling was and we can tell you why you're wrong. The short version is that we (and I honestly thinks this includes Burnham) don't really know the sampling plan.

Tim: I am agnostic on their n = 10 number. Given what we now know (but did not know then), it is plausible. But your n = 0 is almost certainly wrong. Correct? Anyway, the most productive way forward is to start a new thread on the MBS paper. Feel free to just repeat your critique. Or, if you want me to go first, I will send you something to quote. That is the way you make progress in science.

Kevin,

I just checked those links. Any other readers who do so will be confused as to your meaning. So, let's simplify! Here is the exact question I asked last March:

Were all streets included in the sample (including back alleys) or just streets which intersected main streets?

Well? You didn't choose to answer this question a year ago. Would you like to answer it now? And, if you can, please provide a reason for your belief: something from the Lancet paper or the supplementary materials or the authors' public statements.

I don't think you can. And it's not your fault! My honest belief is that even Gilbert Burnham (who I think is a good guy in a tight spot) does not know. He knows what he told his Iraqi colleagues to do. But he has all sorts of reasons for thinking that they didn't actually do that. So, he is in a bind.

By the time I was finishing J School classes, we were heavily intermingled with advertising, marketing and PR students. And the McCardles are the product our system wants to turn out.

Just as tasty and filling as the real thing. People's choice!

Of course I did answer your question, albeit in a different thread.

So, once again, will you answer my question? What is this new information you have obtained which makes it necessary for Tim to open a new thread on the MSB study? You are saying nothing now that you werenât saying in that thread and in many another before it.

You donât seem to notice that you contradict yourself. You say (1) that Burnhamâs team wonât answer questions that nobody else can answer but (2) we have new information about what they did so we need to revisit the topic. Does not compute.

Contrary to your claim (no. 39), science does not progress by running around in circles.

Kevin writes:

So, once again, will you answer my question? What is this new information you have obtained which makes it necessary for Tim to open a new thread on the MSB study? You are saying nothing now that you werenât saying in that thread and in many another before it.

A fair question! What we know now that we did not know is March 2008 is that the Lancet authors are no longer standing by the explanations that they gave before people started looking more closely at their work.

Consider this example from April 16, 2008. Summary: Burnham and Roberts made a bunch of false claims about their data in a letter to the National Journal. They published those claims on their web page. I (and others) pointed out that those claims were false. They then deleted the letter and refused to apologize, pretending as if the entire incident never happened. See the link for full details.

Why is this important? Because it means that we can no longer rely on Burnham's MIT presentation! We now know for a fact that some of things that Burnham believed (in all honesty) to be true pre-April 2008 are not, in fact, true. Even worse, these untrue things are not admitted to. They are washed down the memory hole.

And, even if you ignore this problem for a moment, it is still the case that science (or at least I!) move slowly. Although it is true that my initial impression of Burnham's MIT talk was favorable, it was only in June 2008 that I was able to create a transcript of the talk and look closely at the slides. Although this should have been clearer to me before, it became clear then that his explanation of the sampling scheme was gibberish. To quote myself:

Pages 21-24 highlight a different version of the sampling plan than is described in the paper. Burnham claims that they did not restrict the sample to streets that crossed their main streets. Instead, they made a list of "all the residential streets that either crossed it or were in that immediate area." This is just gibberish.

First, if this was what they actually did, why didn't they describe it that way in the article? Second, given the time constraints, there was no way that the teams had enough time to list all such side streets. Third, even if the interviewers did do it this way, the problem of Main Street Bias would still exist, except it would be more Center Of Town Bias. Some side streets are in the "immediate area" of just one main street (or often in the area of none) and other side streets (especially those toward the center of a town or neighborhood) are near more than one. The later are much more likely to be included in the sample.

Again, maybe these problems should have been obvious to me before, but they only became obvious in June 2008. Apologies for the delay.

Is it really your claim that the Iraqi survey teams had time to drive into a city they had never been to, create a listing (without maps!) of every main street, cross street and back alley, and then randomly select among them?

see david, you are using a typical denialist tactic of a huge list of completely random and independent lines of attack against the lancet papers. you can neither give reason nor evidence of fraud or big errors in the study.

the main street bias attack is one without any substance. you can easily check this for yourself:

just walk down a huge road (why not chose a shopping lane?) and ask every person you meet, whether they live in a road intersecting this one or not.

the yes/no ratio you get, is the "mainstreet bias" ratio for the WORST CASE scenario of polling behaviour by the Lancet team.

i am looking forward to read your numbers!

David Kane: ...the Lancet authors are no longer standing by the explanations that they gave before people started looking more closely at their work.

So Burnham no longer stands by the description he gave at MIT? You haven't produced any evidence of that at all.

Again, maybe these problems should have been obvious to me before, but they only became obvious in June 2008.

As will be apparent from the thread I previously linked to, you started kicking up about this no later than 4 March 2008. You said then that you âbecame convinced that this was a real issueâ some time previous to that. If you want to go on nitpicking Burnhamâs statements for inconsistencies it might be best to get your own story straight. Iâm not suggesting that because you give conflicting descriptions of your researches nothing you say about them can be trusted. But thatâs the kind of deduction you go in for and it does you no credit.

Anyway, at some point you made the startling discovery that the procedure described by Burnham doesnât give all households in a district an equal chance of being selected. Now if that was news to Tim Lambert it might make sense for him to open a new thread to discuss the matter. But he probably saw that difficulty the moment he first set eyes on the relevant paragraph of the Lancet paper.

I donât know where you got your information about which Iraqi cities a group of Iraqis (whom you havenât met) have been to, or what maps they carry in their cars. Iâm quite sure Gilbert Burnham never claimed that they created listings including every back alley.

Kevin writes:

So Burnham no longer stands by the description he gave at MIT? You haven't produced any evidence of that at all.

You don't read my blog closely enough! I first blogged about Burnham's Feb 2007 MIT presentation in March 2007 but didn't get around to creating and posting a transcript until June 2008. Apologies for the delay.

Now, if that were the last word from the Lancet authors as to how the survey was conducted, you might have a point. But it wasn't. Instead, they posted a Q&A about the survey in early 2008. And that contradicts Burnham's presentation. In the Q&A, they claim that:

Sampling in the 2006 study was designed to give all households in Iraq an equal chance of being included.

They might be true, but it is inconsistent with Burham's presentation.

But, if that were the Lancet author's last (and final) statement, we might still make progress. But it isn't! Instead, they took that statement down and replaced it with nothing.

It is impossible to know what the Lancet authors claim the sampling procedure to be. If you have a link, provide it.

Kevin goes on:

Anyway, at some point you made the startling discovery that the procedure described by Burnham doesnât give all households in a district an equal chance of being selected. Now if that was news to Tim Lambert it might make sense for him to open a new thread to discuss the matter. But he probably saw that difficulty the moment he first set eyes on the relevant paragraph of the Lancet paper.

If you feel like diving into the intellectual history about why it takes me so long to figure things out, I am happy to go along for the ride. In the meantime, you are exactly correct. It was/is obvious that there is no way for every household in Iraq to have an equal chance of being sampled. Then why were the Lancet authors claiming the opposite as late as spring 2008?

Best part about this debate? The "alleges" in the title of this post. Is it really a matter of factual dispute that Burnham has violated AAPOR standards? Now, you may argue that Burnham has no obligation to follow these standards, you may argue that the standards are stupid, you may argue that the whole controversy has been spawned by the evil neocon conspiracy, but there is no way to deny to Burnham has violated them. Let me walk Tim through this slowly.

Here are the AAPOR standards.

2. The exact wording of questions asked, including the text of any preceding instruction or explanation to the interviewer or respondents that might reasonably be expected to affect the response.

Burnham et al have not released the exact wording of the questions, much less the interviewer scripts. Ergo, they have violated the standards. There is nothing alleged about it.

Don't believe me? Here is a quote from Mary Losch, AAPOR standards chair. AAPOR

requested the survey instrument, (including consent information) and it was not provided. The template did not appear to be much beyond an outline and certainly was not the instrument in its entirety.

And there you have it. Tim ought to strike out the "alleges" from the post title.