Online Debate

PrintPrint CiteCite
Style: MLAAPAChicago Close


Violence by the Numbers in Iraq: Sound Data or Shaky Statistics?

Discussants: Lawrence J. Korb, Senior Fellow, Center for American Progress, and Stephen D. Biddle, Adjunct Senior Fellow for Defense Policy
Updated: October 1, 2007


As critics and supporters of the Iraq war search for clues to bolster their arguments, the military's measurement of violence has become a source of intense debate. Some military analysts say Gen. David Petraeus, the top U.S. commander in Iraq, cherry-picked numbers to support claims the Pentagon's surge strategy is working. Others insist the numbers are, at the very least, an honest assessment of the situation on the ground.

Lawrence J. Korb, senior fellow at the Center for American Progress, and Stephen Biddle, CFR's senior military analyst, discuss the Pentagon's numerical assessment of violence in Iraq.

Weigh in on this debate by emailing the editors at To view other online debates click here.

Stephen BiddleMost Recent

October 1, 2007

Stephen Biddle

Larry and I agree that Petraeus’ testimony would have been strengthened by discussing the limits of knowledge on the causes of declining violence. But we disagree on the trend: I think the evidence generally supports Petraeus’ claim that violence has declined. And I don’t think Larry can exclude the surge as a cause of this decline, or sustain his claim that any reduction is “mostly a result of the new arbitrary definition” of sectarian killings instead.

There are at least three problems here. First, Larry’s evidence to show that violence is not declining is problematic. He doesn’t address the range of sources examined in my backgrounder, but instead cites only the Defense Department’s most recent report to Congress (PDF), Measuring Stability and Security in Iraq (September 2007), which he says contradicts Petraeus. Larry doesn’t explain why he thinks this, but in fact it doesn’t. The DoD report uses a different measure than Petraeus’ charts (average daily killed and wounded, versus monthly killed for the MNF-I data). Since they’re counting different things, the figures won’t be the same. But the trends are actually quite similar even so: if one breaks out the civilian casualties from the awkward stacked bar chart in the DoD report and plots these over time, the DoD’s figures for civilian killed-and-wounded grow through 2006, then the upward trend is arrested late in the year and declines modestly thereafter. This is essentially the same trend that all sources report for 2006-7, including MNF-I. As with all the other sources with similar trends, the slope of the 2006 increase is steeper than that of the 2007 decline, so of course the average casualty figures for 2007 are greater than for 2006. But that does not mean that the DoD data show a worsening casualty picture in 2007, as Larry suggests. And it certainly does not mean that Petraeus’ findings are an unreasonable outlier contradicted by other, ostensibly more reliable sources.

Second, while much of Larry’s argument turns on claims about sectarian killings and definitions thereof, Petraeus’ testimony showed declines in total fatalities, total attacks, IEDs, Baghdad-only casualties, and “high profile” attacks, as well as just sectarian killings. Yet Larry treats only the sectarian subset and ignores everything else. What about all the rest? How is a claim about one definition supposed to challenge testimony about trends in a much wider range of phenomena?

Third, even for sectarian violence per se, one can’t prove that the surge was irrelevant by disputing a definition of “sectarian.” The MNF-I definition could be wrong and the surge could still account for all, some, or none of any real change in violence, sectarian or otherwise. To show any specific ground-truth role for the surge in any outcome would require evidence on causation that no one has yet produced. MNF-I cannot prove causation, but neither can Larry—and that means he can’t sustain his claim that the surge was unimportant any more than his opponents can prove that it was central. We just don’t know, hence neither can be excluded. I happen to think that MNF-I’s definition is a good deal less arbitrary than Larry does, but either way the definition dispute cannot bear the weight of the claims Larry bases on it.

Again, none of this proves or disproves any position on the war. But it’s important to be clear on what we know and don’t know if any position is to be defensible.

Lawrence J. KorbLawrence J. Korb

September 27, 2007

In his thoughtful response, Steve Biddle makes several good points and challenges a couple of my assertions. He and I both agree that aggregate violence data alone are an insufficient basis to sustain a policy position and that a total withdrawal from Iraq is a desirable policy.

However, Steve also argues that there is little evidence to back up my claim that “violence is not going down.” What about the Pentagon’s latest quarterly report that came out last week? It shows that civilian casualties were lower in the months preceding the surge and higher in the months after the surge, the exact opposite of what Petraeus testified.

Steve also contends that I need to give the surge some (or more) credit for the apparent decline in sectarian violence. I believe that the decline is mostly a result of the new arbitrary definition of what constitutes sectarian violence. While the MNF may no longer be characterizing deaths as sectarian based upon which side of the head the bullet entered, they employ equally arbitrary criteria. As the Washington Post notes, Iraqis killed in a car bomb on August 25 in a Shiite neighborhood in Baghdad are labeled sectarian while Iraqis killed in a car bomb a week later near Ramadi were not. Or if victims are sprayed with gunfire, as opposed to a single shot, they are not considered sectarian.

How will the Petraeus MNF-I characterize the recent killing of Abu Risha, the Sunni Arab leader of the tribal Awakening Council in Anbar presumably by al-Qaeda in Iraq? Or the attacks on September 24 and 25 by Sunni Arab militants against police chiefs, ministry officials and tribal leaders? These attacks occurred primarily in mixed areas of Shiites and Sunnis or in exclusively Sunni areas. The MNF-I team leader for sectarian analysis captured the arbitrariness of these criteria when he told Karen DeYoung of the Washington Post, “if you and I pulled from the same database, and I pulled one day and you pulled the next, we would have totally different numbers.”

If General Petraeus were an attorney defending a client, he could be forgiven for trying to “prove” that the surge has reduced violence in general and sectarian violence in particular. But, as Steve acknowledges, this causation cannot be established with confidence. The General owes it to the troops he commands and the nation he serves to make it clear that there is a great deal of doubt about his interpretation of the data and about whether the surge is achieving any real security gains, let alone its strategic or political objectives which after all is its only purpose.

Stephen BiddleStephen Biddle

September 25, 2007

Larry Korb and I may or may not disagree on Iraq policy (I’ve argued that either the Petraeus strategy or a total withdrawal from Iraq are defensible). But we clearly disagree on recent security trends and the data on these. Given space limitations I’ll highlight just three points of divergence.

Larry claims that “violence in Iraq is not going down.” I address the debate on Iraqi violence data in this Backgrounder. But the bottom line is that multiple, independent sources show essentially the same trend that Gen. Petraeus presented: total civilian casualties grew through 2006, crested late in that year, then declined in 2007. This decline may or may not continue, and either way, aggregate violence data alone are an insufficient basis to sustain a policy position on Iraq. But there is little evidence in other sources to back Larry’s view on this or to contradict MNF-I’s (Multi-National Force-Iraq).

Larry also contests the causes of any local decline, and rules out any role for the surge, substituting ethnic cleansing instead. Here, too, I treat the issue in this Backgrounder; suffice to say here that there has been a lot of sectarian cleansing in Iraq, but there has also been a 33 percent increase in US combat brigades, whose mission has shifted to population security. The data is insufficient to attribute a decline in civilian deaths to either cause – but it would be surprising if either cleansing or the surge could be wholly excluded. (Larry also repeats a widespread error holding that MNF-I characterizes deaths as sectarian based on which side of the head the bullet entered; this is simply not true, as Petraeus pointed out in his testimony).

Larry similarly holds that since MNF-I’s data show a decline in casualties prior to the completion of the surge, the surge cannot have been the cause. Yet things are not so simple. Muqtada al Sadr’s decision to order his militia to stand down shortly after the surge was announced – but long before it was fully in place – probably played an important role in reducing civilian fatalities early in 2007. Without a trustworthy account from Sadr himself we cannot know for sure why he did this. But a plausible possibility is a recalculation of his self-interest in light of the announced US troop increase, which would implicate the surge in any subsequent decline in violence. For now, though, causation cannot be established with confidence – hence one cannot rule out an important role for the surge in reducing Iraqi civilian casualties, as Dr. Korb does.

Lawrence J. KorbLawrence J. Korb

September 25, 2007

Not since Secretary of Defense Robert McNamara has any defense official tried to dazzle the Congress and the American people with a bewildering set of selected statistics to prove that the U.S. is making military progress as General David Petraeus did. Using no less than 13 charts to back up his prepared testimony, Petraeus would have us believe that, as a result of the surge, violence in Iraq in general, and Baghdad in particular, is declining, sectarian violence is dropping, car bombings are falling, and the Iraq security forces are improving. But close examination of the data shows that Petraeus’ claims are misleading and contradict the independent analysis of the National Intelligence Estimate (NIE), the GAO, and the independent commission headed by General James Jones.

Violence in Iraq is not going down. In August, according to Iraq’s Interior ministry, 2,318 civilians died compared to 1,980 in July. Petraeus claimed that only about 1,500 died in August. Similarly violence in Baghdad is down because Baghdad has been ethnically cleansed, turning from a 65 percent Sunni city to 75 percent Shia. More than half of all Baghdad’s neighborhoods are now Shia dominated as compared to only a couple a year ago.

Sectarian deaths are down only because the definition has been narrowed. Petraeus’ numbers do not include Shia-on-Shia violence or Sunni-on-Sunni violence. Moreover, because it is very difficult to gauge the sectarian intent of particular murders the military often uses odd categorizations such as not counting people being shot in the head from the front. Even according to Petraeus’ own numbers the violence dropped before February of this year – but after the surge actually went into effect there was little change. And Petraeus did not tell us that the number of car bombings in Iraq was five percent higher in July than in December 2006.

Nor are the Iraqi security forces improving. The Jones commission concluded that the Iraqi National Police are so ineffective that they should be disbanded and reorganized and that the other security forces will not be able to fulfill an independent security role within the next 12 to 18 months and that the number of Iraqi Army units capable of operating independently dropped from 10 to 6 between March and July.

If things in Iraq are going as well as Petraeus claims, why do nearly 80 percent of Iraqis say that things are going badly and 70 percent believe that the escalation has worsened rather than improved security? And why has the number of people leaving the country each month doubled since the surge began? And most importantly, why have more American servicemen and women died this year than in any previous year of this mindless, senseless, and unnecessary war?

More on This Topic