Important New Violent Death Estimates for the War in Peru with Implications Beyond just Peru: Part 2

This is a follow up to last week’s post about new estimates,  published by Silvio Rendon, of human losses in the war in Peru, 1980 to 2000.

Important context is that the Truth and Reconciliation Commission (TRC) of Peru published estimates that were quite surprising in at least two respects:

  1.  The TRC’s estimate for the total number of people killed in the war, 69,000 with a 95% uncertainty interval of 61,000 to 78,000, was far higher than the roughly 25,000 documented deaths spread across the three lists that the TRC worked with.
  2. The TRC estimated that the number of people killed by the left-wing Shining Path (SP) guerrillas was much higher than the number of people killed by the Peruvian State, reversing the perpetrator pattern for the roughly 25,000 documented deaths that formed the basis for the TRC’s estimates.

My previous post addressed only point 2, the TRC’s transfer of primary responsibility from the State to the SP, which the Rendon estimates transfer back to the State.

Let’s now consider point 1.

Rendon’s main estimate for total deaths is 48,000 with a 95% uncertainty interval of 43,000 to 53,000.  This is substantially below the TRC’s estimate but still well above the 25,000 documented deaths on the TRC’s lists.

The TRC’s estimate of 69,000 (61,000 – 78,000) total deaths in the war comes from an MSE analysis (also known as capture-recapture) of the three lists held by the TRC.  Crucially, the non-State  component of this estimate, covering SP killings plus killings attributed to “other” groups, derive from an unusual indirect method that is meant to sidestep the problem of excessively sparse data in many strata (See my earlier post for details).

Rendon’s estimate breaks down into three main steps:

  1. Use multiple imputation to (randomly) assign perpetrators to deaths attributed to unknown perpetrators in the TRC data.
  2. Make direct capture-recapture estimates for all of the TRC’s geographical strata that admit direct estimation after step 1, i.e., all strata for the State, 11 strata for the SP (covering about 1/2 of documented deaths) and 9 strata on average for identified other groups (covering about a 1/3 of documented deaths).
  3. Use kriging, an interpolation method that incorporates spatial  correlation between sampled observations,  to extend the direct estimates for killings attributed to non-State groups to cover the entire country.

In addition, Rendon offers a separate estimate for which he divides the country into just 10 strata, in contrast to the TRC’s 58 strata. This coarser partition enables direct estimation for each perpetrator in each stratum.  Multiple imputation is again used to allocate to known perpetrators deaths attributed by the TRC to unknown perpetrators.  This estimate also comes out to about 46,000 with a standard error of around 3,500.

Rendon provides even a third estimate, using multi-level modelling, that turns out to be similar to the first two.  This one is, however, rather complicated and I will not try to describe it in the present blog post.

In short, all of Rendon’s estimates point toward numbers that are substantially lower than what the TRC estimated but substantially higher than the number of documented deaths.

Finally, consider the estimates of Rendon and the TRC for killings attributed to the State.  Both are direct capture-recapture estimates but Rendon alone uses multiple imputation to account for unknown perpetrators, pushing his  estimates nearly 40% above the TRC’s.  Rendon estimates roughly 28,000 State-caused deaths (standard error = 2,185) compared to roughly 20,500 (standard error = 1,718) for the TRC.  This difference suggest that the State may have evaded responsibility for quite a few deaths in the TRC’s accounting  because of the way the TRC lumped deaths caused by unknown perpetrators in with deaths caused by “other” perpetrators.

In my next post I will examine the response to the Rendon paper coming from two authors of the TRC estimates.

Advertisements

Important New Violent Death Estimates for the War in Peru with Implications Beyond just Peru: Part 1

Silvio Rendon just published an important new paper that challenges statistical work done for the Truth and Reconciliation Commission (TRC) of Peru on violent deaths in the Peruvian conflict, 1980 to 2000.

I can only scratch the surface in one post.  So I plan to focus on a few central points and elaborate later.  Two authors of the original TRC estimates have already posted a rebuttal but, for now, I’ll just consider the original TRC work and Rendon’s paper.

The TRC based its estimates on three (after some consolidation) lists of people documented as killed in the war.  According to the TRC, these lists officially contain roughly 25,000 unique individuals, many of whom appear on two or even all three lists.

The TRC’s estimation method, known as capture-recapture or multiple systems estimation, is based on analyzing list overlaps in an attempt to discover the number of deaths that did not make it onto any list.  Intuitively, heavy overlap across lists suggests that a high percentage of all deaths already made it onto at least one list.  On the other hand, light overlap across lists suggests that only a low percentage of all deaths got listed.  For more background on this method please take my [as of now] free online course  or read this introductory article that I wrote with Nicholas Jewell and Britta Jewell.

The TRC made two especially striking statistical claims:

  1. The true number of people killed in the war was far higher than the roughly 25,000 deaths that were documented across the three TRC lists.  The TRC estimated 69,000 deaths with a 95% uncertainty range of 61,000 to 78,000.
  2. The left-wing Shining Path (SP) guerrillas killed more people (46%)  than the Peruvian State did (30%), contrary to the perpetrator pattern for the roughly 25,000 documented deaths according to which the State killed more people (47%) than the Shining Path did (37%).

Advocates for applying capture-recapture methods lean heavily on the State-SP reversal described in point 2 above.  For example, Megan Price and Patrick Ball write:

… in our work for the truth commission in Perú in 2003, we found that
killings attributed to the government had a much higher probability of documentation than killings attributed to the Shining Path insurgency, yet questions of accountability hinged precisely on determining which group perpetrated the majority of the violence [3]. A naïve analysis of the observed data, without accounting for selection bias, would have incorrectly held the state responsible for a larger proportion of the violence.

The Rendon paper challenges this claim, rather convincingly in my view.  In fact, Rendon estimates that the percentages attributed to the Shining Path (SP) and the State are roughly 31% and 43% respectively.  These numbers are approximately in proportion to their respective percentages among documented deaths.  So it seems that the TRC actually introduced a substantial bias rather than correcting one.

How did this happen?

The TRC divided Peru into 58 geographical strata and made death estimates for the State and the SP in each one.  But all this subdivision rendered the data too sparse in many places to allow for standard capture-recapture estimates of SP-caused deaths to be performed.

The ideal method to estimate the total number of victims for each
perpetrator would be to stratify the data simultaneously by geography
and perpetrator and then choose the model with the best fit for each
perpetrator in each geographic stratum. This method is not possible
because of the sparseness of the data for reported deaths attributed to
the PCP-Shining Path and other perpetrators, as mentioned above. (From the TRC Report)

So, the TRC followed an indirect procedure rather than their ideal direct one:

  1. Estimate the number of deaths caused by either the State or the SP.
  2. Estimate the number of deaths caused by just the State.
  3. Estimate SP-caused deaths as 1 – 2, i.e., deaths caused by either the State or the Shining Path minus deaths caused by just the State

Step three may seem correct simply as a matter of arithmetic (State + SP – State = SP).  But this is not true when the components in this subtraction problem are uncertain estimates.  Theoretically, this indirect method can even lead to negative estimates for unlisted deaths, i.e., estimates for SP deaths that are less than the numbers of documented deaths.

Rendon points out that there are actually nine strata for which you can do standard direct estimates for the SP.  He performs this estimation and finds considerably fewer SP-perpetrated deaths than the TRC’s indirect estimates do.  Moreover, Rendon performs simulations based on these nine strata which show that overestimation by the indirect method, compared to the direct one, is a general phenomenon for data that look similar to the data in these nine strata.

Next, Rendon provides good evidence that the overestimation phenomenon he has identified  applies to the entire country, not just to the nine strata out of the TRC’s 58 that allow for direct estimation of SP-caused deaths.  He accomplishes this by merging strata until there are just ten of them.  Each of these strata is appropriate for direct estimation while still allowing for  considerable geographical heterogeneity in violence patterns.  Again, Rendon finds big  overestimation by the indirect method compared to the direct one and that the State is the main perpetrator.

Rendon also addresses a further problem with the TRC work that I haven’t yet mentioned: incomplete data fields. One manifestation of incompleteness is that nearly 3,000 documented deaths cannot be placed in one of the TRC’s 58 strata.  So these incompletely georeferenced deaths are dropped from the TRC’s estimation.  It turns out, however, that more than 2/3 of these are attributed to the State.  It is likely that this disproportionate exclusion of State-caused deaths artificially tilted the TRC’s estimates away from State and towards SP responsibility.

There are also approximately 3,000 further deaths  listed as caused by unknown perpetrators.  Rendon performs a multiple imputation analysis that randomly assigns these deaths, based on known characteristics, to the State, SP and to other, smaller, groups.  One fruit of this work is that there are now 11 strata that allow for direct estimation of SP-caused deaths.  This may not sound like like a lot of strata but these 11 actually account for roughly half of all the documented SP deaths.  Again, Rendon’s results hold up – the State is the main perpetrator and the indirect method substantially overestimates SP-caused deaths.

Here’s a summary.

  1. The direct method gives much lower estimates for the SP than the indirect one does for the nine strata that allow direct estimates and the direct method assigns primary responsibility to the State.
  2. When known characteristics are used to allocate deaths by unknown perpetrators to known perpetrators then the direct method again gives much lower SP estimates than the indirect one does and assigns primary responsibility to the State.  Now the SP-estimation covers about half of the documented deaths.
  3. When strata are amalgamated sufficiently so that it is possible to do direct estimates for the whole country then, again, the direct method gives much lower SP estimates than the indirect method does and assigns primary responsibility to the State.

These results make me think that the much hyped reversal of responsibility claimed by the TRC, i.e., that the Shining Path rather than the State was the main perpetrator in the war in Peru, is wrong.

OK, that’s it for this post.  More to come.

 

A New Entry into the Decline of War Debate: Yes, there has been a decline

I’ve just posted a new paper with Stijn van Weezel on the decline-of-war debate.  This is part of an ongoing discussion  which involves such people as Steven Pinker, Nassim Nicholas Taleb and Aaron Clauset.

Here is our abstract:

For the past 70 years, there has been a downward trend in the size of wars, but the idea of an enduring ‘long peace’ remains controversial. Some recent contributions suggest that observed war patterns,including the long peace, could have come from a long-standing and unchanging war-generating process, an idea rooted in Lewis F Richardson’s pioneering work on war. Aaron Clauset has tested the hypothesis that the war sizes after the Second World War are generated by the same mechanism that generated war sizes before the Second World War and fails to reject the ‘no-change’ hypothesis. In this paper, we transform the war-size data into units of battle deaths per 100,000 of world population rather than absolute battle deaths – units appropriate for investigating the probability that a random person will die in a war. This change tilts the evidence towards rejecting the no-change hypothesis. We also show that sliding the candidate break point slightly forward in time, to 1950 rather than 1945, leads us further down the path toward formal rejection of the no-change hypothesis. Next, we expand the range of wars to include not just the inter-state wars considered by Clauset (2018) but also intra-state wars. Now we do formally reject the no-change hypothesis. Finally, we show that our resultsdo not depend on the choice between two widely used war datasets.

 

Here is one of the key figures from the paper.

gleditsch_all

The X axis is war sizes expressed as the number of battle deaths divided by world population at the time they occurred.  The Y axis for the figure on the left gives p values for tests of the hypothesis that war sizes after 1945 are generated in the same way that wars sizes before 1945 were generated.  The Y axis on the right gives the same information but using 1950, rather than 1945, as the break point.

We can read off the picture on the right, for example, that wars killing more than 45 people per 100,000 of world population after 1950 have been much less common than such wars were before 1950.  Indeed, the estimated probability that the pre-1950 war generation mechanism continued to operate after 1950 for wars of sizes above 45 per 100,000 is only around 0.05.  So we can even formally reject a hypothesis that nothing changed after 1950 for wars of sizes 45 per 100,00 and above.

Please have a look at these slides for a more leisurely development of what I just said in the last two paragraphs.  The slides explain all the key ideas in the paper.  The only limitation of the slides is that they do not separate out the relative importance of each step in the argument.

There are two broad take-home points from this work:

  1.  The evidence is fairly strong for a decline in war thesis
  2.  We should not focus exclusively on 1945 as the only potential break point.

On point 2 I know of two separate research teams who have papers specifically on the change-point issue.  I believe that neither of these papers are yet in the public domain but I expect that they will be soon.

This topic is heating up!

 

The Hurricane Maria Death Toll Numbers: An Extension of my Piece in The Conversation

I just published a piece on the Hurricane Maria death toll numbers.  

Space is always limited with such pieces so I’ll extend it a little bit here.

Excess deaths estimates tend to have really wide uncertainty intervals and the Harvard study on the Hurricane Maria death toll is no exception.  The reason for such extreme uncertainty is obvious, once you think about it, but very few people seem to have thought about it.

Suppose we estimate that country X suffered 10,000 deaths last year and we put a 95% uncertainty interval of 9,000 to 11,000 around that estimate.  That’s a reasonably tight estimate: plus or minus 10%.

Suppose now that in a normal year country X suffers 8,000 deaths.  However, in the year we do our estimate there was a war, hurricane, tidal wave or something else that seems to elevate the death rate.  The purpose of our estimate is to quantify this elevation.

A standard excess deaths estimate is 2,000.  We obtain this simply by subtracting 8,000, the normal or baseline death rate, from 10,000, the central estimate of the rate in the special year.

If we treat the normal (baseline) rate as a certainty then it is also easy to  place a 95% uncertainty interval around our excess death estimate.  This uncertainty interval runs from 1,000 (9,000 – 8,000) to 3,000 (11,000 – 8,000).  But this is an interval of plus or minus 50%.around our central estimate of 2,000.

The point is that when you subtract off a baseline then you magnify the size of the swing when you measure this swing in percentage terms around your excess death estimate.  

This means that what might be a pretty large sample size for determining the total number of deaths is actually a pretty small sample size for determining excess deaths.

Seen through this lens it’s clear that the Harvard study has a tiny sample size.  So it is no surprise that they published a preposterously wide uncertainty interval of about plus or minus 87%.

The moral of the story is that excess death surveys need very large sample sizes compared surveys aimed just at measuring total deaths.

 

A Zombie Graph Lives on Courtesy of the Lancet

I’ve got a new article just out in The Conversation.  Here’s the short version:

  1. The Lancet publishes a false graph
  2. The problems of the graph are exposed, several even in letters to the Lancet.
  3. The Lancet just leaves the graph up.
  4. A Washington Post reporter stumbles onto the false graph, thinks it’s cool and reprints it.
  5. I tell the reporter that  he just published a false graph
  6. The reporter does a mea culpa and pulls the graph down
  7. I write up this sequence of events for The Conversation
  8. The Conversation sends it to the Lancet
  9. The Lancet declines to comment and leaves the false graph up
  10. The Conversation publishes the piece
  11. Someone else sees and believes in the graph?

An Interesting Job in Casualty Recording!

Check out this job opportunity.

 

Closing date: 12 April 2018

Programme Development Officer

Posted by Every Casualty Worldwide

Save job

Salary £28,000 –  £32,000
Location London, Greater London
Part-time

This is an opportunity to make a unique contribution to changing the way that governments acknowledge the human cost of armed conflicts.

Job Description

We are looking for a person with a track record of effective advocacy in the area of human rights, humanitarian aid, international development or peacebuilding who can use their talents and energies to develop this work and our organisation.  This is a 4-days per week, 12-month role, with a possibility of extension. The post is home-based and the post-holder will be expected to operate primarily in London.

About Every Casualty Worldwide

Every Casualty Worldwide (ECW) works towards a world in which every casualty of armed violence, both civilian and combatant, is properly recognised. This is to give a human face to those who have paid conflict’s highest price, and also to enable a wide range of humanitarian, human-rights and peacebuilding goals.

Our project began in 2007 under the auspices of the London based think tank Oxford Research Group. In 2014 ECW became a standalone not-for-profit company, and in May 2016 was granted registered charity status (charity number 1166974). Our vision is succinctly expressed in the Charter for the Recognition of Every Casualty of Armed Violence launched at the British Academy in 2011 and endorsed by a wide range of supporting organisation.

Applicants should note that while it works closely with practitioners, ECW is not itself a casualty recording organisation but seeks to improve the state of casualty recording across the globe, including by responsible states and other conflict parties, to fulfil the core demands of the Charter that every casualty of armed conflict is ‘promptly recorded, correctly identified and publicly acknowledged’.

ECW has been working to fulfil its charitable objectives through research, capacity building, and advocacy. Our research has focused on better understanding and codifying existing and emerging casualty recording practice around the world and the various humanitarian, human rights, and peacebuilding benefits that it brings. Capacity building has revolved around establishing and maintaining the first international Casualty Recorders Network. These activities have underpinned our advocacy with states, who bear prime responsibility for ensuring compliance with existing norms.

In recent years our efforts have concentrated on the development and eventual publication in late 2016, of an international set of Standards for Casualty Recording, in consultation with and endorsed by casualty recording practitioner NGOs and key international agencies who use casualty data, such as the ICRC, the ICC, and UNOCHA. These have provided new openings for advocacy with governments around the world, as these standards allow the conversation to move from the politics around casualty numbers to the need to do casualty recording in any conflict according to a recognised set of principles and methods.

In the UK context, these developments have been further assisted by the publication in July 2016 of the Chilcot Report (“Iraq Inquiry”). Chapter 17 of the report, entitled “Civilian Casualties” draws heavily on the work of Iraq Body Count, one of ECW’s founding network members, and concludes by recommending that in future conflicts,

Government has a responsibility to make every reasonable effort to identify and understand the likely and actual effects of its military actions on civilians

and that

Government should be ready to work with others, in particular NGOs and academic institutions, to develop such assessments and estimates over time

Maximising the opportunity presented by these recommendations is a key element of ECW’s current priorities.

The immediate challenge (2018-19)

Following the publication of the Chilcot Report, the UK Government committed itself to conducting a ‘lessons learned’ investigation across Whitehall, coordinated by the National Security Adviser, into the substantive criticisms of the government made by the report. To assist this process, the MOD set up a Chilcot implementation unit tasked to find practical ways to meet the recommendations of the report.

ECW has been closely involved in the casualty recording element of this process, and is well placed to make significant contributions towards ensuring that the eventual full government response lives up to the spirit and letter of the Chilcot specific recommendations on casualty recording. In order to fully capitalise on this and other emerging opportunities we need to augment our capacity to respond rapidly and effectively to advance the issue of casualty recording both within government and in public arenas.

The UK is only one relevant actor of many, though well-placed to take an influential lead on this issue if it can be persuaded to do so. On the wider international front we also need to maintain and garner further momentum among key players who have already indicated their support for casualty recording in various ways.

The role of Programme Development Officer

This is a newly created role within the organization and the post-holder is expected to play a key part in expanding the impact and effectiveness of ECW, working alongside the co-directors. The role is focused on three organisational priorities. In order of immediate importance, they are:

  1. Ensuring an effective and appropriate UK government response to the Chilcot Report in the area of Casualty Recording;
  2. Maintaining growing international engagement with Casualty Recording on the relevant fora beyond the UK (e.g. various UN fora);
  3. Enhancing the institutional stability and sustainability of ECW.

The post-holder will report to the co-directors, and initially working closely with them while gaining the experience and subject knowledge to be able to work more independently.

Specific objectives to which the post-holder will contribute are:

  • Working with government officials, NGO and other partners to promote and assist development of policy and implementation in casualty recording;
  • Conducting targeted advocacy to stimulate government action, including identifying effective pressure points and opportunities for contributing to other relevant initiatives;
  • Elevating the importance and relevance of casualty recording in public discourse;
  • Ensuring the Casualty Recorders Network is kept informed of ECW activities and able to contribute effectively.

Specific activities to support work in these areas may include:

  • Engaging with the UK environment on a day to day basis, scanning the scene for advocacy opportunities, reaching out to relevant individuals and institutions;
  • Attending and representing ECW at relevant meetings and networking;
  • Making profile- and awareness-raising interventions ranging from short items (e.g. tweets) through to more substantive commentary (e.g op-eds, interviews with experts, etc.);
  • Monitoring national and international initiatives which the casualty recording agenda can support;
  • Keeping abreast of developments in armed conflict and other news relevant to casualty recording;
  • Assisting in the drafting of expressions of interest and full grant bids where appropriate;
  • Contributing web content to ECW website;
  • Contributing to the strategic planning processes of the organisation.

Person Specification

  • At least 3 years policy and advocacy experience in a human rights, humanitarian, peacebuilding, international development or other relevant contexts (whether government, international agency, or NGO), in a role involving networking and outreach;
  • High level of literacy in English;
  • Proven ability to communicate effectively with a wide range of audiences, including the ability to produce compelling text and presentations;
  • Competency with standard office and communications software;
  • Ability to manage their own time and workload, using their own initiative;
  • Experience in maintaining well-organised records of activity;
  • A good understanding of the role of new and traditional media in advocacy;
  • Commitment to ideals expressed in the Charter for the Recognition of Every Casualty;
  • A good understanding of the UK government and parliamentary system, and its connections to the international system;
  • A working understanding of the international system, and the relevant fora for civil society-led or -initiated change;
  • Existing right to work in the UK.

Further particulars

This is a home-working position and the candidate must have a private space in which to work, with a reliable internet connection. This is a 4-days (30 hours) per week position, starting July 2018 or as soon as possible thereafter, but no later than August 2018. The post runs for 12 calendar months.

We would expect normal working hours, i.e. Monday to Friday, between the hours of 9 and 6, with core working hours of between 10am and 5pm, though some flexibility around working hours could be discussed. Most routine team communications as well as collaborative work will be undertaken remotely using Skype. Allowance towards phone and internet costs will be made available. A suitable laptop will be supplied if needed. The position will require easy access to central London for face-to-face ECW team or stakeholder meetings in Westminster. All travel other than for ECW team meetings will be reimbursed.

Salary will be pro-rata to the full-time range £35,000 – £40,000 per annum (i.e. actual gross salary of £28,000 – £32,000) according to qualifications and experience.  The postholder will be required to make a minimum 2% contributions to a statutory pension scheme (NEST), to which ECW will also contribute pro-rata. Although this is a 12-month position, there is a real prospect of the position’s being extended on a more long-term basis (circumstances permitting).  The current position is made possible through a generous grant from the Joseph Rowntree Charitable Trust.

How to apply

Closing date for applications 12 April 2018.

Interviews will be held in London in early May.

Informal enquiries are welcomed prior to submission of a full application, and potential applicants may send a preliminary email to admin[AT]everycasualty[DOT]org

Full applications should consist of a single file (word or pdf) containing a description of no more than 1000 words demonstrating how the applicant meets the person specification and why they are applying for this position, along with a brief CV (including the names and contact details of two referees).

The client requests no contact from agencies or media sales.