UNICEF Gives us lots of Great Data….and maybe a bit too much Expert Judgement

All readers of this blog should know this UNICEF web site.  Childmortality.org lets you quickly call up a mass of child mortality data for most countries in the world.

Sometimes you learn that the state of our knowledge is a mess (for example, check out Angola) but UNICEF always tries to impose some order by drawing dark blue lines through the middle of the data points.

I just wish that UNICEF was more transparent and rule-oriented about how they create these lines.  They seem to be products of both sophisticated modelling and bargaining among experts.

There is, of course, a certain appeal to the notion that UNICEF people don’t just blindly adopt the curves that their computers spit out at them.  Rather, the masters bring their voluminous knowledge and uncanny intuitions to the table.

However, there is a danger that the practice of tweaking one’s estimates with expert judgements can open the door to all sorts of prejudices and biases.  (This joint interview with Daniel Kahneman and Gary Klein is relevant to this question.)

Take a look at UNICEF’s Democratic Republic of Congo graph.

DRC

To me, the flat blue region seems strangely unmoored from the data.  So I took a crude pass at the underlying data between 1999 and 2011.  (When there were multiple observations in a single year I just averaged them and I rounded month-year observations to the nearest year.)

Here are the data together with a straight-line fit to the points:

DRC_mine

UNICEF’s flat region seems nowhere to be found.

OK, this wasn’t a randomly chosen example.  A little bird tells me that the flat part of the DRC graph will disappear for the next update of childmortality.org.

But why did UNICEF fudge the trend in the first place?  I think I may know the answer.  (Scepticism alert: I will now speculate about motives.  This is always  hazardous for those of us who can’t climb into other peoples’ minds.)

There is an NGO called the International Rescue Committee that stridently insisted for years that there was a spike in child mortality rates in the DRC that started right around the year 2000.  This claimed spike was said to be so massive as to make the DRC conflict the deadliest war since World War II.  Back in 2010 the Human Security Report showed that the IRC was wrong about this.  Could it be that some of UNICEF’s experts have used their influence to mitigate the embarrassment of their friends at the IRC for being so spectacularly wrong about the DRC?

Of course, now that you know about childmortality.org you can see for yourself (without turning to the Human Security Report) that the DRC did not suffer a huge and sustained spike in child mortality in the 2000’s.  Still, the Human Security Report deserve kudos for identifying this issue before the pile up of data that proved its point.

To be clear, child death rates are still very high in the DRC.  Moreover, many people have been violently killed, raped and forced to flee during the course of war.  It has been an awful, nasty war.   Unfortunately, other wars have been awful, nasty and bigger.

Advertisements

5 thoughts on “UNICEF Gives us lots of Great Data….and maybe a bit too much Expert Judgement

  1. Hello. Thank you for this question.

    I think this deserves a whole blog post which might take me a couple of days. So let me give you a quick list here (but without numbers) to hold you while you wait.

    Here are wars that are definitely bigger than the DRC one.

    The Korean War 1950-1953

    The Vietnam Wars 1946-1975 (Some would classify 1955-1975 as two separate wars 1955 -1964 and 1965 – 1975. If so, then both are bigger than DRC. In addition, there is the French war with Vietnam 1946-1954. This also, is probably bigger than the DRC one.

    Cambodia – 1967 -1998 (The 1970’s alone is bigger than DRC)

    The Chinese Civil War (Just the part of it between 1946 and 1949 since we’re only looking at the post-WWII period)

    The Iran-Iraq War 1980 – 1988

    Afghanistan (You could say that this one goes from 1978 to the present but just the Soviet period alone was a bigger war than the DRC one)

    Rwandan Genocide (Some would argue that the genocide was not part of a war so it would be improper to include this on a list of wars. However, if it’s included then Rwanda is bigger than DRC.)

    These are clear cut cases that spring to mind for me.

    There are almost surely others although the numbers are not solid enough to say definitively. Greece and Algeria would be likely candidates.

    Among ongoing wars Iraq and Syria are both possible.

    I’m sure I could find more candidates.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s