On Tuesday I provided some eye-popping comparisons on one Iraq survey fielded by D3/KA against another Iraq survey fielded by another company at exactly the same time. In light of this evidence any reasonable person has to agree that the D3/KA data are fabricated. Nevertheless, today I give you a different window into the same D3/KA survey.
Recall that one of the main markers of fabrication in these surveys is that the respondents to what I’m calling the “focal supervisors” have too many “empty categories”. A response category is “empty” for a group of supervisors if it is offered as a possible choice but zero respondents actually chose it. For example, in Part I to this series we saw that for all public services zero respondents for the focal supervisors said that the service was “unavailable” or that availability was “very good”. These are, therefore, both empty categories for the focal supervisors.
Langer Research Associates tried to rationalize all the empties for the focal supervisors by arguing that other supervisors also have empties. Langer Associates also argued that Steve Koczela and I were unfair to compare the group of focal supervisors with the group of all the other supervisors. This is because the number of empties should be decreasing in the total number of interviews and the all-others group did more interviews than the focal group did. Langer does have a point on this which I addressed in this post. Here I follow up with a couple of pictures based on the same D3/KA survey discussed on Tuesday.
Each picture takes a bunch of different combinations of supervisors and for each combination plots the number of empties against the number of interviews. The first plot graphs the data on 100 combinations of three supervisors plus the focals. The second plot graphs the data on 100 combinations of four supervisors plus the focals.
You can see that:
1, The number of empties is, indeed, decreasing in the number of interviews.
2. Even after adjusting for this fact the focal supervisors still have overwhelmingly more empties than they should have, given the number of interviews they have conducted.