Well, after yesterday's revelations, blogspace seems to be split. On the one hand, we have Clayton Cramer, Steve Verdon, Jane Galt, Glenn Reynolds and Marie Gryphon who think Lott has been exonerated. On the other hand Kevin Drum and Tom Spencer are not yet convinced. On the gripping hand we have Jim Henley and John Quiggin who think that it has been established that Lott conducted a survey, but the small sample size means that Lott could not properly use it make his "98% brandishing" claim. Julian Sanchez's friend also makes the point about the sample size being too small.
I appreciate the concerns of those who think that one person saying that he had been surveyed is not sufficient to settle the matter. After all, anybody in the USA could have come forward and say that they had been surveyed. However, Lindgren questioned him carefully about what questions he was asked and they corresponded pretty well to what Lott said they were (with one major discrepancy which I will comment on in the next paragraph). The only way this could have happened is if he had conspired with Lott, and given the blundering way Lott has defended himself, I just don't buy it.
The major discrepancy is that since Lott was only counting defensive gun uses that occurred in the year before the survey and this gentleman's one was four years before the survey, it should not have been counted. Lott himself argues that the explanation for this is that his student interviewer screwed up! Well, if a sample size of 0.5 is good enough for Lott to conclude that 75% of firings are warning shots, a sample size of one should be more than sufficient for us to conclude, using the Lott method, that 100% of Lott's interviews were not conducted correctly.
Anyway, I now believe that Lott did conduct some sort of survey. The next question is, did the 98% figure come from that survey? The evidence suggests that it did not. Lott put forward the 98% figure well before the survey was complete. He repeatedly attributed the figure to other surveys or Kleck, only changing his story after Duncan pointed out that the other surveys either said nothing on the subject or gave radically different results. (And note that Lott actually still denies he has changed his story about the attribution of the figure.) And even with his small sample size, it is still extremely unlikely that a survey could have come up with a brandishing figure as high as Lott's. (Though this last one could be explained by arithmetic errors on Lott's part.)
Furthermore, even if the 98% did come from his survey, Lott is still in trouble. Proper scientific conduct requires that if you make a claim, you had better have the evidence to back it up. By his own admission, Lott does not. He should have withdrawn his 98% figure long long ago instead of repeating it over 50 times.
And, even if by some miracle Lott found all the missing data from his survey and was able to recreate all his calculations, he would still be in trouble. As others have noted, the sample size of his survey (25 DGUs it would seem) is just too small to give a statistically reliable estimate. Other surveys, such as Kleck's have a much larger sample of defensive gun uses, but Lott never ever mentions the brandishing number from Kleck (unless you want to count the times when he falsely attributes the 98% brandishing number to Kleck). Lott's excuse here (from his 14 Jan response is
"As to so-called technical problems, I am have always acknowledged that these are small samples, especially when one breaks down the composition of those who use guns defensively. Even the largest of the surveys have few observations in this category."
Meanwhile Kleck and Gertz write about their survey:
While estimates of DGU frequency are reliable because they are based on a very large sample of 4,977 cases, results pertaining to the details of DGU incidents are based on 213 or fewer sample cases, and readers should treat these results with appropriate caution.
No similar caution appears in any of the over 50 times Lott advanced his 98% figure based on a mere 25 DGU cases.