Journal of Quantitative Criminology, Vol. 19, No. 2, June 2003 (ß 2003 )
Measurement and Other Errors in County-Level UCR
Data: A Reply to Lott and Whitley
Michael D. Maltz1,2 and Joseph Targonski1
Lott and Whitley note that our analyses of the errors in the county-level UCR data
used in More Guns, Less Crime (J. R. Lott, University of Chicago Press, Chicago,
1998, 2000) ignore the fact that all data have measurement error, that the largest
errors were in counties with low populations, and that population-weighted
regressions were used. We agree that this mitigates some of the effects of the errors,
but does not take them fully into account. We also note that this is but one of the
problems associated with the analysis. We therefore find no reason to alter our
original conclusion, ‘‘that in their current condition, county-level UCR crime
statistics cannot be used for evaluating the effects of changes in policy.’’
1. INTRODUCTION
In their critique of our paper, which described the effect of missing data
on the analyses in More Guns, Less Crime (Lott, 1998, 2000) that use
county-level UCR data, Lott and Whitley (2003) make a number of state-
ments. They include:
. we ignored the effect of population-weighted regression;
. all data sets have measurement error, and survey data have much more
missing data than are found in Lott’s (1998, 2000) analyses;
. we used the wrong states in our analyses;
. state-level analyses are similarly affected by missing data;
. state- and city-level analyses show the same pattern as the county-level
analysis.
1
Department of Criminal Justice, University of Illinois at Chicago, 1007 W. Harrison Street,
Chicago, Illinois 60607-7140.
2
To whom correspondence should be addressed. E-mail: mikem@uic.edu
199
0748-4518/03/0600-0199/0 ß 2003 Plenum Publishing Corporation
, 200 Maltz and Targonski
Our original paper dealt primarily with the problems inherent in the
UCR, not with the specifics of the analyses in More Guns, Less Crime
(MGLC). This is the primary reason that we did not ‘‘directly test if it affects
the results’’ (Lott and Whitley, 2003).
In this paper, however, we respond to the Lott and Whitley with respect
to each of the above points. In Section 2 we discuss the effect of weighting
the observations by population. In Section 3 we examine the effect of
errors, including missing data. Section 4 discusses the choice of states we
used. In Section 5 we discuss state-level analyses. Section 6 contains our
conclusion.
2. POPULATION-WEIGHTED REGRESSION
Lott and Whitley are, of course, correct in their statement that a
regression weighted by population would diminish the effect of the small
counties that (generally) have less consistent reporting histories. As we
stated in our paper, Lott (2000, pp. 143, 155) criticized Black and Nagin’s
(1998) analysis for ignoring the smaller counties and focusing their analysis
on counties with 100,000 or more population, so we included all counties in
our paper. Their reanalysis of our data shows that when weighted by
population the problem is reduced. For example, we note from their Fig. 3
that, although there are 21 states that have over 10% of their observations
with extensive missing data (defined by 30% or more coverage gaps), only
15 states have over 10% of their population affected by this magnitude of
coverage gaps. This is still a substantial amount of error.
In both editions of MGLC the data were taken at face value, assumed
to be error-free. Now that the nature of the errors is more thoroughly
understood, Lott and Whitley suggest that they do not affect the results
substantially. They reanalyzed the MGLC data removing the 16 most error-
laden states; however they do not state how (or whether) they dealt with the
errors that remained in the data of the remaining 44 states. If they did not
adjust for the errors in those states, this step is inadequate.
3. ERRORS IN THE COUNTY-LEVEL DATA SET
The errors are not insignificant. Moreover, there are a number of dif-
ferent types of error that affect analyses, not just the ones to which Lott and
Whitley refer. They are correct in their assertion that ‘‘[v]irtually all data
have measurement error,’’ in referring to the problems with UCR missing
data that we documented (Maltz and Targonski, 2002). Actually, this is only
one type of error that affects their findings; instrumentation error and error
The benefits of buying summaries with Stuvia:
Guaranteed quality through customer reviews
Stuvia customers have reviewed more than 700,000 summaries. This how you know that you are buying the best documents.
Quick and easy check-out
You can quickly pay through credit card or Stuvia-credit for the summaries. There is no membership needed.
Focus on what matters
Your fellow students write the study notes themselves, which is why the documents are always reliable and up-to-date. This ensures you quickly get to the core!
Frequently asked questions
What do I get when I buy this document?
You get a PDF, available immediately after your purchase. The purchased document is accessible anytime, anywhere and indefinitely through your profile.
Satisfaction guarantee: how does it work?
Our satisfaction guarantee ensures that you always find a study document that suits you well. You fill out a form, and our customer service team takes care of the rest.
Who am I buying these notes from?
Stuvia is a marketplace, so you are not buying this document from us, but from seller nimrit. Stuvia facilitates payment to the seller.
Will I be stuck with a subscription?
No, you only buy these notes for $8.49. You're not tied to anything after your purchase.