The Case of the Distorted Crime Statistics

Almost everyone wants low crime rates. Low amounts of crime can enhance residents’ quality of life, reduce need for community services, and attract businesses and tourists.

For the last few years Buckeye, Arizona has been touted as the “safest” city in the Phoenix metropolitan area because of its low crime statistics. In 2017 the Arizona Republic reported: “Buckeye and Gilbert are some of the safest communities in metro Phoenix among cities with a population larger than 25,000…. Buckeye leaders take its safe status as a win, considering the city of about 65,500 residents also is the seventh fastest-growing city in the nation.”

But it now appears that those statistics were too good to be true. According to a June 2019 story in the Arizona Republic, a private investigation firm has found evidence that Buckeye’s crime statistics have been misreported since 2014. The investigation was launched after police department employees sent an anonymous letter about crime recording practices.

According to the Arizona Republic story, crime codes were changed to report serious offenses as minor offenses. “For example, burglary reports would be labeled as theft or lost property. Assault or robbery cases would be labeled as disorderly conduct.” The statistics sent to the Federal Bureau of Investigation (FBI) for the Uniform Crime Reports (UCR) were based on the altered crime codes.

Misclassifying serious offenses as lesser ones, not surprisingly, reduces a city’s UCR crime rates. The UCR violent crime statistics include four types of crime: murder, rape, robbery, and aggravated assault (an assault involving a weapon or with the potential to cause severe injury). The UCR violent crime statistics do not include disorderly conduct or other types of assault. Thus, misclassifying an aggravated assault as a simple assault or disorderly conduct keeps the offense out of the official FBI crime statistics and gives a perception that the city has fewer aggravated assaults than it really does.

The investigators noted that the violent crime rate in Buckeye had dropped nearly 41 percent from 2013 to 2014 and wrote: “This staggering drop in crime, and the continuing downward trend the following years, at a time when Buckeye was steadily increasing in population and neighboring cities were experiencing crime increases, cast doubt on the validity of those figures.”

But would the low crime statistics and the drop in 2014 have been sufficient evidence to cast doubt on the figures? Let’s look at the data.

Were Buckeye’s Crime Statistics Unusually Low?

Figure 1. Violent crimes per 100,000 residents from UCR data, 2010 – 2017.

Figure 1 shows the UCR violent crime rates for 15 cities in the Phoenix metropolitan area between 2010 and 2017. The lines represent the number of violent crimes reported by the city divided by the city’s population, then multiplied by 100,000 to obtain the number of violent crimes per 100,000 residents. The legend lists the cities in order of decreasing population, with Phoenix and Mesa having the largest populations and Paradise Valley having the smallest. Phoenix is the brown line at the top of the graph; Buckeye is the dashed red line at the bottom.

The FBI cautions against ranking cities by their UCR crime rates, and I heartily agree. Many factors affect a city’s crime rates, including the composition of the population, citizens’ practices of reporting crimes to the police, climate, economic conditions, and cultural factors. You cannot conclude that one police department is doing a better job than another by comparing their UCR crime rates. (There’s another, mathematical, reason to be wary about comparing crime rates among cities, and I’ll discuss that later in this column.)

Figure 2. Violent crime rates for cities with population between 50,000 and 100,000, for 2015. The red rectangle indicates Buckeye’s position.

That said, let’s look at Figure 1 with a view to detecting possible “outliers” for crime rates. Although Buckeye is usually near the bottom it does not appear outlandishly unusual when compared with the other cities (assuming the other cities report accurate statistics).

Figure 2 shows a histogram of UCR crime rates for the 474 cities in the 2015 UCR data with population between 50,000 and 100,000 (Buckeye’s population in 2015 was about 62,000). Buckeye, indicated by the red rectangle, is among the five percent of cities with the lowest UCR crime rates. The violent crime rate is on the low side, but not astonishingly so.

What about the “41 percent drop” in violent crime in Buckeye from 2013 to 2014? Well, Figure 1 shows that every city has fluctuations in the violent crime rate from year to year. In fact, one would be suspicious of the record-keeping of a city that had exactly the same number of violent crimes every single year. Buckeye reported 49 violent crimes in 2013 and 29 in 2014. Even though this represents a large percentage drop, these are relatively small numbers, coming from a city with fewer than 60,000 residents in 2014. We would expect the number of violent crimes to vary from year to year, and this drop is consistent with fluctuations we would expect to see from random variation. So the 41 percent drop by itself does not give reason for suspicion.

Let’s look at one more measure. Homicide statistics are usually considered to be more reliable than police agency statistics for other types of violent crime. Homicides are more difficult to misclassify as a different crime type than are aggravated assaults, rapes, and robberies. They also are more likely to become known to the police and they receive more news coverage than other crimes.

Figure 3. Percentage of violent crimes that are homicides (diamond) along with 95 percent confidence interval (horizontal line), 2014 to 2017. The vertical line shows the national statistic of 1.3 percent.

Nationally, about 1.3 percent of all violent crimes between 2014 and 2017 were homicides. We would not expect to have exactly this percentage in every city, but an unusually high percentage of violent crimes that are homicides might raise questions about the classification of other types of violent crime. If aggravated assaults, rapes, and robberies are routinely misclassified as lesser offenses, the percentage of violent crimes that are homicides will be higher than it should be.

But small cities usually have small numbers of homicides, so one needs to combine several years of data to obtain stable statistics. Between 2014 and 2017, Buckeye reported 124 violent crimes, of which 9 (7.3 percent) were homicides. Figure 3 displays the percentage of violent crimes that were homicides for the 15 cities in the Phoenix metropolitan area, with the highest population cities at the top and the lowest population cities at the bottom. The diamond for each city indicates the percentage, and the horizontal line shows a 95 percent confidence interval* for the percentage. Buckeye had an unusually high percentage of homicides among the violent crimes.

Five Percent of the Members of Any Set are in the Lowest Five Percent

Buckeye consistently had among the lowest violent crime rates in the Phoenix metropolitan area (Figure 1), had a violent crime rate below the fifth percentile among cities in its size class (Figure 2), and had more homicides among the violent crimes than typical in the area (Figure 3). None of these statistical analyses, however, could be used by itself to prove malfeasance in the crime statistics. After all, some city in any group will have the lowest crime rate. And even if everyone is keeping scrupulous records, through the magic of mathematics there will always be five percent of the cities in the bottom five percent.

Also, although misclassifying aggravated assaults as lesser offenses leads to a too-high percentage of violent crimes that are homicides, one cannot conclude that a high percentage means that crimes have been misclassified. The high percentage can have other causes, such as an unusually high number of homicides in a particular year.

You need confirmatory evidence, not just statistical analyses by themselves, to demonstrate that a low crime rate statistic is invalid. In this case, confirmatory evidence was supplied by the private investigation firm and also by local news reporting that found dozens of records of persons arrested for violent crimes that were not included in the UCR statistics.

Why Hub Cities May Exhibit Higher Crime Rates

I want to repeat the warning about comparing crime rates among cities. I displayed the rates (Figure 1) and percentage of homicides (Figure 3) for 15 cities in the metropolitan area to see whether the statistics indicated that Buckeye might be an outlier. But one should not conclude from these graphs that some police departments are more effective than others, or even that the UCR crime rates reflect the true amount of crime.

Overall, about half of violent crimes are reported to the police, and the percentage of crimes reported to police varies from city to city. A city might exhibit a higher UCR violent crime rate because its reputation for taking complaints seriously and being responsive to crime victims encourages victims to report incidents to the police department. Judging such a police department solely by its UCR crime rate penalizes it for good community relations — the city may have low amounts of crime, but a high percentage of those crimes reported to the police.

There’s another reason to be wary about comparing crime rates among cities, and that stems from how rates are calculated. The FBI divides the number of violent crimes in the city’s UCR report by the population of the city and then multiplies this ratio by 100,000. This seems simple enough, but it can end up giving “hub” cities — centers of economic and cultural activity — higher crime rates than their neighbors because the resident population is smaller than the population at risk of being a crime victim.

Here’s a fictional example that illustrates how a hub city can be disadvantaged in crime statistic comparisons. Suppose City A has lots of workplaces and sports arenas, and every day the 100,000 residents of City A are joined by 50,000 people from other cities who work there or attend a sporting event. This means that every day, City A has 150,000 people who could, conceivably, be crime victims. City B also has 100,000 residents, but no workplaces or facilities that attract visitors from other cities. City A reports 300 violent crimes during the year, 200 of which occurred to residents and 100 of which occurred to visitors. City B also reports 200 violent crimes that occurred to residents, but no crimes against visitors because no one visits City B. City B’s rate is 100,000 x 200/100,000 = 200 violent crimes per 100,000 residents. City A’s rate is 300 violent crimes per 100,000 residents, one and a half times as high as City B’s rate.

City A’s rate is higher than City B’s because the UCR counts crimes in the location where the crime occurs; it does not distinguish between crimes against residents and crimes against visitors. But the denominator for the crime rate is the resident population, and it does not include visitors.** A city with 500 residents, which happened to be home to a large entertainment complex or festival grounds where 10 among the 75,000 annual visitors were victims of assault, would have a violent crime rate of 2,000 violent crimes per 100,000 residents.

So one should not conclude from Figure 1 that Phoenix and Tempe, the two lines at the top, are “doing less” to control crime than the other cities. These two cities, both hubs of economic and cultural activity, attract many daily visitors who are part of the population “at risk” of violent crime but are not counted in the denominator of the violent crime rate. The population served by each city’s police department is substantially larger than the number of residents.

Former Madison, Wisconsin police chief David Couper argues in How to Rate Your Local Police that easily calculated metrics such as crime rates and average time to respond to calls should not be used to measure a police department’s effectiveness. They reduce complex processes to a single number, and that can lead to pressure to reduce that number, no matter how. Crime statistics provide valuable information as part of a system for improving community safety. But only if they accurately reflect the amount of crime.

Footnotes

*The FBI does not report margins of error or confidence intervals for its crime statistics. They view these as exact numbers with no sampling variability. But one can think of the crime statistics, and the percentage of violent crimes that are homicides, as the results of processes that have fluctuations over time. Even if the crime rates are stable over time, one would expect different numbers and rates from year to year. The Centers for Disease Control and Prevention take this view and present confidence intervals along with their mortality statistics. I calculated the confidence intervals in Figure 3 assuming that (number of homicides)/(number of violent crimes) is a Binomial proportion. Because the percentage of violent crimes that are homicides is close to zero, I used Wilson confidence intervals, which are constrained to be positive, instead of the usual Stat 101 symmetric confidence intervals, which would include negative values. The cities with lower numbers of violent crimes in the denominator of the proportion (usually those with smaller populations) have wider confidence intervals.

**For this example, in fact, City B’s residents experienced a larger number of crimes (crimes reported to police) than City A’s residents — they were victims of the 200 crimes that occurred in City B plus some of them were crime victims when working in or visiting City A.

There are also other issues involved in comparing cities’ UCR crime rates. Sometimes multiple agencies have jurisdiction in an area: for example, a university or airport might have its own police department. Some agencies report data for only a few months of the year, not the full 12 months. The FBI has procedures for dealing with missing data, and for assigning crimes and population to police agencies, but these procedures are geared toward producing state and national statistics and do not resolve the inconsistencies for local statistics.