Common Monkeyflower

Edit 2, 12:30pm 12/5/15: lots of traffic to this post–welcome! You should also check out the comment thread on MarkMaynard.com’s discussion of this post; his readers offer a lot of good thoughts on what good reporting of these data *could* look like, and questions that could be asked.  And, to be clear, my point in this post is not to state that things are hunky-dory in Michigan: comparing the Census’ 1-year data for 2014 (from ACS) to 1999 (from the y2k Census), we see that statewide poverty rates climbed from 10.5% to 16.2% in those 15 years. That 547,000 more Michigan residents were living in poverty last year than at the turn of the century is clearly cause for concern. My frustration comes because bad data reporting actually obscures and misleads us from the real picture, preventing us from actually digging in and figuring out what’s going on and how to address it. All of the following commentary doesn’t get into that work–it only serves to get us out of the data dead end and back to square 1, so that we have the chance to set off down the right path.

Yesterday was one of my favorite holidays of the year: New Census Data Day.  Just as Black Friday follows Thanksgiving, though, Bad Data Headlines Day inevitably follows as one of my least favorite days.

Michigan’s poverty rate soars as income drops even in economic rebound, census shows! screams the MLive headline, with lead paragraphs reading,

Median income in three out of every four Michigan cities and villages declined in the past five years, according to new data from the U.S. Census Bureau. At the same time, the share of people living in poverty rose in two-thirds of the state’s communities.

Statewide, more than one out of every six people are living in poverty, a 17 percent increase from the previous 5-year period. The median household income in Michigan from 2010-2014 was $49,087 per year – up a few hundred bucks from the 2005-2009 period, but when adjusted for inflation it’s down 8.7 percent during that time.

Of that, only the last sentence on household income is a clear and accurate explanation of the data; the rest ranges from misleading to flat wrong.

17% more poverty?!

First, the math issue that had people contacting me in some alarm.  From the wording of the article, they were led to believe that a net 17% of Michigan residents had newly fallen into poverty between 2009 and 2014–that if the poverty rate was, say, 10% in 2009, it must be 27% in 2014, because that’s a 17% increase.  (Of course, the statement “one out of every six” is actually only 16.7%, meaning this read of the 17% increase can’t possibly be right, unless the poverty rate was below zero in 2009–which suggests that “one out of every six” was not a useful way to state the current rate.)

In fact, the statewide poverty rate for the period 2010-2014 was only 16.9%, while the statewide poverty rate for the period 2005-2009 was 14.5%.  Since 16.9 is 16.55% larger than 14.5, it’s reasonably accurate to round it up to 17%–but confusing.  It’s like giving a weather report that, “It’s a chilly 35 degrees Fahrenheit out there right now, but the temperature is expected to rise by 7 degrees Celsius by lunchtime.”

Five-year periods

Any place the numbers “2014” and “2009” are used in these reports, they’re wrong. The new Census data is the 5-Year American Community Survey data for the period 2010-2014. These numbers are the result of surveys given over a 5-year period and rolled up (with some weightings) to create a statistically representative sample.

The Census Bureau’s pre-release webinar for media users explicitly states that the data “describe the average characteristics over a specific period of time, not a single point in time”.  (The webinar also states the 5-year data should be used when “No 1-year estimate is available”.  More on that in a minute.)

So what the data actually show is that, during the period 2010-2014, Michigan’s poverty rate was 16.9%, 2.4 percentage points higher than Michigan’s poverty rate of 14.5% during the period 2005-2009.

Those are big blocks of time.  In 2005, the housing boom was in turbo-mode and would be for sometime longer–and that’s averaged in with the 2009 depths of the recession.  Similarly, the “new” data covers a period from 2010–still on the economic rocks–to 2014.  This is not the right data to use to describe changes “in the past five years”, because that entire 5 year period is lumped together as a single data point.

So…do we know anything at all about poverty trends?

Sure.  Actually, much more useful data for this was released back in September, when the 1-year ACS numbers came out. While the 5-year ACS data covers areas down to the “block group” scale (around 1,000 people, as a rule of thumb), the 1-year data only covers areas of at least 60,000 people, in order to have statistically meaningful sample sizes. Fortunately, Michigan’s population is over 60,000 people, despite what decades of national press might lead one to believe.

Looking at the 1-year ACS numbers, Michigan’s poverty rate over time looks like this:

2005 - 13.2%
2006 - 13.5%
2007 - 14%
2008 - 14.4%
2009 - 16.2%
2010 - 16.8%
2011 - 17.5%
2012 - 17.4%
2013 - 17%
2014 - 16.2%

From these one-year numbers, you can see that Michigan’s poverty rate in 2014 was the same as in 2009, at 16.2%, and that the poverty rate peaked in 2011, dropping every year since.  These numbers also clearly show why the five-year bundle of 2005-2009 is a bad data point to use for “the recession”, and 2010-2014 a bad data point for “the recovery”.

So are these numbers good for anything?

If you want to use the 5-year ACS poverty numbers that newspapers are publishing for local communities, think of them as an over/under comparison–how has my community fared over the last 10 years compared to the state?

For example, the state’s inflation-adjusted median household income dropped 8.7% from the 2005-2009 period to the 2010-2014 period.  By contrast, the City of Ypsi’s inflation-adjusted median household income dropped 13.6% between those periods, while the City of Ann Arbor’s increased by 1%–another data point in our local tale of two cities.

edit, 12/5/15: Also, I didn’t get in to margin of error here–the Census Bureau does include that information in the ACS, and the DetNews’ database of poverty rates, the their credit, does include a column stating whether the change over time is significant or not.