Showing posts with label statistics. Show all posts
Showing posts with label statistics. Show all posts

Saturday, March 12, 2011

Has Google found the algorithm for good management?

Not yet. Google has spent the last two years analyzing performance reviews, feedback surveys and nominations for top-manager awards in order to identify the most significant phrases of praise or complaints.
According to the New York Times, "Google executives say they aren’t crunching all this data to develop some algorithm of successful management."
It did however, identify 8 core principles of good management. The results are not shocking: managers who take time to communicate, who respect their employees, who do not micromanage all do best.
What is new here is the idea that a company can use data to garner what makes an effective manager. Will these Google characteristics become the norm because they are "verifiable" through data? Will the company continue to segment its results by department type or employee characteristics? While it's interesting to read the results of studies like this and no one can deny the importance of a good manager in a work environment I can't help but think that bringing too much data in to this is overkill.

Wednesday, March 3, 2010

A New Way to Measure Poverty

The official federal poverty measure, designed in the 1960s, is the most commonly used indicator of economic hardship in America. Unfortunately, it provides no guidance on pressing questions like the role of regional price variations on poverty or the impact of Food Stamps.

Researchers have been aware of these deficiencies for some time. Now the Census Bureau will begin reporting supplemental poverty measures based on recommendations from the National Academy of Sciences. These new measures will take into account the impact of taxation, in-kind public assistance and medical out-of-pocket costs, and will use more realistic poverty thresholds.

While this is a new step at the federal level, the City of New York has been working on alternative poverty measures since 2008. The efforts of New York City's Center for Economic Opportunity are described here in today's New York Times. A copy of CEO's new report is available on CEO's website for download. It describes the development of the measure as well as poverty trends in NYC between 2005 and 2008.

Thursday, August 6, 2009

The way we count stuff matters

NYU Law professor Anthony Thompson points out an interesting quirk in the way the census works: prison inmates are counted as living in the same county as their prision, even if they will be released within a year or two. Why does this matter? It has a direct impact on how federal and state congressional districts are drawn and how government money is channeled:
"...the district of State Senator Elizabeth O'C. Little, a Republican in upstate New York, has 13 prisons, adding approximately 13,500 incarcerated "residents." Without the inmate population, Ms. Little would face an uncertain future. Her district would probably have to be redrawn because it wouldn't have enough residents to justify a Senate seat.

The residence rule raises two fundamental issues:

First, inmates in nearly all states aren't allowed to vote, yet their presence affects electoral representation in places where they do not live permanently.

Second, a disproportionate number of state prison inmates are from urban areas. Most state prisons, however, are in rural areas. As a result, resources and electoral authority are transferred from inner cities to rural jurisdictions.

The effects are plain to see. Cities lose out on funds that could be used both for crime prevention and prisoner rehabilitation; rural areas do their best to thwart reform because they don't want to lose the benefits that prisons confer on them."
As I've written before, most American live in what the Census classifies as urban areas. So the residence rule will skew public funds in an inefficient and non-representative way.

Conducting a census presents myriad conceptual issues, and no strategy is perfect. But there's something fundmentally wrong with counting incarcerated citizens who are legally barred from voting towards a district's population total. This will ultimately give voters in those districts disproportionate voting power and will result in policy choices that do not reflect the will of the majority.

Monday, June 15, 2009

Poverty and income inequality over time

Justin Wolfers posted this graph at Freakonomics, showing income growth by quintile since the 1970s:


His analysis was that economic growth has done little for the poor, accruing mostly (almost entirely) to the upper incomes.

Russ Roberts, however, takes issue with this analysis. In two separate posts (here and here), he criticizes Wolfers' interpretation:
"This alas, is a meaningless chart. It tells you nothing about who got the gains of the last 35 years. Why? Because they're not the same people in the quintiles. Starting in 1973, and it's not a coincidence, the divorce rate in the United States began to rise. The number of families increased dramatically simply because of divorce. There was also an increase in the number of families headed by single women with children. The quintile breaks-points changed, not because the economy was growing or shrinking but simply because of changes in the types of families."
In a sense, Roberts is not wrong. Looking at static graphs of quintiles does not account for differences in composition. If people move up the income ladder, then we will mistake static income growth among the groups for static income growth among individuals. The key here is the degree of social mobility. Basically, what are the odds that a person who is poor today will be poor next year, or in ten years?

Emmanuel Saez, the most recent winner of the John Bates Clark Medal, looks at this question in depth. Using Social Security data, Seaz tracked individuals over time to assess income growth and social mobility in the US over time. He concludes:
"We found that changes in short-term mobility have not substantially affected the evolution of inequality, so that annual snapshots of the distribution provide a good approximation of the evolution of the longer term measures of inequality. In particular, we find that increases in annual earnings inequality are driven almost entirely by increases in permanent earnings inequality with much more modest changes in the variability of transitory earnings. However, our key finding is that while the overall measures of mobility are fairly stable, they hide heterogeneity by gender groups. Inequality and mobility among male workers has worsened along almost any dimension since the 1950s: our series display sharp increases in annual earnings inequality, slight reductions in short-term mobility, large increases in long-term inequality with slight reduction or stability of long-term mobility.

Against those developments stand the very large earning gains achieved by women since the 1950s, due to increases in labor force attachment as well as increases in earnings conditional on working. Those gains have been so great that they have substantially reduced long-term inequality in recent decades among all workers, and actually almost exactly compensate for the increase in inequality for males."
Certainly, some longitudinal studies have found larger amounts of social mobility, but the bulk of the literature has not found the degree of mobility that Roberts implies.

So while the graph posted by Wolfers is static, there is evidence that it is a good approximation of reality. Income growth has accrued largely to wealthier individuals and families over the past 30 years, and it has not been sufficiently counterbalanced by social mobility.

Curiously, Roberts says that liberals lack a causal mechanism for changes in inequality and stagnant incomes among the poor. However, Wolfers was explicitly citing one of the most powerful arguments for this phenomenon, namely "skill-biased technological change". As Harvard's Larry Katz and Claudia Goldin explain, technological change has raised the productivity of skilled workers relative to less skilled workers, causing changes in relative incomes. In manufacturing, for example, advanced machinery has decreased demand for assembly line workers, but increased demand for engineers and mechnical operators. At the same time, the supply of educated workers has not kept up, resulting in large income gains in the upper half of the income distribution and much lower gains at the bottom.

Roberts is correct, however, in asserting that there is a difference between income inequality and absolute well-being. A rising tide can lift all boats, even while it lifts some faster than others. But it's important not to minimize slow income growth among lower income Americans; they may be better off than they were 30 years ago, but they are still struggling.

In a follow-up post, I'll delve more into the measurement, economics and politics of inequality.

Saturday, May 16, 2009

Lies, damn lies, refuted by statistics

According to the early reviews, Selena Robert's new book paints a rather unflattering picture of Yankees' third-baseman Alex Rodriguez. Aside from accusing A-Rod of having taken steroids since high school, Roberts alleges that Rodriguez and other opposing players tipped pitches during blow-outs; knowing what pitches were coming, they could improve their statistics.

Unfortunately, Roberts has little more than anonymous sources to support her story. So baseball statisticians had to delve into past seasons in order to find evidence of A-Rod's malfeasance. The stats cast doubt on Robert's allegations:
"If a tipping conspiracy were in place, one would expect that Rodriguez and rival middle infielders in games he played to have hit better in low-leverage situations than in high-leverage ones. Using a fairly loose definition of high leverage as a L.I. above 1.5 and low leverage as below 0.7, the data provide a resounding answer: either no tipping was going on or it was pathetically ineffective.
Contrary to his reputation as a choker, Rodriguez was actually at his best when the game was on the line as a Ranger. According to data compiled by Sean Forman of Baseball-Reference.com, his combined on-base and slugging percentages (O.P.S.) from 2001 to 2003 was 1.076 in high-leverage situations, compared with 1.017 for medium leverage and .982 in low leverage. Opposing second basemen and shortstops showed the same pattern. They registered an .899 O.P.S. when leverage was high, .825 when it was middling, and .817 when it was low. Unless Rodriguez’s behavior was even more nefarious — tipping only when it mattered most — the numbers give no reason to believe he was involved.

Using a more stringent definition of blowouts yields the same result. In plate appearances in which the teams were separated by seven runs or more, Rodriguez mustered just a .851 O.P.S., compared with 1.021 when the margin was six or fewer runs. His middle infield counterparts compiled a .744 mark in the laughers and .840 the rest of the time."
Of course, we should not mistake absence of evidence for evidence of absence. It is possible that A-Rod tipped pitches on rare occasions; rare enough that one or two extra home-runs wouldn't show up in this kind of analysis. But this does provide strong evidence against the possibility that this kind of behavior was widespread. That conclusion may not restore our collective innocence, but it's nice to know.

Tuesday, April 28, 2009

Putting things in perspective (or trying, at least)

I came across this article from CNN on the current swine flu outbreak. Trying to contextualize the possibility of an influenza pandemic, the author wrote:
"In 1968, a 'Hong Kong' flu pandemic killed about 1 million people worldwide. And in 1918, a 'Spanish' flu pandemic killed as many as 100 million people. Putting those figures into perspective about 36,000 people die from flu-related symptoms each year in the United States, according to the Centers for Disease Control and Prevention."
The first two numbers are world-wide totals, while the third figure is US-specific. Since the US has roughly 5% of the world's population, the yearly world death toll from the seasonal flu would be about 750,000. That would put it close to the 1968 Hong Kong flu and far below the Spanish flu. If you read the paragraph quickly, however you come away with the impression that the 1968 pandemic was significantly worse than the average flu season, while the 1918 pandemic was exponentially worse; in reality, only the latter is true.
Clearly, not all pandemics are equal, ranging from catastrophic to roughly equivalent to a seasonal flu. Either way, if you're going to provide context, it's good to put everything on the same scale.

Friday, April 10, 2009

Making gay marriage count

With Iowa becoming the fourth state in the US to legalize gay marriage, it would seem reasonable to assume that the number of married couples (both heterosexual and homosexual) would increase. Unfortunately, when it does, the federal government won't (officially) take notice. The reason for this, is that the Census Bureau--the government's statistical agency--can't legally count these couples as married:
The federal law banning gay marriage bars the agency from counting same-sex marriages, the Census Bureau says, even though they are legal in Massachusetts, Connecticut, and, most recently, Vermont and Iowa.

Last summer, census officials announced that legally married same-sex couples would be reported as unmarried, same-sex partners.
The 1996 Defense of Marriage Act allows states to refuse recognition of gay marriages from other states, and bars the federal government from recognizing gay couples as "married". This means that when legally married gay couples are surveyed by the Census Bureau, their answers to questions pertaining to marital status will be recoded by Census Bureau statisticians.

It's not all that uncommon for research to be subject to political pressures. But the purpose of the Census Bureau is to be the nation's "leading source of quality data about the nation's people and economy". It's hard to fulfill this role when you're legally bound to deny reality.

And it's not as if this issue is going away anytime soon. According to polling and statistics expert Nate Silver, a majority of states could oppose gay marriage bans by 2012. Momentum is on the side of greater freedom and equality and if the Census Bureau wants to accurately measure marriage rates in the US, they will have to count gay married couples sooner or later.

Fortunately, New York Mayor Mike Bloomberg and City Council Speaker Christine Quinn are advocating for equality and for sanity:
"Although we understand that federal law may not recognize same-sex marriages for the purposes of administration of federal benefits programs," they wrote, "we do not believe it prevents the Census Bureau from reporting statistics from the forms of self-identifying same-sex couples married under state law, like all married couples."

Sunday, March 8, 2009

What does the income distribution really look like?

This great animation shows what it looks like to earn $100,000,000 - as opposed to, say $45,000 per year, which is the median US income:



It's from the election season and it's biased toward the Democrats. But when we talk about increasing taxes on the richest 1%, it's hard to wrap you head around some of the income figures being thrown around. This is what they would look like in 3 dimensions.

Thursday, February 5, 2009

Thursday, January 29, 2009

Getting mugged by a girl named Apple

There must be something about being famous that compels parents to give their children stupid names. These crazy celebrity offspring appellations range from place names (like David and Victoria Beckham's child "Brooklyn") to fake-profession titles (Jason Lee's child "Pilot Inspektor") to the obviously drug-induced (Bob Geldof's child "Fifi Trixibell").

Are stupid baby names a problem? According a new study from two economists at Shippensberg University in Pennsylvania, these oddly-named children are more likely to commit crimes than those of us with boring names, like John or Dan. The authors find that:
"The distribution of first names in the state’s population is different from the names of juvenile delinquents. Our results show that unpopular names are positively correlated with juvenile delinquency for both blacks and whites."
The authors note that it is unlikely that having an unpopular name causes juvenile delinquency, but rather that unpopular names are "correlated with factors that increase the tendency toward juvenile delinquency, such as a disadvantaged home environment and residence in a county with low socioeconomic status." This seems strange to me. Why would they embark on a study to show that something correlated with factors that increase crime also are linked with more crime? It seems a little roundabout.

Further, Steven Levitt critiques the questionable methods used in this paper:

"The authors first compute criminality for each name by taking the ratio of the number of juvenile delinquents with that name and dividing it by the number of children total with that name. The higher that ratio, the more criminal the name. But then the authors take the log of that ratio. The problem is that the log of zero is equal to negative infinity, so any name for which that ratio is equal to zero gets dropped from the analysis.

The kinds of names that will have a ratio of zero are uncommon names for which no one with that name is a juvenile delinquent.

If I understand correctly what they are doing, if exactly one person has a particular name, the only way that the observation for that name will be included in their sample is if that person is a juvenile delinquent! This leads to a powerful bias toward mistakenly concluding that people with uncommon names are more likely to be criminals."
Quantitative research is fraught with pitfalls that can bias your results. It seems weird to me that they would have missed something like that, but then again, this study makes little sense to begin with.

Since the success of Malcom Gladwell's "The Tipping Point" and Levitt's "Freakonomics", there has been a tendency for researchers to look for novel and completely unexpected relationships in data. But if you look hard enough (and use certain methods) you can find almost anything. It's really important to ground this sort of research in some sort of theory that one is trying to prove.

The good news is that we don't have to fear the wave of kids with unusual names eminating out of Hollywood.

Tuesday, November 4, 2008

I hope the TV was wearing protection

Journalists have a tendency to get over-excited when reporting the results of scientific research. So I wasn't surprised yesterday when I saw the following headline:

Just when we thought we could keep our kids safe at home!

According to CNN:
Researchers at the nonprofit organization found that adolescents with a high level of exposure to television shows with sexual content are twice as likely to get pregnant or impregnate someone as those who saw fewer programs of this kind over a period of three years.
So is it time to turn off Gossip Girl? Perhaps not. Studies like this are good for showing correlations, or the fact that a relationship between two variables exists. They are less apt at proving what causes what. Consider this critique from the Tara Parker-Pope:
"...a closer look at the data shows the relationship between television, sexual content and teen pregnancies is complex. The same study, published today in Pediatrics, also found that teens who watch a lot of television in general are less likely to become pregnant. How can that be? The answer may be that kids who watch a lot of television obviously aren’t out dating and socializing with friends. So as unhealthy as it may be to spend hours in front of a screen, the behavior appears to be oddly protective against teen pregnancy...The link between television and teen pregnancy only shows up when a high proportion of the television shows watched by a teen are filled with sexual content."
So TV has two potential effects on teen sexual behavior. But it is also possible that there is some unobserved factor out there, which is actually at work. It is quite likely that teens who are more sexually active have a preference for more sexually explicit shows. This might make sense given the fact that while TV overall has gotten steamier, the overall teen pregnancy rate has gone down.

Either way, the authors conclude with a point that's hard to argue with:
“If the type of sexual portrayals that teens see on TV are the only messages they’re getting about sex, then they’re likely to approach sexual relationships in a way that might not be the healthiest way,” said Steven Martino, study co-author and a behavioral scientist at the RAND Corporation, the nonprofit health care research firm that conducted the study. “It’s important to talk to them about that and see how they’re reacting, and offer other perspectives to them about sex that they might not be getting on television.”

Friday, June 27, 2008

Lies, damn lies and statistics: death penalty edition

My blog partner recently posted a terrific piece on this week's Supreme Court ruling regarding the death penalty and child rape. I thought this was a good chance to talk about how difficult it can be to parse the data on such complex social issues. Warning: this gets a little technical, so please fight the urge to fall asleep.

In discussing the evidence on the deterrent effect of the death penalty on crime, she cites an Amnesty International fact sheet, which provides two graphs showing murder rates in states without the death penalty to be lower than those with the death penalty:





So we should conclude that not only does the death penalty not have a deterrent effect, but that it actually increases murders, right? Well, not really. These graphs show a clear correlation between the death penalty and murder rates, but they provide no information about which causes which. This type of analysis suffers from problems of "endogeneity", where you have unclear or two-way causality. So while the death penalty could be causing differences in murder rates, it is also possible that causality runs the other way; that is, states with already higher murder rates felt the need to implement the death penalty in order to reduce crime. As a result, we need more sophisticated statistical techniques to determine exactly what is happening with the data.

To overcome the endogeneity problem, empirical papers have used an "instrument variables" approach. An instrument variable is an independent variable that is correlated with the independent variable of interest in the model (in this case the use of the death penalty), but is otherwise uncorrelated with the dependent variable (in this case the murder rate). This will control for the endogenous effect of independent variable. Here's a technical explanation of how this works.

Unfortunately, as Justin Wolfers and John Donohue point out, finding a good instrument variable is not so easy. For example, one well known empirical paper on this topic used "percent of Republican voters in the previous election". There's good reason to think that this would be correlated with the explanatory variable (i.e. the death penalty), but there's also good reason to think that this would be otherwise correlated with the murder rate, violating one of the rules for instrument variables. Republicans tend to take tougher stances on crime, favoring harsher sentencing laws and a stronger police presence. As a result, states with more Republican voters may have lower murder rates, but it might not have to do with their use of the death penalty.

Why does this need to be so complicated? Well the only real way to identify the effect of the death penalty is to run a controlled experiment where, for example, you randomly assigned some states to have the death penalty and some not to. That way, the only difference between the groups of states is the death penalty. Of course, we can't do this. So we have to resort to observational studies, despite their difficulty in identifying the direction of causality.

So what does the research say? The evidence is mixed. There's a very interesting discussion between Justin Wolfers, John Donohue and Paul Rubin in The Economists' Voice, an excellent collection of essays on a wide range of public policy issues. The authors debate the results of several papers on the issue of deterrence, all of which contain conflicting results. Wolfers and Donohue, looking at the longest dataset of all the papers involved, find no net lives saved per execution. While Rubin and others disagree with this finding, I think that the preponderance of evidence in this area suggests that the death penalty has no consistent or significant deterrent effect. Even Gary Becker, Nobel Laureate and proponent of the death penalty deterrent theory, notes that, "the evidence is decidedly mixed...the weight of the positive evidence should not be overstated."

When faced with morally charged issues like the death penalty, many people look to empirics to settle the debate. While I feel that that impulse is the right one, there's a danger in overstating or oversimplifying the results of empirical research in the social sciences. Social systems are enormously complex, and sifting through reams of data requires a great deal of time, patience and humility. Statistics has a lot to teach us about politically charged issues like the death penalty; we just shouldn't be in a rush to judge what it is they have to say.