Andrew Gelman Attempts to Preach to the Econometricians
Andrew Gelman:
Everyone’s trading bias for variance at some point, it’s just done at different places in the analyses: Unbiased estimation used to be a big deal…. The basic idea is that you don’t want to be biased; there might be more efficient estimators out there but it’s generally more kosher to play it safe and stay unbiased. But in practice one can only use the unbiased estimates after pooling data…. That’s why you’ll see economists (and sometimes political scientists, who really should know better!) doing time-series cross-sectional analysis pooling 50 years of data and controlling for spatial variation using “state fixed effects.” That’s not such a great model, but it’s unbiased—conditional on you being interested in estimating some time-averaged parameter. Or you could estimate separately using data from each decade but then the unbiased estimates would be too noisy. To say it again: the way people get to unbiasedness is by pooling lots of data. Everyone’s trading bias for variance at some point, it’s just done at different places in the analyses.
It seems (to me) to be lamentably common for classically-trained researchers to first do a bunch of pooling without talking about it, then getting all rigorous about unbiasedness. I’d rather fit a multilevel model…. The choice to pool is no joke; it can have serious consequences. If we can avoid pooling, instead using multilevel modeling and other regularization techniques….
One of the (many) things I like about Rob Tibshirani’s lasso method (L_1 regularization) is that it has been presented in a way that gives non-Bayesians (and, oddly enough, some Bayesians as well) permission to regularize, to abandon least squares and unbiasedness. I’m hoping that lasso and similar ideas will eventually make their way into econometrics--and that once they recognize that the tough guys in statistics have abandoned unbiasedness, the econometricians will follow, and maybe be more routinely open to ideas such as multilevel modeling that allow more flexible estimates of parameters that vary over time and across groups in the population.
The application to Greenlaw et al., "Crunch Time", is left as an exercise for the reader…
