Building Rigorous Evidence to Drive Policy

One of the principles motivating the President’s Budget is that, as a nation, we haven’t been making the right investments to build a new foundation for economic prosperity — and we need smarter investments in education, health care, and social services.

But, in making new investments, the emphasis has to be on "smarter."  Many programs were founded on good intentions and supported by compelling anecdotes, but don’t deliver results.  This is one reason the President directed OMB to review the federal budget.  In the first year of this effort, we identified more than 100 terminations, reductions, or other areas of savings that take nearly $17 billion off the federal government’s bottom line next year alone.

Although this is a good start, our review faces a major challenge: much of the time, it’s hard to say whether a program is working well or not.  Many initiatives drive funds to the local level, but don’t track how they are spent; others spend dollars bit by bit, so the results are hard to see.

Rigorous ways to evaluate whether programs are working exist.  But too often such evaluations don’t happen.  They are typically an afterthought when programs are designed — and once programs have been in place for awhile, evaluating them rigorously becomes difficult from a political economy perspective.

This has to change, and I am trying to put much more emphasis on evidence-based policy decisions here at OMB.  Wherever possible, we should design new initiatives to build rigorous data about what works and then act on evidence that emerges — expanding the approaches that work best, fine-tuning the ones that get mixed results, and shutting down those that are failing.

At one level, we need to invest more in program evaluation — and the President’s Budget does.  For example, we call for an expansion of the Institute for Education Sciences and growth in the evaluation budgets at the Department of Labor and the Corporation for National and Community Service.  We’re working with YouthBuild, a program the President supports, to implement its first-ever rigorous national evaluation.

Beyond investing more in evaluations, we are also designing new initiatives with evaluation standards built into their DNA.  One example is the Department of Education’s "Grow What Works and Innovation Fund." And at the Department of Health and Human Services, the Administration has proposed two new efforts that meet very different needs but have in common the rigorous use of performance data.

One initiative supports home visitation.  Home visitation programs send a trained professional to provide help and support to vulnerable parents — teaching everything from child nutrition to reading at home to anger management.  For the best-researched of these initiatives, rigorous evaluations have shown reductions in the incidence of low birthweight, child abuse and neglect, and even run-ins with the criminal justice system a decade or more later.

The other initiative is about preventing teen pregnancy.  Teen pregnancy is linked to many troubling outcomes for both teenagers and their children.  After a long period of decline, the rate of teen pregnancy rose again in the last two years for which we have data.  Here, too, proven models exist for working with teenagers to bring teen pregnancy down.

For these two very different subjects, we’re using a similar, two-tiered approach.  First, we’re providing more money to programs that generate results backed up by strong evidence.  That’s the top tier.  Then, for an additional group of programs, with some supportive evidence but not as much, we’ve said: Let’s try those too, but rigorously evaluate them and see whether they work.  Over time, we hope that some of those programs will move into the top tier — but, if not, we’ll redirect their funds to other, more promising efforts.

This design differs from the typical approach.  We haven’t simply created a block grant and told states they can do whatever they want, nor have we dictated a particular program design and told everyone to follow it.  Instead, we’ve said that we’re flexible about the details of the program; we only insist that most of the money go toward the programs backed by the best available evidence, and the rest to programs that are promising and willing to test their mettle.

This two-tiered structure will provide objective criteria to inform our decisions on which home visitation and teen pregnancy models to invest in.  It will also create the right incentives for the future.  Organizations will know that to be considered for funding, they must provide credible evaluation results that show promise, and be ready to subject their models to analysis.  As more models move into the top tier, it will create pressure on all the top-tier models to improve their effectiveness, so they continue to receive support.

By instilling a culture of learning into federal programs, we can build knowledge so that spending decisions are based not only on good intentions, but also on strong evidence that carefully targeted investments will produce results.