Further Comments on School Improvement Grants

Andy Smarick says I told you so.

To be clear, I wasn’t the only one raising the alarm. For example, Charlie Barone on the left and Rick Hess on the right were skeptical of the program. But the Duncan-Obama team pushed ahead on SIG with fervor. It’s never been all that clear whether they weren’t aware of the history and research or chose to ignore it. (For an administration that claimed to always follow data, always do what’s right for kids, and only pursue what works, the irony is striking.) But they designed and then implemented a program that did the same things that had disappointed when they were called “comprehensive school reform,” “restructuring,” “reconstitution,” or something else—give more money to the very districts running the persistently failing schools and ask those districts to implement a list of mandated interventions. For some reason this administration was certain this approach would work this time around despite the mountain of evidence telling them otherwise.

On the other hand, Neerav Kingsland cautions on the methods used in the study of SIG’s (in-)effectiveness.

In using a regression-discontinuity design (comparing schools that received the SIG treatment to slightly higher performing schools that did not receive the treatment), the authors were not able to generate a sample size that would be sensitive to positive significant effects that, in my mind, could be considered a success.

The federal government should have either randomized which SIG-eligible schools received funding for the SIG treatment, or the authors should have used a quasi-experimental student-based methodology that allowed for a larger sample

Null Hypothesis Watch

Emma Brown writes in the Washington Post,

One of the Obama administration’s signature efforts in education, which pumped billions of federal dollars into overhauling the nation’s worst schools, failed to produce meaningful results, according to a federal analysis.

Test scores, graduation rates and college enrollment were no different in schools that received money through the School Improvement Grants program — the largest federal investment ever targeted to failing schools — than in schools that did not.

But I imagine that from a teachers’ union perspective, the program has to be considered a success.

Noah Smith on Higher Education Policy

He writes,

As long as the number of available college spots remains roughly fixed, reducing the price of college will have only a very modest effect in creating broad-based economic opportunity.

My recommended solution is to focus on increasing the number of college spots available. Those could be four-year university slots, or vocational education — a mix of both would probably be best. But the key is that supply should go up.

Pointer from Mark Thoma.

Of course, from an economic point of view, Smith’s point is spot on. However, the Kling Theory of Public Choice is that public policy will always choose to subsidize demand and restrict supply. That is what is most in the interest of incumbent suppliers, who are the drivers of public policy.

Some Holiday Cheer

From Peter Gray,

Another reason for the increased ease of Self-Directed Education lies in technology. Today, anyone with a computer and Internet connection can access essentially all the world’s information. Self-directed learners who want to pursue almost any subject can find articles, videos, discussion groups, and even online courses devoted to it. They can gain information and share thoughts with experts and novices alike, throughout the world, who have interests akin to theirs. Students in standard schools must study just what the school dictates, in just the ways that the school decides; but self-directed learners can find subjects and means of study that match their own particular interests and styles of learning. Self-directed learners are not held back by the slow pace of a school course, nor are they rushed ahead when they want more time to think about and delve deeply into any given aspect of the interest they’re pursuing.

The future belongs to the auto-didacts.

Doubting the Null Hypothesis

From a chapter of the 2017 Economic Report of the President.

Zimmerman (2014) compares students whose GPAs are either just above or just below the threshold for admission to Florida International University, a four-year school with the lowest admissions standards in the Florida State University System. This study finds that “marginal students” who are admitted to the school experience sizable earnings gains over those who just miss the cutoff and are thus unlikely to attend any four-year college, translating into meaningful returns net of costs and especially high returns for low-income students. Using a similar methodology, Ost, Pan, and Webber (2016) study the benefit of completing college among low-performing students whose GPAs are close to the cutoff for dismissal at 13 public universities in Ohio. They find substantial earnings benefits for those who just pass the cutoff and complete their degree.

My thoughts:

1. Usually, the economic report of the President comes out in early February. Since the Obama Administration will still be in office in January, I would have expected to see the report come out then. I guess having it come out now makes it easier for the Council members to clip back into academia the second semester, if that is what they want.

2. The second result may not really support the authors’ policy view. If the only difference between Alice and Bob is that Alice just barely got the sheepskin, then Alice did not learn more than Bob. That suggests that Alice’s earnings advantage was due solely to signaling.

3. Remember that regardless of economic theory, government policy tends toward subsidizing demand and restricting supply. Subsidizing demand and restricting supply raises price, not quantity. If you really want to produce a large increase in college attendance and completion, you have to do something to increase supply. The only supply-side policies discusses in the chapter are regulations intended to identify and punish low-quality institutions.

Tyler Cowen on School Choice

He writes,

if you’re reading a critique of vouchers and the critic isn’t willing to tell you up front that parents typically like this form of school choice, I suspect the critic isn’t really trying to inform you.

Perhaps the voucher movement ought to be called the “Make schools accountable to parents” movement. I was opposed to the No Child Left Behind law from the very beginning, on the grounds that it defined “accountability” for schools as accountability to Washington.

Parents will not be perfectly informed consumers of public schools. But bureaucrats in Washington will be much less well informed.

I am not claiming that vouchers will lead to better long-term outcomes. As you know, I believe in the Null Hypothesis. But I like the idea of having teachers and school administrators accountable to people who can observe their performance close up.

College Inefficiency

Steve Pearlstein writes,

“The American university is a grand political accommodation,” says Richard Vedder, an Ohio University economist and founder of the Center for College Productivity and Affordability. College presidents, he argues, appease faculty members by giving them control over what and how they teach. They appease students and parents with high grades and good facilities. They appease alumni with expensive sports teams. They appease politicians with shiny new research centers. “The idea is to buy off any group that might upset the political equilibrium,” Vedder said.

Pearlstein goes on to suggest four steps that colleges could take (but won’t) to cut costs.

Higher education in America is a classic case in which public policy seeks to subsidize demand while restricting supply. Just as in the case of housing, the subsidies largely serve to drive up prices. And as Alex Tabarrok points out,

Prices aren’t rising because costs are rising, however, costs are rising because prices are rising.

Null Hypothesis Watch

1. Pro:
Timothy Taylor writes,

Why do the academic effects of early childhood education so often fade out? Is it lack of lack of follow-up in schools? The importance of peer effects as student who received pre-K assistance are blended in later grades with those who do not? Maybe the pre-K programs themselves vary in some way?

Read the whole post. Not all of the evidence is consistent with the null hypothesis, but it is very difficult to reject.

2. Con: David Leonhardt writes,

“The gains to children in Massachusetts charters are enormous. They are larger than any I have seen in my career,” [education researcher Susan] Dynarski wrote. “To me, it is immoral to deny children a better education because charters don’t meet some voters’ ideal of what a public school should be. Children don’t live in the long term. They need us to deliver now.”

Pointer from Tyler Cowen. For this research to be convincing to me, it would have to show that there is not much fadeout and also that the interventions are scalable.

And Again

Jason Richwine writes,

Last year the Education Department announced that math and reading scores on the National Assessment of Educational Progress had declined since 2013. Though the decline was small (and not much to worry about), the announcement ruined the Obama administration’s case that its education policies were vindicated by rising scores. Now a new report from the Institute of Education Sciences (IES) has questioned the efficacy of one of the most consequential of those policies – the “Race to the Top” grants.

The Null Hypothesis Strikes Again

Stephen L. Morgan and Sol Bee Jung write,

The results demonstrate that expenditures and related school inputs have very weak associations not only with test scores in the sophomore and senior years of high school but also with high school graduation and subsequent college entry. Only for postsecondary educational attainment do we find any meaningful predictive power for expenditures, and here half of the association can be adjusted away by school-level differences in average family background. Altogether, expenditures and facilities have much smaller associations with secondary and postsecondary outcomes than many scholars and policy advocates assume. The overall conclusion of the Coleman Report—that family background is far and away the most important determinant of educational achievement and attainment—is as convincing today as it was fifty years ago.

Pointer from Timothy Taylor, who links to other papers, not all of them as supportive of the null hypothesis.