Null Hypothesis Watch

Two papers that claim to reject it.

1. Michael Lovenheim and Alexander Willen write,

We see consistent evidence that 12 years of exposure to a collective bargaining law negatively impacts both cognitive and noncognitive scores among men. AFQT percentile declines by 10.2, a 20.9% effect relative to the mean.

See also the abstract, quoted by Tyler Cowen.

2. Michael Gilraine, Hugh Macartney, and Robert McMillan write,

California’s statewide class size reduction program of the late-1990s. . .caused marked reductions in local private school shares, consequent changes in public school demographics, and significant increases in local house prices — the latter indicative of the reform’s full impact. Second, using a generalization of the differencing approach, we provide credible estimates of the direct and indirect impacts of the reform on a common scale. These reveal a large pure class size effect of 0.11 SD (in terms of mathematics scores), and an even larger indirect effect of 0.16 SD via induced changes in school demographics. Further, we show that both effects persist positively, giving rise to an overall policy impact estimated to be 0.4 SD higher after four years of treatment (relative to none).

I am skeptical of both papers. I am not convinced that the methods used truly eliminate possible confounding factors. But I have not read either paper closely.

9 thoughts on “Null Hypothesis Watch

  1. In terms of study #2, I was wondering where the increase in public school enrollment was coming from since the 1990s. And California and a couple other blue states could make the public school enrollment go up 2%. (Although I am guessing there was impact of decrease Catholic students as well in the nation.)

    Where this study fits in with impact on house prices is beyond me. I know there is some of this right district stuff but housing prices are up everywhere. This hits me simple correlation due to timing of the late 1990s. In fact higher housing prices probably effects private school enrollment a lot more than school enrollment effects housing prices.

  2. re: Lovenheim and Willen: An income decrease of less than 3% seems pretty insignificant, which means it’s probably noise or an artifact, which we should believe if there’s something else fishy with the results.

    Which, there is.

    They say at one point, “Among men, our estimates point to negative effects of exposure to teacher collective bargaining laws on the longrun labor market outcomes of students who grew up in states with these laws. These results are consistent with the “rent-seeking” hypothesis of teacher unionization.”

    What would actually be consistent with the rent-seeking hypothesis is if stronger work protections from compulsory unionization reduced hiring selectivity and pressure to meet standards, and allowed teachers to get away with being crummier or lazier, which yielded consistently bad results for all groups. Instead, there is apparently no significant effect on women (especially white and Asian women – table 6), and a big disparity between black and Hispanic men on the one hand (-10.6%), and white and Asian men on the other (-1.9%).

    Boy, that rent-seeking hypothesis sure does acts strangely differently on different groups.

    The second article, on the other hand, puts a lot of emphasis on different outcomes for different demographic classes, but has enough free variables studying a peculiar time, sector, and place (the California Housing Market in the late 90’s) that it’s hard to take seriously, as the estimated indirect effects dominate the direct effects.

    • Living in California I think the author has the causation the wrong way. The fact that housing prices shot up is the cause of less private education than education effecting housing prices. (I know certain zip codes were higher price with school districts.)

  3. The first one makes no sense at all. I don’t know what the confounding factor is, but that it exists I have no doubt.

    The second one seems unlikely. California public schools in rich areas are exceptionally good. People who pay for public schools do so because they can afford the status, not because they hate the local schools.

  4. These results would be remarkable if true. There is almost no way that this could have any measurable impact.

  5. http://www.nytimes.com/1988/12/20/science/study-links-intelligence-and-myopia.html

    IN general, people who are nearsighted do better on intelligence tests and achieve higher educational levels than those who are not, but the traditional explanation -that reading promotes nearsightedness in genetically susceptible people – may not be correct, Danish scientists say. Instead, they suggest, ”visual exploration of the near environment” from birth may be associated with both higher intelligence and nearsightedness later in life.

    ——————-

    Us myoptics solve difficult problems when toddlers. Adults do not know how myoptic we are, and to get around we myoptics keep a much more detailed map of our surroundings than normal. Look at this study, tell me about the Null hypothesis.

    • Just to buttress. The opposite study found that top athletes always had perfect vision, since youth.

      There is a bunch of nurture going on, which effects events moving forward. Eyeball shape and intelligence have no direct map, but vision is a complex model.

Comments are closed.