A commenter writes,
surely students who took statistics with you know statistics better than students who didn’t take statistics at all, right? The null hypothesis taken to the extreme would suggest that having taken your- or any- statistics course, should provide no benefit in understanding statistics at all.
If we define the intervention as “took a statistics course” vs. “did not take a statistics course,” then I believe that the intervention worked for some students. However, if we define it as “took a statistics course with Dr. Kling” vs. studied the material some other way, then I would not bet against the null hypothesis.
The same commenter writes,
Arnold, are you of the belief that educational outcomes would be the same if we got rid of schools altogether? That almost seems to be what you’re implying here.
Again, this raises the question of what is the relevant experiment. (I have often not been clear on this.) For example, if you want to compare home schooling with standard schooling, I would not be willing to bet against the null hypothesis. But if you compare standard schooling with no schooling, you are talking about something else entirely.
Learning from other humans is an essential trait of human nature. To hold dogmatically to the null hypothesis, one would have to suggest that the amount and type of learning that children undertake is a function only of their individual characteristics and of the culture in which they are embedded. It does not depend on the way that the institutions of schooling are structured.
The institutional structure does affect resource allocation with respect to teaching and learning. I would speculate that our school system probably makes more efficient use of resources than would a system in which schools did not exist. I would speculate that it makes less efficient use of resources than would a system of vouchers and competition rather than government-managed schools.
Would I go far as to say that the only difference that schools make is in resource allocation, not in outcomes? I doubt that such an extreme position is warranted. But statistically, educational interventions tend to affect resource allocation much more than outcomes. For educational interventions within roughly the current institutional setting, the null hypothesis is not an iron law, but it is an empirical regularity.