the highly dubious notion that college and career skills are the same. On its face, the idea is absurd. After all, do chefs, policemen, welders, hotel managers, professional baseball players and health technicians all require college skills for their careers? Do college students all require learning occupational skills in a wide array of careers?
No, that is not Bryan Caplan. It is Robert Lerman. Read the whole thing.
So, I tried to read my free review copy of The UnStoppables, by Bill Schley. I hate his writing style, but I think that on substance the book, which is a guide/pep-talk for entrepreneurs, is actually good. In talking about how to come up with a business idea, Schley suggests asking yourself these questions (p. 22):
1. I wish I could, so why can’t I?
2. What if?
3. How come no one ever fixed that?
4. Why does this have to be such a pain?
For a long time, I have wished that I could better navigate the world of online learning. What if there were a guide for online learning that students could use to find the best resources and that educators could use to benchmark the competition and share resources? There are lots of great learning videos online, but there is a lot of garbage, and it’s not easy to get straight to the best. How come no one ever fixed that? As a teacher, I find it very difficult to share learning resources with other teachers–using some of their videos, adopting some of their online quizzes, etc. Why does this have to be such a pain?
On my recent vacation, I saw how Rick Steves and tripadvisor.com have gone a long way toward solving these problems for travel. So my latest idea for an education start-up is something like a Rick Steves or tripadvisor.com for online learning resources.
The Rick Steves model ensures consistency of how evaluation takes place, and it gives you the voice of a dedicated, opinionated consumer. The tripadvisor.com model uses crowd-sourcing, so you get less consistency of methodology but broader, timelier coverage.
Let’s assume the Rick Steves model, and take first-year statistics as the prototype. If you were Rick, you would list the topics that you think generally belong in such a course. Then go through all the online materials available from Khan, Kling, Udacity, Coursera, Carnegie-Mellon, etc., and create a model itinerary for students. If one of these brands just dominates in every topic within first-year statistics, then recommend that brand. Otherwise, for each topic, list the top three explanatory videos, the top three sets of interactive exercises, etc.
It is important to remember that your perspective is that of a typical student, not that of someone with an advanced background in statistics. Your advanced background may lead you to over-rate deep, brilliant lecturers (like Udacity’s Thrun) and under-rate folks like Khan who keep it simple and glide past issues that someone pursuing a Ph.D in stats would want to treat more carefully.
Two factors would make the online learning space harder to profit from than the travel space. First, the online learning world changes more rapidly. It takes a couple of years to put up a new hotel. It takes much less time to put up a new lecture or quiz on the central limit theorem. So you couldn’t sell printed books easily, since they would be out of date before they are published. Even with a web site, a lot of the work you do in 2013 will have to be tossed out or re-done in 2014.
The second factor is that it would be harder to generate revenue from advertising. Travel web sites are complementary to the existing bricks-and-mortar folks (hotels, restaurants, rental car companies), who get the concept of advertising. Bricks-and-mortar educators, on the other hand, view the online world as a competitive threat rather than as a pure complementary good. It’s not clear that a for-profit university or textbook publisher would see any point in advertising on the sort of site that I have in mind.
Bradford S. Bell and Jessica E. Federman write,
The meta-analyses reviewed above show that when instructional design characteristics are held constant across delivery conditions, e-learning and classroom instruction generally produce similar learning outcomes.
Pointer from Timothy Taylor–read his whole post.
The null hypothesis is that there is no difference in outcomes, and apparently the “meta analysis” does not reject the null hypothesis. Taylor interprets this as evidence that online learning has “caught up” to classroom learning in terms of quality. My more cynical interpretation is that there was never any catching up required, because what students learn does not appear to depend in any way on how they are taught. I am sure that there is a limit to this: presumably, if you do a randomized experiment in which one group of students gets $50,000 worth of instruction and another group gets zero, you will see some difference in outcomes. However, I would bet that, relative to what we do today, the way to improve cost-effectiveness in education is to slash costs. That is, my view of the null hypothesis is that most of what we spend on education has no marginal impact.
Note, however, that the authors of the study seem convinced that the null hypothesis is false. They believe that empirical evidence shows that pedagogical techniques do affect outcomes.
It is the lead story in today’s Washington Post.
Lawmakers have responded by introducing legislation that would at least temporarily block the standards in at least nine states, including two that have put the program on hold. The Republican governors of Indiana and Pennsylvania quickly agreed to pause Common Core, and Michigan Gov. Rick Snyder (R), a vocal supporter of the plan, is nevertheless expected to accept a budget agreement struck by GOP legislators that would withhold funding for the program pending further debate.
I strongly support the Tea Party on this one. I was an early opponent of national testing. Ten years ago, I wrote
For the standardized testing approach to accountability, success by definition means making schools responsive to top-down control. In the case of the Bush administration “reforms,” standardized testing increases the leverage of the Federal government over local schools. Any conservative ought to think twice about supporting such a trend.
there are clearly countries that spend less per student than you would expect given their level of per capita GDP, like Iceland, which is labelled, and Italy, which is the unlabelled point more-or-less under Spain. There are also countries that spend more per student than you would expect given their GDP, including Ireland, Canada, and especially the United States.
He is referring to higher education.
Returning to the Oregon Medicaid study, Tyler Cowen writes,
The key question here is how we should marginally revise our beliefs, or perhaps should have revised them all along (the results of this study are not actually so surprising, given other work on the efficacy of health insurance). For instance should we revise health care policy toward greater emphasis on catastrophic care, or how about toward public health measures, or maybe cash transfers? (I would say all three.) One might even use this study to revise our views on what should be included in the ACA mandate, yet I haven’t heard a peep on that topic. I am instead seeing a lot of efforts to distract our attention toward other questions.
Nick Schulz and I have referred to health care and education as the new commanding heights. That is, they are as important in the 21st century as steel and electric power were in the 20th. However, steel and electric power had major scale economies that lent themselves to top-down, bureaucratic management. Health care and education do not.
What I think this means that those who want to apply centralized, technocratic solutions in health care and education (“Obamacare,” “No Child Left Behind”) are on the wrong side of history. Perhaps my views are mistaken. But in any case, I wish that people were less emotionally invested in the technocratic approach, so that if it does prove to be dysfunctional they are able to back off.
From the Chronicle of Higher Education.
The graders working for EduMetry, based in a Virginia suburb of Washington, are concentrated in India, Singapore, and Malaysia, along with some in the United States and elsewhere. They do their work online and communicate with professors via e-mail. The company advertises that its graders hold advanced degrees and can quickly turn around assignments with sophisticated commentary, because they are not juggling their own course work, too.
In Swarthmore College’s Honors Program, at least when I went there, grading also was outsourced. The professor sent the syllabus to an outside examiner, who made up the exam and graded it. This was considered a good thing.
Perhaps the graders who work for EduMetry are not as skilled as the outside examiners used by Swarthmore. Still, what is the next best alternative? Multiple-choice tests on Scantrons? Computerized grading?
As a practical matter, the alternative is not careful, skillful feedback provided by distinguished professors. However, I do expect that many professors at lesser-ranked colleges will see this as a threat to “quality,” meaning their incomes.
By the way, if you are looking for yet another article on MOOCs, the Richmond Fed has one. I am quoted a couple of times. This whole education-technology thing is reminding me of the early days of the Web in the mid 1990s, when there were lots of articles about stuff that seemed important at the time but which has long since been forgotten. Magazines like Business Week ran cover stories on “push technology,” “browser wars,” “applets,” etc.
Julie Berry Cullen, Steven D. Levitt, Erin Robertson, and Sally Sadoff write,
our advice to high schools when it comes to underperforming students is to redefine the mission and eschew traditional success metrics like test scores, focusing instead on more pragmatic objectives like keeping kids out of trouble, giving them practical life skills, and helping with labor market integration. That conclusion will no doubt be unsatisfying to many readers. In an ideal world, high schools would perform miracles, bringing struggling students back from the brink schools and launching them towards four-year college degrees.
So far, I could call this Journal of Economic Perspectives symposium on education “Living with the null hypothesis.”
President Obama says that the science is clear. Greg J. Duncan and Katherine Magnuson are not so sure.
We find that the evidence supports few unqualified conclusions. Many early childhood education programs appear to boost cognitive ability and early school achievement in the short run. However, most of them show smaller impacts than those generated by the best-known programs, and their cognitive impacts largely disappear within a few years. Despite this fade-out, long-run follow-ups from a handful of well-known programs show lasting positive effects on such outcomes as greater educational attainment, higher earnings, and lower rates of crime. Since findings regarding
short and longer-run impacts on “noncognitive” outcomes are mixed, it is uncertain what skills, behaviors, or developmental processes are particularly important in producing these longer-run impacts
Andrew Coulson links to a study in the UK. The study reports that
From the analysis of the available data, there is no clear correlation between funding and school average performance; for a given level of funding, there is significant variation in performance the calculated correlation coefficient between the two variables was less than 0.1. In our view, this suggests that the level of funding, per se, is almost irrelevant as a predictor of performance.
Having said that, the study does find a correlation between a measure of school quality and average student performance. That result would contradict the null hypothesis. However, the indicator that the researchers use for school quality is not convincingly exogenous.
the Ofsted ‘overall effectiveness’ indicator has been used as our measure of school quality. This is a combined assessment of schools’ performance that contains specific judgements on:
— pupils’ achievement and the extent to which they enjoy their learning;
— how well do learners achieve;
— pupils’ behaviour;
— the quality of teaching; and
— the effectiveness of leadership and management in embedding ambition and driving improvement.
The ‘overall effectiveness’ judgement is then made on a four-point scale: outstanding (1); good (2); satisfactory (3); and inadequate (4)
Coulson also supplies a chart demonstrating the null hypothesis in American education spending.
From Hal Salzman, Daniel Kuehn, and B. Lindsay Lowell. It covers a lot of ground, but I was struck by the education/employment disconnect.
[1] For every two students that U.S. colleges graduate with STEM degrees, only one is hired into a STEM job.
[2] 36 percent of IT workers do not hold a college degree at all.
[3] The annual number of computer science graduates doubled between 1998 and 2004, and is currently over 50 percent higher than its 1998 level.
Pointer from Tyler Cowen, who focuses on other results.
My questions:
[1] What are the other 50 percent of STEM grads doing? How many are going to grad school? How many are flipping burgers? If the latter, are their degrees maybe not so impressive, for whatever reason?
[2] What counts as an IT worker? The security guard at Google? Or only people who do technical work?
[3] Does everyone agree on this? Using a different endpoint, Alex Tabarrok wrote,
we graduated more students with computer science degrees 25 years ago!
Assuming consistent measurement between Alex and these authors, we must have had a huge drop in computer science degrees between 1985 and 1998. Note that the drop from 2004 to today seems to be large, also.