Category Archives: Economic education and methods

How economic model-building is unique

Posted on by

Consider two rationales for building models:

(a) Build a model in order to clarify the signal by filtering out the noise in a complex causal system. This is a knowledge-seeking endeavor.

(b) Build a model in order to be able to say, “In setting X, I can show how you get outcome Y.” This is just playing a game.

An example of playing a game is Akerlof’s Lemons model. In effect, it says, “In a setting where sellers know the quality of the product and buyers do not, sellers of high-quality products will have to settle for low prices, if they choose to sell at all.”

Some remarks:

1. I am pretty sure that economists are unique in their attachment to model-building as a game. My sense is that in other disciplines, including those that study human behavior and those that use non-mathematical models, researchers are more likely to be building models in order to try to separate the signal from the noise in a complex causal system.

2. Countless papers begin by describing a setting as having two factors of production, capital and labor, before adding further wrinkles to the setting. From the knowledge-seeking perspective, I fear that this is a dubious strategy. The two-factor model gets rid of a lot of signal and introduces a lot of noise. But for playing the game (and getting published) it works well.

3. Economists who work in business (think of Hal Varian at Google) do not have the luxury of playing games. If they want to use models to help the firm, they need to build them with the goal of separating signal from noise.

CBO under attack

Posted on by

Marc Short and Brian Blase write,

The CBO’s methodology, which favors mandates over choice and competition, is fundamentally flawed. As a result, its past predictions regarding health-care legislation have not borne much resemblance to reality. Its prediction about the Senate bill is unlikely to fare much better.

1. Note that both authors work in the Trump Administration.

2. Sherry Glied and others wrote,

This analysis finds that the CBO overestimated marketplace enrollment by 30 percent and marketplace costs by 28 percent, while it underestimated Medicaid enrollment by about 14 percent. Nonetheless, the CBO’s projections were closer to realized experience than were those of many other prominent forecasters.

I would not take the position that there is an obviously better model than what the CBO uses. The problem in the policy environment is that CBO estimates are treated as scientific truth. This misleads participants in the policy process into believing that predicting the outcomes of policy is a science. This in turn biases policy toward aggressive intervention.

The false belief in economic science imposes a real cost. Legislators and bureaucrats become overconfident in their ability to manage market processes.

Catherine Rampell takes the opposite point of view as mine.

Contrary to the predictions of economists everywhere, the HHS propaganda document claims that the Cruz amendment would cause insurance coverage to go up and premiums to fall. Astoundingly, even premiums for people in the Obamacare-compliant plans — which, again, economic theory suggests would get stuck with only the very sickest, most expensive Americans — would allegedly decline relative to current law. (Compare “2020 Current Law Enrollment Weighted Average” to “2020 Silver ACA Compliant” in the chart below.)

This is garbage, and exactly why we need nonpartisan scorekeepers like the CBO.

Rampell is right to attack the memo that she criticizes. But she is wrong to wish to anoint the CBO as a scientific umpire.

Suppose that the CBO were asked to “score” the employment effects of a minimum wage increase, and suppose that their most preferred model projected a large decrease in employment. Would the Catherine Rampells and the Mark Thomas be so eager to say that “this is exactly why we need a CBO”–in order to settle the argument about the minimum wage?

If the science is not definitive with respect to the employment effect of the minimum wage, then it is surely not definitive with respect to the insurance-market effect of allowing health insurance companies to offer less comprehensive policies with lowerpremiums. The CBO should not be the ultimate arbiter of contested economic analysis.

Nancy MacLean: ignoring the central ethical issue

Posted on by

Henry Farrell and Steven Teles write,

MacLean is not only wrong in detail but mistaken in the fundamentals of her account.

I have met both Farrell and Teles, at dinners organized by Teles and Brink Lindsey, for “liberaltarians.” The liberaltarian project always seemed to me to be quixotic, but it did demonstrate overlap between (some) progressives and libertarians on a few economic issues, particularly related to Public Choice. Farrell and Teles strike me as coming more from the liberal camp as opposed to the libertarian camp. But because they are receptive to Public Choice ideas, some progressives might consider them to be heretics.

Historian Andrew Seal writes,

Some of my colleagues and I at the Society for US Intellectual History Blog and I are planning a roundtable to discuss Democracy in Chains as a work of intellectual history, in large part because we feel that the critiques of MacLean’s work have not adequately engaged with its core arguments and because these critiques often seem unfamiliar with the “best practices” of intellectual history.

For me, the central issue is scholarly ethics. I expect that when it comes to history, many books will be written that have narratives that are controversial and have flimsy support. That is acceptable.

The ethical issue is whether the historian has an obligation to make the effort to elevate truth above narrative. Did Nancy MacLean make that effort, as Seal’s use of the phrase “best practices” implies?

For example, I could wish to create a narrative that tries to portray Dr. Martin Luther King as a racist, and I could do so while staying within ethical boundaries. It might not be very persuasive, of course. But if I quote Dr. King as saying “I have a dream that my four little children will one day live in a nation where they will…be judged by the color of their skin” (i.e., leaving out the word “not”), then that is unethical. That seems pretty clear to me. And it seems to me that MacLean’s conduct comes pretty close to that, yet I do not see it condemned outright as unethical by Farrell and Teles, much less by Seal.

Let me put it this way: if MacLean’s actions do not constitute easily-recognized and serious violations of the ethics of the history profession, then that profession has no ethics. And historians on the left ought to be thinking about whether that is what they want.

Math and uncertainty

Posted on by

A commenter writes,

if the math is done right, it should then say precisely that: there isn’t enough data to resolve the parameters you’re trying to impute with any reasonable degreee of confidence. The ‘anti-math’ people seem to forget that uncertainty is itself a quantifiable thing.

This does not address the problem that Richard Bookstaber and others call radical uncertainty. Consider what the CBO director wrote concerning the agency’s evaluation of the ARRA (the 2009 Stimulus bill).

The macroeconomic impacts of any economic stimulus program are very uncertain. Economic theories differ in their predictions about the effectiveness of stimulus. Furthermore, large fiscal stimulus is rarely attempted, so it is difficult to distinguish among alternative estimates of how large the macroeconomic effects would be. For those reasons, some economists remain skeptical that there will be any significant effects, while others expect very large ones.

Note that he did not attempt to quantify this uncertainty, nor could he have done so. Note also that what Congress and the public focused on were the apparently precise numerical estimates of the CBO model, rather than the uncertainty of those estimates.

The CBO uses a standard macro model, in which there is only one type of worker in the economy. I believe that workers in today’s economy are highly specialized, and that this accounts for the difficulty in creating new patterns of trade when old patterns become unprofitable. It is easier to use math to analyze a model with one type of worker than it is to apply math to my model. I think that is an argument against the tyranny of math in economics.

Peter Turchin on mathematical modeling

Posted on by

He writes,

Models clarify the logic of hypotheses, ensure that predictions indeed follow from the premises, open our eyes to counterintuitive possibilities, suggest how predictions could be tested, and enable accumulation of knowledge. The advantage of clarity that mathematical models offer scientists is nicely illustrated in the following quote from Economics Rules: “We still have endless debates today about what Karl Marx, John Maynard Keynes, or Joseph Schumpeter really meant. … By contrast, no ink has ever been spilled over what Paul Samuelson, Joe Stiglitz, or Ken Arrow had in mind when they developed the theories that won them their Nobel.” The difference? The first three formulated their theories largely in verbal form, while the latter three developed mathematical models.

Pointer from Mark Thoma.

Are you kidding me? The meaning of Arrow’s Impossibility Theorem has been endlessly debated.

With mathematical models in economics, the question is whether the conclusions of the model apply in the real world. That is something that cannot be settled mathematically. It often cannot be settled empirically.

If I had chosen to write a review of Turchin’s latest book, Ages of Discord, I would have devoted most of the review to criticism of Turchin’s statistical methods. I am quite confident that if you formulated his project as “Come up with a set of indicators that represent the concepts here,” there would be no consensus, and that almost no one would come up with the indicators that he selected. The overall thesis of the book might turn out to be right. But in terms of methods, it could be held up as the poster child of what Paul Romer calls “mathiness.”

The CBO gets worse

Posted on by

John Taylor writes,

The second CBO procedural change was to discontinue the use of the “alternative fiscal scenario” in the long-term projections

There is more at the link.

I think that CBO modeling is way over-rated and biased toward interventionist policies. I would take them out of the modeling business almost entirely.

For budget projections, you need to do modeling. Budget projections are numbers, after all.

But models are subject to all sorts of errors. GDP growth could turn out to be higher or lower than expected. Interest rates could be higher or lower than expected. Your estimates of the cost of programs could be off, because people may respond to incentives differently than what you expect. Laws may change.

Given these sources of error, scenario analysis is a must. The CBO should be doing more scenario analysis, not less. I am increasingly convinced that the CBO as it currently operates is performing a huge disservice to public policy. Congress should make major reforms to the mission and operations of the CBO.

Richard Bookstaber on Economic Methods

Posted on by

I have a long essay on his book, The End of Theory. One brief excerpt:

In conventional economics, people are assumed to know, now and for the indefinite future, the entire range of possibilities, and the likelihood of each. The alternative assumption, that the future has aspects that are not foreseeable today, goes by the name of “radical uncertainty.” But we might just call it the human condition. Bookstaber writes that radical uncertainty “leads the world to go in directions we had never imagined… The world could be changing right now in ways that will blindside you down the road.” (page 18).

Read the entire essay. It is another attempt to address issues of economic methods.

Wither academic ethics?

Posted on by

If you read David Henderson at EconLog or Don Boudreaux at Cafe Hayek, you know that a history professor named Nancy MacLean claims to have unearthed some sort of right-wing conspiracy involving James Buchanan, and this has gotten her some play in liberal media. However, David, Don, and other critics have pointed out what appear to be pretty blatant instances of MacLean twisting Buchanan’s words, even to the point of making it sound like he favors X when the full context clearly shows the opposite. My thoughts:

1. If the critics are correct, then MacLean’s breech of ethics is quite serious. If you are going to have an academy that claims to be searching for truth, then people have to play by rules. They have to be as open as possible about caveats to their own work. They have to try to be as fair as possible to those with whom they disagree. They have to strive for honesty and objectivity, even if these ideals may not be attainable.

2. However much I dislike mathematical formalism in economics, I have to say that it does impose some discipline. Maybe you can construct biased models and try to pass them off as scientific, as Paul Romer accused others of doing in his “mathiness” critique. But nobody uses x’s and y’s to conduct hit jobs and character assassination. There are some natural boundaries imposed by sticking to formal models. And when someone like Paul Krugman steps outside of those formal boundaries and writes newspaper columns, you can see the results.

3. James Buchanan won a Nobel Prize. Say what you will about the committee that decides on the prize, they do not sell their votes to the Koch brothers. Every year, they evaluate a body of work that is very difficult for non-economists to understand and very well vetted by other economists.

4. You can teach about methods that an ethical academic can use in research and writing. However, I do not believe that you can teach an unethical person to be ethical by offering a course in ethics. Instead, you have to police ethics. I think that the most important factor is a willingness to police your own side. If some economists in the conservative/libertarian orbit look at MacLean’s work and conclude in writing that it is basically sound, then that could help. Conversely, if some economists in the progressive orbit decide that she has indeed violated scholarly norms and they put that view in writing, then that could help. If neither side concedes, then it’s game over. There will be no such thing as a search for truth any more.

5. Michael Munger’s review describes MacLean’s book as “historical fiction,” and he says that she does a good job of it. But he stops short of hitting the ethical issues hard. I wish that a serious critical review would appear in a major academic journal.

Models vs. Verbal Reasoning

Posted on by

John Taylor writes,

The network, which welcomes researchers interested in policy and model comparisons, is one part of a larger project called the Macroeconomic Model Comparison Initiative (MMCI) organized by Michael Binder, Volker Wieland, and me. That initiative includes the Macroeconomic Model Data Base, which already has 82 models that have been developed by researchers at central banks, international institutions, and universities. Key activities of the initiative are comparing solution methods for speed and accuracy, performing robustness studies of policy evaluations, and providing more powerful and user-friendly tools for modelers.

Why limit the comparison to models? Why not compare models with verbal reasoning?

I think that this is a larger question for the profession. I have staked out a claim that policy makers would be better off without the CBO’s models of health insurance coverage or Keynesian multipliers. I believe that policy makers would be better served by verbal reasoning instead.

The dominant view of the profession is that “it takes a model to beat a model.” There are a number of concerns with verbal reasoning. It lacks precision. It cannot be evaluate quantitatively. etc.

I wish to argue that when all is said and done, models often do more harm than good to the decision-making process. What are the best arguments against my view?

The left, the market, and economists

Posted on by

In a recent exchange with Don Boudreaux, Bryan Caplan writes,

The heart of the left is being anti-market.

From the standpoint of the oppressor-oppressed axis, it may make sense to be anti-market. If you look at market outcomes, you see some people do much better than others. It is natural to assume that those doing well are oppressors and those doing not as well are oppressed.

As an economist, I look at the market as impersonal. It is a process. As a process, it has many virtues.
Competition helps to regulate exploitation. The profit motive spurs innovation that helps people in general. You know the drill.

Bryan is among those who believe that teaching people economics can help them to understand the process perspective and to see the market in less personal terms. Hence, if you confront people on the left with economics, their leftism will soften. That indeed has happened to many economists. Vernon Smith and Deirdre McClosky are two prominent ex-socialists.

Unfortunately, I think that going forward we are going to see the opposite effect of confronting leftists with economists. That is, I think that the academic economics will be converted to an oppressor-oppressed view of markets. Not that I think that such a view is more justified now than in the past. Rather, I think that the leftism in academia is stronger than in the past. See my recent essay. As I have pointed out in previous posts, we are already seeing much more focus in academic economics on anti-market perspectives that align with the oppressor-oppressed framing.