The network, which welcomes researchers interested in policy and model comparisons, is one part of a larger project called the Macroeconomic Model Comparison Initiative (MMCI) organized by Michael Binder, Volker Wieland, and me. That initiative includes the Macroeconomic Model Data Base, which already has 82 models that have been developed by researchers at central banks, international institutions, and universities. Key activities of the initiative are comparing solution methods for speed and accuracy, performing robustness studies of policy evaluations, and providing more powerful and user-friendly tools for modelers.
Why limit the comparison to models? Why not compare models with verbal reasoning?
I think that this is a larger question for the profession. I have staked out a claim that policy makers would be better off without the CBO’s models of health insurance coverage or Keynesian multipliers. I believe that policy makers would be better served by verbal reasoning instead.
The dominant view of the profession is that “it takes a model to beat a model.” There are a number of concerns with verbal reasoning. It lacks precision. It cannot be evaluate quantitatively. etc.
I wish to argue that when all is said and done, models often do more harm than good to the decision-making process. What are the best arguments against my view?
It is ambiguous so often meaningless because it leaves out at least as much as it includes, and lacks clarity because it leave uncertainty about the strength of various effects and when and where they become important. It is used to blow smoke to hide the absurdity of the claims being made behind omission and vagueness. It is avoidance of data that may prove threatening in favor of just so stories.
serious question, is this a critique of reasoning or models?
Anyone smart enough to use models to get the answer they want is smart enough to use reasoning. The problem is not models verses reasoning. The problem is the tolerance for economist to use models or reasoning to advance their agenda and the lack of censure for doing so. To cite one simple example, Card and Kruger should be banned from teaching economics and forced out of the economics profession for their minimum wage study.
The statement of what you wish to argue is hard to evaluate, because we can hardly imagine what a non-model-based decision-making-process would look like today. How do we know it wouldn’t be even worse? What other adjustments might me made to respond to existing incentives in different ways?
For example, since ancient times, leaders often find it useful to legitimate their decisions by appealing to some transcendant authority. An exercise of pure arbitrary will exposes leaders to primitive, anti-domination instincts, and makes them fully liable for any bad results later on, which would make them too risk-averse without a an “accountability-hedge.” In Roman rimes an Augur would divine the will of the gods via bird flight or other natural revelation, or a haruspex would inspect some poor animals’ entrails. “It’s the gods, not me, that tell us it is a good day for a battle.”
Today, ‘Science’ and pan-justifying, unfalsifiable economic models helps to play this legitimating role, with the complicated symbols and dense computer simulations creating a needed layer of abstruse obscurity, and the “expert consensus” dispersing responsibility and providing a great alibi / cover-story for the pork barrel ‘stimulus’ projects and politicized credit allocation decision makers wanted to accomplish anyway.
That seems to make these models look bad, lending credibility to a big con. But if one sees the con as both an inevitable and inescapable part of human nature – part of the tragedy of the human condition and the way humans coordinate any group action – then if the models go away, or lose esteem through (perfectly valid) criticism, then it seems reasonable to expect something else to pop up and take their place.
And its also possible that replacement will be a lot worse. Consider, over the last 50 years, Libertarians and free-marketers have been doing an ok job fighting on the economic battlefield with these models and other studies. Partially because of this, the West has been able to avoid the worst kind of Socialist economic disasters as, for example, we’re seeing unfold in Venezuela.
Verbal models seem related to the case study approach common in business and law schools. Quoting from one of the pages at the Harvard Business School site, “Chris Christensen described case method teaching as “the art of managing uncertainty” ” As a possible example of someone who has a quantitative background who grapples a lot with uncertainty Nassim Nicholas Taleb comes to mind.
Another thought: Perhaps there is also a way of characterizing this divide between models and verbal reasoning is the difference between searching for general rules and dealing with particular specific circumstances — The greater the homogeneity among the particular level of units being looked at then the more useful the methods of generalization become. There is that old retort by Tip O’Neil, “All politics is local.” Ultimately, at the finest level of detail, is all economics local? (And therefore particular and specific?) So the use of models to at first paint broad brushstrokes but then for the final applications some effective verbal reasoning is often required?
It seems to me that a great deal of the difficulty comes not from models as such but rather a lack of clarity around what we want the model to do. Any model, economic, statistical, whatever, is a formal aid to make an informal decision. Whether the model is useful for the decision is up to the decision maker. An economic narrative that is verbally argued remains a model.
It seems to me that there is a hierarchy of models:
(I) Causal Deterministic Models: These models can actually make accurate predictions about real events and provide an argument that their constituent concepts may be thought of as “causes”. The most obvious example is Newtonian physics.
(II) Causal Statistical Models: This status is at least claimed on behalf of econometric models. These work in a similar way to the above except the predictions contain an error term. With these, the question sneaks in as to how the details of the probability distribution of the error term affect the decision-making process. Often, these concerns are papered over with Gaussian assumptions.
(III) Pure Statistical Models: possibly useful for predictions but essentially the use of probability models to structure observed coincidences in the strict sense of that term.
(IV)
Mathematical models that produce quantitative predictions that bear some qualitative resemblance to real data. In other words, there exists an analogy between how the data behaves and how the model predictions behave.
(V) Purely narrative models that do not make predictions about a data stream. Aphorisms and fables fall into this category.
From my, admittedly still cursory, understanding of the history of economic thought, many economic models seem to be initially pushed as I or II and then seem to fall, often skipping III. Few economists would use the language I did in V but if you read closely they mean essentially the same thing. I am reminded of defenses of various abstruse physical theories as still of use as a philosophical cosmology. In the current climate, for a model to represent itself as V would mean model death.
Quantitative models have the strength of making relatively tight predictions about observed data. The predictive precision though often comes at the expense of other considerations. The greatest problem seems to come in when it is the model framing the question rather than the question framing the model. The “imprecision” of ordinary language is vastly preferable to false precision in another model. Further, using somewhat more technical but nonetheless simple language, often irons out many of the difficulties. Ultimately, all inductive reasoning must be carried out in ordinary language since there is no way to formalize the analogy between the formalism and the real world; precisely the scientific claim being advanced. This is true even in physics though since physics supports engineering, the analogy is rather more straightforward.
I am sympathetic to Kling’s claim that models often do more harm than good. I think that if one is very clear about the proper use of models of any sort in the reasoning process, many of these problems go away. If I were to name an economic model, I suspect economists would have difficulty classifying it as one of the above, this is evidence of a lack of clarity about the role of models in the reasoning process and consequently, a muddying of the decision making.
I should give credit where it is due. My model hierarchy is lightly adapted from that given by WM Briggs in his excellent Uncertainty: the Soul of Modeling, Probability, and Statistics.
So where does the work of someone like Bryan Caplan fit into your classification scheme? His models are typically verbal descriptions of some type of behavior being analyzed with a verbal statement of a hypothesis. Then he uses data and empirical models (which are distinct in the economics literature from theoretical or conceptual models) to test his verbally-stated hypothesis.
I have not read Caplan extensively though I am familiar with some of his work. Firstly, some caution is in order. A mathematical model described verbally remains mathematical. Consider the statement: “income compensated demand curves slope down”. The statement is entirely verbal but the underlying description is mathematical even if its interpretation is economic. In this sense, the foundations of price theory are mathematical. It seems to me that so long as we stay within neoclassical boundaries, the argument is not so much over whether the foundations of economics are in some way mathematical but rather over the extent to which the heavy usage of symbolic mathematics is a hindrance or a help to economic rhetoric. At this level of generality, it seems to me the question is ill-posed as a concrete purpose has not been stated. My own best guess is that the heavy use of symbolic mathematics often hinders communication between economists. This is exacerbated when economists equivocate between features of formal mathematical models and observations of the external world. The result is the worst of both worlds.
This is a round about way of answering your question. I think you can find in Bryan Caplan’s work mostly models of class II, IV and V. He describes these models verbally but he does make predictions about reality, though rarely exact point predictions about some future measurement. More often, his models aim to replicate some qualitative feature of a particular data set.
Well… let me answer your question by first proposing a model of the decision-making process..