Category Archives: Economic education and methods

Improving Economic Research

Posted on by

Garrett C. Christensen and Edward Miguel have a new paper ($). They conclude

There are many potential avenues for promoting the adoption of new and arguably preferable practices, such as the data sharing, disclosure and pre-registration approaches described at length in this article. One issue that this article does not directly address is how to most effectively – and rapidly – shift professional norms and practices within the economics research community. Shifts in graduate training curricula, journal standards (such as the Transparency and Openness Promotion Guidelines), and research funder policies might also contribute to the faster adoption of new practices, but their relative importance remains an open question. The study of how social norms among economists have shifted, and continue to evolve, in this area is an exciting social science research topic in its own right, and one that we hope is also the object of greater scholarly inquiry in the coming years.

I have been thinking quite a bit about this recently. To make a long story short:

1. Economic phenomena are rife with causal density. Theories make predictions assuming “other things equal,” but other things are never equal.

2. When I was a student, the solution was thought to be multiple regression analysis. You entered a bunch of variables into an estimated equation, and in doing so you “controlled for” those variables and thereby created conditions of “other things equal.” However, in 1978, Edward Leamer pointed out that actual practice diverges from theory. The researcher typically undertakes a lot of exploratory data analysis before reporting a final result. This process of exploratory analysis creates a bias toward finding the result desired by the researcher, rather than achieving a scientific ideal of objectivity.

3. In recent decades, the approach has shifted toward “natural experiments” and laboratory experiments. These suffer from other problems. The experimental population may not be representative. Even if this problem is not present, studies that offer definitive results are more likely to be published but consequently less likely to be replicated.

I agree with Christensen and Miguel that the norms and incentives within the economics profession are the key. For a long time, both the norms and the incentives have pulled researchers in the direction of getting certain types of results, which has pulled them away from the direction of following robust methods. That culture is very difficult to change.

Note that I recently discovered the web site The Replication Network.

The Psychology of Politics

Posted on by

Maria Konnikova surveys some of the literature.

Lord and his colleagues asked people to read a series of studies that seemed to either support or reject the idea that capital punishment deters crime. The participants, it turned out, rated studies confirming their original beliefs as more methodologically rigorous—and those that went against them as shoddy.

This and other studies serve to highlight confirmation bias, which helps to reinforce tribalism in politics.

I would like to point out that this form of confirmation bias is a very important problem in academia. That is why I think that studies should be evaluated methodologically before the results are known. A referee should be asked whether the study is capable of producing results that influence someone to change their minds. Could the results turn out to be against your prior beliefs? If so, would that influence your prior beliefs?

Ideology and Polarity

Posted on by

Jordan Peterson says,

In a sophisticated religious system, there is a positive and negative polarity. Ideologies simplify that polarity and, in doing so, demonize and oversimplify.

That sentence really bolsters my approach in the Three Axes Model. The whole interview is interesting.

In fact, I have been binge-watching his lectures. Reviews of his book suggested that it might be inaccessible, but his lectures are very accessible, albeit with a big investment of time. If you don’t have the patience for his style, you might want to jump to lecture 5, part 1. But my view is that you should have patience for his style.

Peterson, like Jung, believes that ancient myths tell us a lot about how we are wired. In my eBook, I say that the Progressive oppressor-oppressed axis can be found in the Exodus story. I think that Peterson would locate what I call the civilization-barbarism axis in a lot of ancient myths in which the death of a king or the emergence of a terrible king leads to chaos until a hero fights the chaos and is crowned the new king.

The libertarian liberty-coercion axis may be more modern. In Peterson’s terms, government (and our cultural inheritance in general) always enbodies both the good father who provides order and the tyrant who chains people. The liberty-coercion axis sees the tyrant and not the good father. Peterson probably would find libertarian utopianism to be akin to other utopianisms. In that sense, he would view a really dogmatic libertarian as dangerous, the way that Whitaker Chambers famously remarked that reading Ayn Rand made him feel as though there was lurking a “To a gas chamber–go!” mindset.

I think that embedded in his course is a philosophy of science that is profound. I think it can be applied usefully as a perspective on economic models. I will say more about that when I finish the course.

Peter Turchin’s Latest Book

Posted on by

It is called Ages of Discord: A Structural-Demographic Analysis of American History, and I received a review copy. I am not very far into it. An alternative title might be “Average is over. . .and maybe so is everything else.” From the back cover:

Historical analysis shows that long spells of equitable prosperity and internal peace are succeeded by protracted periods of inequity, increasing misery, and political instability. These crisis periods–“Ages of Discord”–have recurred in societies throughout history. Modern Americans may be disconcerted to learn that the US right now has much in common with the Antebellum 1850s and, more surprisingly, with ancien regime France on the eve of the French revolution.

I will have some problems with his approach to history, if what he says on p.6-7 is any indication.

What we need is theory in the broadest sense, which includes general principles that explain the functioning and dynamics of societies and models that are built on these principles, usually formulated as mathematical equations or computer algorithms. Theory also needs empirical content that deals with discovering general empirical patterns, determining the empirical adequacy of key assumptions made by the models, and testing model predictions with the data from actual historical societies.

This sounds like it borrows some of the more dubious methodological doctrines of economics. I have been arguing that mental processes are important in explaining social outcomes. I fear that the emphasis on mathematical equations and data leads instead to a focus on physical processes, to the neglect of mental processes. I do not think that Turchin will turn out to be such a physical determinist. But we will see.

Somewhat related: Yascha Mounk on indicators of fragility in democracy.

The first factor was public support: How important do citizens think it is for their country to remain democratic? The second was public openness to nondemocratic forms of government, such as military rule. And the third factor was whether “antisystem parties and movements” — political parties and other major players whose core message is that the current system is illegitimate — were gaining support.

Pointer from Tyler Cowen.

Turchin also uses indicators, but his set is different.

Defining Terms in the Social Disciplines

Posted on by

Chelsea Leu writes,

Chemists don’t squabble about what oxygen is, but if psychologists convene a conference on a fuzzier concept like “trust,” says Colin Camerer, an economist at Caltech, they’ll spend the first two days disagreeing about what the word actually means.

Pointer from Tyler Cowen.

I agree that many important terms in the social definitions are poorly defined. Examples include trust, culture, and happiness. Ethnicity, which is very important in political studies, may not be well defined. I am not sure that such terms as extroversion or openness are well defined. Even IQ may not be that well defined.

Security prices are well defined. As a result, theory and empirical research in finance tends to be more robust than elsewhere in economics. However, even finance has its less well-defined concepts, such as “expectations.”

I should point out that the problem of poorly-defined terms arises not because social scientists are less intelligent or careful than natural scientists. The problem is that social science has to deal with phenomena of the mind. Oxygen is part of the physical world. Trust and extroversion exist in the mental world.

I have noted that in all of the social disciplines there is a bias in favor of explaining outcomes on the basis of physical phenomena, such as natural resources and physical capital, rather than on the basis of mental phenomena, such as culture and institutions. In part this may reflect frustration with the challenge of defining terms when describing mental phenomena.

Although exhortations to social scientists to define terms may be helpful, I think it is important to understand that exhortations alone cannot solve the problem. Mental phenomena are harder to pin down.

Social Science is Mis-named

Posted on by

John Tierney writes,

In a classic study of peer review, 75 psychologists were asked to referee a paper about the mental health of left-wing student activists. Some referees saw a version of the paper showing that the student activists’ mental health was above normal; others saw different data, showing it to be below normal. Sure enough, the more liberal referees were more likely to recommend publishing the paper favorable to the left-wing activists. When the conclusion went the other way, they quickly found problems with its methodology.

I prefer the term social disciplines for economics, political science studies, sociology, anthropology, psychology, and history.

I believe that it is true in general that one’s instinct is to focus on methodological flaws when results conflict with your priors and to ignore methodology and instead focus on results when a study finds congenial results. Probably this affects natural science as well, but my guess is that in natural science results are often more robust, so that if you do not like the results you cannot just carp about the methods.

In fact, it probably would be a good idea of journal editors would send out papers stripped of their results for peer review. That is, the referee should recommend for of against publication of a paper based on the methods used and the question asked, not on the answer obtained.

Outline for an Econduel

Posted on by

The topic is, “Should we think of economics as a science?”

Against

Thinking of economics as a science is incorrect as a description and unwise as a prescription. As a description, the claims that economists make are not as robust as claims made in natural sciences. As a prescription, claiming that economics is a science leads to a belief in the capability of economic policy-making that is unwarranted and dangerous.

For

It is true that it is not possible in economics to make claims that are as precise and verifiable as in chemistry of physics. But economics still contains valuable laws, such as the law of supply and demand. And economists still should attempt to follow the scientific method as best they can. If no one believes that economics is a science, then that opens things up for all sorts of bad intuition and nonsense to enter the policy debates.

I have more thoughts, but nobody reads blogs on Thanksgiving, anyway.

Interpretive Frameworks and the Election

Posted on by

Robby Soave wrote,

Trump won because of a cultural issue that flies under the radar and remains stubbornly difficult to define, but is nevertheless hugely important to a great number of Americans: political correctness.

Read the whole essay. I was not persuaded.

After the financial crisis, it was remarkable how many economists found their world view confirmed by it. Keynesians said that it proved Keynesianism, Those who thought that loose monetary policy is the root of all evil felt vindicated. Scott Sumner and others put the blame on tight money. Economists on the left blamed neoliberalism and deregulation. Economists on the right blamed government-sponsored enterprises and affordable housing goals.

I’m getting the sense that last week’s election is going to have a similar follow-up. Everyone is going to use it to climb on to their favorite hobby horse. It proves that America is racist. It proves that the economy is not working for some people. It proves that nationalism is more popular than globalism. It proves that elites have failed. It proves that the American political process is flawed. It proves that new media have altered the electoral landscape. It proves that Obamacare is not working. It proves Hillary Clinton is unpopular. It proves that the American people do not care about facts. It proves that Americans are still mad at Wall Street. It proves that Americans want to get out of the Middle East. It proves that terrorism is a major issue. It proves that issues don’t matter, and that it’s all identity politics. Americans wanted anyone but another Bush or another Clinton. It was a repudiation of Barack Obama. It shows that democracy is flawed.

I am going to climb on to my own hobby horse in order to offer a meta-interpretation. We face a blooming, buzzing confusion of interpretive frameworks for the election. Because it is only one event, we are not going to be able to sort it out.

I would argue that the same holds true for many economic phenomena. There are many plausible interpretive frameworks. If we are looking at singular events, like the financial crisis, we have no chance of definitively sorting them out. When we look at macroeconomics in general, too many factors change to enable us to draw firm conclusions. With microeconomics, there is enough similarity across markets and across time to develop more confidence in crude interpretive frameworks, such as basic supply and demand. But attempts to get more refined and precise are likely to fail.

Confirmation Bias in Macroeconomics

Posted on by

Prakash Loungani writes,

The evidence shows the “cycs” have been proved largely right. U.S. unemployment has fallen pretty much in line with the recovery in output. U.S. states where growth was stronger than the national average had declines in unemployment greater than the national average. And looking across the globe, countries which experienced more rapid growth than the global average — a group which includes the United States and the United Kingdom — had declines in unemployment greater than the global average.

Pointer from Mark Thoma. His point is that this vindicates the theory of aggregate demand. My response: as opposed to what? A theory that output and employment are totally unrelated? Whose theory is that? In fact, employment and output should be correlated whether one is telling a PSST story or an aggregate demand story.

The usual notion of confirmation bias is that people over-rate findings that favor their preferred point of view. But macroeconomists go beyond this. They take findings that are completely neutral in their implications for one point of view vs. another and claim that these findings confirm their preferred point of view.

Let’s Eliminate “Culture” from Social Science (er, Disciplines)

Posted on by

Joel Mokyr, in his new book A Culture of Growth, says on p.8

Culture means various things to different people, and to begin, we need to clarify the concept and our use of it. Given the rather astonishing popularity of the concept of culture in the social sciences and the humanities and the mind-boggling number of definitions employed. . .

What follows from this is that social scientists should not use the term “culture” and instead replace it with a word or phrase that is less loaded with alternative definitions and connotations. Mokyr goes on to explain what he means by culture, pointing out that it is similar to a definition that can be found in Boyd and Richerson’s book Culture and the Evolutionary Process. Mokyr offers

Culture is a set of beliefs, values, and preferences, capable of affecting behavior, that are socially (not genetically) transmitted and that are shared by some subset of society.

My recommendation would be to replace the term “culture” with the phrase “socially communicated knowledge and behavior.” I think it is pretty obvious that a large subset of what we know is socially communicated through conversation, writing, teaching, on-the-job training, and such. A large subset of our behavior also is socially communicated. We imitate prestigious people. We obey authorities. We covet praise and fear being shamed by friends, family, and strangers.

I am not saying that everyone should define culture as “socially communicated knowledge and behavior.” Other people may wish to define it differently. Rather, I am suggest that Mokyr and others who use the term “culture” as he does should instead use the phrase “socially communicated knowledge and behavior.”

If you want to say that economic growth and development are affected by culture, some people will be inclined to resist. But if you say that economic growth and development are affected by socially communicated knowledge and behavior, my guess is that you will have pretty much everyone on your side immediately. If you say that the market is an institution that contributes to culture, again some people will resist. But if you say that the market is an institution that contributes to socially communicated knowledge and behavior, people will be ready to listen to your account of that process.

In short, the best way to get “culture” appreciated as an important factor in economics and other social sciences disciplines is to stop using the term “culture.”