Probability and Causal Density

“Scott Alexander” writes,

If ten different factors caused the decline in crime, that would require that ten different things suddenly changed direction, all at the same time in 1994. That’s a pretty big coincidence. . .we should give some credibility penalty to a story with ten factors.

I do not buy this argument. I do not think that one should automatically penalize a study more for claiming that there are ten factors rather than one prominent factor.

My view would be that when there is a lot of causal density, one should be skeptical of any study that claims to have the answer, whether the answer consists of one factor or many. Take as an example the financial crisis of 2008. There are many plausible causal factors. Should we prefer a study that attributes the crisis entirely to one factor rather than a study that attributes it to a combination of factors? I think not. Instead, given the non-experimental nature of the problem, I think we need to accept the fact that we will have to live with some uncertainty about what exactly caused the crisis.

For a phenomenon that is amenable to replicable experiments, it may be possible to obtain evidence against causal density and in favor of an explanation based on one or two factors. But not for something like the drop in crime over the past two decades.

A Question about Inequality Within States

Salil Mehta writes,

Let’s start by looking at this chart below. It shows the differences in state-level ratios, contrasting the typical incomes at the top 1% versus the typical incomes at the bottom 1%… the Economic Policy Institute (EPI) chart above has a clear concordance between income dispersion and the population size itself.

Pointer from Tyler Cowen.

My question is this. Suppose that we ignored the actual geography of states, and instead we produced artificial pseudo-states by taking random samples of all U.S. data. We took one sample the size of Texas, and called it pseudo-Texas. Another the size of Vermont, and called it pseudo-Vermont. etc. My guess is that the pattern of inequality across pseudo-states would look a lot like the pattern across actual states. If that is true, then there is not really much information in the pattern of inequality across states.

Sentences I Could Have Written

From James Manzi.

If we really could build regressions that would reliably predict what the impacts of various policies would be, it would be a powerful argument against certain political and economic freedoms. Why go to all the trouble of having a messy and expensive market, or states as laboratories of democracy, when we could just have a couple of professors build us a model?

Judith Rich Harris 1, Nurture Assumption 0

Kevin M. Beaver and others write,

The role of parenting in the development of criminal behavior has been the source of a vast amount of research, with the majority of studies detecting statistically significant associations between dimensions of parenting and measures of criminal involvement. An emerging group of scholars, however, has drawn attention to the methodological limitations—mainly genetic confounding—of the parental socialization literature. The current study addressed this limitation by analyzing a sample of adoptees to assess the association between 8 parenting measures and 4 criminal justice outcome measures. The results revealed very little evidence of parental socialization effects on criminal behavior before controlling for genetic confounding and no evidence of parental socialization effects on criminal involvement after controlling for genetic confounding.

Pointer from Jason Collins. A caveat is that this is an example of the statisical fallacy of using absence of evidence to imply evidence of absence.

Family Structure Matters

Timothy Taylor looks at various meta-analyses of studies of the possible causal role of the absence of a father on outcomes for the children. He quotes a meta-analysis by Sara McLanahan and others

The research base examining the longer-term effects of father absence on adult outcomes is considerably smaller, but here too we see the strongest evidence for a causal effect on adult mental health, suggesting that the psychological harms of father absence experienced during childhood persist throughout the life course. The evidence that father absence affects adult economic or family outcomes is much weaker. A handful of studies find negative effects on employment in adulthood, but there is little consistent evidence of negative effects on marriage or divorce, on income or earnings, or on college education.

Read Taylor’s whole post. He and the authors he cites are quite aware of the difficulty of distinguishing correlation from causation in this sort of research.

Andrew Gelman is Too Glib

Not in general, but in this post, where he writes,

I’d like to flip it around and say: If we see something statistically significant (in a non-preregistered study), we can’t say much, because garden of forking paths. But if a comparison is not statistically significant, we’ve learned that the noise is too large to distinguish any signal, and that can be important.

Pointer from Mark Thoma. My thoughts:

1. Just as an aside, economists are sometimes (often?0 guilty of treating absence of evidence as evidence of absence. For example, if you fail to reject the efficient markets hypothesis, can you treat that as evidence in favor of the EMH? Many have. Similarly, when Bob Hall could not reject the random walk model of consumer spending, he said that this was evidence in favor of rational expectations and consumption smoothing.

2. I think that a simpler way to make Gelman’s point would be to say that passing a statistical significance test is a necessary but not a sufficient condition for declaring the evidence to be persuasive. In particular, one must also address the “selection bias” problem, which is that results that pass significance tests are more likely to be written up and published than results that fail to do so.

The Omniscient Voyeur

Bryan Caplan writes,

In the GSS, males report an average of 14.19, women an average of 4.76.* If you mean the median, then males report a median of 3, woman a median of 2.

In my statistics course, I use this as a classic example of biased statistics. Here is why I suspect bias.

Suppose that the number of men is M and the number of women is W. Suppose that an omniscient voyeur can count all of the heterosexual relationships. Call this number n. Then the average number of sex partners for men is n/M. For women, it is n/W. Assuming that W and M are about the same, then the omniscient voyeur will know that the averages are about the same. So if the reported averages are different, then the reported averages are biased statistics.

Heritability of g

Plomin and Deary write,

(i) The heritability of intelligence increases from about 20% in infancy to perhaps 80% in later adulthood. (ii) Intelligence captures genetic effects on diverse cognitive and learning abilities, which correlate phenotypically about 0.30 on average but correlate genetically about 0.60 or higher. (iii) Assortative mating is greater for intelligence (spouse correlations ~0.40) than for other behavioural traits such as personality and psychopathology (~0.10) or physical traits such as height and weight (~0.20). Assortative mating pumps additive genetic variance into the population every generation, contributing to the high narrow heritability (additive genetic variance) of intelligence.

Pointer from Kyle Griffin. Read the whole thing. The authors’ discussion of Genome-wide Complex Trait Analysis, a new method of examining genetic influence, was new to me. Deary is the author of Intelligence: A Very Short Introduction, which doubles as an excellent introduction to statistical methods in research.

In my appearance in St. Louis in six weeks, I will be talking about the forces that cause stratification of earnings to accelerate. One of those factors is assortative mating based on g.

Taleb, Evolution, and GMOs

The podcast with Russ Roberts is here.

Taleb’s point is that “science” cannot prove that GMO’s are safe. We know that when organisms evolve using a trial-and-error process of gradual tweaking, that process is safe. But direct intervention to create GMO’s is not the same process.

Taleb says.

The big risk is what can happen when you have two things going together–which is, what happens, Soviet style, is a combination of monopoly of some plants over others, that it’s too large a system; and of course creation of other species that will themselves also be too powerful and then you may kill the GMOs or one may kill the other and you may have huge imbalances in nature. And these imbalances in nature can produce large deviations.

I think I get Taleb’s point. But it strikes me that to cause catastrophic harm, a GMO has to be both weird enough to cause unprecedented things to happen but not so weird that it fails to function as a living organism. That may be an impossible combination.

Annual Physicals vs. Evidence

Ezekiel Emanuel writes,

Those who preach the gospel of the routine physical have to produce the data to show why these physician visits are beneficial. If they cannot, join me and make a new resolution: My medical routine won’t include an annual exam.

He cites controlled experiments showing that the Null Hypothesis is true for the routine physical exam.

Not surprising, really. Ask Robin Hanson.

Pointer from Jason Collins.