April L. Bleske-Rechek writes,
Mayviewed a random sample of poster abstracts that had been accepted for presentation at an annual convention of the premier professional organization in psychology, the Association for Psychological Science. We were disappointed to find that over half of the abstracts that included cause and effect language did so without warrant (i.e., the research was correlational). Of course, poster presentations are held to a less rigorous standard than are formal talks or published journal articles, so in a follow-up study, we reviewed 660 articles from 11 different well-known journals in the discipline. Our findings replicated: over half of the articles with cause and effect language described studies that were actually correlational; in other words, the causal language was not warranted.
In biological families, children resemble their parents in vocabulary and verbal ability; in adoptive families, they do not. The key implication is that Hart and Risley’s finding of a link between parents’ verbal behavior and their children’s verbal ability does not warrant an inference that parents’ verbal behavior influences their children’s verbal ability.
Somebody should put together a YouTube course on “How to be skeptical of statistical studies.” I nominate Russ Roberts.
After the replication scandals, I would have thought that the journals would have strengthened their technical reviews of articles. A better than 50% failure rate undermines the peer review model.
A little bit of knowledge is a dangerous thing?
Researchers bend data/stats when they have emotional investment in supporting a particular conclusion, they let their emotions override their rational thinking, and nothing puts that in check.
Nassim Nicholas Taleb is the man when it comes to debunking the use of statistics in studies.
I love this video of him generating correlations of two random variables:
Randomness of Correlation & Its Hacking by BigDataists
https://www.youtube.com/watch?v=D6CxfBMUf1o
Of course there is no correlation between the two random variables!
He also just posted the following, but I haven’t read it yet.
Common Misapplications and Misinterpretations of Correlation in Social “Science”
https://www.academia.edu/39797871/Common_Misapplications_and_Misinterpretations_of_Correlation_in_Social_Science_
On the one hand, it’s well known that all psychological traits show a substantial genetic influence, and any study that fails to consider genetic confounders is fatally flawed. On the other hand, the sorts of double-blind controlled experiments that could provide evidence of causation would likely be stopped by ethical review boards. ERBs are often overzealous, but they do serve a useful function in preventing researchers from causing permanent harm to kids.