a response to Taleb
How scientific is the five-factor model of personality psychology? On a scale of 1 to 10, where 1 is not scientific at all and 10 is very scientific, I would place the five-factor model at 4. In this case, I use scientific to mean the ability to make useful, reliable predictions. By that standard, IQ, much as some people hate it, is scientific. Astrology, much as some people like it, is not scientific.
To be useful, a psychometric instrument has to be predictive in two ways. First, the measure taken on one person using one instrument should be useful in predicting that same measure on that person at a different time or using a similar instrument. That indicates robustness or reliability.
Second, personality psychology also must be able to make predictions about differences in behavior and life outcomes. That would make it useful, for example, in assigning tasks in business or in improving online dating services.
To understand the requirements for prediction, consider the famous Stanford marshmallow experiment. The first question we should ask about this experiment is how consistent the results would be with repetition. Suppose that you give the test on Monday, and Alice delays gratification but Bobby eats the marshmallow right away. On Tuesday, will the result be the same, or will it be Bobby who delays gratification and Alice who eats the marshmallow? Does a child’s performance on the marshmallow test predict what the child will do with a different treat? If not, then the marshmallow test is not a reliable instrument for measuring a personality trait.
What made the marshmallow test famous was the follow-up work which suggested that a child’s ability to defer gratification on the test helped predict future outcomes, such as SAT scores. These correlations with longer-term outcomes speak to the usefulness of the test in revealing some important trait.
IQ vs. Astrology
Using prediction as a standard, IQ is a scientific measure. Measuring IQ at different points in time or using different instruments tends to yield very similar results. Also, IQ is useful in predicting outcomes in life.
On the other hand, some people use astrology to predict life outcomes. But are astrological predictions accurate? Quickly consulting Google, I found an article by Kelly Katera.
There haven’t been many studies that investigate the science behind astrology, but of the few that have, the results have failed to support the validity of astrological views. For instance, a study tested the accuracy of astrological charts in describing the personality traits of 193 study participants, and the results indicated that the scores were at a level consistent with chance.
This would tend to confirm my view. Astrology is not scientific.
The popularity of astrology nonetheless shows that we are inclined to see people as having innate personality differences. We think that we can model these personality differences and use those models to predict behavior and outcomes. Moreover, we are capable of over-estimating the ability of a model, such as astrology, at making such predictions, perhaps because we are guilty of confirmation bias. That is, we are inclined toward noticing when the model succeeds and ignoring or rationalizing instances where the model fails.
Personality psychology as a science
We know that people differ in their brain chemistry. There is a field known as personality neuroscience that links brain chemistry to personality traits. The five-factor model has been examined from this perspective. Timothy A. Allen, the author of the survey article, concludes that the research shows promise yet leaves many important questions unanswered.
Personality psychology will always have scientific limitations. The traits measured in the five-factor model are not nearly as robust as IQ. The traits are measured using self-assessment questionnaires, and they are not as consistent over time or across measurement instruments as what one finds with IQ testing.
The five-factor model also has limits in predicting behavior and life outcomes. Those are determined by many factors other than innate personality characteristics: environmental influences, life history, and cultural context. I expect that rigorous studies might find small correlations between measured personality traits and behavior, but it is unlikely that we will find that any single personality trait overwhelmingly determines particular behaviors.
Some of us may be guilty of over-estimating the value of personality psychology, just as many people seem to over-estimate the value of astrology. But personality psychology is not total BS.
Astrology may not work in all circumstances, but I have found star-charting effective, relatively speaking, in macroeconomics.
For example, the US orthodox macroeconomics professions spent the last 40 years warning about pending higher interest rates and inflation. See Martin Feldstein for the past, and we have James Grant for now. So, the orthodox macroeconomics profession was 100 percent wrong.
Using my astro charts, and special chanting, I was wrong 50% of the time. I confess I am skilled at chanting, so due to modesty I may be overrating astrology.
Do you give consultations? I can use some help.
Glad you note that personality stuff is not BS.
But you have a very weak quantitative issue — 5-factor is 4 (scale 1-10), but what is IQ? 10? 6? You don’t say.
Reasonable definition: I use scientific to mean the ability to make useful, reliable predictions
However, not enough actual quantification –
IQ is useful in predicting outcomes in life. Where’s an example? What do you really mean? I agree with this idea, but I’m not really sure what you’re saying.
There is some quantitative difference between ordering the 50 grads randomly (pseudo-random alphabetically, or reverse with Z first), with ordering them according to IQ. And a different (slight? ) difference ordering them by GPA, but the GPA order is also far better than random, and perhaps better than IQ.
Just linear orders; by random (reverse alpha, Z first); by IQ (lowest first); by GPA (lowest first); by starting salary (lowest first).
What are the differences between “actual” starting salary order, and X predicted. Where all the lists go from 1-50, and 50 is the highest possible difference for any one person, with 0 being the lowest, and we calculate the sum of [abs] differences.
I’d guess IQ would be just a little better than GPA, tho it could be the reverse. A perfect predictor would be 10. Random would be 1; it might be possible to misorder so as to deliberately have worse scores than random, perhaps in reverse IQ, or instance. Perhaps from 1 down to 0 would be decimals of misorder.
One skilled in using the 5-factor personality analysis could also order the 50 grads. Their order’s difference would be, probably(?), higher than IQ. If that difference is given a predictor value of 4, I’d guess the IQ predictor value would be about 8 (of 10).
And I suspect that if the personality profilers were willing to do more IQ integration with their big-5 stuff, they get an order with a predictor value of 9 or closer to perfect 10.
I learned about the very popular Myers’-Briggs personality tests, plus the great book “Please understand me”, which went thru 4 pairs of personality traits that seem relevant to understanding other people, like Introvert-Extrovert; open-ended (P – perceiving) vs closure, ordered (J-judging), plus others. I’ve often been tested and test myself and remain xNTP, where my Introvert score = Extrovert. (I’m an introvert with reasonably good extrovert skills.) All of the people I know who’ve taken the test and are “J” have clean desks and are orderly; those who are “P” have messy desks. There’s a lot of other similar predictions possible. It doesn’t seem to have much “starting salary” predictive power.
Jordan Peterson and his popularity have increased the public visibility about the alternative, more preferred (by shrinks) Big-5 tests, which are different than the MBTI personalities. In particular, the Big-5 trait of “Agreeableness” seems well correlated with lower salaries in companies. Women also have a generally higher Agreeableness score, and perhaps lower salaries due to that personality trait, rather than due to their being women.
The traits measured in the five-factor model are not nearly as robust as IQ.
I’m sure this is true about each trait, but IQ, too, is not determinative of life’s outcomes, nor even the starting salary (or perhaps more importantly the salary 10 years after graduating from college; now I’m thinking, 15 years after graduating from high school would be an even better time to measure).
Because there is a known, but seldom explicitly talked about racism as part of IQ, personality traits and other indications will be more publicly used. The racist IQ order, highest IQs first:
Jews,
Chinese
Brahma Indians (few discussions of this)
Whites
Latinos & Hispanics
Blacks.
Blacks are poor; blacks have low IQs. On average.
There’s lots who are richer and smarter than me.
Charles Murray is called a racist for noting these truths.
Public figures are not allowed to talk about this.
I’m retired, or else I’d probably not even write it now.
Missing 4th paragraph somehow, the model set-up:
Take one college where 50 students are graduating from Economics in that year.
Their starting salaries go from $45k up to $90k.
Their IQs are from 110 to 145.
Their GPAs are from 2.8 up to 4.0.
They have an actual ordering from lowest starting salary to highest starting salary — this is the base “actual” which other orderings are trying to predict.
https://www.articlegateway.com/index.php/AJM/article/view/2392
A study of non-cognitive big-5 traits and their impact on wages, with some comparison of the cognitive ability impact on wages.
Rolf Degen tweets continuously about such psych studies, with nice copies of interesting factoids.
https://twitter.com/DegenRolf/status/1204267316629913600
Openness % is not noted in the excerpt. (OCEAN 5-factors)
Conscientiousness (only significant positive trait): 6.2%-7.4%
Extraversion: -3.4% (surprise to me; I’d guess positive)
Agreeableness has estimated negative return: -4.8%
Neuroticism: -3.6%
Cognitive ability is positive: 13-16%
I understand this to be the wage difference for those who have a one-standard deviation more of any trait. Not sure how much “cognitive ability” is merely another phrase to describe IQ.
This seems like a good way to quantitatively compare, and try to understand, differences. Better than Arnold’s “scientific” rating of “4”.
This Canada study used the Longitudinal and International Study of Adults.
I’d guess other studies, in other countries, states, professions, among graduates of colleges, etc., would be coming up with more insights. Then meta-analyses of the various studies would increase the likelihood of limited reality being within the scientifically predicted range.
Rolf is often linking various meta-analyses.