Referring to Hayek’s claim that local knowledge favors decentralized decision processes, Handle comments,
IT and increasingly capable and sophisticated management information systems, which themselves benefit from massive economies of scale, and the management techniques they enable, has invalidated this argument. If anything, big companies now seem to have a clear advatange with regards to acquiring and leveraging ‘local knowledge’, and combined with the other advantages of brand recognition, size and sophistication and capacity for, e.g., rent-seeking and bearing the burden of compliance overhead, that leaves “the little, genuinely-independent guy” with zero chance in the long run
1. I wonder if this applies to government. If the U.S. federal government took advantage of Big Data, could we be as well-run as Singapore or Norway? I tend to doubt it. Perhaps someone wants to argue that we could be that well-run if we had an epistocracy.
2. Once again, I am reminded of a Diamond Age world. Technology can allow giant enterprises to meet everyone’s needs cheaply. By the same token, luxury consists of goods and services provided by small artisan craftspeople.
3. This is another instance in which the Internet vision of the 1990s, the days of the “hippie Internet,” is turning out to be wrong.
4. Will there be sufficient dynamism in this big-firm economy? Where will competition come from? From small firms that can suddenly get big and unseat big incumbents? From big incumbents trying to encroach on one another’s turf?
If physical goods that can be shipped to you via Amazon are cheap…nobody cares.
If luxury artisan goods are expensive…nobody who can’t afford them will care.
The tricky goods are the expensive but necessary. Housing, education, healthcare. Some because they involve skilled labor (thus can’t be cheap, even if we did it better then today) or are inherently zero sum in some respect (housing in great centralization hubs). These goods are how we can be “rich” but feel “poor”, all the way up and down the income ladder.
What we see with those sectors is that its a mix of local entrepreneurs, medium size operators, and large behemoths (often acting through a medium sized local operator).
So take healthcare. I work for a behemoth. However, they have lots of people at the local level sorting out the complicated local issues. And of course hospitals and doctors are local, even if they are owned by a behemoth each is going to operate at a very independent level.
Real estate is all “location, location, location” and knowing which hip neighborhood will gentrify next (or creating it proactively).
Education we’ve all had to deal with the local school board.
Generally this will leave the small independently owned operator (who is not just a franchiser in a bigger organization) out of the loop, except in those instances where something is niche enough to be ignored or they are a regulatory protected professional.
3. This is another instance in which the Internet vision of the 1990s, the days of the “hippie Internet,” is turning out to be wrong.
TBH, this was not surprising as the Dotcom boom was an over-investment in these various websites that did not have a business case. And in 1997 there was just a lot of new crazy stuff to see and do. I remember ordering a rare P-Funk live disc in 1998 that would have never turned up in the stores. It was new and exciting.
However, the internet companies DID NOT KILL local operators! It felt like Hippie Internet because they were competing against consolidated retail markets. It was Barnes & Noble or Borders that killed the local bookstores and now Amazon is killed Borders and weakening Barnes & Noble. And now Amazon is going after Wal-Mart and Target both of whom quite large.
The local economy dominated by small operators was already lost to most of America in the 1980s and 1990s. So any nostalgia for local economy with small store business before Amazon is simply not true. (And Wal-Mart still sells a lot more stuff than Amazon!)
Anyway, I assume Big Internet companies are dominating because:
1) The marginal cost of another megabyte is exceptionally small so these firms will never see rising average cost.
2) With such small marginal costs, successful companies can pay better and hire the best. If you want to work in tech Google is like joining the NFL for coding students.
3) The average citizens can only process so many sites and looking for convenience. If you believe than why are the Big 4 banks gaining customers? I hate Wells Fargo but they have the most ATMs in our area and good local banks would be a pain to use.
4) If you don’t believe me on the local economy stuff and cheaper internet, what would happen to impact to car sales if car dealership state laws were rolled back. (In terms of why car dealerships did not corporate consolidate, remember the manufacturers sell the cars for the same price to all dealers so they could not grow like Wal-Mart who buys their cheaper than their competitors.)
I don’t think that really addresses Hayek’s point about knowledge problems. The reason Hayek thinks you need decentralized markets isn’t just that there’s a lot of data out their and it’s computationally difficult to account for all of them. The issue is that the information you need doesn’t exist but for the decentralized market process that creates it.
Technology (“big data,” etc.) can make it easier for a firm to get its hands on the kinds of information that is sitting around waiting to be discovered, and that could give the firm e.g. better market research. But the real knowledge that you’d need to solve Hayek’s knowledge problem can’t be acquired in that way because it’s “defined in the process of its emergence.”
This is also essentially Coase’s point: Firms don’t solve the knowledge problem; they just exist in scenarios in which solving it via markets is even more costly than not solving it at all.
So technology probably increases economies of scale that makes consolidation more efficient on some level, but it’s important to keep clear what kind of knowledge we’re talking about. It also cuts both ways: there’s also a good chance that advances in IT substantially reduce transaction costs so that more decentralization makes sense.
I’m going to push back a bit against your claim.
Let me step back first and consider the questions of the economic problems one is trying to solve: what should be produced and consumed, in what quantities, by whom, how are resources to be allocated, etc.
But that necessarily raises a logically prior question which is, “Optimizing for what?”
I think it’s fair to paraphrase Hayek’s basic claim as the Socialist Central Planner trying to optimize for net social welfare over some long time horizon (or some alternative or proxy index based on overall production), and operating without prices or markets, simply cannot do as well as a “markets and prices”-based system, even under some idealized conditions, and assuming empowered decision-makers in either model are just as uncertain about future developments.
Now that’s not necessarily true for some total-war, command-economy that has conscripted the entire population into a near slave-labor force. That’s because the central planner doesn’t care about what anybody wants and is using single objective valuations. He cares about the number of tanks being fielded, and assuming he has access to all the upstream production functions, it’s sometimes theoretically possible to solve the constrained economic optimization problem for something like tank numbers.
But as soon as one starts caring about what people want and subjective valuations, then one needs access to knowledge about individual preferences, opportunity costs, elasticities, demand curves, etc.
The point about markets is that they allow the individuals to optimize their own welfare and act on those wants by choosing and making bids and the prices that emerge from those processes signal information that indirectly allows us to infer some details about those wants. Those prices and some insight into local individuals and conditions allow entrepreneurs to observe those local details and notice discrepancies and arbitrage opportunities, and the in the past, all of those observations were too difficult to collect, aggregate and analyze.
But two things.
First, even when prices are uniform and everyone is a price-taker, they actually don’t communicate very much of the desired granular knowledge about individuals. On average maybe they tell us something like the ratio of the “value of the highest valued use of the marginal unit” (MVHVU for short) of a particular kind of (more or less fungible) thing to the MVHVU of every other kind of thing at a particular time and place.
But any entrepreneur or planner wants to know much more than that, he wants to know what’s going on in everybody’s heads. He wants some insight into the metaphysical construct of “utility”, he wants to know, by direct or indirect observation or inference, every consumer’s demand curves, consumer surpluses, elasticities, preferences, and so forth. He doesn’t necessarily want other to be able to use a uniform “price system” and wants to be able to price discriminate, that is, make prices for each person according to that granular, individualized level of detail.
He doesn’t want to broadcast copies of the same message to everyone, but to use individualized information fed into an automated algorithm which can tailor-make messages calculated to influence a particular individual based, for example, on their profile, peer and social groups, and the subcultural celebrities that are most likely to be effective as mimicked ‘taste-makers’ for that particular person, e.g., “Three of your friends, that girl our algorithm thinks you like based on your cyberstalking, and your favorite signer whose style we know you try to imitate based on X, Y, and Z, all bought the iPhoneX … “ (Obviously not delivered in that kind of explicit manner.) Entrepreneurs don’t want to be “preference takers” either, they want to shape and alter preferences to the extent possible, but they need to be able to accurately model psychological processes to do so and to have some insight into the particular psychological profile they are trying to influence at any particular moment, combined with other individualized details about habits and life circumstances and so forth.
The point is, market prices communicate information and knowledge the market-less Socialist Central Planner does not have, but not nearly as much knowledge as is useful and desired for economic purposes. The little guys can’t succeed anymore because all they have is prices, but a few big guys now have much, much more than prices.
Which leads up to the second point which is that a state of affairs which might have been utterly inconceivable except as speculative science fiction in Hayek’s day has actually come to pass very recently, which is the currently technically possible ability to feasibly and economically maintain and update in real-time a comprehensive database of nearly all individual behavior and activity, which means the ability to make individualized predictions instead of relying of aggregated averages and models using those constructs. Indeed, there are some indications that the gold that can be algorithmically mined from all that big data can help certain companies know more about individuals than they know about themselves, which to the extent this knowledge is useful in terms of making accurate predictions is analogous to “actionable intelligence.”
Here’s a hypothetical example. Let’s say someone is coming down with a cold, but hasn’t quite noticed it themselves yet because severe symptoms haven’t risen to the level of conscious awareness. Nevertheless, when people get sick, their online (or other digitally observable) behavior changes in ways that algorithms can detect with good accuracy. And the record of this individual’s behavior when sick is that he always buys a particular brand of cold medicine. So, theoretically, a firm with access to this information could try to anticipate the future demand that the individual himself does not even yet know he will want, and put the logistical machine in motion so that the right bottle of cold medicine will arrive in the vicinity of the customer just in time, at which point his device could give a push notification, “Feeling sick? Here’s a 20% off coupon and the bottle of medicine could be at your door (or car, or whatever) in just 15 minutes.”
In such a system many consumer goods and services may not even have predictable or generally advertised “prices” anymore, since they ideal would be for perfect price discrimination.
But the point is that the kind of information and knowledge observable and inferable by just a few special firms today puts the mere market-price system to absolute shame. It’s not just “market research”, it’s the theoretical ability to observe, model, and influence all the brains of all the individual autonomous actors in the marketplace, the aggregation of which was thoroughly unobservable in Hayek’s presentation of an economic system, and which also has deep implications for the potential future accuracy of economic observations and statistics.
In this “private panopticons” world, the question becomes which kind of entities have the advantage in collecting, observing, analyzing, and exploiting this kind of information? Is it decentralized arbitrageurs of local scope and scale, or is it monolithic global giants that have established entrenched positions in the digital economy which allow them to acquire, store, process, correlate, and use a large amount of the personal information of billions of individuals?
It seems clear that, as with many IT enterprises, there are tremendous quantities of scale and the ability to benefit from network effects and quasi-monopoly state deriving from social “focal points” which cause the names of certain leading (or first-mover) firms to become directly associated with particular kinds of services in a way that puts any competitor at a severe disadvantage (e.g., Google, Uber.)
The point is that the ultimate form of decentralized observation is an invisible spy following everyone around and recording everything. Which is kind of what just a few companies are able to do with smartphones. And the implication of those economies of scale and particular market position that enables digital surveillance across individual devices and correlated internet sessions means that maximally decentralized observation will be occurring under the auspices of maximally centralized – and thus consolidated – firms.
Handle’s comment echoes a similar argument that Ben Thompson at Stratechery has been touting for some time now…
https://stratechery.com/2015/aggregation-theory/
…With near zero transaction costs for many internet-enabled businesses, distribution no longer becomes a constraint. In the past, a company could only supply services within range of their physical distribution. As ‘physical’ becomes ‘digital’, supply becomes increasingly abundant; customer acquisition costs (enhanced by the access to customer data/’local knowledge’) have very strong economies of scale.
It is for that reason that I come to the opposite conclusion as Joe K: Diminishing transaction costs will likely increase the winner-take-all effect. This is already underway in many segments of the tech industry (Amazon in retail, Google in search, Netflix in cord-cutting home entertainment, etc.). I wonder how long until we see similar effects across all industries?
A related question is what should we infer from what we observe from the big companies in terms of how much they pay for access to information about individuals, and how much they invest in trying to obtain comprehensive records of everything people do. Apple, Facebook, Google, and Amazon all seem to spare no expense and to be pulling out all the digital stops to track everything they can about people and to match all kinds of online sessions and activities to particular real life identities and profiles. It’s not just these corporations, most companies are trying to dive deep into the information game and are willing to pay or offer discounts for it, e.g. Costco and most groceries stores.
When you walk into a brick and mortar store it’s possible you are arriving as a complete stranger and unknown quantity. But when you interact with these companies in any way (often indirectly and unwittingly), you are a very, very well-known quantity, and everything about your experience can be immediately tailored to your unique signature.
There is just no doubt that these databases of personal history are extremely valuable, but that raises the question of which entities can collect this information in the cheapest, most efficient, and most comprehensive manners?
My local grocery store chain only has access to few tiny pieces of the puzzle. Maybe my local mechanic only knows about the maintenance I’ve done at his one particular location, or with his one particular chain.
But just a few certain companies will be able to leverage their special position to maintain a kind of “persistent presence” (to use malware terminology) on one’s devices and keep close track of all the things one is doing, or to otherwise form relationship with all the third parties out there to also “snoop and rat” on you in exchange for preferable treatment.
It seems to me that the emergence of such extremely valuable comprehensive databases and the fact that in the nature of things they will be owned in that fully comprehensive and mine-able form as proprietary records by just a few internet companies means that these companies will retain a significant and permanent advantage over any entity that cannot tailor its business operations with the same knowledge base.
As for government, it seems perfectly clear to me that a few obvious jurisprudential reforms would make crime a thing of the past with existing technology and surveillance systems. Many commentators are fond of touting how low crime has fallen from the insane peaks of the 60s-90s, as if it were time to ease off the intrusiveness and severity of the criminal justice system, but the reality is that practically all traditional crimes could be easily and predictably detected with perfect reliability (and thus largely prevented to less than 10% of present levels) using information that is already being collected and which only requires the authority and initiative to correlate in an automated fashion.
Big Data is hard to use.
https://www.wsj.com/articles/retailers-emails-are-misfires-for-many-holiday-shoppers-1511778600
You intellectual idiots ought to talk to people who actually Make Useful Things for a living rather than to people who write about things, move things around, trade things for other things, “manage” the making of Useful Things, and so on.
You really have no clue what it means to “make a decision” and the world you talk about -the world you see- is a parasite on the real world. I’m not saying that’s bad, I very much like your fantasy, but it’s 11 days away from cannibalism.
This seems like an interesting hypothesis, but it’s just an assertion. Went back and read the whole comment and I don’t see Handle offering any evidence to back it up.
On the contrary, here’s one data point: I work for a large corporation, where we use “big data” and ML etc and I don’t see evidence that our middle managers or IT has made us particularly good at solving issues related to local knowledge or that’s made us particularly better at overcoming classic corporate issues related to bureaucracy etc.