David Rosenthal writes,
very powerful economic forces drive centralization of a successful decentralized system. . .
the fundamental problem is that decentralized systems inherently provide users a worse experience than centralized systems along the axes that the vast majority of users care about.
Pointer from Tyler Cowen.
The argument over whether software should be centralized or decentralized is analogous to the argument about command vs. a market. In Specialization and Trade, I describe two forms of coordination, or resource allocation. A command system is used within a firm. A price system is used in the market.
I discovered during my business career that software does not evolve independently of the context in which it is created. I used to say that every organization gets the information system it deserves. Tightly-run organizations end up with very reliable systems. More free-flowing organizations end up with very fragmented systems.
People’s needs differ from and conflict with one another. In a command system, a central planner determines which needs will be met. In a market system, the price and profit system directs entrepreneurs to which needs will be met.
A command system is fine if the pattern of needs is given, or if you have enough power over people to treat their needs as given. A central planner can seek to optimize to meet a given set of needs. But a market system works better at discovering needs.
The original communication network–the telephone system–was centralized. That is because switches were expensive relative to bandwidth. But as computers took over switching, the cost of switching plummeted, obeying Moore’s Law. This opened the way for the Internet to take over communications around the turn of the 21st century.
When the Web first arrived, people did not know how it was going to be used. The challenge was one of discovering needs, and decentralization was most appropriate.
Eventually, some needs coalesced, and we started to see well-worn paths through the Internet jungle. So there emerged big, centralized systems, such as caching servers and search engines.
Amazon, Google, Facebook, and Apple are able to take people’s needs as given. They try to optimize to meet those needs.
One of them could falter if and only if it gets caught flat-footed by a new service that has discovered needs that its customers have that are not being met.
If you don’t know exactly what your software will need to do, then a decentralized architecture might make sense. But once you find a clear pattern of usage, you will want to optimize the software for that pattern, and one can predict that the architecture will evolve in a centralized direction.