THE SOFTWARE PRODUCTION FUNCTION
"Arguing in My Spare Time," No. 2.07
Arnold KlingJune 4, 1999
May not be redistributed commercially without the author's permission.
A test question I recall from undergraduate economics asked:
"What law is illustrated by the fact that not all of the world’s wheat can be grown in a single flower pot?"
The answer is the Law of Diminishing Returns. If we fix the quantity of land, adding more and more labor brings proportionately less and less wheat.
Frederick Brooks’ classic essay, "The Mythical Man-Month," also illustrates the Law of Diminishing Returns. If you have a project that will take one programmer six months to complete, Brooks says that you cannot reduce the time to three months by adding a second programmer. You can reduce the time to some extent by adding staff, but adding more and more programmers brings proportionately less and less reduction in time-to-completion. Furthermore, Brooks argues that at some point the marginal return from additional staff turns negative: adding staff beyond a certain amount actually will increase the time needed to complete the project.
While Brooks evidently is correct that the Law of Diminishing Returns applies to software development, the reasons that this should be the case are not obvious. For example, it is not explained by the fact that some programmers are more productive than others.
An often-cited study done 30 years ago showed that some programmers can produce 10 times as many lines of code per day as other programmers whose level of experience appeared to be equal. There is some indirect evidence that these differences in programming ability can be found today as well. Some staff programmers earn about $30,000 a year, which works out to $15 an hour for a 2000-hour work year. Meanwhile, some consultants earn $300 an hour, or 20 times as much. Some of this differential represents a return to specialized knowledge and experience. Some of it represents a risk premium (the consultant may experience more gaps in employment). Some of it represents compensation for adverse working conditions (the consultant probably will be expected to work evenings and weekends and perhaps to work in a difference city for long periods). Nonetheless, these large differences in wage rates probably are indicative of productivity differences.
However, differences in programming productivity do not explain why software development should exhibit diminishing returns. Suppose that we have one efficient programmer, and we find that adding an inefficient programmer fails to double output. Is this the Law of Diminishing Returns? No. It does not explain why adding a second programmer who is as efficient as the first would not double output. Even if we add inefficient programmers, as long as the increase in output is proportionate to the increase in staff (given their ability) then the Law of Diminishing Returns does not apply.
The Law of Diminishing Returns is based on the assumption that there is an important factor of production whose quantity is fixed. In the case of wheat, holding the quantity of land fixed, there are diminishing returns to labor.
What is the fixed factor in software production? It seems safe to rule out land as being a critical factor of production in software. The amount of physical capital (machinery) is not a very plausible candidate, either.
Brooks himself views the software designer as a fixed factor. In another classic essay, "No Silver Bullet," he argues that the actual coding of computer programs represents a relatively small fraction of the work needed to develop software. As a result, he argues, even if one could achieve a dramatic improvement in productivity as measured by lines of code written per day, this would not be a "silver bullet" that would speed the overall software development process by an order of magnitude.
Brooks views software design as having extremely diminishing returns. His ideal team would have one chief designer, with other members of the team organized to support that designer. In his view, the design process exhibits diminishing returns because of the costs of communication. The time available to the potential users of a system to communicate their requirements to designers is limited. The time that designers have to communicate with one another and with the developers who will implement the design is limited. The more designers you add, the more communication is required. Therefore, increasing the number of designers leads to a less-than-proportional increase in completed design work.
Because design involves communication, it is difficult to draw boundaries around the design process. Users are involved in design. So are programmers. On a given project, no one person may even have the title of "designer."
In fact, the productivity of the technical designer may depend largely on external forces. Because the designer’s success requires communication, if the users do not articulate their needs in a clear and consistent way there is not much chance of success.
What all of the foregoing suggests is that what economists call the production function (a well-defined relationship between the inputs and outputs of a production process) may be a difficult concept to apply to software. We know that software requires design, and that design requires communication. Software engineering experts DeMarco and Constantine have written that "sociology" plays a major role in determining the success or failure of systems projects. That is a sobering observation.
Robert Solow, a Nobel laureate from MIT, famously warned his economics colleagues about the temptations of practicing "amateur sociology" when problems lie outside the economic paradigm. The explanation for the fact that the Law of Diminishing Returns applies to software development seems to require us to take that step.