In 2003, Nicholas Carr shook up an increasingly irrelevant community of CIOs by publishing the article “IT Doesn’t Matter”. I believe that he got it half right: the irreversable trend of information technology was toward commodity economics, and thus the idea of paying rents for proprietary software was preposterous. What he did not quite get right was to properly recognize that his insight was itself a strategic enabler for those intelligent enough to understand the competitive consquences of the trend he identified. It did matter that open source software provided to users the benefits of commodity economics without the fetters of proprietary software. And it did matter that CIOs wake up and recognize that the open source dynamic was going to be as world-changing to company strategy and execution as, say, the use of commodity energy, commodity communications, commodity raw materials, all traded in commodity currencies such as the US Dollar, Euro, and/or Japanese Yen. (And another thing missing from his analysis, but later provided by Brent Williams, is the remarkable way in which Open Source software companies, when done right, can deliver commodity benefits while still enjoying non-commodity financial returns, but I’ve already blogged about that.) And so again, Nicholas Carr gives us another article half-full of wonderful insights about open source, this time titled The Ignorance of Crowds. The half that Nicholas gets completely right is that the open source development model is generally far better at improving code than the proprietary model has been. Indeed, what Eric Raymond observed with the Linux kernel was finally the answer to Fred Brooks’s “The Mytical Man Month”. Namely, that the organic model of “many eyeballs” trumps the industrial efforts of “divide and conquor”, perhaps because the former is self-organizing and the latter is centrally organized, resourced, and managed. Economists from Adam Smith to Milton Friedman should both be thrilled to see their economic theories finally becoming mainstream in the world of software development! The part that Nicholas gets wrong, however, is the analysis of the limitations of the open source model, and in particular where he asserts (without evidence) that the weaknesses he ascribes to the open source model are therefore strengths of the proprietary model. There is no logical reason to assume that if a problem is difficult to solve with one approach, it is necessarily easier or even trivial with a different approach. Let’s look at the details, though, of his arguments, for a better look at the half he gets wrong. First, he argues that the bazaar model is essentially limited to fixing stuff that’s broken, and no good at innovation or creating something truly new. Of course, he argues against himself because he himself recognizes that the key enabler for the development of something like Linux (which he considers to be a non-innovative clone of Unix) was…the Internet. But the Internet is a creation of the open source community (long before it was called open source). The free, fair, and flat bargain of reading, sharing, and modifying collective and interoperable code was a tremendous innovation whose effects are still being felt. I remember being at a CEO conference in 1995 when Microsoft’s great “innovation” was to launch “Bob”, while Jim Clarke and Marc Andressen were showing the world what they could do on top of an infrastructure of open source (namely, the Mosasic web browser). Tell me: which proved to be the greater innovation? Indeed, the Internet continues to be a tremendous font of innovation, innovation that’s a direct result of the free, fair, and flat structure of open source and open standards and a direct result of the unlimited participation the Internet encourages. He then attempts to use the Wikipedia to support his case that collective development is doomed to producing second-rate results. As a frequent user of Wikipedia (one of 150M visitors per month Carr himself provides), I have to disagree. While Britannica has been around for more than 250 years, the Wikipedia has been around for scarcely more than 6, and in that short time, Wikipedia has become by far the more relevant reference for the work I do and the things I want to learn about. I have found Wikipedia to be a great way to learn about nearly every topic that strikes my fancy: acoustics, aperiodic tiling, digital filters, analog synthesizers, 3D rendering techniques, and military history (including the history that’s being written as I write this) and climate change. As much as I enjoyed reading Britannica articles when I was a child, the Wikipedia and its cousin, the MIT Open Coureware Project, have truly become my resource of choice for knowledge in the 21st century. And Wikipedia’s constant growth (in terms of breadth, depth, and readership) proves to me that it has become the competitively superior product, Raymond’s comments to the New Yorker notwithstanding. Which one do you really think will be more relevant 6 or 10 or 20 years hence? For me, Wikipedia, because when I have something relevant to add, I add it! But it is in the last part that Nicholas goes farthest off the track, where he completely misaprehends the fundamental distinction between open and proprietary mindsets. When the quantum physicist looks at a single atom in isolation, is the atom a solid, a liquid, or a gas? The question is meaningless. A single developer is not, a priori, a Cathedral or a Bazaar, which is why Raymond introduced a third term: a Wizard. Now, whether the Wizard intends to conjure either a Cathedral or a Bazaar, that’s an interesting question. But to analyze a small number of people and say “the smallness defines the type” is as absurd as saying that “George Washington, prior to the Second Continental Congress, was just like Caesar.” It misses the entire point of the American Revolution. A far more instructive way to understand the question of how creation takes place among the few is to read the case study of Herbsleb and Mockus comparing and contrasting open source and proprietary software development projects. What they found was that both were Wizard-heavy: the top developer in both cases added, deleted, or changed about 20% of all the code, and the next 4-5 developers did the next 30%-50% of everything. (If all could contribute 20%, every software project would be fully complete and functional with just 5 people.) But then they observed a tail–a long one in the case of Apache, and a surgically shortened one in all the proprietary software projects. In other words, the proprietary projects were self-limiting compared with the inclusive model of open source. The conclusion that Herbsleb and Mockus derive, which fits with the years of experience I have talking with developers all over the world, is that the long tail of open source is strictly a benefit, not a bug, and that conversely, the necessary absence of the long tail in proprietary software development is strictly a defect, not a feature. Thus, Nicholas should not have concluded “In other words, keep the bazaar out of the cathedral.” but rather “keep the bazaar from becoming a cathedral.” And the way to do that is to start with a Wizard who is interested in creating bazaars, not defending their cathedrals.