Cathedral And The Bazaar Essay, Research Paper

In his essay, The Cathedral and the Bazaar, Eric Raymond says:

Perhaps in the end the open-source culture will triumph not because cooperation is morally right or software “hoarding” is morally wrong…but simply because the closed-source world cannot win an evolutionary arms race with open-source communities that can put orders of magnitude more skilled time into a problem.

Probably the best way to begin, is by giving a little background into the man who wrote this quote. While researching this paper, the following quote was found. It seems to describe Eric Raymond well.

Eric S. Raymond is a wandering anthropologist and troublemaking philosopher who happened to be in the right place at the right time, and has been wondering whether he should regret it ever since.

He has been involved with Internet and part of the hacker culture since the 1970’s. Several of his projects are now carried by all of the major Linux distributions. This includes fetchmail, and his contribution to GNU emacs. Also, his essay, “The Cathedral and the Bazaar” is considered to be the catalyst that lead to Netscape opening up its browser’s source code.

In some ways the first half of the opening quote is rather meaningless. It seems unfathomable that anyone could dispute the fact that the more people you have working on a problem, the quicker it will get fixed. If a company such as Microsoft could have 5000 employees working on the same problem at the same time, we would likely never see buggy software come out the door again. At the same time, one has to argue whether open source (OS) model is one that can be profitable. The software industry is a profit driven industry, so it is debatable that sharing on such a high level is beneficial for any industry that is driven by profit. However, large organizations are beginning to embrace the OS movement. There must be some benefit for an organization to open up some or all of their source code or else it wouldn’t be happening. Netscape cites “The Cathedral and the Bazaar as being pivotal in convincing Netscape management to release the Mozilla source code. Netscape’s underlying feeling about going OS, was:

Since Communicator’s code was so tightly integrated with Java and HTML, most recognized an emerging truth: It wasn’t such a huge jump to make.

In a press release released by Netscape, Jim Barksdale, president and chief executive officer said:

“By giving away the source code for future versions, we can ignite the creative energies of the entire Net community and fuel unprecedented levels of innovation in the browser market. Our customers can benefit from world-class technology advancements; the development community gains access to a whole new market opportunity; and Netscape’s core businesses benefit from the proliferation of the market-leading client software.”

Another major player in the technology industry that appears to support the OS movement is Hewlett Packard. At a recent keynote address at the Networld+Interop Conference, HP’s CEO, Carly Fiorina stated:

The open source movement is natural, inevitable and creates huge benefits. It’s part of the next wave of computing, and that will involve participants and users within the industry in open source.

This is more proof that large organizations are seeing the benefits if the OS movement. And, as more organization jump on board, others will follow, even if it’s just for fear of being left behind. Of course, whether HP stands behind that statement and supports these ideals remains to be seen.

Make no mistake, large organizations like HP and Sun are not exactly excited about the situation. Sun is getting nervous enough that they announced that Solaris will now be available free for non-commercial use. The OS concept is destroying their current licensing models. However, they have placed restrictions on their source code that hamper its effectiveness. As Linus Torvalds commented, “a window is not open just because you can see through it.”

Obviously the company with the most to lose is Microsoft. A decent program interface on these open source products could pose a new threat to the server operating system empire Microsoft is currently building. One would have to agree that if the giant machine known as Microsoft Corporation deems the Open Source movement, and more specifically Linux as a threat to its business, then it is obviously something to be taken seriously. On October 31st 1998, an internal Microsoft memorandum was leaked. This document, then dubbed “the Halloween Document”, outlined the threat that Open Source Software and in particular Linux, could be to Microsoft Corp. The following is an excerpt from the executive summary of the “Halloween Document:”

Open Source Software (OSS) is a development process which promotes rapid creation and deployment of incremental features and bug fixes in an existing code / knowledge base. In recent years, corresponding to the growth of Internet, OSS projects have acquired the depth & complexity traditionally associated with commercial projects such as Operating Systems and mission critical servers.

Consequently, OSS poses a direct, short-term revenue and platform threat to Microsoft — particularly in server space. Additionally, the intrinsic parallelism and free idea exchange in OSS has benefits that are not replicable with our current licensing model and therefore present a long term developer mindshare threat.

This “Halloween Document” was proof that Microsoft is aware of the potential “threat” of OS and Linux. A movement that is deemed a threat to the worlds largest software company is obviously something to be taken seriously. In fact, even after the hype of the “Halloween Document” had died down Microsoft chose to re-open the “war” putting up a page on its web site, which exposes “Linux Myths” . The open source movement must be doing something right.

One interesting statistic says that;

“Open source UNIX is now running over 50% of the servers on the Internet. The Apache HTTP Server is the most popular Internet server application today, also holding a commanding 50% Internet share.”

Of course, Microsoft is not the only entity that would like to disagree with Eric Raymond and his view of why the OS model works. The opening quote in this paper echoes the sentiment of another section of The Cathedral and the Bazaar. Raymond refers to an idea attributed to Linus Torvalds, that “Given enough eyeballs, all bugs are shallow.”

Raymond expands on that quote by stating that;

In the bazaar view, on the other hand, you assume that bugs are generally shallow phenomena – or, at least, that they turn shallow

pretty quick when exposed to a thousand eager co-developers pounding on every single new release.

Some people would disagree with the concept that all you need is to throw a multitude of developers at a problem, and they will fix it. In the opinion of Jamie Zawinski, formerly of the Netscape and Mozilla project, the majority of open-source projects (including Linux) usually have a small core group of hardcore coders and developers working on a project, and a large number of people who contribute a small amount here-and-there:

If you have a project that has five people who write 80% of the code, and a hundred people who have contributed bug fixes or a few hundred lines of code here and there, is that a ‘105-programmer project?’”

Basically what is comes down to is that there has to be a balance between the Bazaar and Cathedral style. A pure Bazaar style is not really sustainable, because their needs to be some sort of control on the whole project. Eric Allman states:

In my experience, the “cathedral” and the “bazaar” are endpoints on a continuum, and neither really works well in its pure form. Successful bazaar products have a “pope” of some sort, an arbiter of good taste, if nothing else. This leader may be a small group (such as the Apache core team) or an individual (such as Linus Torvalds), but without a person or small team providing focus, such projects tend to splinter. The bazaar model can fail when software authors graduate or change jobs or just get bored. There isn’t always the stick-to-it ethos that a company can engender.

But the cathedral has its own set of problems. Companies may fail, cut off low-performing product lines or be acquired by companies that chop out competing products, which leaves customers in the lurch. I’m hoping that the hybrid model will give us the best of both worlds.

Nikolai Bezroukov, a Senior Internet Security Analyst at BASF Corporation feels there are four questions that come up in regards to the OS theory that “Given enough eyeballs, all bugs are shallow.” Bezroukov’s theory would fall more under the category of “too many cooks spoil the broth.”

o Is this the best way to utilize talented developers?

Why waste their skills on debugging? In some commercial environments, professional testers provide an important edge. Improved project management could be more beneficial than throwing ten programmers after a single bug.

o Are all bugs created equal?

There are at least three major types of bugs – code errors, logical errors and architectural problems. The problem of stagnation of early architecture solutions, wasting time and effort in debugging instead of solving underling architectural problems, is common to OSS projects.

o Is it easy to force gifted programmers to search for the same bug?

Talented developers are first programmers not testers; they usually prefer making their own bugs to fixing bugs of others.

o Does it make sense to fix bugs in ugly source code?

If the source is ugly (and as Ken Thompson pointed out, some parts of Linux are) additional bugs could be easily introduced by fixing an existing one. Rewriting, not fixing, is a more viable option here.

Raymond felt so strongly about Bezroukov’s interpretation of The Cathedral and the Bazaar that he felt it necessary to respond to it. Raymond’s web site states the following:

Nikolai Bezroukov’s article in First Monday, unfortunately, adds almost nothing useful to the debate. Instead, Mr. Bezroukov has constructed a straw man he calls “vulgar Raymondism” which bears so little resemblance to the actual content of my writings

and talks that I have to question whether he has actually studied the work he is attacking. If “vulgar Raymondism” existed, I would be its harshest critic.

I tend to side with Raymond and Torvalds on this issue, because I agree with Raymond in regard to the fact that if “Linus’s Law” was false, then the Linux kernel would have fallen apart from all of the unforeseen interaction between new code and these undiscovered “deep” bugs. Bezroukov states, “..additional bugs could be easily introduced by fixing an existing one. Rewriting, not fixing, is a more viable option here.” This does not make sense. Even in the closed source world, you do not usually rewrite code rather than fix. Does Bezroukov expect software to be completely rewritten every single time a bug is found? Clearly, if a bug is caused by really bad code architecture then it may need to be rewritten, but only if that is the case. One good point that Bezroukov does make in his paper is concerning Raymond’s statement:

When you start community-building, what you need to be able to present is a plausible promise. Your program doesn’t have to work particularly well. It can be crude, buggy, incomplete, and poorly documented. What it must not fail to do is convince potential co-developers that it can be evolved into something really neat in the foreseeable future.

Bezroukov argues that this so-called open source concept is common practice in the closed source world. Microsoft has been releasing buggy software for years on the premise that it’s ok, as long as it looks as thought it has potential to be a standard setter or useful product somewhere down the road. Just get it out to the public, fix it later.

In the end, Eric Raymond’s quote is hard to dispute. As with any problem, the more people you have working towards finding a solution, the more likely and quickly you are to succeed. The real challenge in an open-source project is finding a balance between a pure Bazaar system, and the so-called closed source system. A pure Bazaar style system is not sustainable because every project needs a core person or group to keep the project from flying apart. Someone has to have final say in the project development, especially when dealing with core aspects of the project such as a kernel. Linus Torvalds is that person for Linux. Projects like Linux and Apache are making waves in the software industry and open source is gaining momentum. Large organizations like Netscape, Sun and HP are testing the waters, and Microsoft seems to be on the attack to protect its monopoly. Whether open source continues its momentum, or reaches a less spectacular then anticipated peak remains to be seen.