Crash Test Dummies

Beta Blues

       They just don't make beta software the way they used to.
       That's the conclusion that I've reached over the past few months after playing with the beta versions of Netscape's Navigator, Microsoft's Internet Explorer and dozens of plug-ins that unashamedly crash my system at the slightest provocation.
       In the old days (before computer companies started distributing their programs on the Internet), most people agreed on what the word "beta" meant. Software developers followed more or less the same cycle of creating and releasing their programs to the general public. Back in those days, most people wouldn't be caught dead running beta software.
       First came the prototype. A programmer or two would hack for a few weeks and come up with a model of what they wanted to build. The prototype might be slow; it might crash. It might even be written in Macromind Director. It didn't really matter: The purpose of the prototype wasn't to get customers, it was to get ideas and possibly funding.
       Successful companies threw away their prototype and started developing a 1.0 release. This new program would take shape slowly: a menu here, a new option there. Eventually, a skeletal application would emerge that sort of worked. At some point, the programmers would create an "alpha" release. The alpha would be formally distributed within the company and perhaps shown to a few trusted clients (but never left for them to use). Most feedback would focus on the new features that people wanted to see.
       Next would come the "feature freeze," in which the product's menus, commands, options and so forth were finalized for the 1.0 release. From this point on, the main goal of the programmers was to fix bugs.
       Beta followed the feature freeze. A beta was a special version of the program distributed to the company's favorite customers, whose mission was to help the company find bugs in return for early access to the technology or for a substantial discount. Beta testing was hard work: some companies even paid their beta testers. A friend of mine, Pascal Chesnais at the Massachusetts Institute of Technology Media Lab, once won a color inkjet printer from a computer company because he found more bugs in their beta release than anybody else. (Icame in second.)
       At the end of the beta period, programmers would categorize, rank and fix all of bugs that had been uncovered. Fixes would start, but at some point, the company would institute a "code freeze," after which no more changes would be made in the final shipping program. Bugs would still be reported and ranked, but instead of being fixed, they would merely be documented in the "Release Notes."
       Why not fix a bug that somebody knows about? Sometimes, when you fix one bug, you actually introduce another. Unlike a hastily-written patch, a hastily-written set of release notes won't crash if somebody makes a stupid mistake, like leaving out an important comma. Freezing code avoids tempting fate.


       That's the way software used to be developed. The Internet has fundamentally changed the software and the whole process. Gone are the traditional notions of prototype, alpha, beta and release. Instead, now we have a coding frenzy and a steady stream of "betas" available by FTP.
       These days, the very first words of advice given to any startup -- "throw away your prototype" -- are wrong. Companies that take the time to reinvent their core technology before hitting the market shrivel up and die, steamrollered by others who decided to hit the market instead of taking the time to "do it right." These hacks then use the revenue and mindshare that they got from that first burst of publicity to fund further development of their product. It's become a winning strategy to ship inferior code, provided that you can follow it up with a slightly improved version in a few weeks' time.
       The alpha -- that is, the first version of the program that sort of works -- is now put up for download. Why hire a staff of quality assurance testers when there's a volunteer army on the Internet who will do your testing for free? Today's betas are really alpha-quality code, and that nifty Netscape plug-in you download will probably crash your computer after it has run for a few minutes (just long enough to get you hooked).
       As things turn out, today's 1.0 releases are really just alphas too. The industry has lost the concept of "feature freeze" and "code freeze." Just look at the so-called betas for Netscape's 2.0 (and now 3.0) Navigator. With each new "release," fundamentally new features were added. JavaScript! Java! Certificates! But realistically, there's no way to fix bugs when you constantly add new features.
       With the pace of the industry today, anybody who code freezes should have their head examined: Why document your bugs, when users are too lazy to read documentation? If somebody has a problem, just tell them to download the newest version! It won't fix their problem, but who cares? Look at all the new features.


homeback to archives