| "Beowulf is just starting. We haven’t yet seen what we can get out of it." |
The PC revolution put a computer on everyone’s desk. Now, the open source revolution is making it possible to give supercomputers to the masses. With both a no-cost open source operating system in Linux, and the open source Beowulf clustering system, virtually anyone can own one. "All you need is a few nodes and a basic network to plug them together," says Dr. Walter Ligon, associate professor in the Electrical and Computer Engineering department at Clemson University in Clemson, S.C. "100-bit networking and switched hubs are cheap enough that students in a dorm can build a Beowulf, and they’re doing it."
| "So far this year, Atipa has done 30% more Beowulf business than it did in all of 1999." |
When he’s not advising would-be supercomputer manufacturers, Ligon helps maintain the Beowulf-underground.org portal, and is putting the finishing touches on a proposed Beowulf system for his department that will have over 500 CPUs running Beowulf, to test new software as well as to work on projects like plant genome sequencing. And like his not-yet-installed system, Ligon says the best is yet to come for Beowulf users. "Beowulf is just starting. We haven’t yet seen what we can get out of it."
Good for business
Beowulf may be in its infancy, but it is a baby that’s getting a lot of attention. Even companies like Compaq Computer Corp., which has its own high-end clustering systems — thanks to its acquisition of Digital Equipment Corp. — aggressively markets Beowulf. While Compaq will gladly sell a customer TruCluster for Tru64 UNIX, Beowulf systems, particularly those based around the floating point-friendly Alpha chip, have been good business for Compaq. Some customers are using "literally thousands of Compaq Alpha servers with Beowulf," says Glenn Johnson, Director of Compaq’s Linux Program Office.
Compaq’s participation in the Beowulf market is as easy to understand as the appeal of the clusters to its customers. "Any industry that has a computational problem that can be parallelized is looking at Beowulf because you can assemble a hell of a lot of gigaflops for not a lot of money," says Johnson.
But Beowulf’s price and open source advantages aren’t always enough to win the business. To some extent, its open source development and distribution model have hindered its acceptance outside the academic and pure research realms. Even within those traditionally strong Beowulf markets, some major projects have opted for traditional supercomputers instead of Beowulf. Mark Seager, principal investigator for Lawrence Livermore Laboratory’s ASCI White supercomputer says that Beowulf wasn’t in the running for that system. ASCI White runs on RS/6000 servers, AIX operating system software, and Parallel Environment cluster code, all from IBM. At 8,192 processors, it is one of the world’s largest supercomputers. Seager says that the "vendor solution" IBM could provide outweighed any possible advantages of a Beowulf-based supercomputer. "Beowulf is not a cohesive product," he says. "It’s not what we would call production quality."
Companies such as Atipa Linux Solutions, Linux NetworX Inc., and VA Linux Systems Inc. are trying to remedy this, in part by providing end-to-end support for the Beowulf systems they sell, from hardware to OS to Beowulf itself. Still, even Beowulf founding developer Donald Becker says there is something to Seager’s argument. Becker, who was part of the team that developed Beowulf at NASA beginning in 1993, says that when customers are looking for an off-the-shelf supercomputer, Beowulf is at a disadvantage.
The lack of integrated turnkey solutions has been "a real problem" for Beowulf, he says. Not every company interested in supercomputing has a resident expert. "In the academic and scientific community, there are a lot of people who can construct and maintain their own Beowulf systems, but people who do that are rather specialized, and you don’t find that in the broader commercial world."
But some vendors are now seeing corporate customers who are taking a serious look at Beowulf for their "heavy hardware" needs. "We are seeing a big shiftit’s the product lifecycle," says Brad Rutledge, public relations director for Linux NetworX. "We think Linux as well as clustering is going from the techie to the early-adopter stage," attracting companies that can afford to take technology bets in exchange for the price advantages and power of Beowulf systems.
And while no one can tell for sure, since the necessary software can be downloaded from the Internet for free, Beowulf clusters appear to be growing in number. Becker estimates that there are now thousands of Beowulf clusters, ranging from hobbyist 4 and 8-computer setups to monster systems with hundreds or thousands of nodes.
Three of these large systems are the "Los Lobos" Beowulf clusters, funded by the National Science Foundation. Like the ASCI White supercomputer, the NSF turned to IBM for hardware, but rather than paying top dollar for AIX and IBM Parallel Environment, Los Lobos will run Red Hat Linux and Beowulf. The first system, rolled out at the University of New Mexico, consists of 256 IBM Netfinity servers. Assuming all goes well, additional clusters will be deployed at Argonne National Laboratory, of Argonne, Ill., and the University of Illinois.
The Los Lobos systems will require additional software to accomplish their goals of modeling everything from quantum particle behavior to weather systems, but much of the new software will be released back into the Beowulf and Linux open source communities, including better I/O and filesystem interfaces for Beowulf. Project managers hope to make the three clusters available on demand to their research clientele, allowing a remote researcher to start a job on the cluster best suited to their needs.
Real-world Beowulf
Atipa Linux Solutions is pulling customers in from beyond academia. The company characterizes Beowulf systems as a "significant percentage" of its revenues, and counts Bloomberg L.P., Conoco Inc., Exxon Mobil Corp., and Motorola Inc., as well as scientific and research institutions, among its Linux-cluster clients. "I deal with Beowulf clientele on almost a daily basis," says David Grant, Atipa sales manager. Last year, Atipa’s sales organization was only able to come up with two or three Beowulf leads per month. Now, they’re pulling more in every week.
Atipa, for example, is working with a large telecommunications client interested in a Beowulf cluster for record processing. Like many newcomers to Linux clustering, the telecom company is buying a cluster with only a small number of nodes at first. That’s fine with Atipa: if the system is a success, it expects follow-on orders. "A lot of the industry is in trial mode, and that’s the beauty of clustering: you can trial it," and add extra nodes as the need increases, said Maria Goulet, Atipa’s director of marketing. All of that business has translated to year-to-date Beowulf revenues for Atipa 30% greater than the entire ’99 total, with even better performance expected for the next six months.
Beowulf business was so promising for Alta Technology Corp., of Sandy, Ut., primarily known as a system board manufacturer, that it formed Linux NetworX, a subsidiary dedicated to Beowulf cluster sales and support. The company has sold several major clusters, including a 256-CPU setup, and a 128-CPU cluster used by Lockheed Martin Corp. Linux NetworX also develops and markets an easier-to-use management suite for its Beowulf clients. Dubbed ClusterWorX, the program is currently only sold as part of a Beowulf hardware package, and is proprietary, although customers do receive source code for their internal use only.
As for the mainstream open source Beowulf package, look for a new revision to be released through Scyld Computing Corp., of Annapolis, Md., by fall that incorporates an easier installation system, new graphical administration tools, and tighter integration with the Linux kernel. With the new software, Beowulf administrators will be closer to the experience of managing a unified supercomputer from a single terminalusers will be able to run standard system status tools and see all of the cluster jobs as though they were truly managing a supercomputer instead of a stack of PCs acting in concert.
One direction you won’t see Beowulf going, if Donald Becker has anything to say about it, is across the greater Internet. Becker now maintains and manages the Beowulf source code while providing commercial development and support for Beowulf systems through Scyld Computing. While distributed computing projects such as SETI@Home have people excited about the prospects of a true Internet supercomputer, Becker says that Beowulf will stay squarely focused on linking machines on private, high-speed, high-reliability LANs. According to Becker, accounting for the latency and security risks over the Internet makes an Internet Beowulf cluster unworkable. "That security translates into a lot of overhead," he says.
Keep in mind that Beowulf isn’t even the only open-source game in town. While Beowulf is generally considered the leader in high-performance cluster computing for Linux, projects such as Linux Virtual Server and Kimberlite, from Mission Critical Linux, focus on "high-availability" applications, such as Web and database servers. In the more commercial space, TurboCluster from TurboLinux looms large in high-availability, and it remains to be seen what Caldera will do with SCO’s UnixWare NonStop high-availability clustering softwarea Linux port would certainly not come as a shock. Beowulf, like its heroic namesake, will have to remain vigilant.
Jason Compton is an Evanston, IL-based technology journalist. He is a regular contributor to the Chicago Tribune, Linux Magazine, and Smart Business.
