It seems that everyone today is either running a cluster or wants to run a cluster. One of the more popular operating systems for implementing clusters is Linux. Linux offers the performance and cost benefits that other operating systems such as Solaris and UnixWare can not compete with.
Clustering and its Benefits
A cluster is a series of multiple machines that are designed to work together as one. As an example, you could purchase a 4 CPU machine from Penguin Computing for approximately $20,000, or you could build four separate machines that basically equate to a $20,000, 4-Way SMP machine for $8,000. The upside to a Linux cluster is the ability to build large powerful machines using commodity hardware. The downside is that your applications have to be written with specific libraries such as MPI and PVM.
If you are writing a custom application, using alternate libraries is fine; but what if you are running a stock application such as Povray for 3D rendering? What if you are running a high-end threaded Java servlet application for e-commerce? What if you want to increase the development and decrease compile times of a large project? In all these cases, a linux server cluster can help you out.
This is where MOSIX comes in. The MOSIX cluster is designed to run like an SMP machine. It utilizes a “fork and forget” method of dealing with large nodes. MOSIX handles the optimizing of the resources across multiple machines.
MOSIX is an open source project that runs on Linux. The current supported version of the kernel is 2.2.16. It will compile cleanly on a vanilla system and is most widely tested on RedHat and SuSe. It can handle up to 65,535 individual nodes. Mosix is available from its Web site, http://www.mosix.org. It is a small download including the source code to patch your Linux kernel and the secondary programs such as tune and mon.
The MOSIX installation process is very simple. The only real requirement is that the kernel is still a virgin kernel source. In other words, you have a source that is not modified from the original Linux distribution of the kernel. Even if you installed the kernel source from your distribution, I suggest getting a new kernel source from http://www.kernel.org. After unpacking and installing the new kernel source, you can proceed with the MOSIX installation.
The MOSIX installation is based on a file called mosix.init. The file will be in the same directory as the unpacked MOSIX distribution. I was surprised when I executed mosix.init. The program, although terminal-based, is straightforward. The first question it asks is the type of installation you would like to perform. Since I had not installed MOSIX before, I chose the Compile and Install MOSIX from Scratch option.
After selecting the option, it will ask you some other questions, such as where your kernel source is located. It also prompts you for the type of configuration you would like to use to configure your kernel. As I don’t have TCL/TK installed, I choose menuconfig. Once you select your configure option, mosix.init will spawn the correct configuration program.
When menuconfig starts, the first option on the screen was the choice of MOSIX parameters. After selecting the MOSIX option, you have the choice to select each of the parameters that are applicable to the MOSIX kernel patch. Every option is described adequately enough with the online help documentation that is part of menuconfig.
After you complete and save your kernel configuration, the mosix.init program will take over again. It provides a nice little status indicator on the kernel compilation and the compilation of the external user space utilities.
Once mosix.init has completed its tasks, it installs the software and drops you back to the prompt. This is the one part where I had trouble. The software walks you through everything so well that I felt like I had done something wrong. It had not yet asked me what machines I wanted to be part of the cluster, nor was I sure what else it had done. Of course, it was all explained in the README files–files that I hadn’t read yet.
As I am never content to just trust software, I also performed a manual installation. The manual installation is basically the same, and provides some insight into why the automated version is so easy. After performing a manual patching of the kernel and configuring through the menuconfig program, the only things remaining were the file configuration and the user space utility compilation.
The basic configuration of MOSIX was simple. I only had to modify some standard system files such as /etc/inittab and /etc/inetd.conf. The reason I edited these files was to make sure that the services I did not want to cluster did not get migrated. Migration is the term that MOSIX uses to describe a process that is moved (migrated) to another node within the cluster. If a process is migrated, the node it is migrated to takes responsibility for processing the request. Therefore, services such as shutdown should not be migrated.
I had recompiled the kernel and a reboot was in order. The reboot went as expected until the very end. At the end of the boot, right before it asks you for login, a little program popped up and said that I had not placed any hosts into the /etc/mosix.map file. It then asked me which editor I would like to use to edit this file. After selecting my editor (joe) the file opened with plain English instructions on how to enter the correct data. I placed my machine and our development machine, which I had not yet installed MOSIX on, into the file. After saving the file, the machine came up clean.
Installing MOSIX on the Development Machine
From this point, I began to install MOSIX on a development machine that’s a completely different machine in terms of hardware. My machine is a K6/2-550 with 256 megabytes of RAM. The development machine is a Intel Celeron 333 (overclocked to 405) with 128 megabytes of RAM. I expected at least one bump in the installation process, but was denied the satisfaction. The installation went just as clean as the first installation. Again, on reboot it asked me to edit the /etc/mosix.map file. Again, it came up clean to the login prompt.
I was at a loss. I actually just sat in my chair staring at the machine for a minute. I had to have forgotten something. I logged into my machine (the K6) and proceeded to fire up KDE2 final beta.
Testing the Installation
MOSIX comes with several user space programs that are designed to allow you to monitor, administer, and tune the MOSIX cluster. The simple but effective mon program is used to monitor the utilization of each node via a curses-based graph. Once I had mon running, I tried to execute a couple of programs and see if mosix would migrate them to the other machine.
The program I first tried was Netscape. As Netscape is traditionally a very bulky program, I thought it would be interesting to run the Netscape process on a different CPU rather than tie up my CPU cycles. Netscape did not migrate on its own, but I was able to force the migration using the runon command.
In the picture above, my machine is number 1 and the development machine is number 3. As you can see by the graph, machine number 3 is grinding away. It is grinding away on the execution of Netscape. It didn’t last long. As soon as Netscape finished the start-up, the process migrated back to my machine. However, I was able to take the first step and migrate the process.
I then proceeded to optimize MOSIX by running the tune program. The tune program performs several optimizations to help the migration algorithms work more effectively with the different machines. I noticed something when running the program: MOSIX will require machines that have good Floating Point Units (FPU). The K6/2, although a great CPU, suffers in this arena and the Celeron easily stomped the K6/2, even though the K6/2 is 145MHZ faster and has more RAM. If you prefer AMD, I would make use the Athlon/Duron series because it has more efficient FPU.
I would have to explore MOSIX a great deal more than this article allows to truly test the limits of this software, but from the experience that I have had with it and the experience that I have read from other people’s comments on the mailing list, MOSIX has great potential. If you are looking for a way to increase your computing horsepower in a cost-effective manner for a shared environment or distributed application, I strongly suggest you look into MOSIX.
— Instead of asking why a piece of software is using “1970s technology,” start asking why software is ignoring 30 years of accumulated wisdom.
Joshua Drake is an e-commerce and Linux consultant who owns his own company, Command Prompt. He has been using Linux for almost nine years and is the Linux Documentation Project’s Webmaster. His other projects include the LinuxPorts.com Website and the OpenDocs publishing company.