Grids, Clusters, Virtualized Environment, and All of That1

Computing 101

Whatever it is that you are trying to do, one thing is certain and that is the need for more compute power is always increasing. You want to do more in less time, be more efficient, cheaper, and more reliable. If you have ‘x’ amount of compute power today, you want ‘2x’ tomorrow, and so on and so forth. The more you do, the more you realize what else you want do; thus, you crave more. This cycle forces you to come up with innovative ways of achieving your goals, whether it being multi-processor systems, multi-single processor systems, or many other solutions that have surfaced over the past few decades.

I want to briefly and rather brutally cover the very basics of computing. To decrease the overall runtime, one needs to decrease one or more of the following components:

1. Communication time
1. This is the time that it takes to get the data and the command from point A to point B. Remember, if you telling a processor to do something, first you need to tell it what that thing is that you want it to do, and the data that the command needs to run against. The time that it takes to get these two pieces of information to the processor is crucial in the overall runtime. This is especially true if you are talking about distributed systems where the components are dispersed maybe even globally.
2. Startup time
1. This is the time that it takes for the ‘manager’ to assign the CPU to do whatever it is that you are asking it to do. As you can imagine, this is probably one of the most difficult parts of the system. This is where the Operating System or the Resource Manager is usually the bottleneck. Imagine, if you will, that you have 1000s of requests and only a limited number of resources at your disposal, the Resource Manager (or also called the Scheduler) needs to handle all of these requests and ensure completion on a very limited number of resources.
2. Another scenario that I would like to pose here is that if you only have four jobs and you also had four CPUs, would you need a resource manager? The answer is No! The resource manager or scheduler is required only if you are trying to ‘timeshare’ your scarce resources among a number of users. You see the scenario I just prescribed in embedded realtime systems where you need to know when a task will run and exactly for how long.
3. Processing time
1. This is where Moore’s law comes into play. This is the classic processor problem where faster processors are sought after to decrease processing time.
2. Distributed processing is also another way of looking at this problem. Instead of one very powerful processor, you can use a large number of small processors. There is a number of problems where the sheer number of operations is slowing you down, and not necessarily the speed that a single operation takes. Processors are, generally speaking, fast enough these days that much of the focus is being diverted to multi-processing systems.

As you can see, there is more than just the processing time that will determine how long a task will run. There are many technologies that will decrease some of these components, and I have inserted an example for your viewing:

Table 1: The Three Classes of Problem Realization

There are many other vendors, and these vendors target more than one class of problem, but I have shown only the cream of the crop. The chances are that you will need to implement a solution that covers this spectrum, and that is not easy. I am talking about grand problems, and the solutions to these problems are grander than the problems themselves. One thing is certain: More unconventional methods are emerging. Solutions that were not possible a few years back or never made it out of the research labs are resurfacing due to the advance in technology, open source software, predefined interfaces, and others.

… and All of That

Depending on whom you speak with, you will get a different explanation for one or even all of these “key phrases” that you read in this article. I don’t pretend to clear up all the confusion, but rather give you a perspective what each one [might] be perceived as and why. Clusters are what I like to cover first, mainly due to the fact that clusters in the most classical² sense have been around for many years. If you have been working for a number of years, the name Tandem or Stratus rings a bell. These two computer manufacturers (Tandem now part of HP NonStop, and Stratus still in business) truly understood the concept of cluster computing. Each of the server classes that these two companies made was composed of a number (16 or more) tightly coupled computers interconnected by a communication backbone.

When Intel started supporting Multiprocessing systems with the introduction of the Pentium Pro, one needed to “cluster” four processors into one cluster and have multiple clusters connected via a bridge. Today, when you speak of clusters, you refer to mostly a large number of homogenous computers interconnected in close proximity. If you have a datacenter filled by Dell blades and they all have mostly the same or very similar Operating System version, you have built yourself a cluster. You might think that this explanation is very narrow and restrictive, but think about it for one second, is it really? If you are building a new datacenter, wouldn’t you rather have one Operating System to worry about? Wouldn’t you rather have a single type of vendor to deal with? Whether you are aiming for a cluster or cluster is the end result, this is what you have.

Don’t get confused, as is normally the case, between the meaning of a cluster and a grid. This is where things start getting very interesting as the meaning of these two concepts get a little clouded. As I mentioned, a cluster is normally sought as a homogenous environment and all in close proximity. Depending on whom you ask, one of these two features is more important than the other. For the research community mostly, as long as I am talking about nodes in very close proximity (in other words, a datacenter), I am talking about a cluster. For the commercial world, if I am talking about anything other than the true and classic definition of a cluster, I am talking about a grid. What is the reason for this? I believe the notion of cluster is annotated with reliability and resiliency; after all, the two companies (Tandem and Stratus) tht sold clustered machines had very high reliability built in. When you walk away from that level of tight coupling with off the shelf blades interconnected, you are unable to ensure 99.999% reliability (5-9’s).

This is where Virtualization comes to the rescue. The idea is that you might not be able to have 99.999% reliability at the node level, but can you achieve this level of reliability at the system level across multiple nodes? Distributed virtual environments have the capability to add reliability atop of unreliable environments.

Figure 1: Added Reliability of a Virtualized Environment

Interconnected nodes are rather unreliable. If a job is running on a node, and the node crashes, your job is left in an unknown state at best. Due to the fact that Virtualization adds a layer of indirection for you to access your environment, it can add a certain level of reliability to the environment. I have shown this in Figure 1, with the color red representing a job that is failed due to a box crashing. The virtualized environment reschedules the same job on another node (green) and job continues. You might not be able to recover your work (no check-pointing), but you can be assured that your job will be completed in face of a failure.

The added layer provided by the virtualization software can essentially keep the job (yellow) in an uncompleted state until the result returns and the job can be marked as such. There are a number of side effects to this, as you can imagine. You are adding another component to the picture and it can fail; the added component adds delay to the system, which may not be acceptable; and so forth…. The good thing is that the software or hardware packages that you get today to assist you with this problem have already tried to answer these shortcomings of virtualization with methods such as redundant scheduling where the scheduler schedules the task on multiple nodes simultaneously, synchronized node managers where a primary and backup node are in sync and if one fails the other takes over, reliable interconnects, and many other techniques.

… but Now What?

What you see today in the industry is that many emerging technologies surrounding interconnects, processing technologies, virtualized environment, and so on, are moving toward more coupled environments. The reason for this is rather simple: Web services, standardized communication protocols, open Operating Systems, open source projects, and many other factors have led this “open coupled system” movement. During much of the 70s thru 90s, performance was equated with proprietary systems such IBM’s mainframe, Sun’s powerful workstations, HP’s servers, and the like. But now, the same systems are open: Sun’s release of Solaris into open source, IBM’s introduction of Linux on the mainframe, and HP’s desire to add more to commodity processors are only some the trends that you have seen over the past decade. Many up and coming vendors are taking advantage of this situation as well. If you are able to conform to open standards and provide a common interface to our users, you can take advantage of innovative ways of providing environments that can meet the needs and complexities of the business.

Conclusion

This article covered many of the terms you hear today in relation to High Performance Computing (HPC). You need to spend a ton of time realizing what I covered in Table 1 because every single project that you come across will focus on one or more these classes. You need to realize the problem that you are tackling; otherwise, you will be spinning your wheels and money solving a problem that was not a problem to begin with.

The HPC industry has gone through many changes over the years, and it is now stronger and more important than ever. It is certain that the need for more computing power is increasing, but achieving this power at a cost that won’t break your back or your wallet is the key.

… till next time

End Notes

For those of you who suffered like I did through Vector Calculus, this title might seem very familiar to you. For those who were lucky enough to not have to take such a class, you should still check out Div, grad, curl, and all that by H. M. Schey; it was a life saver.
1. It’s funny to use the term “classical” when you speak about the computers and IT, granted that the industry itself is less than 100 years old, and that goes to show the fast growth of this industry.

About the Author

Mr. Sedighi is currently the CTO and founder of SoftModule. SoftModule is a startup company with engineering and development offices in Boston and Tel-Aviv and a Sales and Management office in New York. He is also the Chief Architect for SoftModule’s Grid Appliance product that has risen from a current need in the market to manage excessive demands of computing power at a lower cost.

Before SoftModule, Mr. Sedighi held a Senior Consulting Engineer position at DataSynapse, where he designed and implemented Grid and Distributed Computing fabrics for the Fortune 500. Before DataSynapse, Mr. Sedighi spent a number of years at TIBCO Software, where he implemented high-speed messaging solutions for organizations such as the New York Stock Exchange, UBS, Credit Suisse, US Department of Energy, US Department of Defense, and many others. Mr. Sedighi received his BS in Electrical Engineering and MS in Computer Science, both from Rensselaer Polytechnic Institute.