Prepare for your Cluster Build

Today, clusters can be built by hobbyists and professionals alike. Cluster sizes can vary from the small, consisting of only 2 nodes to the very large with 100's of nodes.

Why Build a Cluster

Computers are getting faster all the time but most applications still do take advantage of single CPU systems.

Moores Law

• The observation made in 1965 by Gordon Moore, co-founder of Intel, that the number of transistors per square inch on integrated circuits had doubled every year since the integrated circuit was invented. Moore predicted that this trend would continue for the foreseeable future. In subsequent years, the pace slowed down a bit, but data density has doubled approximately every 18 months, and this is the current definition of Moore's Law, which Moore himself has blessed. Most experts, including Moore himself, expect Moore's Law to hold for at least another two decades.
  • Taken from: http://www.webopedia.com/TERM/M/Moores_Law.html

To keep up with and even exceed the speed of todays CPUs, clusters can be built to take advantage of using many CPUs across a network, with more and more software developers designing their applications to use SMP technology.

Symmetric Multi-Processing

• A multiprocessing architecture in which multiple CPUs, residing in one cabinet, share the same memory. SMP systems provide scalability. As business increases, additional CPUs can be added to absorb the increased transaction volume.
  • Taken From: http://support.softwareshelf.com/dictionary/default.asp?l=s

Using various opensource applications, a network of 2 or more computers can act as if its a SMP supercomputer and upgrading the speed is as simple as adding another node onto the network.

openMosix

• openMosix allows you to join together multiple computers running the Linux operating system, and have them appear to the user as one large multiple-processor computer. For example, suppose you had two computers, A and B joined in an openMosix cluster. Without openMosix, if you ran two programs on A they would only get 50% of the CPU time each. With openMosix, one of the programs could migrate 'automagically' to B, so both processes would run at 100% CPU. As far as the user is concerned, A now behaves like a two-CPU SMP computer with twice the CPU power available.
• For example, if your computer could convert a WAV to a MP3 in a minute, then buying another nine computers and joining them in a ten-node openMosix cluster would NOT let you convert a WAV in six seconds. However, what it would allow you to do is convert 10 WAVs simultanously. Each one would take a minute, but since you can do lots in parallel you'd get through your CD collection much faster :).
  • Taken From: http://howto.x-tend.be/openMosixWiki/index.php/FAQ

A process running on the cluster could only become as fast as the fastest node on the cluster but clustering allows you to run multiple processes off one of the nodes and then migrate them across the network.

Make Room

Putting the cluster together can consume space. For a very simple cluster, 2 desktop PC's in 2 rooms in your house can be used and clustered together. Each computer can pass processes to the other computer BUT the disadvantage of just 2 desktop nodes become apparent while you are running processes on your desktop and your work collegue or partner decides to do the same on the other node.

Then It Begins

Scalability is the key word... You add another node, then another, then another...

Some initial thought in where to keep these nodes becomes apparent.

Power Requirements

Each node within the cluster needs power.

Although you do not need to have all the cluster powered up all the time if you dont want to, It is very important that mains wiring is able to stand the cluster running for long periods without causing a fire hazard.

Getting Warm

Running all those nodes produces heat.

Good ventilation is needed as a large cluster can become the equivalent of running multiple electric heaters on full power even during the summer.

My combined systems have kept my room above 23C throughout this winter.
I actually clocked up a temp of 27C today, and we're still in early spring
--Fremen 22:17, 18 Mar 2005 (GMT)
got up to 30C today while it was 27C outside
--Fremen 20:39, 27 May 2005 (BST)

Oooh The Noise

Nodes can be noisy.

If you are building a large cluster at home, the whirring of fans (expecially in older hardware) could drive you mad. (do not attempt to build a giant cluster in your bedroom unless you can put up with the noise)

Repairing/Upgrading a Node

Computers inevitably break down, and need upgrading from time to time.

Even stacking computers one on top of another can cause problems. If you need to access a node low down in your stack, will you need to power down the entire stack to gain access to dismantle and perform any repairs/upgrades necessary!

The solution to this would be to either use industry standard 19inch rackmount units, and invest in a rackmount cage, or to design and build your own enclosure that enables you to access each of the machines you will be using in your cluster separately without disrupting the others' operation.

Happy Clustering

If the above has not put you off, then clustering is for you. Good luck with building your new cluster.

With even more clusters appearing, opensource software developers are creating more and more applications to take advantage of clustering. Why not help by becoming a member of the OSCG.