Build your own Linux Cluster

I only used open-source software to build the cluster.  So everything 
has been downloaded from the web.  Disclaimer: I did this back in 2006, 
so versions will have changed, and procedure might also have become simplified.
The aim of this page is only to give a general idea!

Hardware Requirements:
----------------------

Essentially any linux box with reasonable amount of RAM and hard disk would do.
To make life easier, it is best if one uses standard hardware and if all the
compute nodes are identical.  Though this is not a necessity, heterogeneous
clusters are very much a reality.

Software Requirements:
----------------------

 Operating System - LINUX.  I used CentOS 4.3
 TORQUE - Terascale Open Source Resource and Queue Manager. I used version 2.1.2. 
 MAUI - Cluster Scheduler
 NIS 
 NFS 
 OpenMPI - MPI complier for distributed computing


That's it!  The core software is TORQUE, which is a open source fork of the PBS.


Head Node Configuration:
------------------------
Step 0:  Assuming the head node is installed with the LINUX operating system.

Step 1:  Start NIS service.

Step 2:  Start NFS service.

Step 3:  Download TORQUE (tar.gz) file, and uninstall into /usr/local/

	cd /usr/local
	tar -xvzf torque-2.1.2.tar.gz
	cd torque-2.1.2
	./configure
	make
	make install
	make packages

	./torque.set root
	cd /var/spool/torque/server_priv
	vi node
	   #Add node names one per line, for example:
		node1
		node2
		node3

	#To start PBS server, execute the command:
	pbs_server	#starts server

	#Other PBS commands
	qterm -t quick	#shuts down server
	pbs_mom		#starts pbs client daemon.
	pbs_sched	#Starts native Scheduler. NOTE this is not required if 
			 you are going to use MAUI.

	pbsnodes -a 	# check node status  



Client Node Configuration:
-------------------------

STEP 0:	 Make the node a NIS and NFS client of the Head node.

STEP 1:	 Copy these files "torque-package-mom-linux-i686.sh" and
	 "torque-package-clients-linux-i686.sh" to the clients.

STEP 2:	
	./torque-package-mom-linux-i686.sh --install
	./torque-package-clients-linux-i686.sh --install

STEP 3:
	vi /var/spool/torque/mom_priv/config
	#Add the following two lines
	$pbsserver	headnode
	$usecp 	*:/home1	/home1


	#here /home1 is the common NFS directory, exported by the headnode.  
	# The user home directories
		sit in this partition.

STEP 4:	
	Make sure the client host name and the server host names are given
	in the /etc/hosts  file.

STEP 5:	Start the PBS client daemon
	pbs_mom


OpenMPI Configuration:
----------------------
  In order to make full use of the cluster for distributed computing, you 
need to install OpenMPI.  
In Server:
	cd /usr/local/openmpi.x.x.x/
	./configure CC=gcc CXX=g++ F77=g77 FC=gfortran --with-tm=/var/spool/torque --disable-shared --enable-static
	

	The following line needs to be added to the user .bashrc
	export  LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib/"

In clients:
	cd /usr/local/openmpi.x.x.x/
	./configure --disable-shared --enable-static


	

SUBMISSION of JOBS:
------------------
  In order to submit a job to the cluster, you have to make use of a "batch script".  Essentially , this is a file containing the details of your job, how many
nodes it requires, how much memory, how long it will take, etc..  A sample script can be found here.


Essentially, that's it!!

The part that I have skipped is the automated client 
installation.  That means you install one "golden" client and using some
software (which I don't deal with here) copy it completely into the other 
clients.  It can be pretty tricky.  So if you have only a few clients, and
just want to get a feel for the clustering, then the shortcut method is to 
manually install all the clients!

Back to my Linux Page