High Performance Linux Clusters with OSCAR, Rocks, OpenMosix, and MPI [Electronic resources] نسخه متنی

اینجــــا یک کتابخانه دیجیتالی است

با بیش از 100000 منبع الکترونیکی رایگان به زبان فارسی ، عربی و انگلیسی

High Performance Linux Clusters with OSCAR, Rocks, OpenMosix, and MPI [Electronic resources] - نسخه متنی

Joseph D. Sloan

| نمايش فراداده ، افزودن یک نقد و بررسی
افزودن به کتابخانه شخصی
ارسال به دوستان
جستجو در متن کتاب
بیشتر
تنظیمات قلم

فونت

اندازه قلم

+ - پیش فرض

حالت نمایش

روز نیمروز شب
جستجو در لغت نامه
بیشتر
لیست موضوعات
توضیحات
افزودن یادداشت جدید








8.2 Automating Installations


There are two real benefits from
an automated installationit should save you work, and it will
ensure the consistency of your installation, which will ultimately
save you a lot more work. There are several approaches you can take,
but the key to any approach is
documentation.
You'll first want to work through one or more manual
installations to become clear on the details. You need to determine
how you want your system configured and in what order the
configuration steps must be done. Create an install and a
post-install checklist.

If you are only doing a few machines, you can do the installations
manually from the checklist if you are very careful. But this can be
an error-prone activity, so even small clusters can benefit from
automated installs. If you are building a large cluster,
you'll definitely need some tools. There are many.
This chapter focuses on three fairly representative
approachesRed Hat's Kickstart, g4u, and
SystemImager.

Each of the tools described in this chapter has it place. Kickstart
does a nice job for repetitive installations. It is the best approach
if you have different hardware. You just create and edit a copy of
the configuration file for each machine type. However, Kickstart may
not be the best tool for post-installation customizations.

With image software like g4u or SystemImager, you can install
software and reconfigure systems to your heart's
delight before cloning. If you prepare your disk before using it, g4u
images use less space than SystemImager, and it is definitely faster.
g4u is the simplest tool to learn to use and is largely operating
system independent. SystemInstaller is the more versatile tool, but
comes with a significant learning curve. Used in combination with
rsync, it provides a mechanism to maintain your systems as well as
install them. In the long run, this combination may be your best
choice.


8.2.1 Kickstart


Red Hat's Kickstart
is a system designed to automate the installation of a large number
of identical Linux systems. Similar programs exist for other
releases, such as
DrakX
for Mandrake Linux and Fully Automatic Installation
(FAI)
for Debian. A Kickstart installation can be done
using a local CD-ROM or hard drive, or over a network using FTP, NFS,
or HTTP. We'll look at using a local CD-ROM and
using NFS over a network. NFS is preferable when working with a large
number of machines.


Warning! With any network-based approach, if you have problems, the
first thing to check is
your firewall setting for your
servers!

Anaconda
is the Red Hat installation program. It is written in Python with
some custom modules in C. Anaconda is organized in stages. The first
stage is an installer which loads kernel modules needed later. It is
this loader that goes to the appropriate installation source.
Finally, Anaconda has an auto-install mechanism, Kickstart, that
allows installs to be scripted via the Kickstart configuration file.


8.2.1.1 Configuration file

The first step in using Kickstart is to
create a Kickstart configuration file. Once you have the
configuration file, you'll create a boot disk and
start the installation. You have two options in creating a
configuration fileyou can edit an existing configuration file
or you can use Red Hat's Kickstart
Configurator
program to create a new file. While
the configuration program has a nice GUI and is easy to use, older
versions don't give you the option of reopening an
existing configuration file. So with the older version,
you'll need to get everything right the first time,
start over from scratch, or manually edit the file that it creates
after the fact.

Using Kickstart Configurator is straightforward.
Since it provides a GUI, you'll need to be running
the X Window System. You can start it from a console window with the
command /usr/sbin/ksconfig or, if you are using
gnome, from Main Menu Button
Programs System
Kickstart Configurator. Figure 8-1 shows the initial window.


Figure 8-1. Kickstart Configurator

Simply work your way down the lists on the left setting the fields on
the right as needed. Most of what you'll see will be
familiar questions from a normal installation, although perhaps in
slightly more detail. On the second screen, Installation
Method
, you'll be asked for the
installation methodCD-ROM, FTP, etc. The last two screens ask
for pre-installation and post-installation scripts, allowing you to
add additional tasks to the install. When you are done, save the
file.

Alternatively, you could use an existing configuration file. The Red
Hat installation program creates a Kickstart file for the options you
select when you do an installation. This is saved as
/root/anaconda-ks.cfg. (There is also a template
for a configuration file on the Red Hat documentation disk called
sample.ks, but it is a bit sparse.) If you have
already done a test installation, you may have something very close
to what you need, although you may want to tweak it a bit.

Once you have a
configuration
file, you may need to make a few changes. Often, manually editing an
existing configuration file is the easiest way to get exactly what
you want. Since the configuration is a simple text file, this is a
very straightforward process. The configuration file is divided into
four sections that must be in the order they are described here.

The command section comes first and contains basic system information
such as keyboard and mouse information, the disk partition, etc. Here
is part of a command section with comments explaining each command:

# Kickstart file
# Do a clean install rather than an upgrade (optional).
install
# Install from a CD-ROM; could also be nfs, hard drive, or
# a URL for FTP or HTTP (required).
cdrom
# language used during installation (required)
lang en_US
# languages to install on system (required)
langsupport --default en_US.iso885915 en_US.iso885915
# type of keyboard (required)
keyboard us
# type of mouse (required)
mouse genericps/2 --device psaux -emulthree
# X configuration (optional)
xconfig --card "Matrox Millennium G200" --videoram 8192 --hsync 30.0-60.0
--vsync 47.5-125.0 --resolution 1024x768 --depth 16 --startxonboot
# network setup (optional)
network --device eth0 --bootproto dhcp
# root password (required)
rootpw --iscrypted $1$ÌZ5ÙÏÍÙÑ$Ulh7W6TkpQ3O3eTHtk4wG1
# firewall setup (optional)
firewall --medium --dhcp --port ssh:tcp
# system authentication (required)
authconfig --enableshadow --enablemd5
# timezone (required)
timezone --utc America/New_York
# bootloader (required)
bootloader --md5pass=$1$Åq9erÒbE$HoYKj.adlPZyv4mGtc62W.
# remove old partitions from disk (optional)
clearpart --all --drives=had
#partition information (required)
part /boot --fstype ext3 --size=50 --ondisk=hda
part / --fstype ext3 --size=1100 --grow --ondisk=hda
part swap --size=256 --ondisk=hda

Other options and details can be found in the first chapter of

The Official Red Hat Linux Customization Guide
on the Red Hat documentation disk.


If you omit any of the required commands, the install will pause and
prompt you for that information, which is not what you want for an
automatic installation.

The second part of the
configuration file lists the packages that
will be installed. This section begins with the line
%packages. Here is a part of a sample listing for
this section:

%packages
@ Printing Support
@ Classic X Window System
@ X Window System
@ GNOME
@ Sound and Multimedia Support
@ Network Support
@ Software Development
@ Workstation Common
...
balsa
gnumeric-devel
esound-devel
ImageMagick-c++-devel
mozilla-chat
...

Often you need to list only a component, not the individual packages.
In this example, the lines starting with @ are all
components. The remaining lines are all individual packages.

The last two sections, the pre-install and post-install configuration
sections, are optional. These are commands that are run immediately
before and immediately after installation. Here is an example that
adds a user:

%post
/usr/sbin/useradd sloanjd
chfn -f 'Joe Sloan' sloanjd
/usr/sbin/usermod -p '$1$ÎgùyUDî$oyWJSirX8I0XElXVGXesG2.' Sloanjd

Note that a pre-install section is not run in a
chroot environment, while a post-install section
is.[2]
Basically, these sections provide a primitive way of doing custom
configurations. This can be useful for small changes but is awkward
for complex tasks. For more details about the configuration file, see
the Red Hat documentation.

[2] A chroot environment
restricts access to the part of the filesystem you are working in,
denying access to the remainder of the filesystem.



8.2.1.2 Using Kickstart

Once
you have the Kickstart file, you need to place the file where it will
be available to the system you are configuring. This can be done in
several ways depending on how you will boot the system. For a CD-ROM
installation, you could simply copy the file over to a floppy.

[root@amy root]# mount /mnt/floppy
[root@amy root]# cp ks.cfg /mnt/floppy/ks.cfg
[root@amy root]# umount /mnt/floppy

Reboot your system from an installation CD-ROM. (If your system
won't boot from a CD-ROM, you could create a floppy
boot disk and copy the configuration file onto it.) With this
approach, you'll need to tell the system where to
find the configuration file. At the boot prompt, enter the command

boot: linux ks=floppy

While you will be able to complete the installation without typing
anything else, you will still need to swap CD-ROMs. This probably
isn't what you had in mind, but it is a good, quick
way to test your Kickstart file.

If you want to do a network
installation, you can provide the installation files via FTP, NFS, or
HTTP. You will need to set up the corresponding server, make the
appropriate changes to the Kickstart configuration file and copy it
to the server, and create a network boot disk. (A network or PXE boot
is also an option.) If you want to do an unattended installation, you
will also need a DHCP server to provide both the IP address and the
location of the Kickstart configuration file. Using a boot disk with
an NFS server is probably the most common approach.

To set up a NFS server,
you'll need to identify a machine with enough free
space to hold all the installation CD-ROMS, copy over the contents of
the CD-ROMs, and configure the NFS server software. For example, to
install Red Hat 9, you might begin by creating the directory
/export/9.0 and copying over the distribution
files.

[root@fanny root]# mkdir -p /export/9.0
[root@fanny root]# mount /mnt/cdrom
[root@fanny root]# cp -arv /mnt/cdrom/RedHat /export/9.0
...
[root@fanny root]# eject cdrom

You'll repeat the last three steps for each CD-ROM.

To configure NFS, you'll need to install the NFS
package if it is not already installed, edit
/etc/exports so that the target can mount the
directory with the files, e.g., /export/9.0, and
start or restart NFS. For example, you might add something like the
following lines to /etc/exports.

/export/9.0     george hector ida james
/kickstart george hector ida james

This allows the four listed machines access to the installation
directory and the directory holding the Kickstart configuration file.
You'll start or restart NFS with either
/sbin/service nfs start or
/sbin/service nfs restart.

Since you are doing a network install, you'll need
to replace the entry CDROM in
ks.cfg with information about the NFS server
such as

nfs --server 10.0.32.144 --dir /export/9.0
network --device eth0 --bootproto dhcp

The second line says to use DHCP, which is the default if this
information isn't provided. While not always
necessary, it may be safer in some circumstances to use IP addresses
rather than host names.

If you aren't using PXE, you'll
need a network boot disk. It is tempting to think that, since we have
specified an NFS install in the Kickstart file, any boot disk should
work. Not so! Put a blank floppy in your floppy drive, mount the
first distribution CD-ROM, change to the images
subdirectory, and then use the following command:

[root@amy images]# dd if=bootnet.img of=/dev/fd0 bs=1440k
...

If you don't need to do an unattended installation,
the simplest approach is to copy the configuration file to the boot
floppy and tell the boot loader where to find the file, just as you
did with the CD-ROM installation. If you want to do an unattended
installation, things are a little more complicated.

For an unattended installation, you will need to copy the Kickstart
configuration file onto your NFS server and edit the boot disk
configuration file. While you can place the file in the installation
directory of your NFS server, a more general approach is to create a
separate directory for Kickstart configuration files such as
/kickstart. You'll need to
export this directory via NFS as shown earlier. If you only need one
configuration file, ks.cfg is the usual choice.
However, if you create multiple
Kickstart configuration files,
you can use a convention supported by Kickstart. Name each machine
using the format
IP-number-kickstart
where IP-number is replaced by the IP
address of the target node such as
10.0.32.146-kickstart. This allows you to
maintain a different configuration file for each machine in your
cluster.

To access the file, you need to tell the client where to find the
configuration file. For testing, you can do this manually at the boot
loader. For example, you might enter something like

boot: linux ks=nfs:10.0.32.144:/kickstart/

This tells the loader to use the NFS server 10.0.32.144 and look in
the /kickstart directory. It will look for a
file using the name format
IP-number-kickstart.
Alternatively, you could give a complete file name.

For an unattended installation, you will need to edit
syslinux.cfg on the boot disk, changing the line

default

to something like

default linux ks=nfs:10.0.32.144:/kickstart/

You might also shorten the timeout. Once done, you just insert the
floppy and power up the node. The remainder of the installation will
take place over your network.

While Kickstart does what it was designed to do
quite well, there are some severe limitations to what it can do. As a
package-based installation, there is no easy way to deal with needs
that aren't packaged-based. For example, if you
recompile your kernel, modify configuration files, or install
non-package software, you'll need to do some kind of
scripting to deal with these special cases. That may be OK for one or
two changes, but it can become tedious very quickly. It you need to
make a number of customizations, you may be better served with an
image-based tool like g4u or SystemImager.


8.2.2 g4u


Image
copying is useful in any context where you have a large number of
identical machines. While we will be using it to clone machines in a
high-performance cluster, it could also be used in setting up a web
server farm, a corporate desktop environment, or a computer
laboratory. With image copying, you begin by building a sample
machine, installing all the software needed, and doing any desired
customizations. Then you copy over an image of the disk to other
machines, causing all the added software and customizations to get
copied as well.

g4u is a simple disk image installer. It allows you to copy the image
of a computer's disk to a server and then install
that image on other machines in your cluster. The design philosophy
for g4u is very simple. g4u is indifferent to what is on the
diskit just copies bits. It doesn't matter
what version of Unix or what file system you use. It
doesn't care if a disk sector is unusedit
still gets copied. The image is compressed while on the server, but
otherwise is an exact copy of the disk. If the image includes
configuration files that are specific to the original machine, e.g.,
a static IP address or a host-name file, you will have to correct
these after installing the image. (You can avoid most problems of
this sort if you use DHCP to configure your systems.) g4u works best
when used with disks with the same size and geometry but, under
limited circumstances, it may be finessed to work with other disks.
Image copying is the simplest approach to learn and to use and is
usable with almost any operating system.

There are three things you will need to do before using g4u. If you
don't already have an FTP server, you will need to
create one to store the images. You will need to download the g4u
software. And, while not strictly required, you should prepare your
source system for cloning. All of these are very straightforward.

To set up an FTP server,
you'll need to install the software, edit the
configuration files, and start the daemon. Several FTP server
implementations are available. Select and install your favorite. The
vsftpd (Very Secure FTP)
package is a good choice for this purpose. You'll
need to edit the appropriate configuration files,
/etc/vsftpd/vsftpd.conf,
/etc/vsftpd.ftpusers
, and
/etc/vsftpd.user_list. Then start the service.

[root@fanny etc]# /etc/init.d/vsftpd start
Starting vsftpd for vsftpd: [ OK ]

(When you are through cloning systems, you may want to disable FTP
until you need it again because it poses a security risk. Just
replace start with stop in the
above.) Consult the documentation with your distribution or the
appropriate manpages.

The g4u software consists of a NetBSD boot disk with the
image-copying software. While it is possible to download the sources,
it is much simpler if you just download a disk image with the
software. You can download either a floppy image or a CD-ROM ISO
image in either zipped or uncompressed format from http://www.feyrer.de/g4u/. (The uncompressed
ISO image is smaller than 1.5 MB so downloads go quickly.) Once you
have downloaded the image, unzip it if it is compressed and create
your disk. With a floppy, you can use a command similar to the
following, adjusting the version number as needed:

[root@fanny root]# cat g4u-1.16.fs > /dev/fd0

(With Windows, you can use rawrite.exe, which
can also be downloaded from the web site.) For a CD-ROM, use your
favorite software.

Since g4u creates a disk image, it copies
not only files but unused sectors as well. If there is a lot of
garbage in the unused sectors on the disk, they will take up space in
the compressed image, and creating that image will take longer. You
can minimize this problem by writing zeros out to the unused sectors
before you capture the image. (Long strings of zeros compress quickly
and use very little space.) The g4u documentation recommends creating
a file of zeros that grows until it fills all the free space on the
system, and then deleting that file.

[root@ida root]# dd if=/dev/zero of=/0bits bs=20971520 
dd: writing `/0bits': No space left on device
113+0 records in
112+0 records out
[root@ida root]# rm /0bits
rm: remove `/0bits'? y

Once the file is deleted, the unused sectors will still contain
mostly zeros and should compress nicely. While you
don't have to do this, it will significantly reduce
storage needs and transfer time.

To use g4u, you will need to capture the original disk and then copy
it to the new machines. Begin by shutting down the source machine and
then booting it with the g4u disk. As the system boots,
you'll see some messages, including a list of
commands, and then a command-line prompt. To capture and upload the
disk, use the
uploaddisk command. For example,

# uploaddisk sloanjd@fanny.wofford.int ida.g4u

The arguments to uploaddisk are the
user's FTP server and the saved images.
You'll see a few more messages and then the system
will prompt you for the user's FTP password. As the
disk image is captured and uploaded to the FTP server, the software
will display dots on the screen. When the upload is complete, the
software will display some statistics about the transfer.

To create new systems from the image, the process is almost the same.
Boot the new system from the g4u disk and use the
slurpdisk command, like so:

# slurpdisk sloanjd@fanny.wofford.int ida.g4u

You'll be prompted for a password again and see
similar messages. However, the download tends to go much faster than
the upload. When the user prompt returns, remove the g4u disk and
reboot the system. Log in and make any needed configuration changes.
That's really all there is to it!


8.2.3 SystemImager


SystemImager
is a part of the Systems Installation Suite
(SIS)
, a set of tools
for building an image for a cluster node and then copying it to other
nodes. In many ways it is quite similar to g4u. However, there are
several major differences in both the way it works and in the added
functionality it provides. It is also a much more complicated tool
how to learn to use. These differences will be apparent as you read
through this section.

As with g4u, with SIS you will set up a single node as a model,
install the operating system and any additional software you want,
and configure the machine exactly the way you want it. Next, copy the
image of this machine to a server, and then from the server to the
remaining machines in the cluster.

SystemImager is also useful in maintaining clusters since it provides
an easy way to synchronize files among machines. For example, if you
have a security patch to install on all the machines in the cluster,
you could install it on your model computer and then update the
cluster. Since SIS uses rsync, this is very
efficient. Only the files changed by the patch will be copied.

The Systems Installation Suite is made up of three tools,
SystemConfigurator,
SystemImager, and
SystemInstaller. From a pragmatic perspective,
SystemImager is the place to begin and, depending upon your needs,
may be the only part of the suite you will need to master.

SystemInstaller
is generally used to build a pre-installation image on the image
server without having to first create a model system. For example,
OSCAR uses SystemInstaller to do just this. But if you are happy
building the model system, which is strongly recommended since it
gives you an opportunity to test your configuration before you copy
it, there is no reason to be in a hurry to learn the details of
SystemInstaller.

SystemConfigurator
allows you to do a post-installation configuration of your system.
While it is a useful standalone tool, it is integrated into
SystemImager so that its use is transparent to the user. So while you
will need to install SystemConfigurator, you don't
need to learn the details of SystemConfigurator to get started using
SIS. Consequently, this section focuses on SystemImager.

Since SystemImager uses client-server
architecture, you will need to set up two machines initially before
you can begin cloning systems. The image server
manages the installation, holds the clone image, and usually provides
other needed services such as DHCP. You will also need to set up the
model node or golden
client
. Once you have created the golden
client, its image is copied to the server and can then be installed
on the remaining machines within the cluster.

The installation of SystemImager can be divided into four multistep
phasessetting up the image server, setting up the golden
client, transferring the image to the image server, and copying the
image to the remaining nodes in the cluster. Each of these phases is
described in turn. If you installed OSCAR, this setup has already
been done for you. However, OSCAR users may want to skim this
material to get a better idea of how OSCAR works and can be used.


8.2.3.1 Image server setup

In setting up the image server, you will need to select a server,
install Linux and other system software as needed, install the
SystemImager software on the server, and determine both how you will
assign IP addresses to clients and how you will start the download.

You'll want to take care in selecting your server.
Typically, the SystemImager server will also act as the head node for
your cluster and will provide additional network services such as
DHCP. While it is possible to distribute some of this functionality
among several machines, this isn't usually done and
won't be discussed here. If you already have a
server, this is a likely choice provided it has enough space.

Unlike g4u, the images SystemImager creates are stored as
uncompressed directory trees on the server. This has a number of
advantages. First, it works nicely with rsync.
And as a live filesystem, you can chroot to it
and make changes or even install packages (if you are a brave soul).
You'll only copy useful files, not unused sectors.
While this approach has a number of advantages, even a single image
can take up a lot of space. Heterogeneous clusters will require
multiple images. Taken together, this implies you'll
want a speedy machine with lots of disk space for your server.

Because of dependencies, you should install all of SIS even if you
plan to use only SystemImager. You have a couple of choices as to how
you do this. There is a Perl installation script that can be
downloaded and run. It will take care of downloading and installing
everything else you need. Of course, you'll need
Internet access from your cluster for this to work. Alternatively,
you can download DEB or RPM packages and install. Downloading these
packages and burning them onto a CD-ROM is one approach to setting up
an isolated cluster. This chapter describes the installation process
using RPM packages.

Since SIS supports a wide variety of
different Linux releases, you'll need to select the
correct packages for your distribution, and you'll
need a number of packages to install SystemImager. These can be
downloaded from SourceForge. Go to http://sisuite.sourceforge.net and follow the
links to SystemConfigurator, SystemImager, and SystemInstaller, as
needed, to download the individual packages. If in doubt, you can
read the release notes for details on many of the packages.

There may be additional dependencies that you'll
also need to address. For a simple Red Hat install,
you'll need to install the following packages, if
they are not already on your system, in this order:
rsync, perl-AppConfig,
perl-XML-Simple,
systemconfigurator,
systemimager-common,
systemimager-server,
perl-MLDBM, and
systeminstaller. You'll also
need a boot package specific to your architecture. For example, you
would use systemimager-boot-i386-standard for
the Intel 386 family. rsync is usually already
installed. Install these as you would install any RPM.

[root@fanny sysimager]# rpm -vih perl-AppConfig-1.52-4.noarch.rpm
Preparing... ########################################### [100%]
1:perl-AppConfig ########################################### [100%]

Repeat the process with each package. There is also an X interface to
SystemInstaller called tksis. If you want to
install this, you will need to install perl-DBI,
perl-TK, and
systeminstall-x11. (If you have problems with
circular dependencies, you might put the package names all on the
same line and use rpm -Uvh to install them.)

The SIS installation will create a directory
/etc/systemimager containing the configuration
files used by SystemImager. By default, SystemImager is not started.
You can use the command service systemimager
start
to manually start it. SystemImager
starts the rsync daemon using the configuration
file in /etc/systemimager, so if
rsync is already running on your system,
you'll need to turn it off first. As with any manual
start, if you restart the system, you'll need to
restart SystemImager. (With a recent release, the names of several
services have changed. To ensure you are using the appropriate names,
look in /etc/init.d to see what is installed.)

There are a couple of other things you might want to set up on your
server if you don't already have them. With SIS,
there are four installation methods. You can boot the machine you are
installing the image on from a floppy, from CD-ROM, from its hard
drive, or over the network using a PXE-based network adapter. (The
hard drive option is used for upgrading systems rather than for new
installs.)

If you are going to do a network boot, you will need a TFTP server.
SIS includes a command,
mkbootserver, which will handle the configuration
for you, but you must first install some
packagestftp-server,
tftp, and pxe. Once these
packages are installed, the script mkbootserver
will take care of everything else. As needed, it will create the
/tftpboot directory, modify
/etc/services, modify
/etc/inetd.conf or
/etc/xinetd.d/tftp, verify that the TFTP server
works, configure PXE creating /etc/pxe.conf,
verify the pxe daemon is running, verify the
network interface is up, and pass control to the
mkdhcpserver command to configure a DHCP server.
Once mkbootserver has been run, your server
should be appropriately configured for booting clients and installing
images via PXE. Of course, you'll need a PXE-enabled
network adapter in your client.

Even if you aren't
booting via PXE, you will probably still want to use DHCP to assign
IP addresses. This isn't absolutely necessary since
you can create a configuration diskette for each machine with the
appropriate information, but it is probably the easiest way to go.
Using DHCP implies you'll need a DHCP server, i.e.,
both server software and a configuration file. Setting up the
software is usually just a matter of installing the
dhcp package.

[root@fanny root]# rpm -vih dhcp-3.0pl1-23.i386.rpm
warning: dhcp-3.0pl1-23.i386.rpm: V3 DSA signature: NOKEY, key ID db42a60e
Preparing... ########################################### [100%]
1:dhcp ########################################### [100%]

To create a configuration file, typically
/etc/dhcpd.conf, use the
mkdhcpserver script. You'll need to
collect information about your network such as the IP address range,
broadcast address, network mask, DNS servers, and the network gateway
before you run this script. Here is an example of using
mkdhcpserver for a simple network.

[root@fanny root]# mkdhcpserver
Welcome to the SystemImager "mkdhcpserver" command. This command will
prepare this computer to be a DHCP server by creating a dhcpd.conf file
for use with your ISC DHCP server (v2 or v3).
If there is an existing file, it will be backed up with the
.beforesystemimager extension.
Continue? (y/[n]): y
Type your response or hit <Enter> to accept [defaults]. If you don't
have a response, such as no first or second DNS server, just hit
<Enter> and none will be used.
What is your DHCP daemon major version number (2 or 3)? [2]: 2
Use of uninitialized value in concatenation (.) or string at /usr/sbin/
mkdhcpserver line 202, <STDIN> line 2.
What is the name of your DHCP daemon config file? [ ]: /etc/dhcpd.conf
What is your domain name? [localdomain.domain]: wofford.int
What is your network number? [192.168.1.0]: 10.0.32.0
What is your netmask? [255.255.255.0]: 255.255.248.0
What is the starting IP address for your dhcp range? [192.168.1.1]: 10.0.32.145
What is the ending IP address for your dhcp range? [192.168.1.100]: 10.0.32.146
What is the IP address of your first DNS server? [ ]: 10.0.80.3
What is the IP address of your second DNS server? [ ]: 10.0.80.2
What is the IP address of your third DNS server? [ ]:
What is the IP address of your default gateway? [192.168.1.254]: 10.0.32.2
What is the IP address of your image server? [192.168.1.254]: 10.0.32.144
What is the IP address of your boot server? [ ]: 10.0.32.144
What is the IP address of your log server? [ ]:
Will your clients be installed over SSH? (y/[n]): y
What is the base URL to use for ssh installs? [http://10.0.32.144/
systemimager/boot/]:
What... is the air-speed velocity of an unladen swallow? [ ]:
Wrong!!! (with a Monty Python(TM) accent...)
Press <Enter> to continue...
Ahh, but seriously folks...
Here are the values you have chosen:
#######################################################################
ISC DHCP daemon version: 2
DHCP daemon using fixed-address patch: n
ISC DHCP daemon config file: /etc/dhcpd.conf
DNS domain name: wofford.int
Network number: 10.0.32.0
Netmask: 255.255.248.0
Starting IP address for your DHCP range: 10.0.32.145
Ending IP address for your DHCP range: 10.0.32.146
First DNS server: 10.0.80.3
Second DNS server: 10.0.80.2
Third DNS server:
Default gateway: 10.0.32.2
Image server: 10.0.32.144
Boot server: 10.0.32.144
Log server:
Log server port:
SSH files download URL: http://10.0.32.144/systemimager/boot/
#######################################################################
Are you satisfied? (y/[n]): y
The dhcp server configuration file (/etc/dhcpd.conf) file has been
created for you. Please verify it for accuracy.
If this file does not look satisfactory, you can run this command again
to re-create it: "mkdhcpserver"
WARNING!: If you have multiple physical network interfaces, be sure to
edit the init script that starts dhcpd to specify the interface that
is connected to your DHCP clients. Here's an example:
Change "/usr/sbin/dhcpd" to "/usr/sbin/dhcpd eth1".
Depending on your distribution, you may be able to set this with the
"INTERFACES" variable in either "/etc/default/dhcp" or in your dhcpd
initialization script (usually "/etc/init.d/dhcpd").
Also, be sure to start or restart your dhcpd daemon. This can usually
be done with a command like "/etc/init.d/dhcpd restart" or similar.
Would you like me to restart your DHCP server software now? (y/[n]): y
Shutting down dhcpd: [FAILED]
Starting dhcpd: [ OK ]

As you can see, the script is very
friendly. There is also important information buried in the output,
such as the warning about restarting the DHCP daemon. Be sure you
read it carefully. If you already have a DHCP configuration file, it
is backed up, usually as
/etc/dhcpd.conf.beforesystemimager. You may need
to merge information from your old file into the newly created file.

As previously noted, you don't have to use DHCP. You
can create a configuration disk with a file
local.cfg for each machine with the information
provided by DHCP. Here is an example.

HOSTNAME=hector
DOMAINNAME=wofford.int
DEVICE=eth0
IPADDR=10.0.32.146
NETMASK=255.255.248.0
NETWORK=10.0.32.0
BROADCAST=10.0.39.255
GATEWAY=10.0.32.2
IMAGESERVER=10.0.32.144
IMAGENAME=ida.image

Regardless of how you are booting for your install, the software will
look for a floppy with this file and use the information if provided.
In this example, the client names that have been automatically
generated are not being used, so it is necessary to rename the
installation scripts on the image server. We'll come
back to this.


8.2.3.2 Golden client setup

The golden client is a model for the other
machines in your cluster. Setting up the golden client requires
installing and configuring Linux, the SystemImager software, and any
other software you want on each client. You will also need to run the
prepareclient script to collect image information and
start the rsync daemon for the image transfer.

Because you are using an image install, your image should contain
everything you want on the cluster nodes, and should be compatible
with the node's hardware. In setting up the client,
think about how it will be used and what you will need. Doing as much
of this as possible will save you work in the long run. For example,
if you generate SSH keys prior to cloning systems, you
won't have to worry about key distribution. However,
getting the software right from the start isn't
crucial. SystemImager includes a script to update clients, and since
it uses rsync, updates go fairly quickly.
Nonetheless, this is something of a nuisance, so
you'll want to minimize updates as much as possible.
If possible, set up your client and test it in the environment in
which it will be used.

Getting the hardware right is more important. The hardware
doesn't have to be identical on every node, but it
needs to be close. For network and video adapters,
you'll want the same chipset. Although disk sizes
don't have to be identical, it is better to select
for your golden client a machine with the smallest disk size in your
cluster. And you can't mix IDE and SCSI systems.
Having said all this, remember that you can have multiple images. So
if you have a cluster with three different sets of hardware, you can
create three images and do three sets of installs.[3]

[3] To
some extent, you can install an image configured for different
hardware and use kudzu to make corrections once the system reboots.
For example, I've done this with network adapters.
When the system boots for the first time, I delete the
image's adapter and configure the actual adapter in
the machine. (Actually, SystemConfigurator should be able to manage
NIC detection and setup.)


Once you have built your client, you'll need to
install the SystemImager client software. This is done in much the
same manner as with the server but there is less to install. For a
typical Red Hat install, you'll need
perl-AppConfig,
systemconfigurator,
systemimager-common, and
systemimager-client packages at a minimum.

Once all the software has been installed and configured, there is one
final step in preparing the client. This involves collecting
information about the client needed to build the image by running the
prepareclient script. The script is very
friendly and describes in some detail what it is doing.

[root@ida sis]# prepareclient
Welcome to the SystemImager prepareclient command. This command may modify the
following files to prepare your golden client for having its image retrieved by
the imageserver. It will also create the /etc/systemimager directory and fill
it with information about your golden client. All modified files will be
backed up with the .before_systemimager-3.0.1 extension.
/etc/services:
This file defines the port numbers used by certain software on your system.
I will add appropriate entries for rsync if necessary.
/etc/inetd.conf:
This is the configuration file for the inet daemon, which starts up certain
server software when the associated client software connects to your
machine. SystemImager needs to run rsync as a standalone daemon on your
golden client until it's image is retrieved by your image server. I will
comment out the rsync entry in this file if it exists. The rsync daemon will
not be restarted when this machine is rebooted.
/tmp/rsyncd.conf.13129:
This is a temporary configuration file that rsync needs on your golden client
in order to make your filesystem available to your image server.
See "prepareclient -help" for command line options.
Continue? (y/[n]): y
*********************************** WARNING ***********************************
This utility starts an rsync daemon that makes all of your files accessible
by anyone who can connect to the rsync port of this machine. This is the
case until you reboot, or kill the 'rsync --daemon' process by hand. By
default, once you use getimage to retrieve this image on your image server,
these contents will become accessible to anyone who can connect to the rsync
port on your imageserver. See rsyncd.conf(5) for details on restricting
access to these files on the imageserver. See the systemimager-ssh package
for a more secure method of making images available to clients.
*********************************** WARNING ***********************************
Continue? (y/[n]): y
Signaling xinetd to restart...
Using "sfdisk" to gather information about /dev/hda... done!
Starting or re-starting rsync as a daemon.....done!
This client is ready to have its image retrieved. You must now run
the "getimage" command on your imageserver.

As you
can see from the output, the script runs the
rsync server daemon on the client. For this
reason, you should wait to run this script until just before you are
ready to transfer the image to the image server. Also, be sure to
disable this rsync server after copying the
client image to the image server.


8.2.3.3 Retrieving the image

This is perhaps the simplest phase
of the process. To get started, run the
getimage script. You'll need to
specify the name or address of the client and a name for the image.
It should look something like this:

[root@fanny scripts]# getimage -golden-client ida -image ida.image
This program will get the "ida.image" system image from "ida"
making the assumption that all filesystems considered part
of the system image are using ext2, ext3, jfs, FAT, reiserfs, or xfs.
This program will not get /proc, NFS, or other filesystems
not mentioned above.
*********************************** WARNING ***********************************
All files retrieved from a golden client are, by default, made accessible to
anyone who can connect to the rsync port of this machine. See rsyncd.conf(5)
for details on restricting access to these files on the imageserver. See the
systemimager-ssh package for a more secure (but less effecient) method of
making images available to clients.
*********************************** WARNING ***********************************
See "getimage -help" for command line options.
Continue? ([y]/n): y
Retrieving /etc/systemimager/mounted_filesystems from ida to check for mounted
filesystems...
------------- ida mounted_filesystems RETRIEVAL PROGRESS -------------
receiving file list ... done
/var/lib/systemimager/images/ida.image/etc/systemimager/mounted_filesystems
wrote 138 bytes read 114 bytes 504.00 bytes/sec
total size is 332 speedup is 1.32
------------- ida mounted_filesystems RETRIEVAL FINISHED -------------
Retrieving image ida.image from ida
------------- ida.image IMAGE RETRIEVAL PROGRESS -------------
...

At this point you'll see the names of each of the
files whiz by. After the last file has been transferred, the script
will print a summary.

...
wrote 92685 bytes read 2230781 bytes 10489.69 bytes/sec
total size is 1382212004 speedup is 594.89
------------- ida.image IMAGE RETRIEVAL FINISHED -------------
Press <Enter> to continue...
IP Address Assignment
---------------------
There are four ways to assign IP addresses to the client systems on an
ongoing basis:
1) DHCP
----------------------------------------------------------------
A DHCP server will assign IP addresses to clients installed with
this image. They may be assigned a different address each time.
If you want to use DHCP, but must ensure that your clients
receive the same IP address each time, see "man mkdhcpstatic".
2) STATIC
----------------------------------------------------------------
The IP address the client uses during autoinstall will be
permanently assigned to that client.
3) REPLICANT
----------------------------------------------------------------
Don't mess with the network settings in this image. I'm using
it as a backup and quick restore mechanism for a single machine.
Which method do you prefer? [1]:
You have chosen method 1 for assigning IP addresses.
Are you satisfied? ([y]/n): y
Would you like to run the "addclients" utility now? (y/[n]): n

Unless you have edited
/etc/systemimager/systemimager.conf, the image
will be stored in the directory
/var/lib/systemimager/images as the subdirectory
ida.image.

The getimage command runs
mkautoinstallscript,
which creates the auto-install script
/var/lib/systemimager/scripts/ida.image.master
in this case, and gives you the option to move onto the next step.
But before you do, you may want to kill the
rsync daemon on the golden client.

[root@ida sysconfig]# ps -aux | grep rsync | grep -v grep
root 13142 0.0 0.4 1664 576 ? S 15:46 0:00 rsync
--daemon --
[root@ida sysconfig]# kill 13142


8.2.3.4 Cloning the systems

The final steps of distributing the
image to the clients require creating the installation scripts for
the clients, preparing any needed boot media, and then booting the
clients to initiate the process.[4]

[4] The latest release
of SIS includes a program flamethrower. This is
use to multicast images speeding the file distribution process on
multicast enabled networks. flamethrower is not
discussed in this chapter.


As noted above, you should now have an initial auto-install script.
The next script you'll run is
addclients, which does three thingsit
automatically generates host names for each node, it creates symbolic
links to the auto-install script, one for each client, and it
populates the /etc/hosts table.

[root@fanny root]# addclients
Welcome to the SystemImager "addclients" utility
...

A copy of the host table and the install scripts for the individual
machines are located in the directory
/var/lib/systemimager/scripts. If you
don't want to use the automatically generated names,
you'll need to edit /etc/hosts
and /var/lib/systemimager/scripts/hosts,
replacing the automatically generated names with the names you want.
You'll also need to rename the individual install
scripts in /var/lib/systemimager/scripts to
match your naming scheme. Of course, if you are happy with the
generated names, you can skip all this.

If you are using a network or PXE boot, you can restart the clients
now. If you are booting from a floppy or CD-ROM,
you'll first need to make a boot disk. You can use
the scripts mkautoinstalldiskette or
mkautoinstallcd to make, respectively, a boot
diskette or boot CD-ROM. Here is an example of making a CD-ROM.

[root@fanny root]# mkautoinstallcd -out autoinstall.iso
Here is a list of available flavors:
standard
Which flavor would you like to use? [standard]:
...

Note that the default or standard flavor was used.
This was created when the package
systemimager-boot-i386-standard was installed.
With the CD-ROM script, an ISO image is generated that can be used to
burn a CD-ROM. Fortunately, this is a relatively small file, so it
can easily be moved to another system with a CD-ROM burner. If you
elect to use the diskette script instead, it will mount, format, and
record the diskette for you. If you don't want to
use DHCP, put the file local.cfg on a separate
diskette even if you are using a CD-ROM to boot. When booting from a
diskette, you'll need to put
local.cfg on that diskette. Be warned, you may
run out of space if you use a diskette. If you
aren't using a local configuration file, you need
only one boot disk. You need a diskette for each machine, however, if
you are using the local configuration file. If you upgrade
SystemImager, remember to regenerate your boot disks as they are
release dependent.

Now that you have the boot disk, all you need to do is reboot the
client from it. The client will locate the image server and then
download and run the installation script. You can sit back and watch
the magic for a while. After a short time, your systems should begin
to beep at you. At this point, you can remove any diskettes or
CD-ROMs and reboot the systems. Your node is installed.

There is one last script you may want to run if you are using DHCP.
The script
mkdhcpstatic
can update your DHCP configuration file, associating IP addresses
with MAC addresses. That is, if you run this script, each IP address
will be tied to a specific machine based on the MAC address of the
machine to which it was first assigned. Since IP addresses are handed
out in numerical order, by booting the individual machines in a
specific order and then running mkdhcpstatic,
you can control IP assignments.


8.2.3.5 Other tasks

As if building your network isn't enough,
SystemImager can also be used to maintain and update your clients.
The script
updateclient is used to resynchronize a client with
an image. Its calling syntax is similar to
getimage.

[root@hector root]# updateclient -server fanny -image ida.image
Updating image from module ida.image...
receiving file list ... done
...

You'll see a lot of file names whiz by at this point.

...
wrote 271952 bytes read 72860453 bytes 190201.31 bytes/sec
total size is 1362174476 speedup is 18.63
Running bootloader...
Probing devices to guess BIOS drives. This may take a long time.
Installation finished. No error reported.
This is the contents of the device map /boot/grub/device.map.
Check if this is correct or not. If any of the lines is incorrect,
fix it and re-run the script `grub-install'.
(fd0) /dev/fd0
(hd0) /dev/hda
Probing devices to guess BIOS drives. This may take a long time.
Probing devices to guess BIOS drives. This may take a long time.

It should be noted that the script is fairly intelligent. It will not
attempt to update some classes of files, such as log files, etc.

SystemInstaller also provides several commands for manipulating
images. The commands cpimage,
mvimage, lsimage, and
rmimage are, as you might guess, analogous to
cp, mv,
ls, and rm.


/ 142