Building the Software

The general process for building most software is the same. We run three commands, ./configure, make, and make install. The difficult part is in the ./configure step where you tell the software how you want it set up. Getting the ./configure step correct for your environment is a process of trial and error. That's why I keep the build directories around. If I find that I needed an extra option to ./configure, or I specified something that I don't want, it's easy to go back to the directory, run ./configure again, and rebuild the software. The make utility is very good about only rebuilding the parts that need to be rebuilt.

Typically, you can ask configure what options are available with the --help argument to configure. As an example, here's the output from ./configure --help for the Modules software:

admin@baker:~> cd /apps/source/modules-4.2.0
admin@baker:/apps/source/modules-4.2.0> ./configure --help
Usage: ./configure [OPTION]...
Check requirements then set variables and create Makefiles.

Defaults for the options are specified in square brackets.

Configuration:
  -h, --help              display this help and exit

Installation directories:
  --prefix=PREFIX         install files in PREFIX [/usr/local/Modules]

By default, `make install' will install all the files in
`/usr/local/Modules/bin', `/usr/local/Modules/libexec', etc. You
can specify an installation prefix other than `/usr/local/Modules'
using `--prefix', for instance `--prefix=$HOME'.

For better control, use the options below.

Fine tuning of the installation directories:
  --bindir=DIR            user executables [PREFIX/bin]
  --libexecdir=DIR        program executables [PREFIX/libexec]
  --etcdir=DIR            program configurations [PREFIX/etc]
  --initdir=DIR           environment initialization scripts [PREFIX/init]
  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
  --mandir=DIR            man documentation [DATAROOTDIR/man]
  --docdir=DIR            documentation root [DATAROOTDIR/doc]
  --modulefilesdir=DIR    system modulefiles [PREFIX/modulefiles]

Optional Features:
  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
  --enable-set-manpath    set mandir to MANPATH in init scripts [yes]
  --enable-append-manpath append rather prepend mandir to MANPATH [no]
  --enable-set-binpath    set bindir to PATH in init scripts [yes]
  --enable-append-binpath append rather prepend bindir to PATH [no]
  --enable-dotmodulespath configure modules path in .modulespath file rather
                          than modulerc file [no]
  --enable-doc-install    install documentation files in the documentation
                          directory defined with \'docdir' (no impact on
                          man pages installation) [yes]
  --enable-example-modulefiles
                          install in 'modulefilesdir' some modulefiles
                          provided as examples [yes]
  --enable-compat-version also build and install Modules compatibility (C)
                          version and enable switching capabilities between
                          the two versions [yes]
  --enable-versioning     append modules version to installation prefix and
                          deploy a `versions' modulepath, shared between all
                          versioning enabled Modules installation, containing
                          modulefiles that enable to switch from one Modules
                          version to another [no]
  --enable-silent-shell-debug-support
                          generate code in module function definition and
                          initialization scripts to add support for silencing
                          shell debugging properties [yes]
  --enable-quarantine-support
                          generate code in module function definition and
                          initialization scripts to add support for the
                          environment variable quarantine mechanism [yes]
  --enable-auto-handling  set modulecmd.tcl to automatically apply automated
                          modulefiles handling actions, like loading the
                          pre-requisites of a modulefile when loading this
                          modulefile [no]

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-bin-search-path=PATHLIST
                          list of paths to look at when searching the location
                          of tools required to build and configure Modules
                          [/usr/bin:/bin:/usr/local/bin]
  --with-tclsh=BIN        name or full path of Tcl interpreter shell [tclsh]
  --with-pager=BIN        name or full path of default pager program to use to
                          paginate informational message output (can be super-
                          seeded at run-time by environment variable) [less]
  --with-pager-opts=OPTLIST
                          settings to apply to default pager program [-eFKRX]
  --with-modulepath=PATHLIST
                          default modulepaths to set in default configuration
                          file to be enabled (each path in PATHLIST should
                          be separated by `:') [PREFIX/modulefiles or
                          BASEPREFIX/$MODULE_VERSION/modulefiles if versioning
                          installation mode enabled]
  --with-loadedmodules=MODLIST
                          default modulefiles to set in default configuration
                          file to be loaded (each modulefile name in MODLIST
                          should be separated by `:') []
  --with-quarantine-vars='VARNAME[=VALUE] ...'
                          environment variables to put in quarantine when
                          running the module command to ensure it a sane
                          execution environment (each variable should be
                          separated by ` '). a value can eventually be set to
                          a quarantine variable instead of emptying it. []

Depending on the above configuration options the files are approximately
placed in the following directory structure:
  PREFIX/
    bin/
    etc/
    init/
    libexec/
    share/
      doc/
      man/
        man1/
        man4/
    modulefiles/
admin@baker:/apps/source/modules-4.2.0>

There's a lot of output there, and this is probably the simplest package we're going to build. Don't get overwhelmed or discouraged at all the opions you have.

First, there are a number of options that are mostly consistent across all configure commands. One is the --prefix option, which specifies where the software gets installed. If you want to further modify the standard directory structure, you have options such as --bindir to change just the directories where binaries are installed, or --mandir to change just the directory where the man pages get installed.

Second, as mentioned previously, if you build your software and later find that you really needed a different set of options to configure, you can come back to the build directory, run configure again with the options you want, and rebuild the package.

Third, if you're not sure, you can just run configure without any options and use the defaults. A word of caution here, though: you probably want to at least use the --prefix argument to change where the software gets installed. Most packages will install into /usr/local by default, and that's almost always not where you want your software to go.

For our software, we're going to install everything under the /apps file system, and name each package by its name and version number. When we configure Modules, for example, the path will be /apps/modules-4.2.0.

The order of the software builds is mostly unimportant, with the exception that SLURM depends on munge, so we have to build munge first. When we start building libraries, there will be many more dependencies. For the sake of brevity, I'm only going to list the commands I run to configure and build the software here. You can find the full output in the sub-page called Command Output

The process of building the software often requires a few tries as we find missing dependencies. Sometimes we have to go back and change our configure options as well. It's an on-going process of trial and error, and your needs may change as your user base changes and advances. If you want some in-depth analysis of how and why this software was configured this way, take a look at the HPC For Science section of my site.

To save you a bit of time, there are a few dependencies that we need for these software packages that haven't been installed yet. You can install all your software dependencies for these packages with this command:

zypper install -y tcl-devel readline-devel openssl-devel libnuma-devel

These are the commands I ran to build all the software:

cd /apps/source
cd modules-4.2.0
CPPFLAGS="-DUSE_INTERP_ERRORLINE" ./configure --prefix=/apps/modules-4.2.0 -with-tcl-lib=/usr/lib64 |& tee configure.log
make |& tee make.log
make install

cd ../pdsh-2.33/
./configure --prefix=/apps/pdsh-2.33 --without-rsh --with-ssh --with-machines=/etc/dsh/nodes --with-slurm --with-dshgroups --with-readline |& tee configure.log
make |& tee make.log
make install

cd ../munge-0.5.13
./configure --prefix=/apps/munge-0.5.13 --localstatedir=/var |& tee configure.log
make |& tee make.log
make install

cd ../slurm-19.05
./configure --prefix=/apps/slurm-19.05 --with-munge=/apps/munge-0.5.13 |& tee configure.log
make |& tee make.log
make install

Something to note in the above commands is that I pipe the output of the make command to tee make.log. Most of these builds produce a huge number of messages and warnings. Usually the warnings are okay, but if things don't work correctly when you run the software, you can go back and look at make.log and see all the output from the build. I do the same with the ./configure step.

As we try to configure and test our software builds, we'll find that we need to come back and make some changes, either because the change is required for functionality, or because we want to add some particular feature. One particular error that shows up during the software build and install comes from the make install step for munge. This error message shows up in the output:

/usr/bin/install: cannot change permissions of ‘/var/lib/munge’: No such file or directory

The problem here is that we're building the software as a non-priveleged user (admin) and the install step wants to create a new directory under /var/lib which is privileged. We could run the install as root, but we don't want to do so unless it's necessary. In the case of munge, it isn't necessary, so let's see if we can fix this.

For starters, let's create the problem directory as root, and then change the ownership to the admin user. According to the QUICKSTART file in the munge source directory, this directory should have permissions 0711. It also gives us a list of other directories and their permissions, so let's set all of these up and try our install again.

baker:~ # mkdir /var/lib/munge
baker:~ # chown admin:users /var/lib/munge
baker:~ # chmod 0711 /var/lib/munge
baker:~ # mkdir /etc/munge
baker:~ # chown admin:users /etc/munge
baker:~ # chmod 0700 /etc/munge
baker:~ # mkdir /var/log/munge
baker:~ # chown admin:users /var/log/munge
baker:~ # chmod 0700 /var/log/munge
baker:~ # mkdir /var/run/munge
baker:~ # chown admin:users /var/run/munge
baker:~ # chmod 0755 /var/run/munge

You can see the output from the second try in the Command Output sub-page. This time it looks like everything worked correctly, however, there's another problem lurking here. When the munged daemon runs, it needs to run under its own, dedicated user ID and group ID. We'll fix this when we configure the service. For now, our install seems to have finished successfully, so let's move on and try to configure and test our software stack.