Biswajit Banerjee

### Compiling and running the MPM code Vaango on a Cray

Copper and Excalibur

#### Introduction

One of the reasons I switched to cmake for my builds was the need to compile my Vaango code on a BlueGene/Q system. The code was previously configured using autoconf and m4, and because I hadn’t done that implementation myself it was nightmare to change things when I wanted to add some new feature or library. After moving to cmake things have become decidedly easier. In this article I will discuss how I ported my code to a couple of Cray XE6m machines and how I run the code on that machine. You can download the latest version of the code from GitHub.

#### Authentication

The authentication system for the Cray machine uses Kerberos and requires a Yubikey. The typical process for authentication is as follows:

Here kssh is a Kerberized version of ssh.

#### Building the Vaango code on Copper (Cray XE6m)

The Vaango code is written in C++ and has over the past two years slowly added several C++11 and more recently C++14 features. I first had to make sure that wget and git were available on the Cray machines.

To download the code I used the standard procedure

I then made sure that my googletest and json submodules had been downloaded correctly:

To start the process is usually look at my CMakeLists.txt file to figure out what I need. In this case:

The json submodule downloads a lot of unnecessary data and will be removed at a future date.

##### Checking needed third party packages

Looking at the root-level CMakeLists.txt file tells us that the following external packages are needed to build Vaango:

• boost : for some MPI and serialization code
• cmake : to build the makefiles
• eigen3 : for some matrix operations
• gcc : to build the C++ code
• gfortran : to build the Fortran code
• libxml2 : for XML input/output of Vaango format data
• openmpi : for MPI code
• perl : for Perl scripts
• zlib : for compression code

We avoid continuous integration and testing in the build; so googletests is not used even though it is downloaded.

A module avail command typically lists a large number of potential packages that can be used. In our case, we loaded the following modules and environments:

##### Installing Boost and Eigen3

I couldn’t locate boost or eigen3 on the machine and decided to download and build them:

To build boost I did the following:

For eigen3, the process was

##### Compiling Vaango

I had to modify CMakeLists.txt to automatically detect the MPI compilers and settings. After that, to compile the Vaango code, all I needed was

#### Running the Vaango code on Copper

To check whether the build produced a working executable, I had to start an interactive session with

and then

Larger jobs require the qsub queue system and PBS scripts.

#### Building the Vaango code on Excalibur (Cray XC40)

Another Cray system called Excalibur is also used to run Vaango once in a while. The pre-installed packages on this machine vary with time and the following is what had to be done to get Vaango to run on that machine around a year ago.

##### Installing cmake

We should use at least version 3.2.2 but earlier versions may may with the latest Vaango code. If the build fails, one may need to load some missing modules.

##### Installing boost and eigen3

These can be installed in a manner similar to that for Copper. The installation directory was chosen to be localpackages.

##### Compiling Vaango

The cmake script may need the full set of options but typically works with just the locations of boost and eigen3 provided in the command line, if the correct environment is chosen (in our case, gnu). The process should be identical to that used in Copper.

The full set of path and library options to cmake when using a local mpich installation is given below.

#### Remarks

The process of building Vaango on Cray machines has become considerably simpler over time if implicit codes are not needed. That is still not true for IBM machines such as BlueGene/Q.

If you have questions/comments/corrections, please contact banerjee at parresianz dot com dot zen (without the dot zen).