Compiling Python, NumPy and SciPy with Intel Compilers and Intel MKL

Compute clusters are shared machines so they often contain very outdated versions of software in the name of backward compatibility. While the environment modules project goes a long way of alleviating this problem, I often encounter problems with the Python installation on clusters that I'm working on.

In such a situation, it's often cleanest to just start over with your own mini-installation of Python and all packages that you have control over but that will open a whole new can of snakes^[1]. Since I want to spare you the trial-and-error, I've decided to write things up for you.

Note: This guide assumes that you have a 64-bit system with the Intel compilers installed with binaries icc, icpc, ifort, xiar and xild available in your path and the Intel MKL reachable via a correctly configured $LD_LIBRARY_PATH and $INCLUDE. You will not need root permissions to install the software and all installation will be performed relative to a $PREFIX.

Let's start by setting that prefix variable - a good choice is $HOME/.local:

export PREFIX=~/.local

You also want to make sure that the following is sourced when you log in (by placing it in your .bashrc, .zshrc, etc.):

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib
export C_INCLUDE_PATH=$C_INCLUDE_PATH:$HOME/.local/include
export PATH=$HOME/.local/bin:$PATH

export CC=icc
export CXX=icc
export FC=ifort

Now we're ready to roll! 🤘

Python 3.5.1

Grab a Python tarball from the official webpage, extract it and change into the new directory. Then, run:

./configure --prefix=$PREFIX --with-computed-gotos --without-gcc \
	--with-libm=-limf --with-cxx-main=`which icpc` --with-threads \
	--enable-ipv6 --with-signal-module \
	CC=icc CXX=icpc LD=xild AR=xiar \
	LIBS="-lpthread -limf -lirc" \
	CFLAGS="-O3 -fp-model strict -fp-model source -xHost -ipo -prec-div -prec-sqrt" \
	LDFLAGS="-ipo" \
	CPPFLAGS="" \
	CPP="icc -E"
make
make install

This will compile Python^[2] and install it into the $PREFIX/bin and $PREFIX/lib directories. If you want your custom-compiled version of Python be available under the python and pip, you can create some symlinks:

ln -s $PREFIX/bin/python{3,}
ln -s $PREFIX/bin/pip{3,}

We can now check if things are working:

which python

~/.local/bin/python

python

Python 3.5.1 (default, Mar 15 2016, 11:12:26) 
[GCC Intel(R) C++ gcc 4.8 mode] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

NumPy 1.10.4

Next up is NumPy! Grab a tarball from PyPi, extract and change into the new directory.

In order to use Intel MKL with NumPy, we'll have to create a site.cfg file from the template at site.cfg.example:

cp site.cfg{.example,}

Now edit the site.cfg to reflect the location of the MKL on your system in library_dirs and include_dirs:

[mkl]
library_dirs = /opt/intel/compilers_and_libraries_2016/linux/mkl/lib/intel64
include_dirs = /opt/intel/compilers_and_libraries_2016/linux/mkl/include
mkl_libs = mkl_rt
lapack_libs =

Now we can compile and install:

python setup.py build --compiler=intelem 
python setup.py install

Again, we check if things are working correctly:

import numpy as np
print(np.ones((4, 4)) + np.eye(4))

[[ 2.  1.  1.  1.]
 [ 1.  2.  1.  1.]
 [ 1.  1.  2.  1.]
 [ 1.  1.  1.  2.]]

SciPy 0.17.0

Finally, compiling and installing SciPy is straightforward. Again, grab a tarball from PyPi, extract and change into the new directory. Then run:

python setup.py build --compiler=intelem --fcompiler=intelem
python setup.py install

More packages with pip

If you need more packages for your installation, you should be able to install them using the pip command bundled with Python 3 by default. Packages will be placed relative to your PYTHONHOME, or more specifically in $PREFIX/.local/lib/python3.5/site-packages.

For example, pip install six

Sources

• BinaryBuilder: Building Python 2.7 on 64bit Linux, with Intel compilers
• Intel: Numpy/Scipy with Intel MKL