.. index:: CMTH SciPy Stack
.. _CMTH SciPy Stack:

CMTH SciPy Stack
================

Description
-----------

This is a full `SciPy stack <https://www.scipy.org/stackspec.html>`_ compiled against the Intel compilers and MKL libraries and made available as a module. It contains python 3.6.2 as well as the most recent version at the time of install of the following packages and tools:

- python
- numpy
- scipy
- matplotlib
- cython
- pandas
- ipython
- sympy
- nose
- jupyter
- numba
- bokeh
- tensorflow
- scikit-learn

User instructions
-----------------

The module containing the latest versions of all packages at the beginning of September 2017 can be loaded as:

.. code-block:: bash

     module load cmth-scipy-stack/2017.09

Source
------

- http://www.python.org
- https://www.scipy.org
- https://pypi.python.org

License
-------

Python has the PSF License (Python Software Foundation). From version 2.2 on this is GPL compatible.

Admin notes
-----------

Compilation was all done on a workstation. The following additional dependencies were installed: ``libssl-dev``, ``libsqlite3-dev`` and ``tk-dev``.
First python was compiled. This was done with the Intel compilers as follows

.. code-block:: bash

    ml purge
    ml intel
    ./configure \
        --prefix=/common/debian/9.1/Compiler/intel/2017.4/cmth-scipy-stack/cmth-scipy-stack-2017.09 \
        --with-icc --without-gcc \
        --with-computed-gotos \
        --with-libm=-limf --with-cxx-main=icpc --with-threads \
        --enable-ipv6 \
        --enable-shared \
        --enable-optimizations \
        CC=icc CXX=icpc LD=xild AR=xiar \
        LIBS="-lpthread -limf -lirc" \
        CFLAGS="-O3 -fp-model strict -fp-model source -xHost -ipo -prec-div -prec-sqrt" \
        LDFLAGS="-ipo" \
        CPPFLAGS="" \
        CPP="icc -E" &> configure.log &
    ulimit -s unlimited
    make -j4 &> make.log &
    make install

This was made into a module before proceeding with the rest.

Then for NumPy the following ``site.cfg`` was used to compile it from source::

    [mkl]
    library_dirs = /common/debian/9.1/Core/intel/compilers_and_libraries_2017.4.196/linux/mkl/lib/intel64
    include_dirs = /common/debian/9.1/Core/intel/compilers_and_libraries_2017.4.196/linux/mkl/include
    mkl_libs = mkl_rt
    lapack_libs =

It was then compiled with:

.. code-block:: bash

    ml purge
    ml intel mkl cmth-scipy-stack
    python3 setup.py config --compiler=intelem build_clib --compiler=intelem build_ext --compiler=intelem install --prefix=/common/debian/9.1/Compiler/intel/2017.4/cmth-scipy-stack/cmth-scipy-stack-2017.09

SciPy was then compiled from source with:

.. code-block:: bash

    python3 setup.py config --compiler=intelem --fcompiler=intelem build_clib --compiler=intelem --fcompiler=intelem build_ext --compiler=intelem --fcompiler=intelem install --prefix=/common/debian/9.1/Compiler/intel/2017.4/cmth-scipy-stack/cmth-scipy-stack-2017.09

Everything else was installed from pip3 with the stack module loaded:

.. code-block:: bash

   package_list="matplotlib cython pandas ipython sympy nose jupyter numba bokeh tensorflow scikit-learn"
   for package in $package_list; do
     CC=icc CXX=icpc FC=ifort F77=ifort PYTHONUSERBASE=/common/debian/9.1/Compiler/intel/2017.4/cmth-scipy-stack/cmth-scipy-stack-2017.09 pip3 install --upgrade $package
   done