.. index:: hdf5, managing scientific data
.. _hdf5:

HDF5
====

Description
-----------

HDF5 is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed for flexible and efficient I/O and for high volume and complex data. HDF5 is portable and is extensible, allowing applications to evolve in their use of HDF5.

User instructions
-----------------

hdf5 is available via the modules system.  To add the serial version to your environment run:

.. code-block:: bash

    $ module load hdf5/1.8.19

and to add the MPI version:

.. code-block:: bash

    $ module load hdf5-mpi/1.8.19


Please see the `HDF5 website <http://www.hdfgroup.org/HDF5/>`_ for user tutorials and full documentation.

Source
------

http://www.hdfgroup.org/HDF5/

License
-------

BSD-style open source license.  See `here <http://www.hdfgroup.org/products/licenses.html>`_ for more details.

Admin notes
-----------

The build system uses a standard autoconf approach.  Given growing interest in using HDF5, I compiled both serial and parallel versions for two recent releases and for several compiler and openmpi combinations. Note not all tests pass with the 1.10 version as noted below so the 1.8 version is set as default.

.. code-block:: bash

    mkdir build-1.8.19-gcc63; cd build-1.8.19-gcc63
    module load gcc/6.3.0
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/Compiler/gcc/6.3/hdf5/hdf5-1.8.19 --enable-fortran --enable-fortran2003 --enable-cxx
    ( make -j4 && make check && make install ) > make.log 2>&1 &

    module load gcc/6.3.0 openmpi/2.1.1
    cd ..; mkdir build-1.8.19-gcc63ompi21; cd build-1.8.19-gcc63ompi21
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/MPI/gcc/6.3/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.8.19 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install

    cd ..; mkdir build-1.8.19-gcc71; cd build-1.8.19-gcc71
    module load gcc/7.1.0
    module unload openmpi
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/Compiler/gcc/7.1/hdf5/hdf5-1.8.19 --enable-fortran --enable-fortran2003 --enable-cxx
    ( make -j4 && make check && make install ) > make.log 2>&1 &

    module load gcc/7.1.0 openmpi/2.1.1
    cd ..; mkdir build-1.8.19-gcc73ompi21; cd build-1.8.19-gcc73ompi21
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/MPI/gcc/7.1/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.8.19 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install

    module load gcc/6.3.0
    module unload openmpi
    cd ..; mkdir build-1.10.1-gcc63; cd build-1.10.1-gcc63
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/Compiler/gcc/6.3/hdf5/hdf5-1.10.1 --enable-fortran --enable-fortran2003 --enable-cxx
    ( make -j4 && make check && make install ) > make.log 2>&1 &
    # There were 3 errors here on make check

    module load gcc/6.3.0 openmpi/2.1.1
    cd ..; mkdir build-1.10.1-gcc63ompi21; cd build-1.10.1-gcc63ompi21
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/MPI/gcc/6.3/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.10.1 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install
    # 1 error on check here

    module load gcc/7.1.0
    module unload openmpi
    cd ..; mkdir build-1.10.1-gcc71; cd build-1.10.1-gcc71
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/Compiler/gcc/7.1/hdf5/hdf5-1.10.1 --enable-fortran --enable-fortran2003 --enable-cxx
    ( make -j4 && make check && make install ) > make.log 2>&1 &
    # 3 errors here on check

    module load gcc/7.1.0 openmpi/2.1.1
    cd ..; mkdir build-1.10.1-gcc71ompi21; cd build-1.10.1-gcc71ompi21
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/MPI/gcc/7.1/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.10.1 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install
    # PHDF5 tests failed here.

    mkdir build-1.8.19-intel2017; cd build-1.8.19-intel2017
    module load intel/2017.4.196
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/Compiler/intel/2017.4/hdf5/hdf5-1.8.19 --enable-fortran --enable-fortran2003 --enable-cxx CC=icc CXX=icpc F77=ifort
    ( make -j4 && make check && make install ) > make.log 2>&1 &

    # Note - the gcc install was missing the mpiCC definition - maybe I should go
    # back and redo it. Docs should be fixed too. Although checks turned out the same.
    module load intel/2017.4.196 openmpi/2.1.1
    cd ..; mkdir build-1.8.19-intel2017ompi21; cd build-1.8.19-intel2017ompi21
    ../hdf5-1.8.19/configure --prefix=/common/debian/9.1/MPI/intel/2017.4/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.8.19 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc CXX=mpicxx F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install

    cd ..; mkdir build-1.10.1-intel2017; cd build-1.10.1-intel2017
    module purge
    module load intel/2017.4.196
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/Compiler/intel/2017.4/hdf5/hdf5-1.10.1 --enable-fortran --enable-fortran2003 --enable-cxx CC=icc CXX=icpc F77=ifort
    ( make -j4 && make check && make install ) > make.log 2>&1 &

    module purge
    module load intel/2017.4.196 openmpi/2.1.1
    cd ..; mkdir build-1.10.1-intel2017ompi21; cd build-1.10.1-intel2017ompi21
    ../hdf5-1.10.1/configure --prefix=/common/debian/9.1/MPI/intel/2017.4/openmpi/2.1/hdf5-mpi/hdf5-mpi-1.10.1 --enable-fortran --enable-fortran2003 --enable-parallel CC=mpicc CXX=mpicxx F77=mpif90
    ( make -j4 && make check ) > make.log 2>&1 &
    make install
    # PHDF5 tests failed.