Scientific I/O Libraries

Contents

• Overview
• I/O Libraries
  • HDF5
  • netCDF-4
  • silo
• Viewers/Visualisation Tools
  • ncview
  • VisIT
  • paraview

This page describes the plethora of Scientific I/O libraries available on the system and how to access them. This is not entirely straight forward since these libraries also contain Fortran bindings which are compiler dependent. This problem is solved by using modules.

If you want to use the system compiler, i.e. gcc, g++ or gfortran, you do not need to do anything. The libraries are installed in the system path.

If you want to use modules from within a script you need to first setup your environment by adding

source /etc/profile.d/modules.sh

to your shell script.

You can get a list of available modules by running

module avail

and you can load a module using

module load NAME_OF_MODULE

First of all you will need to select a compiler you want to use. This affects mostly the Fortran compiler since (where available) the C compilers are generally produce interchangable object code. The following compilers are currently supported

• nag/5.1 The NAGware f95 compiler. The C and C++ parts of the libraries are compiled using the system compiler.
• intel/10.0 The Intel ifort version 10.0 Fortran95 compiler.
• intel/11.0 The Intel Professional Compiler Suite v 11.0 which includes C, C++ and Fortran95 compilers and various libraries. Here is the documentation. Although Intel supplies C and C++ compilers the system compilers where used

HDF5 is a very comprehensive albeit complicated to uses scientific data I/O library. It will store about anything you can think off (which is what makes it so complex to use). The library cannot be built with the NAG f95 compiler. So if you need to use HDF5 you cannot use the NAG compiler.

The module will set various environment variables. In particular it will set HDF_PREFIX to be the path to where the module is installed. You can use this for your compiler settings.

netCDF-4 is less complex than HDF5 and thus not support all the wonderful stuff HDF5 does but it is a lot easier to use. netCDF-4 supports a number of netCDF formats. By default you will get the classic format which means your datafiles will be most portable. You can also specify to use the 64bit version to be able to write larger files or the new HDF5 based file format which offers new features.

Except for the NAG compiler all netCDF packages depend on the HDF5 module. Load it first. Then load the netCDF module using

module load hdf5/1.8.3 netcdf/4.0.1

You can leave the version out if you are not bothered about the particular version. The module with the highest version number will be used

Compile your code using

${FC} -O2 -o foo foo.f -L${NETCDF_PREFIX}/lib -L${HDF5_PREFIX}/lib -I${NETCDF_PREFIX}/include -lnetcdf -lnetcdff -lhdf5 -lhdf5_hl

where ${FC} is your Fortran compiler. Leave out the hdf5 libraries if you are using the NAG compiler.

Checkout the netCDF documentation. I recommend the use of the Fortran90 interface. It hides a lot of the type-dependent pain of the API.

silo is another scientific I/O library which uses HDF5 as its backend data store. I have not used it (yet) but it looks very exciting. You can use it to store all kinds of field data defined on structured and unstructured grids. Load it using

module load silo/4.6.2 netcdf/4.0.1

VisIt is a 3D visualisation package that can read data in all sorts of formats, including HDF4, HDF5, netCDF and silo. Before you can run visit you need to load the module

module load applications/visit

You can find some example data files in the directory /usr/geos/VisIt/visit_data_files. Documentation can be found online.