deal.II can interface to the PETSc and Trilinos software libraries. Both of these libraries provide lots of functions for linear algebra, among other things, for example implementations of a variety of linear solvers, as well as various different sparse and dense matrix and vector formats. Trilinos also has many subpackages that deal with problems that go far beyond linear algebra, for example nonlinear solvers, automatic differentiation packages, uncertainty propagation engines, etc. Of particular interest to deal.II is their ability to provide this functionality both on sequential and parallel (using MPI) computers. PETSc is written in C; Trilinos is written in C++ and can be considered to be a more modern version of PETSc though both packages are under continuing development at their respective national laboratories.
deal.II has wrapper classes to the linear algebra parts of both packages that provide almost the same interfaces as the built-in deal.II linear algebra classes. We use these interfaces for parallel computations based on MPI since the native deal.II linear algebra classes lack this ability.
The use of PETSc and Trilinos is optional. To use the wrapper classes,
you first have to install these packages and
point deal.II's ./configure
to the
installation directories. This happens in similar ways for the two
packages:
PETSc usually requires you to set the
environment variables PETSC_DIR
and PETSC_ARCH
to a path to PETSc and denoting the architecture for which PETSc is
compiled. If these environment variables are set, then
deal.II will pick them up during
configuration, and store them. It will then also recognize that
PETSc shall be used, and enable the wrapper classes.
Alternatively, the --with-petsc=DIR
and
--with-petsc-arch=ARCH
switches to
./configure
can be used to override the values
of PETSC_DIR
and PETSC_ARCH
or if
these environment variables are not set at all. If you do have a
PETSc installation and have set the PETSC_DIR
and
PETSC_ARCH
environment variables but do not wish
deal.II to be configured for PETSc use, you
should specify --with-petsc=no
as a flag during
configuration.
Note that in order to use PETSc with deal.II with the current version of deal.II you will need to have at least PETSc version 2.3.0 installed, earlier releases are no longer supported. deal.II version 6.2 has been tested up to PETSc release 3.0.0-p11.
There is an additional caveat: PETSc appears not to co-operate
well when using threads and some programs crash when deal.II is
compiled in its usual mode supporting multithreading. If you see
this sort of behavior, we recommend to try disabling
multithreading upon configuration of deal.II
using the
--disable-threads
switch to ./configure
.
Installing both PETSc and deal.II together can be a bit of a challenge. A good summary of the relevant steps can be found on the Frequently Asked Questions page.
As above, set the TRILINOS_DIR
environment variable to the path to an existing Trilinos installation,
or use the --with-trilinos=/path/to/trilinos
switch of
the deal.II ./configure
script. It should point to the path
of which the include and lib directories are subdirs. The Trilinos
installation needs to provide the same kind of libraries that
deal.II is configured for, i.e. if you want deal.II
to use shared libraries, then Trilinos needs to use shared libraries as
well.
deal.II and its tutorial programs use several of the Trilinos sub-packages.
Up to version 9.0, Trilinos was configured and built using autoconf and
related tools. Configuring and building Trilinos with a sequence of
commands like the following should do the trick:
Obviously the path names in the first line and at the end of the
configure line need to be adjusted.
cd trilinos-9.0.2
mkdir build
cd build
../configure --enable-shared \
--with-cflags=-fPIC --with-cxxflags=-fPIC --with-fflags=-fPIC \
--enable-thyra --enable-stratimikos --enable-rtop --enable-teuchos \
--enable-sacado --enable-ml --enable-ifpack --enable-epetra \
--enable-belos --enable-aztecoo --enable-amesos \
--prefix=/home/bangerth/bin/trilinos-9.0.2
make
make install
Starting with version 10.0, Trilinos uses cmake to configure and
build. This is a bit cumbersome because you first have to install a
version of cmake that is at least a cmake 2.8 prerelease, and then enter
the following slightly longish set of commands:
Again, the path into which you want to install Trilinos in the second to
last line of the cmake command needs to be adjusted. Obviously, if you
want to use Trilinos with MPI on parallel machines, you also need to
flip the value of the
cd trilinos-10.0.2
mkdir build
cd build
cmake \
-D CMAKE_BUILD_TYPE:STRING=RELEASE \
-D CMAKE_CXX_FLAGS:STRING="-g -O3" \
-D CMAKE_C_FLAGS:STRING="-g -O3" \
-D CMAKE_FORTRAN_FLAGS:STRING="-g -O5" \
-D Trilinos_ENABLE_DEFAULT_PACKAGES:BOOL=OFF \
-D Trilinos_ENABLE_Epetra:BOOL=ON \
-D Trilinos_ENABLE_Tpetra:BOOL=ON \
-D Trilinos_ENABLE_ML:BOOL=ON \
-D Trilinos_ENABLE_Ifpack:BOOL=ON \
-D Trilinos_ENABLE_AztecOO:BOOL=ON \
-D Trilinos_ENABLE_Amesos:BOOL=ON \
-D Trilinos_ENABLE_Teuchos:BOOL=ON \
-D Trilinos_ENABLE_Sacado:BOOL=ON \
-D Trilinos_ENABLE_Thyra:BOOL=ON \
-D Trilinos_ENABLE_Didasko:BOOL=ON \
-D Trilinos_ENABLE_Stratimikos:BOOL=ON \
-D Trilinos_ENABLE_Belos:BOOL=ON \
-D Trilinos_ENABLE_EpetraExt:BOOL=ON \
-D Trilinos_ENABLE_Kokkos:BOOL=ON \
-D Trilinos_ENABLE_CTrilinos:BOOL=ON \
-D Trilinos_ENABLE_ForTrilinos:BOOL=ON \
-D Trilinos_ENABLE_TESTS:BOOL=OFF \
-D Trilinos_ENABLE_EXAMPLES:BOOL=OFF \
-D TPL_ENABLE_MPI:BOOL=OFF \
-D Trilinos_EXTRA_LINK_FLAGS:STRING="-lgfortran" \
-D CMAKE_VERBOSE_MAKEFILE:BOOL=FALSE \
-D Trilinos_VERBOSE_CONFIGURE:BOOL=FALSE \
-D BUILD_SHARED_LIBS:BOOL=ON \
-D CMAKE_INSTALL_PREFIX:PATH=/w/bangerth/share/x86_64/trilinos-10.0.2 \
`pwd`/../
make
make install
TPL_ENABLE_MPI
flag above.
The above scheme works if Trilinos is installed in its own directory
and header and library files are in subdirectories of the directory
given to --with-trilinos
. This scheme doesn't work,
however, if Trilinos has been installed as a regular package on a
system, for example into the /usr
or /opt
directories.
In that case, paths to include and library files may be specified
separately using the --with-trilinos-include
and --with-trilinos-libs
switches
to ./configure
. Alternatively, this information can also
be passed to the configuration script by setting the environment
variables TRILINOS_INCDIR, TRILINOS_LIBDIR
.