BSC Tools

Tracing jobs with BSC Tools

In this section you will find an introductory guide to get execution traces in Minotauro. The tracing tool Extrae supports many different tracing mechanisms, programming models and configurations. For detailed explanations and advanced options, please check the complete Extrae User Guide.

The most recent stable version of Extrae is always located at:

/apps/BSCTOOLS/extrae/latest/

This package is compatible with the default MPI runtime in MinoTauro (Bull MPI). Packages corresponding to older versions and enabling compatibility with other MPI runtimes (OpenMPI, MVAPICH) can be respectively found under this directory structure:

/apps/BSCTOOLS/extrae/<choose-version>/<choose-runtime>

In order to trace an execution, you have to load the module extrae and write a script that sets the variables to configure the tracing tool. Let's call this script trace.sh. It must be executable (chmod +x ./trace.sh). Then your job needs to run this script before executing the application.

Example for MPI jobs:

#!/bin/bash 
# @ output = tracing.out
# @ error = tracing.err
# @ total_tasks = 4 
# @ cpus_per_task = 1
# @ tasks_per_node = 12
# @ wall_clock_limit = 00:10

module load extrae

srun ./trace.sh ./app.exe

Example for threaded (OpenMP or pthreads) jobs:

#!/bin/bash 
# @ output = tracing.out
# @ error = tracing.err
# @ total_tasks = 1 
# @ cpus_per_task = 1
# @ tasks_per_node = 12
# @ wall_clock_limit = 00:10

module load extrae

./trace.sh ./app.exe

Example of trace.sh script:

#!/bin/bash 

export EXTRAE_CONFIG_FILE=./extrae.xml
export LD_PRELOAD=${EXTRAE_HOME}/lib/<tracing-library>
$*

Where:

EXTRAE_CONFIG_FILE points to the Extrae configuration file. Editing this file you can control the type of information that is recorded during the execution and where the resulting trace file is written, among other parameters. By default, the resulting trace file will be written into the current working directory. Configuration examples can be found at:
${EXTRAE_HOME}/share/examples
\<tracing-library> depends on the programming model the application uses:

Job type	Tracing library	An example to get started
MPI	libmpitrace.so (C codes) libmpitracef.so (Fortran codes)	MPI/ld-preload/job.lsf
OpenMP	libomptrace.so	OMP/run_ldpreload.sh
Pthreads	libpttrace.so	PTHREAD/README
*CUDA	libcudatrace.so	CUDA/run_instrumented.sh
MPI+CUDA	libcudampitrace.so (C codes) libcudampitracef.so (Fortran codes)	MPI+CUDA/ld-preload/job.slurm
OmpSs	-	OMPSS/job.lsf
*Sequential job (manual instrumentation)	libseqtrace.so	SEQ/run_instrumented.sh
**Automatic instrumentation of user functions and parallel runtime calls	-	SEQ/run_dyninst.sh

* Jobs that make explicit calls to the Extrae API do not load the tracing library via LD_PRELOAD, but link with the libraries instead.

** Jobs using automatic instrumentation via Dyninst neither load the tracing library via LD_PRELOAD nor link with it.

For other programming models and their combinations, check the full list of available tracing libraries at section 1.2.2 of the Extrae User Guide

Tracing jobs with BSC Tools​

Tracing jobs with BSC Tools