Visualizing execution of parallel programs

So, in the last post I was surprised by the apparent slow down of the OpenMP version of Linpack with the increasing number of threads.

So I dicided to visualize what was happening on a desktop multicore machine.

After trying (very hard) to install the tracing infrastructure in cygwin, I finally gave up. Lots of problems, that I will detail later.

So I switched to my Ubuntu installation. Here it was fairly easy

First

download, build and install opari2. Opari is a source-to-source compiler to instrument OpenMP applications so that you can intercept interesing OpenMP events (like thread creation, etc.)

cd /usr/local/contrib
wget http://www.vi-hps.org/upload/packages/opari2/opari2-1.1.2.tar.gz
tar -xvf opari2-1.1.2.tar.gz
cd opari2-1.1.2
./configure
make
make install

then, download, build and install Scorep. Scorep is a collection of tools that allow you to easily instrument applications to generate profiling or tracing information. The same tool can be used to generate OTF2 traces (for Vampir analysis) or profiling info (for Scalasca, or Tau)

cd /usr/local/contrib
wget http://www.vi-hps.org/upload/packages/scorep/scorep-1.4.tar.gz
tar -xvf scorep-1.4.tar.gz
cd  scorep-1.4
./configure --with-opari2=/opt/opari2/ --with-shmem=no
make
make install



then I just compiled my OpenMP linpack

export SCOREP_ENABLE_TRACING=true
/opt/scorep/bin/scorep --compiler gcc -fopenmp -DROLL -DDP -DCTimer -DOMPC -DORDER=100 clinpack.c -o

and executed it, obtaining the valuable trace files :-)

then I installed the Vampir tool (https://www.vampir.eu/) to visualize my traces...

...and realized that actually I was getting some speed up. So I quickly realized that I had a problem with the time measuring function, which was collecting cpu time (aggregated per core) instead of wall clock time.

by compiling again with the correct flag, FLOPS numbers were now coherent 

/opt/scorep/bin/scorep --compiler gcc -fopenmp -DROLL -DDP -DGTODay -DOMPC -DORDER=100 clinpack.c -o

...as many times, stupid mistakes take some time to soar to the surface


About the problems compiling in cygwin. Some errors with C syntax, and many errors caused by the fact that the code assume that the file system of the host machine is case sensitive, which it's not the case for cygwin.