GitLab

It turned out that users ignored the warnings of the test programs
if their MPI library did not provide "MPI_THREAD_SERIALIED" or
"MPI_THREAD_MULTIPLE".

Thus for safty reasons, ELPA does from now on during the call to
"elpa_setup" check the provided threading level of the MPI library.
If the provided level is too low, ELPA will do the following

- limit the number of OpenMP threads **internal** to ELPA to 1
- print a warning about this
- ignore the settings of the user via
-- the OMP_NUM_THREADS variable
-- calling the set-method set("omp_threads", value)

These settings will **not** affect the run-time of a threaded
blas and lapack library, **if** the number of threads for these
libraries can be controlled independently of the OMP_NUM_THREADS
variable (for example for Intel's MKL one can use MKL_NUM_THREADS)

- Rename keyword "gpu" -> "nvidia-gpu"
- Add keyword "amd-gpu"

For ELPA 1stage now you can use GPUs in the OpenMP enabled version
of ELPA. Currently this implies that within ELPA1 only _ONE_
OpenMP thread is used, independent of how many threads the calling
application uses

Now "thres_pd" can be set and used in single-precision calculations.

This is to make the check_pd feature more usable in electronic structure
theory software.

When profiling the GPU version, NVTX can be used to highlight the
corresponding regions of the code in the timeline of the profiling tool
(nvvp or nsight systems). This is very useful to correlate what happens
on the GPU with what part of the code we are in.

ELPA can now be linked against a 64bit integer version of MPI and
ScalaPack. This is an experimental feature

The following points are still to be done
- does not work with real QR-decomposition
- generalized routines return wrong results
- the C tests and the C Cannon algorithm implementation do not
  work (no 64bit header files for MPI *at least* with Intel MPI)

also introducing --with-likwid parameter to the configure script

Can be used to compile with likwid and to enable the likwid to measure
performance of individual compute steps

This solves issue #66

The GPU initialization is actually quite constly, e.g. on Minsky it
takes roughly 0.7s. That is hurting performance for small matrices.
Thus a check has been added and now GPU should be initialized only the
first time.