Commit 685e9338 authored by Andreas Marek's avatar Andreas Marek
Browse files

Merge branch 'master_pre_stage' into CUDA_AWARE_MPI

parents bacbba45 def7e7c3
This diff is collapsed.
......@@ -2,7 +2,11 @@ Changelog for next release
- not yet decided
Changelog for ELPA 2021.05.001.rc2
Changelog for ELPA 2021.05.002
- no feature changes
- correct the SO version which was wrong in ELPA 2021.05.001
Changelog for ELPA 2021.05.001
- allow the user to set the mapping of MPI tasks to GPU id per set/get
- experimental feature: port to AMD GPUS, works correctly, performance yet
......
......@@ -3,6 +3,13 @@
For more details and recent updates please visit the online [issue system](https://gitlab.mpcdf.mpg.de/elpa/elpa/issues)
Issues which are not mentioned in a newer release are (considered as) solved.
### ELPA 2021.05.002 release ###
Correct the SO version which wrongly in
release 2021.05.001 went backwards
### ELPA 2021.05.001 release ###
Currently no issues known
### ELPA 2020.11.001 release ###
- fixes a problem with GPU kernels
- fixes a problem with VSX kernels
......
......@@ -2,7 +2,7 @@
## Current Release ##
The current release is ELPA 2021.05.001.rc2 The current supported API version
The current release is ELPA 2021.05.002 The current supported API version
is 20210430. This release supports the earliest API version 20170403.
The release ELPA 2018.11.001 was the last release, where the legacy API has been
......@@ -130,7 +130,7 @@ the possible configure options.
## Using *ELPA*
Please have a look at the [USERS_GUIDE](./documentation/USERS_GUIDE.md) file, to get a documentation or at the [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.001.rc2/html/index.html) doxygen documentation, where you find the definition of the interfaces. You might want to have a look at the [PERFORMANCE tuning document](./documentation/PERFORMANCE_TUNING.md) to avoid some usual pitfalls.
Please have a look at the [USERS_GUIDE](./documentation/USERS_GUIDE.md) file, to get a documentation or at the [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.002/html/index.html) doxygen documentation, where you find the definition of the interfaces. You might want to have a look at the [PERFORMANCE tuning document](./documentation/PERFORMANCE_TUNING.md) to avoid some usual pitfalls.
## Contributing to *ELPA*
......
This file contains the release notes for the ELPA 2021.05.001.rc2 version
This file contains the release notes for the ELPA 2021.05.002 version
What is new?
-------------
For detailed information about changes since release ELPA 2020.11.001 please have a look at the Changelog file
- bugfixes
- experimental feature: support for AMD GPUs
- smaller bugfixes
- fix the SO version which in release 2021.05.001 is wrong
ABI change
......
......@@ -654,6 +654,7 @@ for cc, fc, m, o, p, a, b, g, instr, addr, na in product(
sorted(address_sanitize_flag.keys()),
sorted(matrix_size.keys())):
cov = "no-coverage"
nev = 150
......@@ -744,7 +745,7 @@ for cc, fc, m, o, p, a, b, g, instr, addr, na in product(
continue
#at the moment gpu testing only on AVX machines or minskys
if (g == "with-gpu" and (instr !="avx" and instr !="power8")):
if (g == "with-gpu" and (instr !="avx512" and instr !="power8")):
continue
# #on KNL do only intel tests
......
......@@ -136,7 +136,7 @@ then
if [ "$gpuJob" == "yes" ]
then
cp $HOME/runners/job_script_templates/run_${CLUSTER}_1node_2GPU.sh .
echo "if \[ \$SLURM_PROCID -eq 0 \]" >> ./run_${CLUSTER}_1node_GPU.sh
echo "if \[ \$SLURM_PROCID -eq 0 \]" >> ./run_${CLUSTER}_1node_2GPU.sh
echo "then" >> ./run_${CLUSTER}_1node_2GPU.sh
echo "echo \"process \$SLURM_PROCID running configure\"" >> ./run_${CLUSTER}_1node_2GPU.sh
echo "#decouple from SLURM (maybe this could be removed)" >> ./run_${CLUSTER}_1node_2GPU.sh
......
......@@ -26,7 +26,7 @@ AM_SILENT_RULES([yes])
# by the current interface, as they are ABI compatible (e.g. only new symbols
# were added by the new interface)
#
AC_SUBST([ELPA_SO_VERSION], [15:3:1])
AC_SUBST([ELPA_SO_VERSION], [16:2:1])
# AC_DEFINE_SUBST(NAME, VALUE, DESCRIPTION)
# -----------------------------------------
......@@ -2460,13 +2460,13 @@ if test x"${ax_cv_have_avx512f_cpu_ext}" = x"yes" -a x"${need_avx512}" = x"no";
echo " "
fi
echo " "
echo "***********************************************************************"
echo "* This is the first release candidate of ELPA 2021.05.001.rc2 *"
echo "* There might be still some changes until the final release of *"
echo "* ELPA 2021.05.001 *"
echo "***********************************************************************"
echo " "
#echo " "
#echo "***********************************************************************"
#echo "* This is the first release candidate of ELPA 2021.05.002.rc2 *"
#echo "* There might be still some changes until the final release of *"
#echo "* ELPA 2021.05.002 *"
#echo "***********************************************************************"
#echo " "
if test x"$enable_kcomputer" = x"yes" ; then
echo " "
......
......@@ -2,7 +2,7 @@
## 0. Preamble ##
This file provides documentation on how to build the *ELPA* library in **version ELPA-2021.05.001.rc2**.
This file provides documentation on how to build the *ELPA* library in **version ELPA-2021.05.002**.
With release of **version ELPA-2017.05.001** the build process has been significantly simplified,
which makes it easier to install the *ELPA* library.
......@@ -10,7 +10,7 @@ The release ELPA 2018.11.001 was the last release, where the legacy API has been
enabled by default (and can be disabled at build time).
With the release ELPA 2019.11.001, the legacy API has been deprecated and the support has been closed.
The release of ELPA 2021.05.001.rc2 does change the API and ABI compared to the release 2019.11.001, since
The release of ELPA 2021.05.002 does change the API and ABI compared to the release 2019.11.001, since
the legacy API has been dropped.
## 1. How to install *ELPA* ##
......@@ -212,7 +212,7 @@ It might be necessary to also set the options (please see configure --help)
--with-GPU-compute-capability
```
Please note that with release 2021.05.001.rc2 also GPU support of AMD and Intel GPUS has been introduced.
Please note that with release 2021.05.002 also GPU support of AMD and Intel GPUS has been introduced.
However, this is still considered experimental. Especially the following features do not yet work, or have not
been tested.
......
......@@ -147,7 +147,7 @@ Local documentation (via man pages) should be available (if *ELPA* has been inst
For example `man elpa2_print_kernels` should provide the documentation for the *ELPA* program which prints all
the available kernels.
Also a [online doxygen documentation](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.001.rc2/html/index.html)
Also a [online doxygen documentation](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.002/html/index.html)
for each *ELPA* release is available.
......@@ -13,7 +13,7 @@ Local documentation (via man pages) should be available (if *ELPA* has been inst
For example `man elpa2_print_kernels` should provide the documentation for the *ELPA* program, which prints all
the available kernels.
Also a [online doxygen documentation](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.001.rc2/html/index.html)
Also a [online doxygen documentation](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.002/html/index.html)
for each *ELPA* release is available.
......@@ -180,7 +180,7 @@ The following table gives a list of all supported parameters which can be used t
## III) List of computational routines ##
The following compute routines are available in *ELPA*: Please have a look at the man pages or [online doxygen documentation] (http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.001.rc2/html/index.html) for details.
The following compute routines are available in *ELPA*: Please have a look at the man pages or [online doxygen documentation] (http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.002/html/index.html) for details.
| Name | Purpose | since API version |
......
......@@ -23,7 +23,7 @@ The *ELPA* library consists of two main parts:
Both variants of the *ELPA* solvers are available for real or complex singe and double precision valued matrices.
Thus *ELPA* provides the following user functions (see man pages or [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.001.rc2/html/index.html) for details):
Thus *ELPA* provides the following user functions (see man pages or [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2021.05.002/html/index.html) for details):
- elpa_get_communicators : set the row / column communicators for *ELPA*
- elpa_solve_evp_complex_1stage_{single|double} : solve a {single|double} precision complex eigenvalue proplem with the *ELPA 1stage* solver
......
......@@ -19,7 +19,7 @@
%define with_openmp 0
Name: elpa
Version: 2021.05.001.rc2
Version: 2021.05.002
Release: 1
Summary: A massively parallel eigenvector solver
License: LGPL-3.0
......
......@@ -84,6 +84,36 @@ module mod_check_for_gpu
stop
endif
success = .true.
numberOfDevices = -1
#ifdef WITH_NVIDIA_GPU_VERSION
! call getenv("CUDA_PROXY_PIPE_DIRECTORY", envname)
success = cuda_getdevicecount(numberOfDevices)
#endif
#ifdef WITH_AMD_GPU_VERSION
! call getenv("CUDA_PROXY_PIPE_DIRECTORY", envname)
success = hip_getdevicecount(numberOfDevices)
#endif
if (.not.(success)) then
#ifdef WITH_NVIDIA_GPU_VERSION
print *,"error in cuda_getdevicecount"
#endif
#ifdef WITH_AMD_GPU_VERSION
print *,"error in hip_getdevicecount"
#endif
stop 1
endif
#ifdef WITH_INTEL_GPU_VERSION
gpuAvailable = .false.
numberOfDevices = -1
numberOfDevices = 1
print *,"Manually setting",numberOfDevices," of GPUs"
if (numberOfDevices .ge. 1) then
gpuAvailable = .true.
endif
#endif
if (obj%is_set("use_gpu_id") == 1) then
call obj%get("use_gpu_id", use_gpu_id, error)
if (use_gpu_id == -99) then
......@@ -110,6 +140,12 @@ module mod_check_for_gpu
endif
endif
if (use_gpu_id+1 .gt. numberOfDevices) then
print *,"Task=",myid," wants to use GPU id=",use_gpu_id," allowed (0:#GPUs-1)"
print *,"However, there are only ",numberOfDevices," on the node"
stop 1
endif
success = .true.
#ifdef WITH_NVIDIA_GPU_VERSION
success = cuda_setdevice(use_gpu_id)
......
......@@ -412,8 +412,8 @@ module cuda_c_kernel
integer(kind=c_intptr_t) :: a_dev
integer(kind=c_intptr_t) :: row_group_dev
#ifdef WITH_NVIDIA_GPU_VERSION
call launch_my_pack_c_cuda_kernel_complex_double(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, a_dev, &
row_group_dev)
call launch_my_pack_c_cuda_kernel_complex_double(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, &
a_dev, row_group_dev)
#endif
end subroutine
......@@ -426,8 +426,8 @@ module cuda_c_kernel
integer(kind=c_intptr_t) :: a_dev
integer(kind=c_intptr_t) :: row_group_dev
#ifdef WITH_NVIDIA_GPU_VERSION
call launch_my_pack_c_cuda_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, a_dev, &
row_group_dev)
call launch_my_pack_c_cuda_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, &
a_dev, row_group_dev)
#endif
end subroutine
#endif
......
......@@ -277,12 +277,12 @@ module gpu_c_kernel
integer(kind=c_intptr_t) :: a_dev
integer(kind=c_intptr_t) :: row_group_dev
#ifdef WITH_NVIDIA_GPU_VERSION
call launch_my_pack_c_cuda_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, a_dev, &
row_group_dev)
call launch_my_pack_c_cuda_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, &
a_dev, row_group_dev)
#endif
#ifdef WITH_NVIDIA_GPU_VERSION
call launch_my_pack_c_hip_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, a_dev, &
row_group_dev)
#ifdef WITH_AMD_GPU_VERSION
call launch_my_pack_c_hip_kernel_complex_single(row_count, n_offset, max_idx,stripe_width,a_dim2, stripe_count, l_nev, &
a_dev, row_group_dev)
#endif
end subroutine
#endif
......
......@@ -689,7 +689,8 @@ program test
#if (TEST_GPU_SET_ID == 1) && (TEST_INTEL_GPU == 0)
! simple test
! Can (and should) fail often
gpuID = mod(myid,2)
!gpuID = mod(myid,2)
gpuID = mod(myid,1)
print *,"Task",myid,"wants to use GPU",gpuID
call e%set("use_gpu_id", int(gpuID,kind=c_int), error_elpa)
assert_elpa_ok(error_elpa)
......
AC_PREREQ([2.69])
AC_INIT([elpa_test_project],[2021.05.001.rc2], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.001.rc2"
AC_INIT([elpa_test_project],[2021.05.002], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.002"
AC_CONFIG_SRCDIR([src/test_real.F90])
AM_INIT_AUTOMAKE([foreign -Wall subdir-objects])
......
AC_PREREQ([2.69])
AC_INIT([elpa_test_project],[2021.05.001.rc2], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.001.rc2"
AC_INIT([elpa_test_project],[2021.05.002], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.002"
AC_CONFIG_SRCDIR([src/test_real2.F90])
AM_INIT_AUTOMAKE([foreign -Wall subdir-objects])
......
AC_PREREQ([2.69])
AC_INIT([elpa_test_project],[2021.05.001.rc2], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.001.rc2"
AC_INIT([elpa_test_project],[2021.05.002], elpa-library@mpcdf.mpg.de)
elpaversion="2021.05.002"
AC_CONFIG_SRCDIR([src/test_real.c])
AM_INIT_AUTOMAKE([foreign -Wall subdir-objects])
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment