Merge branch 'master_pre_stage' into ELPA2_GPU

7097a246 · Andreas Marek · 1fb3347b · 919e9ba0 · 7097a246 · 7097a246
Commit 7097a246 authored Mar 06, 2020 by Andreas Marek
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
--- a/Changelog
+++ b/Changelog
@@ -2,6 +2,20 @@ Changelog for upcoming release
 - not yet decided
+Changelog for ELPA 2019.11.001
+- solve a bug when using parallel make builds
+- check the cpuid set during build time
+- add experimental feature "heterogenous-cluster-support"
+- add experimental feature for 64bit integer LAS/LAPACK/SCALAPACK support
+- add experimental feature for 64bit integer MPI support
+- support of ELPA for real valued skew-symmetric matrices, please cite:
+  https://arxiv.org/abs/1912.04062 
+- cleanup of the GPU version
+- bugfix in the OpenMP version
+- bugfix on the Power8/9 kernels
+- bugfix on ARM aarch64 FMA kernels
 Changelog for ELPA 2019.05.002

--- a/DEPRECATED_FEATURES.md
+++ b/DEPRECATED_FEATURES.md
@@ -8,37 +8,9 @@ in the (near) future from the *ELPA* library.
 With the release of ELPA 2017.05.001 a new, more general API for the library has
 been published. All new features of ELPA will only be accesible via this new interface.
-For ease of transion, the API as defined in release ELPA 2016.11.001 has been frozen in
+For ease of transion, the old API as defined in release ELPA 2016.11.001 has been frozen in
-and will be still supported for some time, albeit without any new features.
+and will be still supported for some time, albeit without any new features. This old API has been
+deprecated in release ELPA 2019.11.001 and has been removed.
-The old, legacy API will be deprecated with the release ELPA 2019.11.001 !
-Independent of the freezing in of the old, legacy API from the release 2016.11.001 the
-following listed interfaces will be removed at some time.
-In order to unfiy the namespace of the *ELPA* public interfaces, several interfaces
-have been replaced by new names. The old interfaces will be removed
-| Deprecated interface     |        Replacement                    |        Comment                |
-|:------------------------ |:------------------------------------- |:----------------------------- |
-| get_elpa_row_col_coms    | elpa_get_communicators                | (removed since 2017.11.001)   | 
-| get_elpa_communicators   | elpa_get_communicators                | (removed since 2017.11.001)   | 
-| solve_evp_real           | elpa_solve_evp_real_1stage_double     | (removed since 2017.11.001)   | 
-| solve_evp_complex        | elpa_solve_evp_complex_1stage_double  | (removed since 2017.11.001)   | 
-| solve_evp_real_1stage    | elpa_solve_evp_real_1stage_double     | (removed since 2019.05.001)   | 
-| solve_evp_complex_1stage | elpa_solve_evp_complex_1stage_double  | (removed since 2019.05.001)   | 
-| solve_evp_real_2stage    | elpa_solve_evp_real_2stage_double     | (removed since 2019.05.001)   | 
-| solve_evp_complex_2stage | elpa_solve_evp_complex_2stage_double  | (removed since 2019.05.001)   | 
-| mult_at_b_real           | elpa_mult_at_b_real_double            | (removed since 2019.05.001)   | 
-| mult_ah_b_complex        | elpa_mult_ah_b_complex_double         | (removed since 2019.05.001)   | 
-| invert_trm_real          | elpa_invert_trm_real_double           | (removed since 2019.05.001)   | 
-| invert_trm_complex       | elpa_invert_trm_complex_double        | (removed since 2019.05.001)   | 
-| cholesky_real            | elpa_cholesky_real_double             | (removed since 2019.05.001)   | 
-| cholesky_complex         | elpa_cholesky_complex_double          | (removed since 2019.05.001)   | 
-| solve_tridi              | elpa_solve_tridi_double               | (removed since 2019.05.001)   |
-For all symbols also the corresponding "_single" routines are available

--- a/Doxyfile.in
+++ b/Doxyfile.in
@@ -885,6 +885,7 @@ EXCLUDE                = @top_srcdir@/src/GPU/check_for_gpu.F90 \
 			@top_srcdir@/src/elpa_c_interface.c \
 			@top_srcdir@/src/elpa2/mod_pack_unpack_cpu.F90 \
 			@top_srcdir@/src/elpa2/elpa2_symm_matrix_allreduce_real_template.F90 \
+			@top_srcdir@/src/elpa2/elpa2_ssymm_matrix_allreduce_real_template.F90 \
 			@top_srcdir@/src/elpa2/elpa2_tridiag_band_template.F90 \
 			@top_srcdir@/src/elpa2/mod_redist_band.F90 \
 			@top_srcdir@/src/elpa2/pack_unpack_cpu.F90 \
@@ -1001,6 +1002,7 @@ EXCLUDE                = @top_srcdir@/src/GPU/check_for_gpu.F90 \
 			@top_srcdir@/src/elpa1/elpa_solve_tridi_impl_public.F90 \
 			@top_srcdir@/src/elpa1/elpa1_trans_ev_template.F90 \
 			@top_srcdir@/src/elpa1/elpa_transpose_vectors.F90 \
+			@top_srcdir@/src/elpa1/elpa_transpose_vectors_ss.F90 \
 			@top_srcdir@/src/elpa1/elpa1_auxiliary.F90 \
 			@top_srcdir@/src/elpa1/elpa1_tridiag_template.F90 \
 			@top_srcdir@/src/elpa1/elpa1_tools_template.F90 \

--- a/INSTALL.md
+++ b/INSTALL.md
@@ -2,7 +2,7 @@
 ## Preamble ##
-This file provides documentation on how to build the *ELPA* library in **version ELPA-2019.05.002**.
+This file provides documentation on how to build the *ELPA* library in **version ELPA-2019.11.001**.
 With release of **version ELPA-2017.05.001** the build process has been significantly simplified,
 which makes it easier to install the *ELPA* library.
@@ -14,9 +14,11 @@ The release ELPA 2018.11.001 was the last release, where the legacy API has been
 enabled by default (and can be disabled at build time).
 With release ELPA 2019.05.001 the legacy API is disabled by default, however,
 can be still switched on at build time.
-Most likely with the release ELPA 2019.11.001 the legacy API will be deprecated and not supported anymore.
+With this release ELPA 2019.11.001, the legacy API will be deprecated and not supported anymore.
+The release of ELPA 2019.11.001 does change the API and ABI compared to the release 2019.05.002, since
+the legacy API has been droped.
-The release of ELPA 2019.05.002 does not change the API and ABI compared to the release 2019.05.001
 ## How to install *ELPA* ##
 First of all, if you do not want to build *ELPA* yourself, and you run Linux,
@@ -84,7 +86,11 @@ An excerpt of the most important (*ELPA* specific) options reads as follows:
 |  --disable-Fortran2008-features      | disable Fortran 2008 if compiler does not support it  |
 |  --enable-pyhton                     | build and install python wrapper, default no          |
 |  --enable-python-tests               | enable python tests, default no.                      |
+|  --enable-skew-symmetric-support     | enable support for real valued skew-symmetric matrices |
+|  --enable-store-build-config         | stores the build config in the library object |
+|  --64bit-integer-math-support        | assumes that BLAS/LAPACK/SCALAPACK use 64bit integers (experimentatl) |
+|  --64bit-integer-mpi-support         | assumes that MPI uses 64bit integers (experimental) |
+|  --heterogenous-cluster-support      | allows ELPA to run on clusters of nodes with different Intel CPUs (experimental) |
 We recommend that you do not build ELPA in its main directory but that you use it
 in a sub-directory:

--- a/ISSUES.md
+++ b/ISSUES.md
@@ -4,13 +4,21 @@ For more details and recent updates please visit the online [issue system] (http
 Issues which are not mentioned in a newer release are (considered as) solved.
 ### ELPA 2019.11.001 release ###
- same issues as in ELPA 2017.11.001
+- memory leak in in GPU version has been fixed
+- no other issues currently known
+### ELPA 2019.05.002 release ###
+- memory leak in GPU version
+### ELPA 2019.05.001 release ###
+- memory leak in GPU version
 ### ELPA 2018.11.001 release ###
- same issues as in ELPA 2017.11.001
+- on (officially not supported) 32bit systems hangs in MPI can occur
 ### ELPA 2018.05.001 release ###
- same issues as in ELPA 2017.11.001
+- on (officially not supported) 32bit systems hangs in MPI can occur
+- printing of autotune has been implemented
 ### ELPA 2017.11.001 release ###
 - the elpa autotune print functions cannot print at the moment

--- a/Makefile.am
+++ b/Makefile.am
@@ -21,15 +21,6 @@ libelpa@SUFFIX@_public_la_SOURCES = \
  src/elpa_api.F90 \
  src/elpa_constants.F90
-if ENABLE_LEGACY
-libelpa@SUFFIX@_public_la_SOURCES += \
-  src/elpa_driver/legacy_interface/elpa.F90 \
-  src/elpa1/legacy_interface/elpa1.F90 \
-  src/elpa2/legacy_interface/elpa2.F90 \
-  src/elpa2/legacy_interface/elpa2_utilities.F90 \
-  src/elpa1/legacy_interface/elpa1_auxiliary.F90
-endif
 # internal parts
 noinst_LTLIBRARIES += libelpa@SUFFIX@_private.la
 libelpa@SUFFIX@_private_la_FCFLAGS = $(AM_FCFLAGS) $(FC_MODOUT)private_modules $(FC_MODINC)private_modules
@@ -64,29 +55,15 @@ libelpa@SUFFIX@_private_la_SOURCES = \
  src/elpa1/elpa1.F90 \
  src/elpa2/elpa2.F90 \
  src/elpa_generalized/cannon.c \
-  #src/elpa_generalized/test_c_bindings.c \
  src/helpers/matrix_plot.F90 \
+  src/general/mod_elpa_skewsymmetric_blas.F90 \
  src/elpa_index.c
 libelpa@SUFFIX@_private_la_SOURCES += src/elpa_c_interface.c
-# elpa_utilities.F90 is private in new API, public in legacy
-if ENABLE_LEGACY
-libelpa@SUFFIX@_public_la_SOURCES += \
-  src/general/elpa_utilities.F90
-else
 libelpa@SUFFIX@_private_la_SOURCES += \
  src/general/elpa_utilities.F90
-endif
-if ENABLE_LEGACY
-libelpa@SUFFIX@_private_la_SOURCES += \
-  src/elpa_driver/legacy_interface/elpa_driver_c_interface.F90 \
-  src/elpa1/legacy_interface/elpa_1stage_c_interface.F90 \
-  src/elpa2/legacy_interface/elpa_2stage_c_interface.F90
-endif
 EXTRA_libelpa@SUFFIX@_private_la_DEPENDENCIES = \
  src/elpa1/elpa_reduce_add_vectors.F90 \
@@ -122,28 +99,12 @@ EXTRA_libelpa@SUFFIX@_private_la_DEPENDENCIES = \
  src/elpa1/elpa_invert_trm.F90 \
  src/elpa1/elpa_multiply_a_b.F90 \
  src/elpa1/elpa_solve_tridi_impl_public.F90 \
+  src/general/elpa_ssr2_template.F90 \
+  src/general/elpa_ssmv_template.F90 \
  src/general/precision_macros.h \
  src/general/precision_typedefs.h \
  src/general/precision_kinds.F90
-if ENABLE_LEGACY
-EXTRA_libelpa@SUFFIX@_private_la_DEPENDENCIES += \
-  src/elpa1/legacy_interface/elpa1_template.F90 \
-  src/elpa2/legacy_interface/elpa2_template.F90 \
-  src/elpa1/legacy_interface/elpa1_c_interface_template.F90 \
-  src/elpa2/legacy_interface/elpa2_c_interface_template.F90 \
-  src/elpa_driver/legacy_interface/elpa_driver_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_solve_tridi_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_mult_at_b_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_mult_ah_b_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_invert_trm_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_cholesky_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_cholesky_template.F90 \
-  src/elpa1/legacy_interface/elpa_invert_trm.F90 \
-  src/elpa1/legacy_interface/elpa_multiply_a_b.F90 \
-  src/elpa1/legacy_interface/elpa_solve_tridi.F90
-endif
 if HAVE_DETAILED_TIMINGS
 libelpa@SUFFIX@_private_la_SOURCES += \
  src/ftimings/ftimings.F90 \
@@ -155,11 +116,6 @@ libelpa@SUFFIX@_private_la_SOURCES += \
  src/ftimings/virtual_memory.c \
  src/ftimings/papi.c
-if ENABLE_LEGACY
-libelpa@SUFFIX@_private_la_SOURCES += \
-  src/helpers/timer.F90
-endif
 else
 libelpa@SUFFIX@_private_la_SOURCES += \
  src/helpers/timer_dummy.F90
@@ -176,6 +132,11 @@ if !HAVE_DETAILED_TIMINGS
 endif
 endif
+if HAVE_HETEROGENOUS_CLUSTER_SUPPORT
+  libelpa@SUFFIX@_private_la_SOURCES +=  src/helpers/get_cpuid_set.c src/helpers/mod_simd_kernel.F90
+endif
 if WITH_REAL_GENERIC_KERNEL
  libelpa@SUFFIX@_private_la_SOURCES +=  src/elpa2/kernels/real.F90
 endif
@@ -479,16 +440,15 @@ nobase_elpa_include_HEADERS = \
  $(wildcard modules/*) \
  src/helpers/lapack_interfaces.h \
  src/helpers/scalapack_interfaces.h \
+  elpa/elpa_simd_constants.h \
  elpa/elpa.h \
-  elpa/elpa_generic.h \
+  elpa/elpa_generic.h
-  elpa/elpa_legacy.h
 nobase_nodist_elpa_include_HEADERS = \
  elpa/elpa_version.h \
  elpa/elpa_constants.h \
  elpa/elpa_generated.h \
-  elpa/elpa_generated_c_api.h \
+  elpa/elpa_generated_c_api.h
-  elpa/elpa_generated_legacy.h
 dist_man_MANS = \
  man/elpa2_print_kernels.1 \
@@ -498,6 +458,8 @@ dist_man_MANS = \
  man/elpa_setup.3 \
  man/elpa_eigenvalues.3 \
  man/elpa_eigenvectors.3 \
+  man/elpa_skew_eigenvalues.3 \
+  man/elpa_skew_eigenvectors.3 \
  man/elpa_generalized_eigenvectors.3 \
  man/elpa_generalized_eigenvalues.3 \
  man/elpa_cholesky.3 \
@@ -511,57 +473,11 @@ dist_man_MANS = \
  man/elpa_autotune_save_state.3 \
  man/elpa_autotune_load_state.3 \
  man/elpa_autotune_print_state.3 \
-  man/elpa_uninit.3
-if ENABLE_LEGACY
-dist_man_MANS += \
-  man/solve_evp_real_1stage_double.3 \
-  man/solve_evp_complex_1stage_double.3 \
-  man/solve_evp_real_2stage_double.3 \
-  man/solve_evp_complex_2stage_double.3 \
-  man/elpa_solve_evp_real_1stage_double.3 \
-  man/elpa_solve_evp_complex_1stage_double.3 \
-  man/elpa_solve_evp_real_2stage_double.3 \
-  man/elpa_solve_evp_complex_2stage_double.3 \
-  man/elpa_get_communicators.3 \
-  man/elpa_mult_at_b_real_double.3 \
-  man/elpa_mult_at_b_real_single.3 \
-  man/elpa_mult_ah_b_complex_double.3 \
-  man/elpa_mult_ah_b_complex_single.3 \
-  man/elpa_invert_trm_real_double.3 \
-  man/elpa_invert_trm_real_single.3 \
-  man/elpa_invert_trm_complex_double.3 \
-  man/elpa_invert_trm_complex_single.3 \
-  man/elpa_solve_evp_real_double.3 \
-  man/elpa_solve_evp_real_single.3 \
-  man/elpa_solve_evp_complex_double.3 \
-  man/elpa_solve_evp_complex_single.3 \
  man/elpa_autotune_setup.3 \
  man/elpa_autotune_step.3 \
  man/elpa_autotune_set_best.3 \
-  man/elpa_autotune_deallocate.3
+  man/elpa_autotune_deallocate.3 \
-endif
+  man/elpa_uninit.3
-if WANT_SINGLE_PRECISION_REAL
-if ENABLE_LEGACY
-dist_man_MANS += \
-  man/solve_evp_real_1stage_single.3 \
-  man/elpa_solve_evp_real_1stage_single.3 \
-  man/elpa_solve_evp_real_2stage_single.3 \
-  man/solve_evp_real_2stage_single.3
-endif
-endif
-if WANT_SINGLE_PRECISION_COMPLEX
-if ENABLE_LEGACY
-dist_man_MANS += \
-  man/solve_evp_complex_1stage_single.3 \
-  man/elpa_solve_evp_complex_1stage_single.3 \
-  man/elpa_solve_evp_complex_2stage_single.3 \
-  man/solve_evp_complex_2stage_single.3
-endif
-endif
 dist_doc_DATA = README.md USERS_GUIDE.md USERS_GUIDE_DEPRECATED_LEGACY_API.md INSTALL.md CONTRIBUTING.md LICENSE Changelog COPYING/COPYING COPYING/gpl.txt COPYING/lgpl.txt
@@ -583,6 +499,9 @@ test_program_fcflags = $(AM_FCFLAGS) $(FC_MODOUT)test_modules $(FC_MODINC)test_m
 noinst_LTLIBRARIES += libelpatest@SUFFIX@.la
 libelpatest@SUFFIX@_la_FCFLAGS = $(test_program_fcflags)
 libelpatest@SUFFIX@_la_SOURCES = \
+  test/shared/tests_variable_definitions.F90 \
+  test/shared/mod_tests_scalapack_interfaces.F90 \
+  test/shared/mod_tests_blas_interfaces.F90 \
  test/shared/test_util.F90 \
  test/shared/test_read_input_parameters.F90 \
  test/shared/test_check_correctness.F90 \
@@ -610,10 +529,6 @@ elpa2_print_kernels@SUFFIX@_FCFLAGS = $(AM_FCFLAGS) $(FC_MODINC)modules
 include test_programs.am
-if ENABLE_LEGACY
-include legacy_test_programs.am
-endif
 #include test_programs_manual.am
 noinst_PROGRAMS += validate_double_instance@SUFFIX@
@@ -714,10 +629,6 @@ CLEANFILES = \
  elpa1_real* \
  elpa*.sh \
  test*.sh \
-  legacy_real* \
-  legacy_complex* \
-  legacy_single_real* \
-  legacy_single_complex* \
  single_real* \
  single_complex* \
  real* \
@@ -730,7 +641,6 @@ CLEANFILES = \
 clean-local:
 	-rm -rf modules/* private_modules/* test_modules/* .fortran_dependencies/*
 	-rm -rf validate_*.sh
-	-rm -rf legacy_*.sh
 	-rm -rf real_2stage*.sh
 	-rm -rf complex_2stage*.sh
 	-rm -rf single_complex_2stage*.sh
@@ -752,12 +662,13 @@ EXTRA_DIST = \
  elpa.spec \
  elpa/elpa.h \
  elpa/elpa_generic.h \
-  elpa/elpa_legacy.h \
  fdep/fortran_dependencies.mk \
  fdep/fortran_dependencies.pl \
  manual_cpp \
  nvcc_wrap \
  remove_xcompiler \
+  src/helpers/fortran_blas_interfaces.F90 \
+  src/helpers/fortran_scalapack_interfaces.F90 \
  src/GPU/cuUtils_template.cu \
  src/elpa_api_math_template.F90 \
  src/elpa_impl_math_template.F90 \
@@ -807,8 +718,11 @@ EXTRA_DIST = \
  src/general/precision_macros.h \
  src/general/precision_typedefs.h \
  src/general/precision_kinds.F90 \
+  test/shared/test_precision_kinds.F90 \
  src/general/prow_pcol.F90 \
  src/general/sanity.F90 \
+  src/general/elpa_ssr2_template.F90 \
+  src/general/elpa_ssmv_template.F90 \
  test/Fortran/assert.h \
  test/Fortran/elpa_print_headers.F90 \
  test/shared/test_check_correctness_template.F90 \
@@ -832,38 +746,7 @@ EXTRA_DIST = \
  test_project_C/fdep \
  test_project_C/m4 \
  test_project_C/src/test_real.c \
-  test_project_C/src/test_blacs_infrastructure.F90\
+  test_project_C/src/test_blacs_infrastructure.F90
-  test_project_1stage_legacy_api/Makefile.am \
-  test_project_1stage_legacy_api/autogen.sh \
-  test_project_1stage_legacy_api/configure.ac \
-  test_project_1stage_legacy_api/fdep \
-  test_project_1stage_legacy_api/m4 \
-  test_project_1stage_legacy_api/src/test_real.F90 \
-  test_project_2stage_legacy_api/Makefile.am \
-  test_project_2stage_legacy_api/autogen.sh \
-  test_project_2stage_legacy_api/configure.ac \
-  test_project_2stage_legacy_api/fdep \
-  test_project_2stage_legacy_api/m4 \
-  test_project_2stage_legacy_api/src/test_real2.F90
-if ENABLE_LEGACY
-EXTRA_DIST += \
-  src/elpa1/elpa1_solve_tridi_real_template.F90 \
-  src/elpa1/legacy_interface/elpa1_template.F90 \
-  src/elpa2/legacy_interface/elpa2_template.F90 \
-  src/elpa1/legacy_interface/elpa1_c_interface_template.F90 \
-  src/elpa2/legacy_interface/elpa2_c_interface_template.F90 \
-  src/elpa_driver/legacy_interface/elpa_driver_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_solve_tridi_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_mult_at_b_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_mult_ah_b_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_invert_trm_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_cholesky_c_interface_template.F90 \
-  src/elpa1/legacy_interface/elpa_cholesky_template.F90 \
-  src/elpa1/legacy_interface/elpa_invert_trm.F90 \
-  src/elpa1/legacy_interface/elpa_multiply_a_b.F90 \
-  src/elpa1/legacy_interface/elpa_solve_tridi.F90
-endif
 if WITH_SCALAPACK_TESTS
 EXTRA_DIST += \

--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
 ## Current Release ##
-The current release is ELPA 2019.05.002 The current supported API version
+The current release is ELPA 2019.11.001 The current supported API version
 is 20190501. This release supports the earliest API version 20170403.
 The old, obsolete legacy API will be deprecated in the future !
@@ -13,7 +13,7 @@ The release ELPA 2018.11.001 was the last release, where the legacy API has been
 enabled by default (and can be disabled at build time).
 With release ELPA 2019.05.001 the legacy API is disabled by default, however,
 can be still switched on at build time.
-Most likely with the release ELPA 2019.11.001 the legacy API will be deprecated and
+With the release ELPA 2019.11.001 the legacy API will be deprecated and
 not supported anymore.
 [![Build 
@@ -104,10 +104,15 @@ Nonetheless, we are grateful if you cite the following publications:
  If you use the new API and/or autotuning:
-  Kus; P.; Marek, A.; Koecher, S. S.; Kowalski H.-H.; Carbogno, Ch.; Scheurer, Ch.; Reuter, K.; Scheffler, M.; Lederer, H.
+  Kus, P.; Marek, A.; Koecher, S. S.; Kowalski H.-H.; Carbogno, Ch.; Scheurer, Ch.; Reuter, K.; Scheffler, M.; Lederer, H.
  "Optimizations of the Eigenvaluesolvers in the ELPA Library",
  Parallel Computing 85, 167-177 (2019)
+  If you use the new support for skew-symmetric matrices:
+  Benner, P.; Draxl, C.; Marek, A.; Penke C.; Vorwerk, C.;
+  "High Performance Solution of Skew-symmetric Eigenvalue Problems with Applications in Solving the Bethe-Salpeter Eigenvalue Problem",
+  https://arxiv.org/abs/1912.04062, submitted to Parallel Computing
 ## Installation of the *ELPA* library
@@ -126,7 +131,7 @@ the possible configure options.
 ## Using *ELPA*
-Please have a look at the [USERS_GUIDE](USERS_GUIDE.md) file, to get a documentation or at the [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2019.05.002/html/index.html) doxygen
+Please have a look at the [USERS_GUIDE](USERS_GUIDE.md) file, to get a documentation or at the [online](http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2019.11.001/html/index.html) doxygen
 documentation, where you find the definition of the interfaces.
 ## Contributing to *ELPA*

--- a/RELEASE_NOTES
+++ b/RELEASE_NOTES
-This file contains the release notes for the ELPA 2019.05.002 version
+This file contains the release notes for the ELPA 2019.11.001 version
 What is new?
 -------------
-For detailed information about changes since release ELPA 2018.11 please have a look at the Changelog file
+For detailed information about changes since release ELPA 2019.05.002 please have a look at the Changelog file
- repackaging of src files, to include forgotten files of the legacy interface
+- several new features have been implemented
+- the old, legacy interface has been removed
 ABI change
 ---------------------
-Since release 2019.05.001 the ABI has not changed.
+Since release 2019.05.002 the ABI has changed, since the legacy interface cannot be activated anymore
 Any incompatibilities to previous version?
 ---------------------------------------