USERS_GUIDE.md 22.1 KB
 Andreas Marek committed Apr 29, 2016 1 2 3 4 5 6 7 8 9 10 11 ``````## Users guide for the ELPA library ## This document provides the guide for using the *ELPA* library in user applications. ### Online and local documentation ### Local documentation (via man pages) should be available (if *ELPA* has been installed with the documentation): For example "man get_elpa_communicators" should provide the documentation for the *ELPA* function which sets the necessary communicators. `````` Andreas Marek committed May 20, 2016 12 ``````Also a [online doxygen documentation] (http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2016.05.001/html/index.html) `````` Andreas Marek committed Apr 29, 2016 13 14 ``````for each *ELPA* release is available. `````` Andreas Marek committed Apr 30, 2016 15 ``````### General concept of the *ELPA* library ### `````` Andreas Marek committed Apr 29, 2016 16 `````` `````` Andreas Marek committed Apr 30, 2016 17 18 19 20 21 22 ``````The *ELPA* library consists of two main parts: - *ELPA 1stage* solver - *ELPA 2stage* solver Both variants of the *ELPA* solvers are available for real or complex valued matrices. `````` Andreas Marek committed May 20, 2016 23 ``````Thus *ELPA* provides the following user functions (see man pages or [online] (http://elpa.mpcdf.mpg.de/html/Documentation/ELPA-2016.05.001/html/index.html) for details): `````` Andreas Marek committed Apr 30, 2016 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 `````` - get_elpa_communicators : set the row / column communicators for *ELPA* - solve_evp_complex_1stage : solve a complex valued eigenvale proplem with the *ELPA 1stage* solver - solve_evp_real_1stage : solve a real valued eigenvale proplem with the *ELPA 1stage* solver - solve_evp_complex_2stage : solve a complex valued eigenvale proplem with the *ELPA 2stage* solver - solve_evp_real_2stage : solve a real valued eigenvale proplem with the *ELPA 2stage* solver Furthermore *ELPA* provides the utility binary "print_available_elpa2_kernels": it tells the user which *ELPA 2stage* compute kernels have been installed and which default kernels are set If you want to solve an eigenvalue problem with *ELPA*, you have to decide whether you want to use *ELPA 1stage* or *ELPA 2stage* solver. Normally, *ELPA 2stage* is the better choice since it is faster, but there a matrix dimensions where *ELPA 1stage* is supperior. Independent of the choice of the solver, the concept of calling *ELPA* is always the same: #### MPI version of *ELPA* #### `````` Andreas Marek committed Apr 29, 2016 41 42 43 44 45 46 47 48 `````` In this case, *ELPA* relies on a BLACS distributed matrix. To solve a Eigenvalue problem of this matrix with *ELPA*, one has 1. to include the *ELPA* header (C case) or module (Fortran) 2. to create row and column MPI communicators for ELPA (with "get_elpa_communicators") 3. to call *ELPA 1stage* or *ELPA 2stage* for the matrix. `````` Andreas Marek committed Apr 30, 2016 49 ``````Here is a very simple MPI code snippet for using *ELPA 1stage*: For the definition of all variables `````` Andreas Marek committed May 19, 2016 50 51 ``````please have a look at the man pages and/or the online documentation (see above). A full version of a simple example program can be found in ./test_project/src. `````` Andreas Marek committed Apr 29, 2016 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 `````` ! All ELPA routines need MPI communicators for communicating within ! rows or columns of processes, these are set in get_elpa_communicators success = get_elpa_communicators(mpi_comm_world, my_prow, my_pcol, & mpi_comm_rows, mpi_comm_cols) if (myid==0) then print '(a)','| Past split communicator setup for rows and columns.' end if ! Determine the necessary size of the distributed matrices, ! we use the Scalapack tools routine NUMROC for that. na_rows = numroc(na, nblk, my_prow, 0, np_rows) na_cols = numroc(na, nblk, my_pcol, 0, np_cols) !------------------------------------------------------------------------------- ! Calculate eigenvalues/eigenvectors if (myid==0) then print '(a)','| Entering one-step ELPA solver ... ' print * end if `````` Andreas Marek committed Apr 30, 2016 78 79 `````` success = solve_evp_real_1stage(na, nev, a, na_rows, ev, z, na_rows, nblk, & matrixCols, mpi_comm_rows, mpi_comm_cols) `````` Andreas Marek committed Apr 29, 2016 80 81 82 83 84 85 86 `````` if (myid==0) then print '(a)','| One-step ELPA solver complete.' print * end if `````` Andreas Marek committed Apr 30, 2016 87 ``````#### Shared-memory version of *ELPA* #### `````` Andreas Marek committed Apr 29, 2016 88 `````` `````` Andreas Marek committed May 09, 2016 89 ``````If the *ELPA* library has been compiled with the configure option "--with-mpi=0", `````` Andreas Marek committed Apr 29, 2016 90 91 92 93 ``````no MPI will be used. Still the **same** call sequence as in the MPI case can be used (see above). `````` Andreas Marek committed Apr 30, 2016 94 ``````#### Setting the row and column communicators #### `````` Andreas Marek committed Apr 29, 2016 95 `````` `````` Andreas Marek committed Apr 30, 2016 96 97 98 ``````SYNOPSIS FORTRAN INTERFACE use elpa1 `````` Andreas Marek committed Apr 30, 2016 99 `````` `````` Andreas Marek committed Apr 30, 2016 100 `````` success = get_elpa_communicators (mpi_comm_global, my_prow, my_pcol, mpi_comm_rows, mpi_comm_cols) `````` Andreas Marek committed Apr 30, 2016 101 `````` `````` Andreas Marek committed Apr 30, 2016 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 `````` integer, intent(in) mpi_comm_global: global communicator for the calculation integer, intent(in) my_prow: row coordinate of the calling process in the process grid integer, intent(in) my_pcol: column coordinate of the calling process in the process grid integer, intent(out) mpi_comm_row: communicator for communication within rows of processes integer, intent(out) mpi_comm_row: communicator for communication within columns of processes integer success: return value indicating success or failure of the underlying MPI_COMM_SPLIT function C INTERFACE #include "elpa_generated.h" success = get_elpa_communicators (int mpi_comm_world, int my_prow, my_pcol, int *mpi_comm_rows, int *Pmpi_comm_cols); int mpi_comm_global: global communicator for the calculation int my_prow: row coordinate of the calling process in the process grid int my_pcol: column coordinate of the calling process in the process grid int *mpi_comm_row: pointer to the communicator for communication within rows of processes int *mpi_comm_row: pointer to the communicator for communication within columns of processes int success: return value indicating success or failure of the underlying MPI_COMM_SPLIT function #### Using *ELPA 1stage* #### After setting up the *ELPA* row and column communicators (by calling get_elpa_communicators), only the real or complex valued solver has to be called: SYNOPSIS FORTRAN INTERFACE use elpa1 success = solve_evp_real_1stage (na, nev, a(lda,matrixCols), ev(nev), q(ldq, matrixCols), ldq, nblk, matrixCols, mpi_comm_rows, mpi_comm_cols) With the definintions of the input and output variables: integer, intent(in) na: global dimension of quadratic matrix a to solve integer, intent(in) nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated real*8, intent(inout) a: locally distributed part of the matrix a. The local dimensions are lda x matrixCols integer, intent(in) lda: leading dimension of locally distributed matrix a real*8, intent(inout) ev: on output the first nev computed eigenvalues real*8, intent(inout) q: on output the first nev computed eigenvectors integer, intent(in) ldq: leading dimension of matrix q which stores the eigenvectors integer, intent(in) nblk: blocksize of block cyclic distributin, must be the same in both directions integer, intent(in) matrixCols: number of columns of locally distributed matrices a and q integer, intent(in) mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) logical success: return value indicating success or failure C INTERFACE #include "elpa.h" success = solve_evp_real_1stage (int na, int nev, double *a, int lda, double *ev, double *q, int ldq, int nblk, int matrixCols, int mpi_comm_rows, int mpi_comm_cols); With the definintions of the input and output variables: int na: global dimension of quadratic matrix a to solve int nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated double *a: pointer to locally distributed part of the matrix a. The local dimensions are lda x matrixCols int lda: leading dimension of locally distributed matrix a double *ev: pointer to memory containing on output the first nev computed eigenvalues double *q: pointer to memory containing on output the first nev computed eigenvectors int ldq: leading dimension of matrix q which stores the eigenvectors int nblk: blocksize of block cyclic distributin, must be the same in both directions int matrixCols: number of columns of locally distributed matrices a and q int mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) int mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) int success: return value indicating success (1) or failure (0) DESCRIPTION Solve the real eigenvalue problem with the 1-stage solver. The ELPA communicators mpi_comm_rows and mpi_comm_cols are obtained with the get_elpa_communicators(3) function. The distributed quadratic marix a has global dimensions na x na, and a local size lda x matrixCols. The solver will compute the first nev eigenvalues, which will be stored on exit in ev. The eigenvectors corresponding to the eigenvalues will be stored in q. All memory of the arguments must be allocated outside the call to the solver. FORTRAN INTERFACE use elpa1 success = solve_evp_complex_1stage (na, nev, a(lda,matrixCols), ev(nev), q(ldq, matrixCols), ldq, nblk, matrixCols, mpi_comm_rows, mpi_comm_cols) With the definintions of the input and output variables: integer, intent(in) na: global dimension of quadratic matrix a to solve integer, intent(in) nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated complex*16, intent(inout) a: locally distributed part of the matrix a. The local dimensions are lda x matrixCols integer, intent(in) lda: leading dimension of locally distributed matrix a real*8, intent(inout) ev: on output the first nev computed eigenvalues complex*16, intent(inout) q: on output the first nev computed eigenvectors integer, intent(in) ldq: leading dimension of matrix q which stores the eigenvectors integer, intent(in) nblk: blocksize of block cyclic distributin, must be the same in both directions integer, intent(in) matrixCols: number of columns of locally distributed matrices a and q integer, intent(in) mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) logical success: return value indicating success or failure C INTERFACE #include "elpa.h" #include success = solve_evp_complex_1stage (int na, int nev, double complex *a, int lda, double *ev, double complex*q, int ldq, int nblk, int matrixCols, int mpi_comm_rows, int mpi_comm_cols); With the definintions of the input and output variables: int na: global dimension of quadratic matrix a to solve int nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated double complex *a: pointer to locally distributed part of the matrix a. The local dimensions are lda x matrixCols int lda: leading dimension of locally distributed matrix a double *ev: pointer to memory containing on output the first nev computed eigenvalues double complex *q: pointer to memory containing on output the first nev computed eigenvectors int ldq: leading dimension of matrix q which stores the eigenvectors int nblk: blocksize of block cyclic distributin, must be the same in both directions int matrixCols: number of columns of locally distributed matrices a and q int mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) int mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) int success: return value indicating success (1) or failure (0) DESCRIPTION Solve the complex eigenvalue problem with the 1-stage solver. The ELPA communicators mpi_comm_rows and mpi_comm_cols are obtained with the get_elpa_communicators(3) function. The distributed quadratic marix a has global dimensions na x na, and a local size lda x matrixCols. The solver will compute the first nev eigenvalues, which will be stored on exit in ev. The eigenvectors corresponding to the eigenvalues will be stored in q. All memory of the arguments must be allocated outside the call to the solver. The *ELPA 1stage* solver, does not need or accept any other parameters than in the above specification. #### Using *ELPA 2stage* #### The *ELPA 2stage* solver can be used in the same manner, as the *ELPA 1stage* solver. However, the 2 stage solver, can be used with different compute kernels, which offers more possibilities for configuration. It is recommended to first call the utillity program elpa2_print_kernels which will tell all the compute kernels that can be used with *ELPA 2stage*". It will also give information, whether a kernel can be set via environment variables. ##### Using the default kernels ##### If no kernel is set either via an environment variable or the *ELPA 2stage API* then the default kernels will be set. ##### Setting the *ELPA 2stage* compute kernels ##### If the *ELPA* installation allows setting ther compute kernels with enviroment variables, setting the variables "REAL_ELPA_KERNEL" and "COMPLEX_ELPA_KERNEL" will set the compute kernels. The environment variable setting will take precedence over all other settings! It is also possible to set the *ELPA 2stage* compute kernels via the API. SYNOPSIS FORTRAN INTERFACE use elpa1 use elpa2 success = solve_evp_real_2stage (na, nev, a(lda,matrixCols), ev(nev), q(ldq, matrixCols), ldq, nblk, matrixCols, mpi_comm_rows, mpi_comm_cols, mpi_comm_all, THIS_REAL_ELPA_KERNEL, useQr=useQR) With the definintions of the input and output variables: integer, intent(in) na: global dimension of quadratic matrix a to solve integer, intent(in) nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated real*8, intent(inout) a: locally distributed part of the matrix a. The local dimensions are lda x matrixCols integer, intent(in) lda: leading dimension of locally distributed matrix a real*8, intent(inout) ev: on output the first nev computed eigenvalues real*8, intent(inout) q: on output the first nev computed eigenvectors integer, intent(in) ldq: leading dimension of matrix q which stores the eigenvectors integer, intent(in) nblk: blocksize of block cyclic distributin, must be the same in both directions integer, intent(in) matrixCols: number of columns of locally distributed matrices a and q integer, intent(in) mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_all: communicator for all processes in the processor set involved in ELPA logical, intent(in), optional: useQR: optional argument; switches to QR-decomposition if set to .true. logical success: return value indicating success or failure C INTERFACE #include "elpa.h" success = solve_evp_real_2stage (int na, int nev, double *a, int lda, double *ev, double *q, int ldq, int nblk, int matrixCols, int mpi_comm_rows, int mpi_comm_cols, int mpi_comm_all, int THIS_ELPA_REAL_KERNEL, int useQr); With the definintions of the input and output variables: int na: global dimension of quadratic matrix a to solve int nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated double *a: pointer to locally distributed part of the matrix a. The local dimensions are lda x matrixCols int lda: leading dimension of locally distributed matrix a double *ev: pointer to memory containing on output the first nev computed eigenvalues double *q: pointer to memory containing on output the first nev computed eigenvectors int ldq: leading dimension of matrix q which stores the eigenvectors int nblk: blocksize of block cyclic distributin, must be the same in both directions int matrixCols: number of columns of locally distributed matrices a and q int mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) int mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) int mpi_comm_all: communicator for all processes in the processor set involved in ELPA int useQR: if set to 1 switch to QR-decomposition int success: return value indicating success (1) or failure (0) DESCRIPTION Solve the real eigenvalue problem with the 2-stage solver. The ELPA communicators mpi_comm_rows and mpi_comm_cols are obtained with the get_elpa_communicators(3) function. The distributed quadratic marix a has global dimensions na x na, and a local size lda x matrixCols. The solver will compute the first nev eigenvalues, which will be stored on exit in ev. The eigenvectors corresponding to the eigenvalues will be stored in q. All memory of the arguments must be allocated outside the call to the solver. SYNOPSIS FORTRAN INTERFACE use elpa1 use elpa2 success = solve_evp_real_2stage (na, nev, a(lda,matrixCols), ev(nev), q(ldq, matrixCols), ldq, nblk, matrixCols, mpi_comm_rows, mpi_comm_cols, mpi_comm_all, THIS_REAL_ELPA_KERNEL) With the definintions of the input and output variables: integer, intent(in) na: global dimension of quadratic matrix a to solve integer, intent(in) nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated complex*16, intent(inout) a: locally distributed part of the matrix a. The local dimensions are lda x matrixCols integer, intent(in) lda: leading dimension of locally distributed matrix a real*8, intent(inout) ev: on output the first nev computed eigenvalues complex*16, intent(inout) q: on output the first nev computed eigenvectors integer, intent(in) ldq: leading dimension of matrix q which stores the eigenvectors integer, intent(in) nblk: blocksize of block cyclic distributin, must be the same in both directions integer, intent(in) matrixCols: number of columns of locally distributed matrices a and q integer, intent(in) mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) integer, intent(in) mpi_comm_all: communicator for all processes in the processor set involved in ELPA logical success: return value indicating success or failure C INTERFACE #include "elpa.h" #include success = solve_evp_complex_2stage (int na, int nev, double complex *a, int lda, double *ev, double complex *q, int ldq, int nblk, int matrixCols, int mpi_comm_rows, int mpi_comm_cols, int mpi_comm_all, int THIS_ELPA_REAL_KERNEL); With the definintions of the input and output variables: int na: global dimension of quadratic matrix a to solve int nev: number of eigenvalues to be computed; the first nev eigenvalules are calculated double complex *a: pointer to locally distributed part of the matrix a. The local dimensions are lda x matrixCols int lda: leading dimension of locally distributed matrix a double *ev: pointer to memory containing on output the first nev computed eigenvalues double complex *q: pointer to memory containing on output the first nev computed eigenvectors int ldq: leading dimension of matrix q which stores the eigenvectors int nblk: blocksize of block cyclic distributin, must be the same in both directions int matrixCols: number of columns of locally distributed matrices a and q int mpi_comm_rows: communicator for communication in rows. Constructed with get_elpa_communicators(3) int mpi_comm_cols: communicator for communication in colums. Constructed with get_elpa_communicators(3) int mpi_comm_all: communicator for all processes in the processor set involved in ELPA int success: return value indicating success (1) or failure (0) `````` Andreas Marek committed Apr 29, 2016 358 `````` `````` Andreas Marek committed Apr 30, 2016 359 360 361 362 363 ``````DESCRIPTION Solve the complex eigenvalue problem with the 2-stage solver. The ELPA communicators mpi_comm_rows and mpi_comm_cols are obtained with the get_elpa_communicators(3) function. The distributed quadratic marix a has global dimensions na x na, and a local size lda x matrixCols. The solver will compute the first nev eigenvalues, which will be stored on exit in ev. The eigenvectors corresponding to the eigenvalues will be stored in q. All memory of the arguments must be allocated outside the call to the solver. `````` Andreas Marek committed Apr 29, 2016 364 365 `````` ``````