VB : Initial commit of ELPA infrastructure to new git repository. This repository should

     now reflect the current status of ELPA for any future developments.

COPYING/COPYING

+Licensing and copyright terms for the ELPA library:
+ELPA Consortium (2011)
+
+****
+
+Copyright of the original code rests with the authors inside the ELPA
+consortium. The copyright of any additional modifications shall rest
+with their original authors, but shall adhere to the licensing terms
+set forth below.
+
+****
+
+The code is distributed under ALMOST all of the the terms of the GNU
+Lesser General Public License version 3 (LGPL). The full text
+can be found in the file "lgpl.txt" in this repository. "lgpl.txt"
+makes reference to the GPL v3, which can also be found in this
+repository ("gpl.txt").
+
+While we are not allowed to alter the license texts as written in those 
+files, IN ADDITION our own license prescribes two important
+modifications / clarifications to the original lgpl.txt:
+
+- In point 2., clause b) - stating that that you may redistribute
+  under the terms of the plain GNU GPL - shall NOT apply. In other
+  words, if you redistribute, you MUST keep the additional permissions
+  of the LGPL v3 in place.
+
+- If you redistribute, you must redistribute under the terms of the
+  LGPL version specified here. Using later or earlier versions
+  published by anyone except the ELPA copyright holders is
+  not allowed. 
+
+****
+
+When in doubt, talk to us. What we would like to ensure is that the ELPA
+code can be used as needed, while providing a strong incentive for
+others to contribute their modifications back to the original development.
+
+****

COPYING/lgpl.txt

+                   GNU LESSER GENERAL PUBLIC LICENSE
+                       Version 3, 29 June 2007
+
+ Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+
+  This version of the GNU Lesser General Public License incorporates
+the terms and conditions of version 3 of the GNU General Public
+License, supplemented by the additional permissions listed below.
+
+  0. Additional Definitions.
+
+  As used herein, "this License" refers to version 3 of the GNU Lesser
+General Public License, and the "GNU GPL" refers to version 3 of the GNU
+General Public License.
+
+  "The Library" refers to a covered work governed by this License,
+other than an Application or a Combined Work as defined below.
+
+  An "Application" is any work that makes use of an interface provided
+by the Library, but which is not otherwise based on the Library.
+Defining a subclass of a class defined by the Library is deemed a mode
+of using an interface provided by the Library.
+
+  A "Combined Work" is a work produced by combining or linking an
+Application with the Library.  The particular version of the Library
+with which the Combined Work was made is also called the "Linked
+Version".
+
+  The "Minimal Corresponding Source" for a Combined Work means the
+Corresponding Source for the Combined Work, excluding any source code
+for portions of the Combined Work that, considered in isolation, are
+based on the Application, and not on the Linked Version.
+
+  The "Corresponding Application Code" for a Combined Work means the
+object code and/or source code for the Application, including any data
+and utility programs needed for reproducing the Combined Work from the
+Application, but excluding the System Libraries of the Combined Work.
+
+  1. Exception to Section 3 of the GNU GPL.
+
+  You may convey a covered work under sections 3 and 4 of this License
+without being bound by section 3 of the GNU GPL.
+
+  2. Conveying Modified Versions.
+
+  If you modify a copy of the Library, and, in your modifications, a
+facility refers to a function or data to be supplied by an Application
+that uses the facility (other than as an argument passed when the
+facility is invoked), then you may convey a copy of the modified
+version:
+
+   a) under this License, provided that you make a good faith effort to
+   ensure that, in the event an Application does not supply the
+   function or data, the facility still operates, and performs
+   whatever part of its purpose remains meaningful, or
+
+   b) under the GNU GPL, with none of the additional permissions of
+   this License applicable to that copy.
+
+  3. Object Code Incorporating Material from Library Header Files.
+
+  The object code form of an Application may incorporate material from
+a header file that is part of the Library.  You may convey such object
+code under terms of your choice, provided that, if the incorporated
+material is not limited to numerical parameters, data structure
+layouts and accessors, or small macros, inline functions and templates
+(ten or fewer lines in length), you do both of the following:
+
+   a) Give prominent notice with each copy of the object code that the
+   Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the object code with a copy of the GNU GPL and this license
+   document.
+
+  4. Combined Works.
+
+  You may convey a Combined Work under terms of your choice that,
+taken together, effectively do not restrict modification of the
+portions of the Library contained in the Combined Work and reverse
+engineering for debugging such modifications, if you also do each of
+the following:
+
+   a) Give prominent notice with each copy of the Combined Work that
+   the Library is used in it and that the Library and its use are
+   covered by this License.
+
+   b) Accompany the Combined Work with a copy of the GNU GPL and this license
+   document.
+
+   c) For a Combined Work that displays copyright notices during
+   execution, include the copyright notice for the Library among
+   these notices, as well as a reference directing the user to the
+   copies of the GNU GPL and this license document.
+
+   d) Do one of the following:
+
+       0) Convey the Minimal Corresponding Source under the terms of this
+       License, and the Corresponding Application Code in a form
+       suitable for, and under terms that permit, the user to
+       recombine or relink the Application with a modified version of
+       the Linked Version to produce a modified Combined Work, in the
+       manner specified by section 6 of the GNU GPL for conveying
+       Corresponding Source.
+
+       1) Use a suitable shared library mechanism for linking with the
+       Library.  A suitable mechanism is one that (a) uses at run time
+       a copy of the Library already present on the user's computer
+       system, and (b) will operate properly with a modified version
+       of the Library that is interface-compatible with the Linked
+       Version.
+
+   e) Provide Installation Information, but only if you would otherwise
+   be required to provide such information under section 6 of the
+   GNU GPL, and only to the extent that such information is
+   necessary to install and execute a modified version of the
+   Combined Work produced by recombining or relinking the
+   Application with a modified version of the Linked Version. (If
+   you use option 4d0, the Installation Information must accompany
+   the Minimal Corresponding Source and Corresponding Application
+   Code. If you use option 4d1, you must provide the Installation
+   Information in the manner specified by section 6 of the GNU GPL
+   for conveying Corresponding Source.)
+
+  5. Combined Libraries.
+
+  You may place library facilities that are a work based on the
+Library side by side in a single library together with other library
+facilities that are not Applications and are not covered by this
+License, and convey such a combined library under terms of your
+choice, if you do both of the following:
+
+   a) Accompany the combined library with a copy of the same work based
+   on the Library, uncombined with any other library facilities,
+   conveyed under the terms of this License.
+
+   b) Give prominent notice with the combined library that part of it
+   is a work based on the Library, and explaining where to find the
+   accompanying uncombined form of the same work.
+
+  6. Revised Versions of the GNU Lesser General Public License.
+
+  The Free Software Foundation may publish revised and/or new versions
+of the GNU Lesser General Public License from time to time. Such new
+versions will be similar in spirit to the present version, but may
+differ in detail to address new problems or concerns.
+
+  Each version is given a distinguishing version number. If the
+Library as you received it specifies that a certain numbered version
+of the GNU Lesser General Public License "or any later version"
+applies to it, you have the option of following the terms and
+conditions either of that published version or of any later version
+published by the Free Software Foundation. If the Library as you
+received it does not specify a version number of the GNU Lesser
+General Public License, you may choose any version of the GNU Lesser
+General Public License ever published by the Free Software Foundation.
+
+  If the Library as you received it specifies that a proxy can decide
+whether future versions of the GNU Lesser General Public License shall
+apply, that proxy's public statement of acceptance of any version is
+permanent authorization for you to choose that version for the
+Library.

README

+Welcome to the git-based distribution of the ELPA eigensolver library.
+
+If you are reading this file, you have obtained the ELPA library
+through the git repository that hosts the source code and also allows
+you to contribute improvements to the project if necessary.
+
+In your use of ELPA, please respect the copyright restrictions
+found below and in the "COPYING" directory in this repository. In a
+nutshell, if you make improvements to ELPA, copyright for such
+improvements remains with you, but we request that you relicense any
+such improvements under the same exact terms of the (modified) LGPL v3
+that we are using here. Please do not simply absorb ELPA into your own
+project and then redistribute binary-only without making your exact
+version of the ELPA source code (unmodified or MODIFIED) available as
+well. 
+
+
+*** Citing:
+
+  A description of some algorithms present in ELPA can be found in:
+
+  T. Auckenthaler, V. Blum, H.-J. Bungartz, T. Huckle, R. Johanni,
+  L. Kr\"amer, B. Lang, H. Lederer, and P. R. Willems, 
+  "Parallel solution of partial symmetric eigenvalue problems from
+  electronic structure calculations", 
+  Parallel Computing [volume], [page], (2011).
+  accepted for publication (May 11, 2011). 
+
+  Please cite this paper when using ELPA. We also intend to publish an
+  overview description of the ELPA library as such, and ask you to
+  make appropriate reference to that as well, once it appears.
+
+
+*** Copyright: 
+
+Copyright of the original code rests with the authors inside the ELPA
+consortium. The code is distributed under the terms of the GNU Lesser General 
+Public License version 3 (LGPL), with two important changes compared
+to the original version: 
+
+- In point 2., clause b) - stating that that you may redistribute
+  under the terms of the plain GNU GPL - shall not apply. In other
+  words, if you redistribute, you must keep the additional permissions
+  of the LGPL v3 in place.
+
+- If you redistribute, you must redistribute under the terms of the
+  LGPL v3. Using later or earlier versions published by anyone except
+  the original and future ELPA authors is not allowed.
+
+Please also note the express "NO WARRANTY" disclaimers in the GPL.
+
+Please see the file "COPYING" for details, which specifies these
+restrictions. 
+
+
+*** Using ELPA: 
+
+ELPA is designed to be compiled (Fortran) on its own, to be later
+linked to your own application. In order to use ELPA, you must still
+have a set of separate libraries that provide
+
+  - Basic Linear Algebra Subroutines (BLAS)
+  - Lapack routines
+  - Basic Linear Algebra Communication Subroutines (BLACS)
+  - Scalapack routines
+  - a working MPI library
+
+Appropriate libraries can be obtained and compiled separately on many
+architectures as free software. Alternatively, pre-packaged libraries
+are usually available from any HPC proprietary compiler vendors.
+
+For example, Intel's ifort compiler contains the "math kernel library"
+(MKL), providing BLAS/Lapack/BLACS/Scalapack functionality. (except on
+Mac OS X, where the BLACS and Scalapack part must still be obtained
+and compiled separately). 
+
+A very usable general-purpose MPI library is OpenMPI (ELPA was tested
+with OpenMPI 1.4.3 for example). Intel MPI seems to be a very well
+performing option on Intel platforms.
+
+Examples of how to use ELPA are included in the accompanying
+test_*.f90 subroutines in the "test" directory. A Makefile in also
+included as a minimal example of how to build and link ELPA to any
+other piece of code. 
+
+
+*** Structure of this repository:
+
+* README file - this file. Please also consult the ELPA Wiki, and
+  consider adding any useful information that you may have.
+
+* COPYING directory - the copyright and licensing information for ELPA.
+
+* src directory - contains all the files that are needed for the
+  actual ELPA subroutines. If you are attempting to use ELPA in your
+  own application, these are the files which you need. 
+
+- elpa1.f90 contains routines for the one-stage solver,
+  The 1 stage solver (elpa1.f90) can be used standalone without elpa2.
+
+- elpa2.f90 - ADDITIONAL routines needed for the two-stage solver
+  elpa2.f90 requires elpa1.f90 and a version of elpa2_kernels.f90, so
+  always compile them together.
+
+- elpa2_kernels.f90 - optimized linear algebra kernels for ELPA.
+  This file is a generic version of optimized linear algebra kernels
+  for use with the ELPA library. The standard elpa2_kernels.f90 runs
+  on every platform but it is optimized for the Intel SSE instruction
+  set. Best perfomance is achieved with the Intel ifort compiler and 
+  compile flags -O3 -xSSE4.2
+
+  For optimum performance on special architectures, you may wish to
+  investigate whether hand-tuned versions of this file give additional
+  gains. If so, simply remove elpa2_kernels.f90 from your compilation
+  and replace with the version of your choice. It would be great if
+  you could contribute such hand-tuned versions back to the
+  repository. (LGPL requirement for redistribution holds in any case)
+
+- elpa2_kernels_bg.f90
+  Example of optimized ELPA kernels for the BlueGene/P
+  architecture. Use instead of the standard elpa2_kernels.f90
+  file. elpa2_kernels_bg.f90 contains assembler instructions for the
+  BlueGene/P processor which IBM's xlf Fortran compiler can handle.
+
+* test directory
+
+- Contains the Makefile that demonstrates how to compile and link to
+  the ELPA routines
+
+- All files starting with test_... are for demonstrating the use
+  of the elpa library (but not needed for using it).
+
+- All test programs solve a eigenvalue problem and check the correctnes
+  of the result by evaluating || A*x - x*lamba || and checking the
+  orthogonality of the eigenvectors
+
+  test_real         Real eigenvalue problem, 1 stage solver
+  test_real_gen     Real generalized eigenvalue problem, 1 stage solver
+  test_complex      Complex eigenvalue problem, 1 stage solver
+  test_complex_gen  Complex generalized eigenvalue problem, 1 stage solver
+  test_real2        Real eigenvalue problem, 2 stage solver
+  test_complex2     Complex eigenvalue problem, 2 stage solver
+

test.txt

-Only a test

test/Makefile

+# ------------------------------------------------------------------------------
+# Please set the variables below according to your system!
+# ------------------------------------------------------------------------------
+# Settings for Intel Fortran (Linux):
+#
+F90=mpif90 -O3 -traceback -g -fpe0
+F90OPT=$(F90) -xSSE4.2
+LIBS = -L/opt/intel/Compiler/11.0/069/mkl/lib/em64t -lmkl_lapack -lmkl -lguide -lpthread \
+-lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64
+#
+# ------------------------------------------------------------------------------
+# Settings for Intel Fortran on MacOSX (home-built BLACS and scalapack):
+#
+#F90=mpif90 -O3 -traceback -g -fpe0
+#F90OPT=$(F90) # -xSSE4.2 ### on Mac OSX, the -xSSE4.2 option is possibly buggy in ifort!
+#LIBS = -L/opt/intel/mkl/lib -I/opt/intel/mkl/include -lmkl_intel_lp64 -lmkl_sequential -lmkl_core \
+#   /usr/local/BLACS/LIB/blacs_MPI-OSX-0.a /usr/local/BLACS/LIB/blacsF77init_MPI-OSX-0.a \
+#   /usr/local/SCALAPACK-1.8.0/libscalapack.a 
+#
+# ------------------------------------------------------------------------------
+# Settings for IBM AIX Power6
+#
+#F90 = mpxlf95_r -q64 -O2 -g -qarch=auto -qtune=auto
+#F90OPT = mpxlf95_r -q64 -O4 -g -qarch=auto -qtune=auto
+#LIBS = -L/usr/local/lib -lscalapack -llapack-essl -lessl -lblacsF77init -lblacs -lblacsF77init -lblacs -lc
+#
+# ------------------------------------------------------------------------------
+# Settings for IBM AIX BlueGene
+#
+#F90 = mpixlf95_r -O3 -g -qarch=auto -qtune=auto
+#F90OPT = mpixlf95_r -O4 -g -qarch=auto -qtune=auto
+#LIBS = -L/usr/local/lib -lscalapack -llapack -lblacsF77init -lblacs -lblacsF77init -lblacs \
+#-L/opt/ibmmath/essl/4.4/lib -lesslbg -lc
+#
+# ------------------------------------------------------------------------------
+
+all: test_real test_complex test_real_gen test_complex_gen test_real2 test_complex2
+
+test_real: test_real.o elpa1.o
+	$(F90) -o $@ test_real.o elpa1.o $(LIBS)
+
+test_complex: test_complex.o elpa1.o
+	$(F90) -o $@ test_complex.o elpa1.o $(LIBS)
+
+test_real_gen: test_real_gen.o elpa1.o
+	$(F90) -o $@ test_real_gen.o elpa1.o $(LIBS)
+
+test_complex_gen: test_complex_gen.o elpa1.o
+	$(F90) -o $@ test_complex_gen.o elpa1.o $(LIBS)
+
+test_real2: test_real2.o elpa1.o elpa2.o elpa2_kernels.o
+	$(F90) -o $@ test_real2.o elpa1.o elpa2.o elpa2_kernels.o $(LIBS)
+
+test_complex2: test_complex2.o elpa1.o elpa2.o elpa2_kernels.o
+	$(F90) -o $@ test_complex2.o elpa1.o elpa2.o elpa2_kernels.o $(LIBS)
+
+test_real.o: test_real.f90 elpa1.o
+	$(F90) -c $<
+
+test_complex.o: test_complex.f90 elpa1.o
+	$(F90) -c $<
+
+test_real_gen.o: test_real_gen.f90 elpa1.o
+	$(F90) -c $<
+
+test_complex_gen.o: test_complex_gen.f90 elpa1.o
+	$(F90) -c $<
+
+test_real2.o: test_real2.f90 elpa1.o elpa2.o
+	$(F90) -c $<
+
+test_complex2.o: test_complex2.f90 elpa1.o elpa2.o
+	$(F90) -c $<
+
+elpa1.o: ../src/elpa1.f90
+	$(F90) -c $<
+
+elpa2.o: ../src/elpa2.f90 elpa1.o
+	$(F90) -c ../src/elpa2.f90
+
+elpa2_kernels.o: ../src/elpa2_kernels.f90
+	$(F90OPT) -c ../src/elpa2_kernels.f90
+
+clean:
+	rm -f *.o *.mod test_real test_complex test_real_gen test_complex_gen test_real2 test_complex2

test/test_complex.f90

+program test_complex
+
+!-------------------------------------------------------------------------------
+! Standard eigenvalue problem - COMPLEX version
+!
+! This program demonstrates the use of the ELPA module
+! together with standard scalapack routines
+! 
+! Copyright of the original code rests with the authors inside the ELPA
+! consortium. The copyright of any additional modifications shall rest
+! with their original authors, but shall adhere to the licensing terms
+! distributed along with the original code in the file "COPYING".
+!-------------------------------------------------------------------------------
+
+   use ELPA1
+
+   implicit none
+   include 'mpif.h'
+
+   !-------------------------------------------------------------------------------
+   ! Please set system size parameters below!
+   ! na:   System size
+   ! nev:  Number of eigenvectors to be calculated
+   ! nblk: Blocking factor in block cyclic distribution
+   !-------------------------------------------------------------------------------
+
+   integer, parameter :: na = 4000, nev = 1500, nblk = 16
+
+   !-------------------------------------------------------------------------------
+   !  Local Variables
+
+   integer np_rows, np_cols, na_rows, na_cols
+
+   integer myid, nprocs, my_prow, my_pcol, mpi_comm_rows, mpi_comm_cols
+   integer i, mpierr, my_blacs_ctxt, sc_desc(9), info, nprow, npcol
+
+   integer, external :: numroc
+
+   real*8 err, errmax
+   real*8, allocatable :: ev(:), xr(:,:)
+   complex*16 :: xc
+   complex*16, allocatable :: a(:,:), z(:,:), tmp1(:,:), tmp2(:,:), as(:,:)
+
+   complex*16, parameter :: CZERO = (0.d0,0.d0), CONE = (1.d0,0.d0)
+
+   integer :: iseed(4096) ! Random seed, size should be sufficient for every generator
+
+   !-------------------------------------------------------------------------------
+   !  MPI Initialization
+
+   call mpi_init(mpierr)
+   call mpi_comm_rank(mpi_comm_world,myid,mpierr)
+   call mpi_comm_size(mpi_comm_world,nprocs,mpierr)
+
+   !-------------------------------------------------------------------------------
+   ! Selection of number of processor rows/columns
+   ! We try to set up the grid square-like, i.e. start the search for possible
+   ! divisors of nprocs with a number next to the square root of nprocs
+   ! and decrement it until a divisor is found.
+
+   do np_cols = NINT(SQRT(REAL(nprocs))),2,-1
+      if(mod(nprocs,np_cols) == 0 ) exit
+   enddo
+   ! at the end of the above loop, nprocs is always divisible by np_cols
+
+   np_rows = nprocs/np_cols
+
+   if(myid==0) then
+      print *
+      print '(a)','Standard eigenvalue problem - COMPLEX version'
+      print *
+      print '(3(a,i0))','Matrix size=',na,', Number of eigenvectors=',nev,', Block size=',nblk
+      print '(3(a,i0))','Number of processor rows=',np_rows,', cols=',np_cols,', total=',nprocs
+      print *
+   endif
+
+   !-------------------------------------------------------------------------------
+   ! Set up BLACS context and MPI communicators
+   !
+   ! The BLACS context is only necessary for using Scalapack.
+   !
+   ! For ELPA, the MPI communicators along rows/cols are sufficient,
+   ! and the grid setup may be done in an arbitrary way as long as it is
+   ! consistent (i.e. 0<=my_prow<np_rows, 0<=my_pcol<np_cols and every
+   ! process has a unique (my_prow,my_pcol) pair).
+
+   my_blacs_ctxt = mpi_comm_world
+   call BLACS_Gridinit( my_blacs_ctxt, 'C', np_rows, np_cols )
+   call BLACS_Gridinfo( my_blacs_ctxt, nprow, npcol, my_prow, my_pcol )
+
+   ! All ELPA routines need MPI communicators for communicating within
+   ! rows or columns of processes, these are set in get_elpa_row_col_comms.
+
+   call get_elpa_row_col_comms(mpi_comm_world, my_prow, my_pcol, &
+                               mpi_comm_rows, mpi_comm_cols)
+
+   ! Determine the necessary size of the distributed matrices,
+   ! we use the Scalapack tools routine NUMROC for that.
+
+   na_rows = numroc(na, nblk, my_prow, 0, np_rows)
+   na_cols = numroc(na, nblk, my_pcol, 0, np_cols)
+
+   ! Set up a scalapack descriptor for the checks below.
+   ! For ELPA the following restrictions hold:
+   ! - block sizes in both directions must be identical (args 4+5)
+   ! - first row and column of the distributed matrix must be on row/col 0/0 (args 6+7)
+
+   call descinit( sc_desc, na, na, nblk, nblk, 0, 0, my_blacs_ctxt, na_rows, info )
+
+   !-------------------------------------------------------------------------------
+   ! Allocate matrices and set up a test matrix for the eigenvalue problem
+
+   allocate(a (na_rows,na_cols))
+   allocate(z (na_rows,na_cols))
+   allocate(as(na_rows,na_cols))
+
+   allocate(ev(na))
+
+   ! For getting a hermitian test matrix A we get a random matrix Z
+   ! and calculate A = Z + Z**H
+
+   ! We want different random numbers on every process
+   ! (otherways A might get rank deficient):
+
+   iseed(:) = myid
+   call RANDOM_SEED(put=iseed)
+
+   allocate(xr(na_rows,na_cols))
+   call RANDOM_NUMBER(xr)
+   z(:,:) = xr(:,:)
+   call RANDOM_NUMBER(xr)
+   z(:,:) = z(:,:) + (0.d0,1.d0)*xr(:,:)
+   deallocate(xr)
+
+   a(:,:) = z(:,:)
+   call pztranc(na, na, CONE, z, 1, 1, sc_desc, CONE, a, 1, 1, sc_desc) ! A = A + Z**H
+
+   ! Save original matrix A for later accuracy checks
+
+   as = a
+
+   !-------------------------------------------------------------------------------
+   ! Calculate eigenvalues/eigenvectors
+
+   call mpi_barrier(mpi_comm_world, mpierr) ! for correct timings only
+   call solve_evp_complex(na, nev, a, na_rows, ev, z, na_rows, nblk, &
+                          mpi_comm_rows, mpi_comm_cols)
+
+   if(myid == 0) print *,'Time tridiag_complex :',time_evp_fwd
+   if(myid == 0) print *,'Time solve_tridi     :',time_evp_solve
+   if(myid == 0) print *,'Time trans_ev_complex:',time_evp_back
+
+   !-------------------------------------------------------------------------------
+   ! Test correctness of result (using plain scalapack routines)
+
+   ! 1. Residual (maximum of || A*Zi - Zi*EVi ||)
+
+   deallocate(a)
+   allocate(tmp1(na_rows,na_cols))
+
+   ! tmp1 =  A * Z
+   call pzgemm('N','N',na,nev,na,CONE,as,1,1,sc_desc, &
+               z,1,1,sc_desc,CZERO,tmp1,1,1,sc_desc)
+
+   deallocate(as)
+   allocate(tmp2(na_rows,na_cols))
+
+   ! tmp2 = Zi*EVi
+   tmp2(:,:) = z(:,:)
+   do i=1,nev
+      xc = ev(i)
+      call pzscal(na,xc,tmp2,1,i,sc_desc,1)
+   enddo
+
+   !  tmp1 = A*Zi - Zi*EVi
+   tmp1(:,:) =  tmp1(:,:) - tmp2(:,:)
+
+   ! Get maximum norm of columns of tmp1
+   errmax = 0
+   do i=1,nev
+      xc = 0
+      call pzdotc(na,xc,tmp1,1,i,sc_desc,1,tmp1,1,i,sc_desc,1)
+      errmax = max(errmax, sqrt(real(xc,8)))
+   enddo
+
+   ! Get maximum error norm over all processors
+   err = errmax
+   call mpi_allreduce(err,errmax,1,MPI_REAL8,MPI_MAX,MPI_COMM_WORLD,mpierr)
+   if(myid==0) print *
+   if(myid==0) print *,'Error Residual     :',errmax