Commit d9a04481 authored by Martin Reinecke's avatar Martin Reinecke
Browse files

Merge branch 'switch_to_ducc' into 'NIFTy_7'

Switch to DUCC

See merge request !530
parents dc9494c9 20577a56
Pipeline #77097 passed with stages
in 13 minutes and 9 seconds
Changes since NIFTy 6 Changes since NIFTy 6
===================== =====================
Change in external dependencies
-------------------------------
Instead of the optional external packages `pypocketfft` and `pyHealpix`, NIFTy
now uses the DUCC package (<https://gitlab.mpcdf.mpg.de/mtr/ducc)>,
which is their successor.
Naming of operator tests Naming of operator tests
------------------------ ------------------------
...@@ -8,6 +16,7 @@ The implementation tests for nonlinear operators are now available in ...@@ -8,6 +16,7 @@ The implementation tests for nonlinear operators are now available in
`ift.extra.check_operator()` and for linear operators `ift.extra.check_operator()` and for linear operators
`ift.extra.check_linear_operator()`. `ift.extra.check_linear_operator()`.
MetricGaussianKL interface MetricGaussianKL interface
-------------------------- --------------------------
......
...@@ -12,7 +12,7 @@ RUN apt-get update && apt-get install -y \ ...@@ -12,7 +12,7 @@ RUN apt-get update && apt-get install -y \
# Optional NIFTy dependencies # Optional NIFTy dependencies
python3-mpi4py python3-matplotlib \ python3-mpi4py python3-matplotlib \
# more optional NIFTy dependencies # more optional NIFTy dependencies
&& pip3 install git+https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git \ && pip3 install ducc0 \
&& pip3 install git+https://gitlab.mpcdf.mpg.de/ift/nifty_gridder.git \ && pip3 install git+https://gitlab.mpcdf.mpg.de/ift/nifty_gridder.git \
&& pip3 install jupyter \ && pip3 install jupyter \
&& rm -rf /var/lib/apt/lists/* && rm -rf /var/lib/apt/lists/*
......
...@@ -49,13 +49,12 @@ Installation ...@@ -49,13 +49,12 @@ Installation
- [SciPy](https://www.scipy.org/) - [SciPy](https://www.scipy.org/)
Optional dependencies: Optional dependencies:
- [pyHealpix](https://gitlab.mpcdf.mpg.de/ift/pyHealpix) (for harmonic - [DUCC0](https://gitlab.mpcdf.mpg.de/mtr/ducc) for faster FFTs, spherical
transforms involving domains on the sphere) harmonic transforms, and non-uniform Fourier transforms
- [nifty_gridder](https://gitlab.mpcdf.mpg.de/ift/nifty_gridder) (for radio - [nifty_gridder](https://gitlab.mpcdf.mpg.de/ift/nifty_gridder) (for radio
interferometry responses) interferometry responses)
- [mpi4py](https://mpi4py.scipy.org) (for MPI-parallel execution) - [mpi4py](https://mpi4py.scipy.org) (for MPI-parallel execution)
- [matplotlib](https://matplotlib.org/) (for field plotting) - [matplotlib](https://matplotlib.org/) (for field plotting)
- [pypocketfft](https://gitlab.mpcdf.mpg.de/mtr/pypocketfft) (for faster FFTs)
### Sources ### Sources
...@@ -78,9 +77,14 @@ Plotting support is added via: ...@@ -78,9 +77,14 @@ Plotting support is added via:
sudo apt-get install python3-matplotlib sudo apt-get install python3-matplotlib
Support for spherical harmonic transforms is added via: The DUCC0 package is installed via:
pip3 install --user git+https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git pip3 install ducc0
If this library is present, NIFTy will detect it automatically and prefer
`ducc0.fft` over SciPy's FFT. The underlying code is actually the same, but
DUCC's FFT is compiled with optimizations for the host CPU and can provide
significantly faster transforms.
Support for the radio interferometry gridder is added via: Support for the radio interferometry gridder is added via:
...@@ -90,15 +94,6 @@ MPI support is added via: ...@@ -90,15 +94,6 @@ MPI support is added via:
sudo apt-get install python3-mpi4py sudo apt-get install python3-mpi4py
Pypocketfft is added via:
pip3 install --user git+https://gitlab.mpcdf.mpg.de/mtr/pypocketfft
If this library is present, NIFTy will detect it automatically and prefer
it over SciPy's FFT. The underlying code is actually the same, but
pypocketfft is compiled with optimizations for the host CPU and can provide
significantly faster transforms.
### Running the tests ### Running the tests
To run the tests, additional packages are required: To run the tests, additional packages are required:
......
...@@ -14,9 +14,14 @@ Plotting support is added via:: ...@@ -14,9 +14,14 @@ Plotting support is added via::
sudo apt-get install python3-matplotlib sudo apt-get install python3-matplotlib
Support for spherical harmonic transforms is added via:: The DUCC0 package is installed via::
pip3 install --user git+https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git pip3 install --user ducc0
If this library is present, NIFTy will detect it automatically and prefer
`ducc0.fft` over SciPy's FFT. The underlying code is actually the same, but
DUCC's FFT is compiled with optimizations for the host CPU and can provide
significantly faster transforms.
Support for the radio interferometry gridder is added via:: Support for the radio interferometry gridder is added via::
...@@ -26,14 +31,6 @@ MPI support is added via:: ...@@ -26,14 +31,6 @@ MPI support is added via::
sudo apt-get install python3-mpi4py sudo apt-get install python3-mpi4py
Pypocketfft is added via:
pip3 install --user git+https://gitlab.mpcdf.mpg.de/mtr/pypocketfft
If this library is present, NIFTy will detect it automatically and prefer
it over SciPy's FFT. The underlying code is actually the same, but
pypocketfft is compiled with optimizations for the host CPU and can provide
significantly faster transforms.
NIFTy documentation is provided by Sphinx. To build the documentation:: NIFTy documentation is provided by Sphinx. To build the documentation::
sudo apt-get install python3-sphinx-rtd-theme dvipng sudo apt-get install python3-sphinx-rtd-theme dvipng
......
...@@ -73,7 +73,7 @@ class GLSpace(StructuredDomain): ...@@ -73,7 +73,7 @@ class GLSpace(StructuredDomain):
# blown up by a factor of self.nlon # blown up by a factor of self.nlon
@property @property
def dvol(self): def dvol(self):
from pyHealpix import GL_weights from ducc0.misc import GL_weights
if self._dvol is None: if self._dvol is None:
self._dvol = GL_weights(self.nlat, self.nlon) self._dvol = GL_weights(self.nlat, self.nlon)
return np.repeat(self._dvol, self.nlon) return np.repeat(self._dvol, self.nlon)
......
...@@ -117,11 +117,11 @@ class LMSpace(StructuredDomain): ...@@ -117,11 +117,11 @@ class LMSpace(StructuredDomain):
e.g. only dependant on theta in radians""" e.g. only dependant on theta in radians"""
from .gl_space import GLSpace from .gl_space import GLSpace
from ..operators.harmonic_operators import HarmonicTransformOperator from ..operators.harmonic_operators import HarmonicTransformOperator
import pyHealpix from ducc0.misc import GL_thetas
# define azimuthally symmetric spaces for kernel transform # define azimuthally symmetric spaces for kernel transform
gl = GLSpace(self.lmax + 1, 1) gl = GLSpace(self.lmax + 1, 1)
lm0 = gl.get_default_codomain() lm0 = gl.get_default_codomain()
theta = pyHealpix.GL_thetas(gl.nlat) theta = GL_thetas(gl.nlat)
# evaluate the kernel function at the required thetas # evaluate the kernel function at the required thetas
kernel_sphere = Field.from_raw(gl, func(theta)) kernel_sphere = Field.from_raw(gl, func(theta))
# normalize the kernel such that the integral over the sphere is 4pi # normalize the kernel such that the integral over the sphere is 4pi
......
...@@ -29,21 +29,20 @@ def set_nthreads(nthr): ...@@ -29,21 +29,20 @@ def set_nthreads(nthr):
try: try:
import pypocketfft import ducc0.fft as my_fft
def fftn(a, axes=None): def fftn(a, axes=None):
return pypocketfft.c2c(a, axes=axes, nthreads=max(_nthreads, 0)) return my_fft.c2c(a, axes=axes, nthreads=max(_nthreads, 0))
def ifftn(a, axes=None): def ifftn(a, axes=None):
return pypocketfft.c2c(a, axes=axes, inorm=2, forward=False, return my_fft.c2c(a, axes=axes, inorm=2, forward=False,
nthreads=max(_nthreads, 0)) nthreads=max(_nthreads, 0))
def hartley(a, axes=None): def hartley(a, axes=None):
return pypocketfft.genuine_hartley(a, axes=axes, return my_fft.genuine_hartley(a, axes=axes, nthreads=max(_nthreads, 0))
nthreads=max(_nthreads, 0))
except ImportError: except ImportError:
import scipy.fft import scipy.fft
......
...@@ -202,15 +202,15 @@ class SHTOperator(LinearOperator): ...@@ -202,15 +202,15 @@ class SHTOperator(LinearOperator):
hspc.check_codomain(target) hspc.check_codomain(target)
target.check_codomain(hspc) target.check_codomain(hspc)
from pyHealpix import sharpjob_d from ducc0.sht import sharpjob_d
self.lmax = hspc.lmax self.lmax = hspc.lmax
self.mmax = hspc.mmax self.mmax = hspc.mmax
self.sjob = sharpjob_d() self.sjob = sharpjob_d()
self.sjob.set_triangular_alm_info(self.lmax, self.mmax) self.sjob.set_triangular_alm_info(self.lmax, self.mmax)
if isinstance(target, GLSpace): if isinstance(target, GLSpace):
self.sjob.set_Gauss_geometry(target.nlat, target.nlon) self.sjob.set_gauss_geometry(target.nlat, target.nlon)
else: else:
self.sjob.set_Healpix_geometry(target.nside) self.sjob.set_healpix_geometry(target.nside)
def __reduce__(self): def __reduce__(self):
return (_unpickleSHTOperator, return (_unpickleSHTOperator,
......
...@@ -423,7 +423,7 @@ def _plot2D(f, ax, **kwargs): ...@@ -423,7 +423,7 @@ def _plot2D(f, ax, **kwargs):
_limit_xy(**kwargs) _limit_xy(**kwargs)
return return
elif isinstance(dom, (HPSpace, GLSpace)): elif isinstance(dom, (HPSpace, GLSpace)):
import pyHealpix from ducc0.healpix import Healpix_Base
xsize = 800 xsize = 800
res, mask, theta, phi = _mollweide_helper(xsize) res, mask, theta, phi = _mollweide_helper(xsize)
if have_rgb: if have_rgb:
...@@ -434,14 +434,15 @@ def _plot2D(f, ax, **kwargs): ...@@ -434,14 +434,15 @@ def _plot2D(f, ax, **kwargs):
ptg = np.empty((phi.size, 2), dtype=np.float64) ptg = np.empty((phi.size, 2), dtype=np.float64)
ptg[:, 0] = theta ptg[:, 0] = theta
ptg[:, 1] = phi ptg[:, 1] = phi
base = pyHealpix.Healpix_Base(int(np.sqrt(dom.size//12)), "RING") base = Healpix_Base(int(np.sqrt(dom.size//12)), "RING")
if have_rgb: if have_rgb:
res[mask] = rgb[base.ang2pix(ptg)] res[mask] = rgb[base.ang2pix(ptg)]
else: else:
res[mask] = f.val[base.ang2pix(ptg)] res[mask] = f.val[base.ang2pix(ptg)]
else: else:
from ducc0.misc import GL_thetas
ra = np.linspace(0, 2*np.pi, dom.nlon+1) ra = np.linspace(0, 2*np.pi, dom.nlon+1)
dec = pyHealpix.GL_thetas(dom.nlat) dec = GL_thetas(dom.nlat)
ilat = _find_closest(dec, theta) ilat = _find_closest(dec, theta)
ilon = _find_closest(ra, phi) ilon = _find_closest(ra, phi)
ilon = np.where(ilon == dom.nlon, 0, ilon) ilon = np.where(ilon == dom.nlon, 0, ilon)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment