Commit 4e671932 authored by Martin Reinecke's avatar Martin Reinecke

Merge remote-tracking branch 'upstream/NIFTy_5' into simplify_for_const

parents 0fd92060 eba1fd40
image: $CONTAINER_TEST_IMAGE
variables:
CONTAINER_TEST_IMAGE: gitlab-registry.mpcdf.mpg.de/ift/nifty-dev:$CI_BUILD_REF_NAME
CONTAINER_TEST_IMAGE: gitlab-registry.mpcdf.mpg.de/$CI_PROJECT_PATH:$CI_BUILD_REF_NAME
OMP_NUM_THREADS: 1
stages:
......@@ -39,9 +39,9 @@ test_serial:
script:
- pytest-3 -q --cov=nifty5 test
- >
python3 -m coverage report --omit "*plot*,*distributed_do*"
python3 -m coverage report --omit "*plot*,*distributed_do*" | tee coverage.txt
- >
python3 -m coverage report --omit "*plot*,*distributed_do*" | grep TOTAL | awk '{ print "TOTAL: "$4; }'
grep TOTAL coverage.txt | awk '{ print "TOTAL: "$4; }'
test_mpi:
stage: test
......@@ -52,17 +52,15 @@ test_mpi:
pages:
stage: release
before_script:
- ls
script:
- python3 setup.py install --user -f
- sh docs/generate.sh
- mv docs/build/ public/
artifacts:
paths:
- public
only:
- NIFTy_4
- NIFTy_5
before_script:
- python3 setup.py install --user -f
......
......@@ -6,7 +6,7 @@ RUN apt-get update && apt-get install -y \
# Packages needed for NIFTy
python3-scipy \
# Documentation build dependencies
python3-sphinx-rtd-theme \
python3-sphinx-rtd-theme dvipng texlive-latex-base texlive-latex-extra \
# Testing dependencies
python3-pytest-cov jupyter \
# Optional NIFTy dependencies
......
NIFTy - Numerical Information Field Theory
==========================================
[![build status](https://gitlab.mpcdf.mpg.de/ift/nifty-dev/badges/NIFTy_5/build.svg)](https://gitlab.mpcdf.mpg.de/ift/nifty-dev/commits/NIFTy_5)
[![coverage report](https://gitlab.mpcdf.mpg.de/ift/nifty-dev/badges/NIFTy_5/coverage.svg)](https://gitlab.mpcdf.mpg.de/ift/nifty-dev/commits/NIFTy_5)
[![build status](https://gitlab.mpcdf.mpg.de/ift/NIFTy/badges/NIFTy_5/build.svg)](https://gitlab.mpcdf.mpg.de/ift/NIFTy/commits/NIFTy_5)
[![coverage report](https://gitlab.mpcdf.mpg.de/ift/NIFTy/badges/NIFTy_5/coverage.svg)](https://gitlab.mpcdf.mpg.de/ift/NIFTy/commits/NIFTy_5)
**NIFTy** project homepage:
[http://ift.pages.mpcdf.de/NIFTy](http://ift.pages.mpcdf.de/NIFTy)
[http://ift.pages.mpcdf.de/nifty](http://ift.pages.mpcdf.de/nifty)
Summary
-------
......@@ -13,23 +13,31 @@ Summary
**NIFTy**, "**N**umerical **I**nformation **F**ield **T**heor<strong>y</strong>", is
a versatile library designed to enable the development of signal
inference algorithms that operate regardless of the underlying spatial
grid and its resolution. Its object-oriented framework is written in
Python, although it accesses libraries written in C++ and C for
efficiency.
inference algorithms that operate regardless of the underlying grids
(spatial, spectral, temporal, …) and their resolutions.
Its object-oriented framework is written in Python, although it accesses
libraries written in C++ and C for efficiency.
NIFTy offers a toolkit that abstracts discretized representations of
continuous spaces, fields in these spaces, and operators acting on
fields into classes. The correct normalization of operations on
fields is taken care of automatically without concerning the user. This
allows for an abstract formulation and programming of inference
these fields into classes.
This allows for an abstract formulation and programming of inference
algorithms, including those derived within information field theory.
Thus, NIFTy permits its user to rapidly prototype algorithms in 1D, and
then apply the developed code in higher-dimensional settings of real
world problems. The set of spaces on which NIFTy operates comprises
point sets, *n*-dimensional regular grids, spherical spaces, their
harmonic counterparts, and product spaces constructed as combinations of
those.
NIFTy's interface is designed to resemble IFT formulae in the sense
that the user implements algorithms in NIFTy independent of the topology
of the underlying spaces and the discretization scheme.
Thus, the user can develop algorithms on subsets of problems and on
spaces where the detailed performance of the algorithm can be properly
evaluated and then easily generalize them to other, more complex spaces
and the full problem, respectively.
The set of spaces on which NIFTy operates comprises point sets,
*n*-dimensional regular grids, spherical spaces, their harmonic
counterparts, and product spaces constructed as combinations of those.
NIFTy takes care of numerical subtleties like the normalization of
operations on fields and the numerical representation of model
components, allowing the user to focus on formulating the abstract
inference procedures and process-specific model properties.
Installation
......@@ -52,7 +60,7 @@ Optional dependencies:
The current version of Nifty5 can be obtained by cloning the repository and
switching to the NIFTy_5 branch:
git clone https://gitlab.mpcdf.mpg.de/ift/NIFTy.git
git clone https://gitlab.mpcdf.mpg.de/ift/nifty.git
### Installation
......@@ -62,18 +70,19 @@ distributions, the "apt" lines will need slight changes.
NIFTy5 and its mandatory dependencies can be installed via:
sudo apt-get install git python3 python3-pip python3-dev
pip3 install --user git+https://gitlab.mpcdf.mpg.de/ift/NIFTy.git@NIFTy_5
pip3 install --user git+https://gitlab.mpcdf.mpg.de/ift/nifty.git@NIFTy_5
Plotting support is added via:
pip3 install --user matplotlib
sudo apt-get install python3-matplotlib
FFTW support is added via:
NIFTy uses Numpy's FFT implementation by default. For large problems FFTW may be
used because of its higher performance. It can be installed via:
sudo apt-get install libfftw3-dev
pip3 install --user pyfftw
To actually use FFTW in your Nifty calculations, you need to call
To enable FFTW usage in NIFTy, call
nifty5.fft.enable_fftw()
......@@ -90,14 +99,13 @@ Support for spherical harmonic transforms is added via:
MPI support is added via:
sudo apt-get install openmpi-bin libopenmpi-dev
pip3 install --user mpi4py
sudo apt-get install python3-mpi4py
### Running the tests
To run the tests, additional packages are required:
sudo apt-get install python3-coverage python3-pytest python3-pytest-cov
sudo apt-get install python3-pytest-cov
Afterwards the tests (including a coverage report) can be run using the
following command in the repository root:
......@@ -108,13 +116,13 @@ following command in the repository root:
### First Steps
For a quick start, you can browse through the [informal
introduction](http://ift.pages.mpcdf.de/NIFTy/code.html) or
introduction](http://ift.pages.mpcdf.de/nifty/code.html) or
dive into NIFTy by running one of the demonstrations, e.g.:
python3 demos/getting_started_1.py
### Acknowledgement
### Acknowledgements
Please acknowledge the use of NIFTy in your publication(s) by using a
phrase such as the following:
......@@ -122,10 +130,10 @@ phrase such as the following:
> "Some of the results in this publication have been derived using the
> NIFTy package [(https://gitlab.mpcdf.mpg.de/ift/NIFTy)](https://gitlab.mpcdf.mpg.de/ift/NIFTy)"
and a citation to one of the [publications](http://ift.pages.mpcdf.de/NIFTy/citations.html).
and a citation to one of the [publications](http://ift.pages.mpcdf.de/nifty/citations.html).
### Release Notes
### Licensing terms
The NIFTy package is licensed under the terms of the
[GPLv3](https://www.gnu.org/licenses/gpl.html) and is distributed
......
......@@ -21,6 +21,8 @@
# 1D (set mode=0), 2D (mode=1), or on the sphere (mode=2)
###############################################################################
import sys
import numpy as np
import nifty5 as ift
......@@ -54,7 +56,11 @@ if __name__ == '__main__':
np.random.seed(42)
# Choose space on which the signal field is defined
mode = 1
if len(sys.argv) == 2:
mode = int(sys.argv[1])
else:
mode = 1
if mode == 0:
# One-dimensional regular grid
position_space = ift.RGSpace([1024])
......@@ -135,6 +141,7 @@ if __name__ == '__main__':
# Plotting
rg = isinstance(position_space, ift.RGSpace)
plot = ift.Plot()
filename = "getting_started_1_mode_{}.png".format(mode)
if rg and len(position_space.shape) == 1:
plot.add(
[HT(MOCK_SIGNAL), GR.adjoint(data),
......@@ -142,10 +149,11 @@ if __name__ == '__main__':
label=['Mock signal', 'Data', 'Reconstruction'],
alpha=[1, .3, 1])
plot.add(mask_to_nan(mask, HT(m - MOCK_SIGNAL)), title='Residuals')
plot.output(nx=2, ny=1, xsize=10, ysize=4, title="getting_started_1")
plot.output(nx=2, ny=1, xsize=10, ysize=4, name=filename)
else:
plot.add(HT(MOCK_SIGNAL), title='Mock Signal')
plot.add(mask_to_nan(mask, (GR(Mask)).adjoint(data)), title='Data')
plot.add(HT(m), title='Reconstruction')
plot.add(mask_to_nan(mask, HT(m - MOCK_SIGNAL)), title='Residuals')
plot.output(nx=2, ny=2, xsize=10, ysize=10, title="getting_started_1")
plot.output(nx=2, ny=2, xsize=10, ysize=10, name=filename)
print("Saved results as '{}'.".format(filename))
......@@ -21,6 +21,8 @@
# 1D (set mode=0), 2D (mode=1), or on the sphere (mode=2)
###############################################################################
import sys
import numpy as np
import nifty5 as ift
......@@ -42,23 +44,26 @@ def exposure_2d():
if __name__ == '__main__':
# FIXME All random seeds to 42
np.random.seed(41)
np.random.seed(42)
# Choose space on which the signal field is defined
mode = 2
if len(sys.argv) == 2:
mode = int(sys.argv[1])
else:
mode = 1
if mode == 0:
# One-dimensional regular grid with uniform exposure
# One-dimensional regular grid with uniform exposure of 10
position_space = ift.RGSpace(1024)
exposure = ift.Field.full(position_space, 1.)
exposure = ift.Field.full(position_space, 10.)
elif mode == 1:
# Two-dimensional regular grid with inhomogeneous exposure
position_space = ift.RGSpace([512, 512])
exposure = exposure_2d()
else:
# Sphere with uniform exposure
# Sphere with uniform exposure of 100
position_space = ift.HPSpace(128)
exposure = ift.Field.full(position_space, 1.)
exposure = ift.Field.full(position_space, 100.)
# Define harmonic space and harmonic transform
harmonic_space = position_space.get_default_codomain()
......@@ -107,9 +112,11 @@ if __name__ == '__main__':
# Plotting
signal = sky(mock_position)
reconst = sky(H.position)
filename = "getting_started_2_mode_{}.png".format(mode)
plot = ift.Plot()
plot.add(signal, title='Signal')
plot.add(GR.adjoint(data), title='Data')
plot.add(reconst, title='Reconstruction')
plot.add(reconst - signal, title='Residuals')
plot.output(name='getting_started_2.pdf', xsize=16, ysize=16)
plot.output(xsize=12, ysize=10, name=filename)
print("Saved results as '{}'.".format(filename))
......@@ -17,10 +17,16 @@
############################################################
# Non-linear tomography
# The data is integrated lines of sight
# Random lines (set mode=0), radial lines (mode=1)
#
# The signal is a sigmoid-normal distributed field.
# The data is the field integrated along lines of sight that are
# randomly (set mode=0) or radially (mode=1) distributed
#
# Demo takes a while to compute
#############################################################
import sys
import numpy as np
import nifty5 as ift
......@@ -28,22 +34,26 @@ import nifty5 as ift
def random_los(n_los):
starts = list(np.random.uniform(0, 1, (n_los, 2)).T)
ends = list(0.5 + 0*np.random.uniform(0, 1, (n_los, 2)).T)
ends = list(np.random.uniform(0, 1, (n_los, 2)).T)
return starts, ends
def radial_los(n_los):
starts = list(np.random.uniform(0, 1, (n_los, 2)).T)
ends = list(np.random.uniform(0, 1, (n_los, 2)).T)
ends = list(0.5 + 0*np.random.uniform(0, 1, (n_los, 2)).T)
return starts, ends
if __name__ == '__main__':
np.random.seed(420)
# Choose between random line-of-sight response (mode=1) and radial lines
# of sight (mode=2)
mode = 1
# Choose between random line-of-sight response (mode=0) and radial lines
# of sight (mode=1)
if len(sys.argv) == 2:
mode = int(sys.argv[1])
else:
mode = 0
filename = "getting_started_3_mode_{}_".format(mode) + "{}.png"
position_space = ift.RGSpace([128, 128])
harmonic_space = position_space.get_default_codomain()
......@@ -62,8 +72,8 @@ if __name__ == '__main__':
# Power-law part of spectrum:
'sm': -5, # preferred power-law slope
'sv': .5, # low variance of power-law slope
'im': .4, # y-intercept mean
'iv': .3 # relatively high y-intercept variance
'im': 0, # y-intercept mean, in-/decrease for more/less contrast
'iv': .3 # y-intercept variance
}
A = ift.SLAmplitude(**dct)
......@@ -79,7 +89,7 @@ if __name__ == '__main__':
signal = ift.sigmoid(correlated_field)
# Build the line-of-sight response and define signal response
LOS_starts, LOS_ends = random_los(100) if mode == 1 else radial_los(100)
LOS_starts, LOS_ends = random_los(100) if mode == 0 else radial_los(100)
R = ift.LOSResponse(position_space, starts=LOS_starts, ends=LOS_ends)
signal_response = R(signal)
......@@ -109,7 +119,7 @@ if __name__ == '__main__':
plot.add(signal(mock_position), title='Ground Truth')
plot.add(R.adjoint_times(data), title='Data')
plot.add([A.force(mock_position)], title='Power Spectrum')
plot.output(ny=1, nx=3, xsize=24, ysize=6, name="setup.png")
plot.output(ny=1, nx=3, xsize=24, ysize=6, name=filename.format("setup"))
# number of samples used to estimate the KL
N_samples = 20
......@@ -125,7 +135,8 @@ if __name__ == '__main__':
plot = ift.Plot()
plot.add(signal(KL.position), title="reconstruction")
plot.add([A.force(KL.position), A.force(mock_position)], title="power")
plot.output(ny=1, ysize=6, xsize=16, name="loop-{:02}.png".format(i))
plot.output(ny=1, ysize=6, xsize=16,
name=filename.format("loop_{:02d}".format(i)))
# Draw posterior samples
KL = ift.MetricGaussianKL(mean, H, N_samples)
......@@ -134,6 +145,7 @@ if __name__ == '__main__':
sc.add(signal(sample + KL.position))
# Plotting
filename_res = filename.format("results")
plot = ift.Plot()
plot.add(sc.mean, title="Posterior Mean")
plot.add(ift.sqrt(sc.var), title="Posterior Standard Deviation")
......@@ -144,4 +156,5 @@ if __name__ == '__main__':
A.force(mock_position)],
title="Sampled Posterior Power Spectrum",
linewidth=[1.]*len(powers) + [3., 3.])
plot.output(ny=1, nx=3, xsize=24, ysize=6, name="results.png")
plot.output(ny=1, nx=3, xsize=24, ysize=6, name=filename_res)
print("Saved results as '{}'.".format(filename_res))
......@@ -44,7 +44,8 @@ def polynomial(coefficients, sampling_points):
class PolynomialResponse(ift.LinearOperator):
"""Calculates values of a polynomial parameterized by input at sampling points.
"""Calculates values of a polynomial parameterized by input at sampling
points.
Parameters
----------
......
# rm -rf docs/build docs/source/mod
rm -rf docs/build docs/source/mod
sphinx-apidoc -e -o docs/source/mod nifty5
sphinx-build -b html docs/source/ docs/build/
......@@ -41,7 +41,6 @@ Abstract base class
One of the fundamental building blocks of the NIFTy5 framework is the *domain*.
Its required capabilities are expressed by the abstract :py:class:`Domain` class.
A domain must be able to answer the following queries:
m
- its total number of data entries (pixels), which is accessible via the
:attr:`~Domain.size` property
......@@ -129,7 +128,7 @@ specify full field domains. In principle, a :class:`~domain_tuple.DomainTuple`
can even be empty, which implies that the field living on it is a scalar.
A :class:`~domain_tuple.DomainTuple` supports iteration and indexing, and also
provides the properties :attr:`~domain_tuple.DomainTuple.shape`,
provides the properties :attr:`~domain_tuple.DomainTuple.shape` and
:attr:`~domain_tuple.DomainTuple.size` in analogy to the elementary
:class:`~domains.domain.Domain`.
......@@ -159,10 +158,11 @@ Contractions (like summation, integration, minimum/maximum, computation of
statistical moments) can be carried out either over an entire field (producing
a scalar result) or over sub-domains (resulting in a field defined on a smaller
domain). Scalar products of two fields can also be computed easily.
See the documentation of :class:`~field.Field` for details.
There is also a set of convenience functions to generate fields with constant
values or fields filled with random numbers according to a user-specified
distribution.
distribution: :attr:`~sugar.full`, :attr:`~sugar.from_random`.
Like almost all NIFTy objects, fields are immutable: their value or any other
attribute cannot be modified after construction. To manipulate a field in ways
......@@ -311,11 +311,15 @@ and ``f1`` and ``f2`` are of type :class:`~field.Field`, writing::
will perform the operation suggested intuitively by the notation, checking
domain compatibility while building the composed operator.
The combined operator infers its domain and target from its constituents,
as well as the set of operations it can support.
The properties :attr:`~LinearOperator.adjoint` and
:attr:`~LinearOperator.inverse` return a new operator which behaves as if it
were the original operator's adjoint or inverse, respectively.
The combined operator infers its domain and target from its constituents,
as well as the set of operations it can support.
Instantiating operator adjoints or inverses by :attr:`~LinearOperator.adjoint`
and similar methods is to be distinguished from the instant application of
operators performed by :attr:`~LinearOperator.adjoint_times` and similar
methods.
.. _minimization:
......@@ -368,8 +372,8 @@ failure.
Sensible stopping criteria can vary significantly with the problem being
solved; NIFTy provides one concrete sub-class of :class:`IterationController`
called :class:`GradientNormController`, which should be appropriate in many
circumstances, but users have complete freedom to implement custom sub-classes
for their specific applications.
circumstances, but users have complete freedom to implement custom
:class:`IterationController` sub-classes for their specific applications.
Minimization algorithms
......@@ -398,7 +402,7 @@ Many minimizers for nonlinear problems can be characterized as
This family of algorithms is encapsulated in NIFTy's
:class:`~descent_minimizers.DescentMinimizer` class, which currently has three
concrete implementations: :class:`~descent_minimizers.SteepestDescent`,
generally usable concrete implementations:
:class:`~descent_minimizers.NewtonCG`, :class:`~descent_minimizers.L_BFGS` and
:class:`~descent_minimizers.VL_BFGS`. Of these algorithms, only
:class:`~descent_minimizers.NewtonCG` requires the energy object to provide
......@@ -424,11 +428,13 @@ the information propagator whose inverse is defined as:
:math:`D^{-1} = \left(R^\dagger N^{-1} R + S^{-1}\right)`.
It needs to be applied in forward direction in order to calculate the Wiener
filter solution. Only its inverse application is straightforward; to use it in
forward direction, we make use of NIFTy's
filter solution, but only its inverse application is straightforward.
To use it in forward direction, we make use of NIFTy's
:class:`~operators.inversion_enabler.InversionEnabler` class, which internally
performs a minimization of a
:class:`~minimization.quadratic_energy.QuadraticEnergy` by means of the
:class:`~minimization.conjugate_gradient.ConjugateGradient` algorithm. An
example is provided in
applies the (approximate) inverse of the given operator :math:`x = Op^{-1} (y)` by
solving the equation :math:`y = Op (x)` for :math:`x`.
This is accomplished by minimizing a suitable
:class:`~minimization.quadratic_energy.QuadraticEnergy`
with the :class:`~minimization.conjugate_gradient.ConjugateGradient`
algorithm. An example is provided in
:func:`~library.wiener_filter_curvature.WienerFilterCurvature`.
......@@ -29,4 +29,6 @@ add_module_names = False
html_theme = "sphinx_rtd_theme"
html_logo = 'nifty_logo_black.png'
exclude_patterns = ['mod/modules.rst', 'mod/*version.rst']
exclude_patterns = [
'mod/modules.rst', 'mod/nifty5.git_version.rst', 'mod/nifty5.logger.rst'
]
......@@ -4,8 +4,7 @@ IFT -- Information Field Theory
Theoretical Background
----------------------
`Information Field Theory <http://www.mpa-garching.mpg.de/ift/>`_ [1]_ (IFT) is information theory, the logic of reasoning under uncertainty, applied to fields.
`Information Field Theory <https://www.mpa-garching.mpg.de/ift/>`_ [1]_ (IFT) is information theory, the logic of reasoning under uncertainty, applied to fields.
A field can be any quantity defined over some space, e.g. the air temperature over Europe, the magnetic field strength in the Milky Way, or the matter density in the Universe.
IFT describes how data and knowledge can be used to infer field properties.
Mathematically it is a statistical field theory and exploits many of the tools developed for such.
......@@ -22,89 +21,18 @@ NIFTy comes with reimplemented MAP and VI estimators.
.. tip:: *In-a-nutshell introductions to information field theory* can be found in [2]_, [3]_, [4]_, and [5]_, with the latter probably being the most didactical.
.. [1] T.A. Enßlin et al. (2009), "Information field theory for cosmological perturbation reconstruction and nonlinear signal analysis", PhysRevD.80.105005, 09/2009; `[arXiv:0806.3474] <http://www.arxiv.org/abs/0806.3474>`_
.. [1] T.A. Enßlin et al. (2009), "Information field theory for cosmological perturbation reconstruction and nonlinear signal analysis", PhysRevD.80.105005, 09/2009; `[arXiv:0806.3474] <https://www.arxiv.org/abs/0806.3474>`_
.. [2] T.A. Enßlin (2013), "Information field theory", proceedings of MaxEnt 2012 -- the 32nd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering; AIP Conference Proceedings, Volume 1553, Issue 1, p.184; `[arXiv:1301.2556] <http://arxiv.org/abs/1301.2556>`_
.. [2] T.A. Enßlin (2013), "Information field theory", proceedings of MaxEnt 2012 -- the 32nd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering; AIP Conference Proceedings, Volume 1553, Issue 1, p.184; `[arXiv:1301.2556] <https://arxiv.org/abs/1301.2556>`_
.. [3] T.A. Enßlin (2014), "Astrophysical data analysis with information field theory", AIP Conference Proceedings, Volume 1636, Issue 1, p.49; `[arXiv:1405.7701] <http://arxiv.org/abs/1405.7701>`_
.. [3] T.A. Enßlin (2014), "Astrophysical data analysis with information field theory", AIP Conference Proceedings, Volume 1636, Issue 1, p.49; `[arXiv:1405.7701] <https://arxiv.org/abs/1405.7701>`_
.. [4] Wikipedia contributors (2018), `"Information field theory" <https://en.wikipedia.org/w/index.php?title=Information_field_theory&oldid=876731720>`_, Wikipedia, The Free Encyclopedia.
.. [5] T.A. Enßlin (2019), "Information theory for fields", accepted by Annalen der Physik; `[DOI] <https://doi.org/10.1002/andp.201800127>`_, `[arXiv:1804.03350] <http://arxiv.org/abs/1804.03350>`_
Discretized continuum
---------------------
The representation of fields that are mathematically defined on a continuous space in a finite computer environment is a common necessity.
The goal hereby is to preserve the continuum limit in the calculus in order to ensure a resolution independent discretization.
+-----------------------------+-----------------------------+
| .. image:: images/42vs6.png | .. image:: images/42vs9.png |
| :width: 100 % | :width: 100 % |
+-----------------------------+-----------------------------+
Any partition of the continuous position space :math:`\Omega` (with volume :math:`V`) into a set of :math:`Q` disjoint, proper subsets :math:`\Omega_q` (with volumes :math:`V_q`) defines a pixelization,
.. math::
\Omega &\quad=\quad \dot{\bigcup_q} \; \Omega_q \qquad \mathrm{with} \qquad q \in \{1,\dots,Q\} \subset \mathbb{N}
, \\
V &\quad=\quad \int_\Omega \mathrm{d}x \quad=\quad \sum_{q=1}^Q \int_{\Omega_q} \mathrm{d}x \quad=\quad \sum_{q=1}^Q V_q
.
Here the number :math:`Q` characterizes the resolution of the pixelization and the continuum limit is described by :math:`Q \rightarrow \infty` and :math:`V_q \rightarrow 0` for all :math:`q \in \{1,\dots,Q\}` simultaneously.
Moreover, the above equation defines a discretization of continuous integrals, :math:`\int_\Omega \mathrm{d}x \mapsto \sum_q V_q`.
Any valid discretization scheme for a field :math:`{s}` can be described by a mapping,
.. math::
s(x \in \Omega_q) \quad\mapsto\quad s_q \quad=\quad \int_{\Omega_q} \mathrm{d}x \; w_q(x) \; s(x)
,
if the weighting function :math:`w_q(x)` is chosen appropriately.
In order for the discretized version of the field to converge to the actual field in the continuum limit, the weighting functions need to be normalized in each subset; i.e., :math:`\forall q: \int_{\Omega_q} \mathrm{d}x \; w_q(x) = 1`.
Choosing such a weighting function that is constant with respect to :math:`x` yields
.. [5] T.A. Enßlin (2019), "Information theory for fields", accepted by Annalen der Physik; `[DOI] <https://doi.org/10.1002/andp.201800127>`_, `[arXiv:1804.03350] <https://arxiv.org/abs/1804.03350>`_
.. math::
s_q = \frac{\int_{\Omega_q} \mathrm{d}x \; s(x)}{\int_{\Omega_q} \mathrm{d}x} = \left< s(x) \right>_{\Omega_q}
,
which corresponds to a discretization of the field by spatial averaging.
Another common and equally valid choice is :math:`w_q(x) = \delta(x-x_q)`, which distinguishes some position :math:`x_q \in \Omega_q`, and evaluates the continuous field at this position,
.. math::
s_q \quad=\quad \int_{\Omega_q} \mathrm{d}x \; \delta(x-x_q) \; s(x) \quad=\quad s(x_q)
.
In practice, one often makes use of the spatially averaged pixel position, :math:`x_q = \left< x \right>_{\Omega_q}`.
If the resolution is high enough to resolve all features of the signal field :math:`{s}`, both of these discretization schemes approximate each other, :math:`\left< s(x) \right>_{\Omega_q} \approx s(\left< x \right>_{\Omega_q})`, since they approximate the continuum limit by construction.
(The approximation of :math:`\left< s(x) \right>_{\Omega_q} \approx s(x_q \in \Omega_q)` marks a resolution threshold beyond which further refinement of the discretization reveals no new features; i.e., no new information content of the field :math:`{s}`.)
All operations involving position integrals can be normalized in accordance with the above definitions.
For example, the scalar product between two fields :math:`{s}` and :math:`{u}` is defined as
.. math::
{s}^\dagger {u} \quad=\quad \int_\Omega \mathrm{d}x \; s^*(x) \; u(x) \quad\approx\quad \sum_{q=1}^Q V_q^{\phantom{*}} \; s_q^* \; u_q^{\phantom{*}}
,
where :math:`\dagger` denotes adjunction and :math:`*` complex conjugation.
Since the above approximation becomes an equality in the continuum limit, the scalar product is independent of the pixelization scheme and resolution, if the latter is sufficiently high.
The above line of argumentation analogously applies to the discretization of operators.
For a linear operator :math:`{A}` acting on some field :math:`{s}` as :math:`{A} {s} = \int_\Omega \mathrm{d}y \; A(x,y) \; s(y)`, a matrix representation discretized with constant weighting functions is given by
.. math::
A(x \in \Omega_p, y \in \Omega_q) \quad\mapsto\quad A_{pq} \quad=\quad \frac{\iint_{\Omega_p \Omega_q} \mathrm{d}x \, \mathrm{d}y \; A(x,y)}{\iint_{\Omega_p \Omega_q} \mathrm{d}x \, \mathrm{d}y} \quad=\quad \big< \big< A(x,y) \big>_{\Omega_p} \big>_{\Omega_q}
.
The proper discretization of spaces, fields, and operators, as well as the normalization of position integrals, is essential for the conservation of the continuum limit.
Their consistent implementation in NIFTy allows a pixelization independent coding of algorithms.
Free Theory & Implicit Operators
--------------------------------
......@@ -205,11 +133,11 @@ NIFTy takes advantage of this formulation in several ways:
3) The response can be non-linear, e.g. :math:`{R'(s)=R \exp(A\,\xi)}`, see `demos/getting_started_2.py`.
4) The amplitude operator may dependent on further parameters, e.g. :math:`A=A(\tau)= F\, \widehat{e^\tau}` represents an amplitude operator with a positive definite, unknown spectrum defined in the Fourier domain.
4) The amplitude operator may depend on further parameters, e.g. :math:`A=A(\tau)= F\, \widehat{e^\tau}` represents an amplitude operator with a positive definite, unknown spectrum defined in the Fourier domain.
The amplitude field :math:`{\tau}` would get its own amplitude operator, with a cepstrum (spectrum of a log spectrum) defined in quefrency space (harmonic space of a logarithmically binned harmonic space) to regularize its degrees of freedom by imposing some (user-defined degree of) spectral smoothness.
5) NIFTy calculates the gradient of the information Hamiltonian and the Fisher information metric with respect to all unknown parameters, here :math:`{\xi}` and :math:`{\tau}`, by automatic differentiation.
The gradients are used for MAP and HMCF estimates, and the Fisher matrix is required in addition to the gradient by Metric Gaussian Variational Inference (MGVI), which is available in NIFTy as well.
The gradients are used for MAP estimates, and the Fisher matrix is required in addition to the gradient by Metric Gaussian Variational Inference (MGVI), which is available in NIFTy as well.
MGVI is an implicit operator extension of Automatic Differentiation Variational Inference (ADVI).
The reconstruction of a non-Gaussian signal with unknown covariance from a non-trivial (tomographic) response is demonstrated in `demos/getting_started_3.py`.
......@@ -296,7 +224,7 @@ Thus, only the gradient of the KL is needed with respect to this, which can be e
We stochastically estimate the KL-divergence and gradients with a set of samples drawn from the approximate posterior distribution.
The particular structure of the covariance allows us to draw independent samples solving a certain system of equations.
This KL-divergence for MGVI is implemented in the class MetricGaussianKL within NIFTy5.
This KL-divergence for MGVI is implemented in the class :class:`~minimization.metric_gaussian_kl.MetricGaussianKL` within NIFTy5.
The demo `getting_started_3.py` for example not only infers a field this way, but also the power spectrum of the process that has generated the field.
......
NIFTy -- Numerical Information Field Theory
===========================================
**NIFTy** [1]_, [2]_, "\ **N**\umerical **I**\nformation **F**\ield **T**\heor\ **y**\ ", is a versatile library designed to enable the development of signal inference algorithms that are independent of the underlying spatial grid and its resolution.
**NIFTy** [1]_, [2]_, "\ **N**\umerical **I**\nformation **F**\ield **T**\heor\ **y**\ ", is a versatile library designed to enable the development of signal inference algorithms that are independent of the underlying grids (spatial, spectral, temporal, …) and their resolutions.
Its object-oriented framework is written in Python, although it accesses libraries written in C++ and C for efficiency.
NIFTy offers a toolkit that abstracts discretized representations of continuous spaces, fields in these spaces, and operators acting on fields into classes.
Thereby, the correct normalization of operations on fields is taken care of automatically without concerning the user.
NIFTy offers a toolkit that abstracts discretized representations of continuous spaces, fields in these spaces, and operators acting on these fields into classes.
This allows for an abstract formulation and programming of inference algorithms, including those derived within information field theory.
Thus, NIFTy permits its user to rapidly prototype algorithms in 1D and then apply the developed code in higher-dimensional settings to real world problems.
NIFTy's interface is designed to resemble IFT formulae in the sense that the user implements algorithms in NIFTy independent of the topology of the underlying spaces and the discretization scheme.
Thus, the user can develop algorithms on subsets of problems and on spaces where the detailed performance of the algorithm can be properly evaluated and then easily generalize them to other, more complex spaces and the full problem, respectively.
The set of spaces on which NIFTy operates comprises point sets, *n*-dimensional regular grids, spherical spaces, their harmonic counterparts, and product spaces constructed as combinations of those.
NIFTy takes care of numerical subtleties like the normalization of operations on fields and the numerical representation of model components, allowing the user to focus on formulating the abstract inference procedures and process-specific model properties.
References
----------
......@@ -21,9 +23,11 @@ Contents
........
.. toctree::