Commit e991506b authored by Torsten Ensslin's avatar Torsten Ensslin

hihgh leveltext for implicit operators and generative models drafted

parent c3db0b31
......@@ -85,4 +85,99 @@ The above line of argumentation analogously applies to the discretization of ope
The proper discretization of spaces, fields, and operators, as well as the normalization of position integrals, is essential for the conservation of the continuum limit. Their consistent implementation in NIFTY allows a pixelization independent coding of algorithms.
Free Theory & Implicit Operators
A free IFT appears when the signal field :math:`{s}` and the noise :math:`{n}` in the data :math:`{d}` are independent, zero-centered Gaussian processes of kown covariances :math:`{S}` and :math:`{N}`, respectively,
.. math::
\mathcal{P}(s,n) = \mathcal{G}(s,S)\,\mathcal{G}(n,N),
and the measurement equation is linear in both,
.. math::
d= R\, s + n,
with :math:`{R}` the measurement response, which maps the continous signal field into the discrete data space.
This is called a free theory, as the information Hamiltonian
.. math::
\mathcal{H}(d,s)= -\log \mathcal{P}(d,s)= \frac{1}{2} s^\dagger S^{-1} s + \frac{1}{2} (d-R\,s)^\dagger N^{-1} (d-R\,s) + \mathrm{const}
is only of quadratic order in :math:`{s}`, which leads to a linear relation between the data and the posterior mean field.
In this case, the posterior is
.. math::
\mathcal{P}(s|d) = \mathcal{G}(s-m,D)
.. math::
m = D\, j
the posterior mean field,
.. math::
D = \left( S^{-1} + R^\dagger N^{-1} R\right)^{-1}
the posterior covariance operator, and
.. math::
j = R^\dagger N^{-1} d
the information source. The operation in :math:`{d= D\,R^\dagger N^{-1} d}` is also called the generalized Wiener filter.
NIFTy permits to define the involved operators :math:`{R}`, :math:`{R^\dagger}`, :math:`{S}`, and :math:`{N}` implicitely, as coputer routines that can be applied to vectors, but which do not require the explicit storage of the matrix elements of the operators.
These implicit operators can be combined into new operators, e.g. to :math:`{D^{-1} = \left( S^{-1} + R^\dagger N^{-1} R\right)^{-1}}`, as well as their inverses, e.g. :math:`{D^{-1} = \left( D^{-1} \right)^{-1}}`.
The invocation of an inverse operator applied to a vector might trigger the execution of a numerical linear algebra solver.
Thus, when NIFTy calculates :math:`{m = D\, j}` it actually solves :math:`{D^{-1} m = j}` for :math:`{m}` behind the scenes.
The demo codes demos/ and demos/Wiener_Filter.ipynb illustrate this.
Generative Models
For more complex measurement situations, involving non-linear measuremnts, unknown covariances, calibration constants and the like, it is recommended to formulate those as generative models as NIFTy provides powerful inference algorithms for such.
In a generative model, all known or unknown quantities are described as the results of generative processes, which start with simple probability distributions, like uniform, iid Gaussian, or delta distributions.
The above free theory case looks as a generative model like the following:
.. math::
s = A\,\xi
with :math:`{A}` the amplitude operator such that it generates signal field with the correct covariance :math:`{S=A\,A^\dagger}` out of a Gaussian white noise field :math:`{\xi}` with :math:`{\mathcal{P}(\xi)= \mathcal{G}(\xi, \mathbb{1})}`.
The joint information Hamiltonian for the whitened signal field :math:`{\xi}` reads
.. math::
\mathcal{H}(d,\xi)= -\log \mathcal{P}(d,s)= \frac{1}{2} \xi^\dagger \mathbb{1} \xi + \frac{1}{2} (d-R\,A\,\xi)^\dagger N^{-1} (d-R\,A\,\xi) + \mathrm{const}.
NIFTy takes advantage of this formulation in several ways:
1) all prior degrees of freedom have now the same variance
2) the amplitude operator can be regarded as part of the response, :math:`{R'=R\,A}`
3) the response can be made non-linear, e.g. :math:`{R'(s)=R \exp(A\,\xi)}`, see demos/demos/
4) the amplitude operator can be made dependent on unknowns as well, e.g. :math:`{A=A(\tau)=\mathrm{FourierTransform}\,\mathrm{DiagonalOperator}(\exp(\tau))}` represents an amplitude model with a flexible Fourier spectrum
5) the gradient of the Hamiltonian and the Fischer information metric with respect to all unknown parameters, here :math:`{\xi} and can be constructed by NIFTy and used for Metric Gaussian Variational Inference.
A demonstration example for reconstructing a non-Gaussian signal with unknown covarinance from a complex (tomographic) response is given by demos/demos/ .
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment