Commit 78b77518 authored by Martin Reinecke's avatar Martin Reinecke
Browse files

merge NIFTY_4

parents 0e8e4be1 82c170d1
%% Cell type:markdown id: tags:
# A NIFTy demonstration
%% Cell type:markdown id: tags:
## IFT: Big Picture
IFT starting point:
$$d = Rs+n$$
Typically, $s$ is a continuous field, $d$ a discrete data vector. Particularly, $R$ is not invertible.
IFT aims at **inverting** the above uninvertible problem in the **best possible way** using Bayesian statistics.
## NIFTy
NIFTy (Numerical Information Field Theory) is a Python framework in which IFT problems can be tackled easily.
Main Interfaces:
- **Spaces**: Cartesian, 2-Spheres (Healpix, Gauss-Legendre) and their respective harmonic spaces.
- **Fields**: Defined on spaces.
- **Operators**: Acting on fields.
%% Cell type:markdown id: tags:
## Wiener Filter: Formulae
### Assumptions
- $d=Rs+n$, $R$ linear operator.
- $\mathcal P (s) = \mathcal G (s,S)$, $\mathcal P (n) = \mathcal G (n,N)$ where $S, N$ are positive definite matrices.
### Posterior
The Posterior is given by:
$$\mathcal P (s|d) \propto P(s,d) = \mathcal G(d-Rs,N) \,\mathcal G(s,S) \propto \mathcal G (m,D) $$
where
$$\begin{align}
m &= Dj \\
D^{-1}&= (S^{-1} +R^\dagger N^{-1} R )\\
j &= R^\dagger N^{-1} d
\end{align}$$
Let us implement this in NIFTy!
%% Cell type:markdown id: tags:
## Wiener Filter: Example
- We assume statistical homogeneity and isotropy. Therefore the signal covariance $S$ is diagonal in harmonic space, and is described by a one-dimensional power spectrum, assumed here as $$P(k) = P_0\,\left(1+\left(\frac{k}{k_0}\right)^2\right)^{-\gamma /2},$$
with $P_0 = 0.2, k_0 = 5, \gamma = 4$.
- $N = 0.2 \cdot \mathbb{1}$.
- Number of data points $N_{pix} = 512$.
- reconstruction in harmonic space.
- Response operator:
$$R = FFT_{\text{harmonic} \rightarrow \text{position}}$$
%% Cell type:code id: tags:
``` python
N_pixels = 512 # Number of pixels
def pow_spec(k):
P0, k0, gamma = [.2, 5, 4]
return P0 / ((1. + (k/k0)**2)**(gamma / 2))
```
%% Cell type:markdown id: tags:
## Wiener Filter: Implementation
%% Cell type:markdown id: tags:
### Import Modules
%% Cell type:code id: tags:
``` python
import numpy as np
np.random.seed(40)
import nifty4 as ift
import matplotlib.pyplot as plt
%matplotlib inline
```
%% Cell type:markdown id: tags:
### Implement Propagator
%% Cell type:code id: tags:
``` python
def Curvature(R, N, Sh):
IC = ift.GradientNormController(iteration_limit=50000,
tol_abs_gradnorm=0.1)
inverter = ift.ConjugateGradient(controller=IC)
# WienerFilterCurvature is (R.adjoint*N.inverse*R + Sh.inverse) plus some handy
# helper methods.
return ift.library.WienerFilterCurvature(R,N,Sh,inverter)
```
%% Cell type:markdown id: tags:
### Conjugate Gradient Preconditioning
- $D$ is defined via:
$$D^{-1} = \mathcal S_h^{-1} + R^\dagger N^{-1} R.$$
In the end, we want to apply $D$ to $j$, i.e. we need the inverse action of $D^{-1}$. This is done numerically (algorithm: *Conjugate Gradient*).
<!--
- One can define the *condition number* of a non-singular and normal matrix $A$:
$$\kappa (A) := \frac{|\lambda_{\text{max}}|}{|\lambda_{\text{min}}|},$$
where $\lambda_{\text{max}}$ and $\lambda_{\text{min}}$ are the largest and smallest eigenvalue of $A$, respectively.
- The larger $\kappa$ the slower Conjugate Gradient.
- By default, conjugate gradient solves: $D^{-1} m = j$ for $m$, where $D^{-1}$ can be badly conditioned. If one knows a non-singular matrix $T$ for which $TD^{-1}$ is better conditioned, one can solve the equivalent problem:
$$\tilde A m = \tilde j,$$
where $\tilde A = T D^{-1}$ and $\tilde j = Tj$.
- In our case $S^{-1}$ is responsible for the bad conditioning of $D$ depending on the chosen power spectrum. Thus, we choose
$$T = \mathcal F^\dagger S_h^{-1} \mathcal F.$$
-->
%% Cell type:markdown id: tags:
### Generate Mock data
- Generate a field $s$ and $n$ with given covariances.
- Calculate $d$.
%% Cell type:code id: tags:
``` python
s_space = ift.RGSpace(N_pixels)
h_space = s_space.get_default_codomain()
HT = ift.HarmonicTransformOperator(h_space, target=s_space)
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
R = HT #*ift.create_harmonic_smoothing_operator((h_space,), 0, 0.02)
# Fields and data
sh = Sh.draw_sample()
noiseless_data=R(sh)
noise_amplitude = np.sqrt(0.2)
N = ift.ScalingOperator(noise_amplitude**2, s_space)
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=noise_amplitude, mean=0)
d = noiseless_data + n
j = R.adjoint_times(N.inverse_times(d))
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
```
%% Cell type:markdown id: tags:
### Run Wiener Filter
%% Cell type:code id: tags:
``` python
m = D(j)
```
%% Cell type:markdown id: tags:
### Signal Reconstruction
%% Cell type:code id: tags:
``` python
# Get signal data and reconstruction data
s_data = HT(sh).to_global_data()
m_data = HT(m).to_global_data()
d_data = d.to_global_data()
plt.figure(figsize=(15,10))
plt.plot(s_data, 'r', label="Signal", linewidth=3)
plt.plot(d_data, 'k.', label="Data")
plt.plot(m_data, 'k', label="Reconstruction",linewidth=3)
plt.title("Reconstruction")
plt.legend()
plt.show()
```
%% Cell type:code id: tags:
``` python
plt.figure(figsize=(15,10))
plt.plot(s_data - s_data, 'r', label="Signal", linewidth=3)
plt.plot(d_data - s_data, 'k.', label="Data")
plt.plot(m_data - s_data, 'k', label="Reconstruction",linewidth=3)
plt.axhspan(-noise_amplitude,noise_amplitude, facecolor='0.9', alpha=.5)
plt.title("Residuals")
plt.legend()
plt.show()
```
%% Cell type:markdown id: tags:
### Power Spectrum
%% Cell type:code id: tags:
``` python
s_power_data = ift.power_analyze(sh).to_global_data()
m_power_data = ift.power_analyze(m).to_global_data()
plt.figure(figsize=(15,10))
plt.loglog()
plt.xlim(1, int(N_pixels/2))
ymin = min(m_power_data)
plt.ylim(ymin, 1)
xs = np.arange(1,int(N_pixels/2),.1)
plt.plot(xs, pow_spec(xs), label="True Power Spectrum", color='k',alpha=0.5)
plt.plot(s_power_data, 'r', label="Signal")
plt.plot(m_power_data, 'k', label="Reconstruction")
plt.axhline(noise_amplitude**2 / N_pixels, color="k", linestyle='--', label="Noise level", alpha=.5)
plt.axhspan(noise_amplitude**2 / N_pixels, ymin, facecolor='0.9', alpha=.5)
plt.title("Power Spectrum")
plt.legend()
plt.show()
```
%% Cell type:markdown id: tags:
## Wiener Filter on Incomplete Data
%% Cell type:code id: tags:
``` python
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
N = ift.ScalingOperator(noise_amplitude**2,s_space)
# R is defined below
# Fields
sh = Sh.draw_sample()
s = HT(sh)
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=noise_amplitude, mean=0)
```
%% Cell type:markdown id: tags:
### Partially Lose Data
%% Cell type:code id: tags:
``` python
l = int(N_pixels * 0.2)
h = int(N_pixels * 0.2 * 2)
mask = np.full(s_space.shape, 1.)
mask[l:h] = 0
mask = ift.Field.from_global_data(s_space, mask)
R = ift.DiagonalOperator(mask)*HT
n = n.to_global_data()
n[l:h] = 0
n = ift.Field.from_global_data(s_space, n)
d = R(sh) + n
```
%% Cell type:code id: tags:
``` python
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
j = R.adjoint_times(N.inverse_times(d))
m = D(j)
```
%% Cell type:markdown id: tags:
### Compute Uncertainty
%% Cell type:code id: tags:
``` python
m_mean, m_var = ift.probe_with_posterior_samples(curv, HT, 200)
```
%% Cell type:markdown id: tags:
### Get data
%% Cell type:code id: tags:
``` python
# Get signal data and reconstruction data
s_data = s.to_global_data()
m_data = HT(m).to_global_data()
m_var_data = m_var.to_global_data()
uncertainty = np.sqrt(m_var_data)
d_data = d.to_global_data()
# Set lost data to NaN for proper plotting
d_data[d_data == 0] = np.nan
```
%% Cell type:code id: tags:
``` python
fig = plt.figure(figsize=(15,10))
plt.axvspan(l, h, facecolor='0.8',alpha=0.5)
plt.fill_between(range(N_pixels), m_data - uncertainty, m_data + uncertainty, facecolor='0.5', alpha=0.5)
plt.plot(s_data, 'r', label="Signal", alpha=1, linewidth=3)
plt.plot(d_data, 'k.', label="Data")
plt.plot(m_data, 'k', label="Reconstruction", linewidth=3)
plt.title("Reconstruction of incomplete data")
plt.legend()
```
%% Cell type:markdown id: tags:
# 2d Example
%% Cell type:code id: tags:
``` python
N_pixels = 256 # Number of pixels
sigma2 = 2. # Noise variance
def pow_spec(k):
P0, k0, gamma = [.2, 2, 4]
return P0 * (1. + (k/k0)**2)**(-gamma/2)
s_space = ift.RGSpace([N_pixels, N_pixels])
```
%% Cell type:code id: tags:
``` python
h_space = s_space.get_default_codomain()
HT = ift.HarmonicTransformOperator(h_space,s_space)
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
N = ift.ScalingOperator(sigma2,s_space)
# Fields and data
sh = Sh.draw_sample()
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=np.sqrt(sigma2), mean=0)
# Lose some data
l = int(N_pixels * 0.33)
h = int(N_pixels * 0.33 * 2)
mask = np.full(s_space.shape, 1.)
mask[l:h,l:h] = 0.
mask = ift.Field.from_global_data(s_space, mask)
R = ift.DiagonalOperator(mask)*HT
n = n.to_global_data()
n[l:h, l:h] = 0
n = ift.Field.from_global_data(s_space, n)
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
d = R(sh) + n
j = R.adjoint_times(N.inverse_times(d))
# Run Wiener filter
m = D(j)
# Uncertainty
m_mean, m_var = ift.probe_with_posterior_samples(curv, HT, 20)
# Get data
s_data = HT(sh).to_global_data()
m_data = HT(m).to_global_data()
m_var_data = m_var.to_global_data()
d_data = d.to_global_data()
uncertainty = np.sqrt(np.abs(m_var_data))
```
%% Cell type:code id: tags:
``` python
cm = ['magma', 'inferno', 'plasma', 'viridis'][1]
mi = np.min(s_data)
ma = np.max(s_data)
fig, axes = plt.subplots(1, 2, figsize=(15, 7))
data = [s_data, d_data]
caption = ["Signal", "Data"]
for ax in axes.flat:
im = ax.imshow(data.pop(0), interpolation='nearest', cmap=cm, vmin=mi,
vmax=ma)
ax.set_title(caption.pop(0))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([0.85, 0.15, 0.05, 0.7])
fig.colorbar(im, cax=cbar_ax)
```
%% Cell type:code id: tags:
``` python
mi = np.min(s_data)
ma = np.max(s_data)
fig, axes = plt.subplots(3, 2, figsize=(15, 22.5))
sample = HT(curv.draw_sample(from_inverse=True)+m).to_global_data()
post_mean = (m_mean + HT(m)).to_global_data()
data = [s_data, m_data, post_mean, sample, s_data - m_data, uncertainty]
caption = ["Signal", "Reconstruction", "Posterior mean", "Sample", "Residuals", "Uncertainty Map"]
for ax in axes.flat:
im = ax.imshow(data.pop(0), interpolation='nearest', cmap=cm, vmin=mi, vmax=ma)
ax.set_title(caption.pop(0))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([.85, 0.15, 0.05, 0.7])
fig.colorbar(im, cax=cbar_ax)
```
%% Cell type:markdown id: tags:
### Is the uncertainty map reliable?
%% Cell type:code id: tags:
``` python
precise = (np.abs(s_data-m_data) < uncertainty )
precise = (np.abs(s_data-m_data) < uncertainty)
print("Error within uncertainty map bounds: " + str(np.sum(precise) * 100 / N_pixels**2) + "%")
plt.figure(figsize=(15,10))
plt.imshow(precise.astype(float), cmap="brg")
plt.colorbar()
```
%% Cell type:markdown id: tags:
# Start Coding
## NIFTy Repository + Installation guide
https://gitlab.mpcdf.mpg.de/ift/NIFTy
NIFTy v4 **more or less stable!**
......
......@@ -10,8 +10,8 @@ if __name__ == "__main__":
p_spec = lambda k: (1. / (k*correlation_length + 1) ** 4)
nonlinearity = Tanh()
#nonlinearity = Linear()
#nonlinearity = Exponential()
# nonlinearity = Linear()
# nonlinearity = Exponential()
# Set up position space
s_space = ift.RGSpace(1024)
......
from .version import __version__
from . import dobj
from .domains import *
from .domain_tuple import DomainTuple
from .operators import *
from .field import Field
from .operators import *
from .probing.utils import probe_with_posterior_samples, probe_diagonal, \
StatCalculator
from .minimization import *
from .sugar import *
......@@ -28,5 +22,5 @@ from .multi import *
__all__ = ["__version__", "dobj", "DomainTuple"] + \
domains.__all__ + operators.__all__ + minimization.__all__ + \
["DomainTuple", "Field", "sqrt", "exp", "log"] + \
["Field"] + sugar.__all__ + \
multi.__all__
......@@ -17,7 +17,6 @@
# and financially supported by the Studienstiftung des deutschen Volkes.
import numpy as np
from ..field import Field
from ..sugar import from_random
__all__ = ["check_value_gradient_consistency",
......@@ -50,15 +49,16 @@ def check_value_gradient_consistency(E, tol=1e-6, ntries=100):
E2 = _get_acceptable_energy(E)
val = E.value
dir = E2.position - E.position
Enext = E2
# Enext = E2
dirnorm = dir.norm()
dirder = E.gradient.vdot(dir)/dirnorm
for i in range(50):
Emid = E.at(E.position + 0.5*dir)
dirder = Emid.gradient.vdot(dir)/dirnorm
if abs((E2.value-val)/dirnorm-dirder) < tol:
break
dir *= 0.5
dirnorm *= 0.5
E2 = E2.at(E.position+dir)
E2 = Emid
else:
raise ValueError("gradient and value seem inconsistent")
# E = Enext
......@@ -67,19 +67,20 @@ def check_value_gradient_consistency(E, tol=1e-6, ntries=100):
def check_value_gradient_curvature_consistency(E, tol=1e-6, ntries=100):
for _ in range(ntries):
E2 = _get_acceptable_energy(E)
val = E.value
dir = E2.position - E.position
Enext = E2
# Enext = E2
dirnorm = dir.norm()
dirder = E.gradient.vdot(dir)/dirnorm
dgrad = E.curvature(dir)/dirnorm
for i in range(50):
gdiff = E2.gradient - E.gradient
if abs((E2.value-E.value)/dirnorm-dirder) < tol and \
Emid = E.at(E.position + 0.5*dir)
dirder = Emid.gradient.vdot(dir)/dirnorm
dgrad = Emid.curvature(dir)/dirnorm
if abs((E2.value-val)/dirnorm-dirder) < tol and \
(abs((E2.gradient-E.gradient)/dirnorm-dgrad) < tol).all():
break
dir *= 0.5
dirnorm *= 0.5
E2 = E2.at(E.position+dir)
E2 = Emid
else:
raise ValueError("gradient, value and curvature seem inconsistent")
# E = Enext
......@@ -721,6 +721,15 @@ class Field(object):
self._domain.__str__() + \
"\n- val = " + repr(self.val)
def equivalent(self, other):
if self is other:
return True
if not isinstance(other, Field):
return False
if self._domain != other._domain:
return False
return (self._val == other._val).all()
for op in ["__add__", "__radd__", "__iadd__",
"__sub__", "__rsub__", "__isub__",
"__mul__", "__rmul__", "__imul__",
......
......@@ -15,10 +15,11 @@ from .energy import Energy
from .quadratic_energy import QuadraticEnergy
from .line_energy import LineEnergy
from .yango import Yango
from .energy_sum import EnergySum
__all__ = ["LineSearch", "LineSearchStrongWolfe",
"IterationController", "GradientNormController",
"Minimizer", "ConjugateGradient", "NonlinearCG", "DescentMinimizer",
"SteepestDescent", "VL_BFGS", "RelaxedNewton", "ScipyMinimizer",
"NewtonCG", "L_BFGS_B", "ScipyCG", "Energy", "QuadraticEnergy",
"LineEnergy", "L_BFGS", "Yango"]
"LineEnergy", "L_BFGS", "EnergySum", "Yango"]
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
# Copyright(C) 2013-2018 Max-Planck-Society
#
# NIFTy is being developed at the Max-Planck-Institut fuer Astrophysik
# and financially supported by the Studienstiftung des deutschen Volkes.
from .energy import Energy
from ..utilities import memo
class EnergySum(Energy):
def __init__(self, position, energies, minimizer_controller=None,
preconditioner=None, precon_idx=None):
super(EnergySum, self).__init__(position=position)
self._energies = [energy.at(position) for energy in energies]
self._min_controller = minimizer_controller
self._preconditioner = preconditioner
self._precon_idx = precon_idx
def at(self, position):
return self.__class__(position, self._energies, self._min_controller,
self._preconditioner, self._precon_idx)
@property
@memo
def value(self):
res = self._energies[0].value
for e in self._energies[1:]:
res += e.value
return res
@property
@memo
def gradient(self):
res = self._energies[0].gradient.copy()
for e in self._energies[1:]:
res += e.gradient
return res.lock()
@property
@memo
def curvature(self):
res = self._energies[0].curvature
for e in self._energies[1:]:
res = res + e.curvature
if self._min_controller is None:
return res
precon = self._preconditioner
if precon is None and self._precon_idx is not None:
precon = self._energies[self._precon_idx].curvature
from ..operators.inversion_enabler import InversionEnabler
from .conjugate_gradient import ConjugateGradient
return InversionEnabler(
res, ConjugateGradient(self._min_controller), precon)
......@@ -97,5 +97,5 @@ class LineEnergy(object):
if abs(res.imag) / max(abs(res.real), 1.) > 1e-12:
from ..logger import logger
logger.warning("directional derivative has non-negligible "
"imaginary part:", res)
"imaginary part: {}".format(res))
return res.real
......@@ -98,7 +98,7 @@ class ScipyMinimizer(Minimizer):
r = opt.minimize(hlp.fun, x, method=self._method, jac=hlp.jac,
hessp=hessp, options=self._options, bounds=bounds)
if not r.success:
logger.error("Problem in Scipy minimization:", r.message)
logger.error("Problem in Scipy minimization: {}".format(r.message))
return hlp._energy, IterationController.ERROR
return hlp._energy, IterationController.CONVERGED
......
......@@ -88,7 +88,7 @@ class Yango(Minimizer):
return energy, controller.ERROR
# Try 1D Newton Step
energy, success = self._line_searcher.perform_line_search(
energy, (rr/rAr)*r, f_k_minus_1)
energy, (rr/rAr)*r, f_k_minus_1)
else:
a = (rAr*rp - rAp*rr)/det
b = (pAp*rr - pAr*rp)/det
......
......@@ -29,6 +29,8 @@ class MultiField(object):
val : dict
"""
self._val = val
self._domain = MultiDomain.make(
{key: val.domain for key, val in self._val.items()})
def __getitem__(self, key):
return self._val[key]
......@@ -44,8 +46,7 @@ class MultiField(object):
@property
def domain(self):
return MultiDomain.make(
{key: val.domain for key, val in self._val.items()})
return self._domain
@property
def dtype(self):
......@@ -71,7 +72,7 @@ class MultiField(object):
return self
def _check_domain(self, other):
if other.domain != self.domain:
if other._domain != self._domain:
raise ValueError("domains are incompatible.")
def vdot(self, x):
......@@ -147,6 +148,17 @@ class MultiField(object):
return MultiField({key: sub_field.conjugate()
for key, sub_field in self.items()})
def equivalent(self, other):
if self is other:
return True
if not isinstance(other, MultiField):
return False
if self._domain != other._domain:
return False
for key, val in self._val.items():
if not val.equivalent(other[key]):
return False
return True
for op in ["__add__", "__radd__", "__iadd__",
"__sub__", "__rsub__", "__isub__",
......
......@@ -130,16 +130,22 @@ class LinearOperator(NiftyMetaBase()):
def __mul__(self, other):
from .chain_operator import ChainOperator
if np.isscalar(other) and other == 1.:
return self
other = self._toOperator(other, self.domain)
return ChainOperator.make([self, other])
def __rmul__(self, other):
from .chain_operator import ChainOperator
if np.isscalar(other) and other == 1.:
return self
other = self._toOperator(other, self.target)
return ChainOperator.make([other, self])
def __add__(self, other):
from .sum_operator import SumOperator
if np.isscalar(other) and other == 0.:
return self
other = self._toOperator(other, self.domain)
return SumOperator.make([self, other], [False, False])
......@@ -148,6 +154,8 @@ class LinearOperator(NiftyMetaBase()):
def __sub__(self, other):
from .sum_operator import SumOperator
if np.isscalar(other) and other == 0.:
return self
other = self._toOperator(other, self.domain)
return SumOperator.make([self, other], [False, True])
......
......@@ -70,6 +70,7 @@ def get_signal_variance(spec, space):
k_field = dist(field)
return k_field.weight(2).sum()
def _single_power_analyze(field, idx, binbounds):
power_domain = PowerSpace(field.domain[idx], binbounds)
pd = PowerDistributor(field.domain, power_domain, idx)
......
......@@ -25,6 +25,7 @@ from test.common import expand
dom = ift.makeDomain({"d1": ift.RGSpace(10)})
class Test_Functionality(unittest.TestCase):
def test_vdot(self):
f1 = ift.from_random("normal", domain=dom, dtype=np.complex128)
......@@ -53,7 +54,8 @@ class Test_Functionality(unittest.TestCase):
assert_equal(val.local_data, f2[key].local_data)
def test_blockdiagonal(self):
op = ift.BlockDiagonalOperator({"d1": ift.ScalingOperator(20., dom["d1"])})
op = ift.BlockDiagonalOperator({"d1":
ift.ScalingOperator(20., dom["d1"])})
op2 = op*op
ift.extra.consistency_check(op2)
assert_equal(type(op2), ift.BlockDiagonalOperator)
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment