Commit 378af2c5 authored by Martin Reinecke's avatar Martin Reinecke
Browse files

Merge branch 'work_on_demos' into 'NIFTy_4'

Work on demos

See merge request ift/NIFTy!218
parents cfe6879c 4e062f6b
Pipeline #24402 passed with stage
in 6 minutes and 11 seconds
%% Cell type:markdown id: tags:
# A NIFTy demonstration
%% Cell type:markdown id: tags:
## IFT: Big Picture
IFT starting point:
$$d = Rs+n$$
Typically, $s$ is a continuous field, $d$ a discrete data vector. Particularly, $R$ is not invertible.
IFT aims at **inverting** the above uninvertible problem in the **best possible way** using Bayesian statistics.
## NIFTy
NIFTy (Numerical Information Field Theory) is a Python framework in which IFT problems can be tackled easily.
Main Interfaces:
- **Spaces**: Cartesian, 2-Spheres (Healpix, Gauss-Legendre) and their respective harmonic spaces.
- **Fields**: Defined on spaces.
- **Operators**: Acting on fields.
%% Cell type:markdown id: tags:
## Wiener Filter: Formulae
### Assumptions
- $d=Rs+n$, $R$ linear operator.
- $\mathcal P (s) = \mathcal G (s,S)$, $\mathcal P (n) = \mathcal G (n,N)$ where $S, N$ are positive definite matrices.
### Posterior
The Posterior is given by:
$$\mathcal P (s|d) \propto P(s,d) = \mathcal G(d-Rs,N) \,\mathcal G(s,S) \propto \mathcal G (m,D) $$
where
$$\begin{align}
m &= Dj \\
D^{-1}&= (S^{-1} +R^\dagger N^{-1} R )\\
j &= R^\dagger N^{-1} d
\end{align}$$
Let us implement this in NIFTy!
%% Cell type:markdown id: tags:
## Wiener Filter: Example
- One-dimensional signal with power spectrum: $$P(k) = P_0\,\left(1+\left(\frac{k}{k_0}\right)^2\right)^{-\gamma /2},$$
with $P_0 = 0.2, k_0 = 5, \gamma = 4$. Recall: $P(k)$ defines an isotropic and homogeneous $S$.
- $N = 0.2 \cdot \mathbb{1}$.
- Number of data points $N_{pix} = 512$.
- reconstruction in harmonic space.
- Response operator:
$$R = FFT_{\text{harmonic} \rightarrow \text{position}}$$
%% Cell type:code id: tags:
``` python
N_pixels = 512 # Number of pixels
def pow_spec(k):
P0, k0, gamma = [.2, 5, 4]
return P0 / ((1. + (k/k0)**2)**(gamma / 2))
```
%% Cell type:markdown id: tags:
## Wiener Filter: Implementation
%% Cell type:markdown id: tags:
### Import Modules
%% Cell type:code id: tags:
``` python
import numpy as np
np.random.seed(40)
import nifty4 as ift
import matplotlib.pyplot as plt
%matplotlib inline
```
%% Cell type:markdown id: tags:
### Implement Propagator
%% Cell type:code id: tags:
``` python
def Curvature(R, N, Sh):
IC = ift.GradientNormController(iteration_limit=50000,
tol_abs_gradnorm=0.1)
inverter = ift.ConjugateGradient(controller=IC)
# WienerFilterCurvature is (R.adjoint*N.inverse*R + Sh.inverse) plus some handy
# helper methods.
return ift.library.WienerFilterCurvature(R,N,Sh,inverter)
```
%% Cell type:markdown id: tags:
### Conjugate Gradient Preconditioning
- $D$ is defined via:
$$D^{-1} = \mathcal S_h^{-1} + R^\dagger N^{-1} R.$$
In the end, we want to apply $D$ to $j$, i.e. we need the inverse action of $D^{-1}$. This is done numerically (algorithm: *Conjugate Gradient*).
<!--
- One can define the *condition number* of a non-singular and normal matrix $A$:
$$\kappa (A) := \frac{|\lambda_{\text{max}}|}{|\lambda_{\text{min}}|},$$
where $\lambda_{\text{max}}$ and $\lambda_{\text{min}}$ are the largest and smallest eigenvalue of $A$, respectively.
- The larger $\kappa$ the slower Conjugate Gradient.
- By default, conjugate gradient solves: $D^{-1} m = j$ for $m$, where $D^{-1}$ can be badly conditioned. If one knows a non-singular matrix $T$ for which $TD^{-1}$ is better conditioned, one can solve the equivalent problem:
$$\tilde A m = \tilde j,$$
where $\tilde A = T D^{-1}$ and $\tilde j = Tj$.
- In our case $S^{-1}$ is responsible for the bad conditioning of $D$ depending on the chosen power spectrum. Thus, we choose
$$T = \mathcal F^\dagger S_h^{-1} \mathcal F.$$
-->
%% Cell type:markdown id: tags:
### Generate Mock data
- Generate a field $s$ and $n$ with given covariances.
- Calculate $d$.
%% Cell type:code id: tags:
``` python
s_space = ift.RGSpace(N_pixels)
h_space = s_space.get_default_codomain()
HT = ift.HarmonicTransformOperator(h_space, target=s_space)
p_space = ift.PowerSpace(h_space)
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
R = HT #*ift.create_harmonic_smoothing_operator((h_space,), 0, 0.02)
# Fields and data
sh = ift.power_synthesize(ift.PS_field(p_space, pow_spec),real_signal=True)
noiseless_data=R(sh)
noise_amplitude = np.sqrt(0.2)
N = ift.ScalingOperator(noise_amplitude**2, s_space)
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=noise_amplitude, mean=0)
d = noiseless_data + n
j = R.adjoint_times(N.inverse_times(d))
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
```
%% Cell type:markdown id: tags:
### Run Wiener Filter
%% Cell type:code id: tags:
``` python
m = D(j)
```
%% Cell type:markdown id: tags:
### Create Power Spectra of Signal and Reconstruction
%% Cell type:code id: tags:
``` python
s_power = ift.power_analyze(sh)
m_power = ift.power_analyze(m)
s_power_data = s_power.val.real
m_power_data = m_power.val.real
# Get signal data and reconstruction data
s_data = HT(sh).val.real
m_data = HT(m).val.real
d_data = d.val.real
```
%% Cell type:markdown id: tags:
### Signal Reconstruction
%% Cell type:code id: tags:
``` python
plt.figure(figsize=(15,10))
plt.plot(s_data, 'g', label="Signal")
plt.plot(d_data, 'k+', label="Data")
plt.plot(m_data, 'r', label="Reconstruction")
plt.title("Reconstruction")
plt.legend()
plt.show()
```
%% Cell type:code id: tags:
``` python
plt.figure(figsize=(15,10))
plt.plot(s_data - s_data, 'g', label="Signal")
plt.plot(d_data - s_data, 'k+', label="Data")
plt.plot(m_data - s_data, 'r', label="Reconstruction")
plt.axhspan(-noise_amplitude,noise_amplitude, facecolor='0.9', alpha=.5)
plt.title("Residuals")
plt.legend()
plt.show()
```
%% Cell type:markdown id: tags:
### Power Spectrum
%% Cell type:code id: tags:
``` python
plt.figure(figsize=(15,10))
plt.loglog()
plt.xlim(1, int(N_pixels/2))
ymin = min(m_power_data)
plt.ylim(ymin, 1)
xs = np.arange(1,int(N_pixels/2),.1)
plt.plot(xs, pow_spec(xs), label="True Power Spectrum", linewidth=.7, color='k')
plt.plot(s_power_data, 'g', label="Signal")
plt.plot(m_power_data, 'r', label="Reconstruction")
plt.axhline(noise_amplitude**2 / N_pixels, color="k", linestyle='--', label="Noise level", alpha=.5)
plt.axhspan(noise_amplitude**2 / N_pixels, ymin, facecolor='0.9', alpha=.5)
plt.title("Power Spectrum")
plt.legend()
plt.show()
```
%% Cell type:markdown id: tags:
## Wiener Filter on Incomplete Data
%% Cell type:code id: tags:
``` python
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
N = ift.ScalingOperator(noise_amplitude**2,s_space)
# R is defined below
# Fields
sh = ift.power_synthesize(ift.PS_field(p_space,pow_spec),real_signal=True)
s = HT(sh)
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=noise_amplitude, mean=0)
```
%% Cell type:markdown id: tags:
### Partially Lose Data
%% Cell type:code id: tags:
``` python
l = int(N_pixels * 0.2)
h = int(N_pixels * 0.2 * 2)
mask = ift.Field(s_space, val=1)
mask.val[ l : h] = 0
R = ift.DiagonalOperator(mask)*HT
n.val[l:h] = 0
d = R(sh) + n
```
%% Cell type:code id: tags:
``` python
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
j = R.adjoint_times(N.inverse_times(d))
m = D(j)
```
%% Cell type:markdown id: tags:
### Compute Uncertainty
%% Cell type:code id: tags:
``` python
sc = ift.probing.utils.StatCalculator()
for i in range(200):
print i
sc.add(HT(curv.generate_posterior_sample()))
m_var = sc.var
m_mean, m_var = ift.probe_with_posterior_samples(curv, m, HT, 200)
```
%% Cell type:markdown id: tags:
### Get data
%% Cell type:code id: tags:
``` python
s_power = ift.power_analyze(sh)
m_power = ift.power_analyze(m)
s_power_data = s_power.val.real
m_power_data = m_power.val.real
# Get signal data and reconstruction data
s_data = s.val.real
m_data = HT(m).val.real
m_var_data = m_var.val.real
uncertainty = np.sqrt(np.abs(m_var_data))
d_data = d.val.real
# Set lost data to NaN for proper plotting
d_data[d_data == 0] = np.nan
```
%% Cell type:code id: tags:
``` python
plt.figure(figsize=(15,10))
plt.plot(s_data, 'g', label="Signal", linewidth=1)
plt.plot(d_data, 'k+', label="Data", alpha=1)
plt.axvspan(l, h, facecolor='0.8', alpha=.5)
plt.title("Incomplete Data")
plt.legend()
```
%% Cell type:code id: tags:
``` python
fig = plt.figure(figsize=(15,10))
plt.plot(s_data, 'g', label="Signal", alpha=1, linewidth=1)
plt.plot(s_data, 'g', label="Signal", alpha=1, linewidth=4)
plt.plot(d_data, 'k+', label="Data", alpha=.5)
plt.plot(m_data, 'r', label="Reconstruction")
plt.axvspan(l, h, facecolor='0.8', alpha=.5)
plt.fill_between(range(N_pixels), m_data - uncertainty, m_data + uncertainty, facecolor='0')
plt.title("Reconstruction of incomplete data")
plt.legend()
```
%% Cell type:markdown id: tags:
# 2d Example
%% Cell type:code id: tags:
``` python
N_pixels = 256 # Number of pixels
sigma2 = 2. # Noise variance
def pow_spec(k):
P0, k0, gamma = [.2, 2, 4]
return P0 * (1. + (k/k0)**2)**(- gamma / 2)
s_space = ift.RGSpace([N_pixels, N_pixels])
```
%% Cell type:code id: tags:
``` python
h_space = s_space.get_default_codomain()
HT = ift.HarmonicTransformOperator(h_space,s_space)
p_space = ift.PowerSpace(h_space)
# Operators
Sh = ift.create_power_operator(h_space, power_spectrum=pow_spec)
N = ift.ScalingOperator(sigma2,s_space)
# Fields and data
sh = ift.power_synthesize(ift.PS_field(p_space,pow_spec),real_signal=True)
n = ift.Field.from_random(domain=s_space, random_type='normal',
std=np.sqrt(sigma2), mean=0)
# Lose some data
l = int(N_pixels * 0.33)
h = int(N_pixels * 0.33 * 2)
mask = ift.Field(s_space, val=1)
mask.val[l:h,l:h] = 0
R = ift.DiagonalOperator(mask)*HT
n.val[l:h, l:h] = 0
curv = Curvature(R=R, N=N, Sh=Sh)
D = curv.inverse
d = R(sh) + n
j = R.adjoint_times(N.inverse_times(d))
# Run Wiener filter
m = D(j)
# Uncertainty
sc = ift.probing.utils.StatCalculator()
IC = ift.GradientNormController(iteration_limit=50000,
tol_abs_gradnorm=0.1)
inverter = ift.ConjugateGradient(controller=IC)
curv = ift.library.wiener_filter_curvature.WienerFilterCurvature(R,N,Sh,inverter)
for i in range(20):
print i
sc.add(HT(curv.generate_posterior_sample()))
m_var = sc.var
m_mean, m_var = ift.probe_with_posterior_samples(curv, m, HT, 20)
# Get data
s_power = ift.power_analyze(sh)
m_power = ift.power_analyze(m)
s_power_data = s_power.val.real
m_power_data = m_power.val.real
s_data = HT(sh).val.real
m_data = HT(m).val.real
m_var_data = m_var.val.real
d_data = d.val.real
uncertainty = np.sqrt(np.abs(m_var_data))
```
%% Cell type:code id: tags:
``` python
cm = ['magma', 'inferno', 'plasma', 'viridis'][1]
mi = np.min(s_data)
ma = np.max(s_data)
fig, axes = plt.subplots(1, 2, figsize=(15, 7))
data = [s_data, d_data]
caption = ["Signal", "Data"]
for ax in axes.flat:
im = ax.imshow(data.pop(0), interpolation='nearest', cmap=cm, vmin=mi,
vmax=ma)
ax.set_title(caption.pop(0))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([0.85, 0.15, 0.05, 0.7])
fig.colorbar(im, cax=cbar_ax)
```
%% Cell type:code id: tags:
``` python
mi = np.min(s_data)
ma = np.max(s_data)
fig, axes = plt.subplots(2, 2, figsize=(15, 15))
data = [s_data, m_data, s_data - m_data, uncertainty]
caption = ["Signal", "Reconstruction", "Residuals", "Uncertainty Map"]
for ax in axes.flat:
im = ax.imshow(data.pop(0), interpolation='nearest', cmap=cm, vmin=mi, vmax=ma)
ax.set_title(caption.pop(0))
fig.subplots_adjust(right=0.8)
cbar_ax = fig.add_axes([.85, 0.15, 0.05, 0.7])
fig.colorbar(im, cax=cbar_ax)
```
%% Cell type:markdown id: tags:
### Is the uncertainty map reliable?
%% Cell type:code id: tags:
``` python
precise = (np.abs(s_data-m_data) < uncertainty )
print("Error within uncertainty map bounds: " + str(np.sum(precise) * 100 / N_pixels**2) + "%")
plt.figure(figsize=(15,10))
plt.imshow(precise.astype(float), cmap="brg")
plt.colorbar()
```
%% Cell type:markdown id: tags:
# Start Coding
## NIFTy Repository + Installation guide
https://gitlab.mpcdf.mpg.de/ift/NIFTy
NIFTy v4 **more or less stable!**
......
......@@ -108,5 +108,5 @@ if __name__ == "__main__":
ift.plot(ift.Field(plot_space,val=data.val), name='data.png', **plotdict)
ift.plot(ift.Field(plot_space,val=m.val), name='map.png', **plotdict)
# sampling the uncertainty map
mean, variance = ift.probe_with_posterior_samples(wiener_curvature, ht, 10)
mean, variance = ift.probe_with_posterior_samples(wiener_curvature, m_k, ht, 10)
ift.plot(ift.Field(plot_space, val=ift.sqrt(variance).val), name="uncertainty.png", **plotdict)
......@@ -72,7 +72,7 @@ if __name__ == "__main__":
sample_mean = ift.Field.zeros(signal_space)
n_samples = 10
for i in range(n_samples):
sample = ht(wiener_curvature.generate_posterior_sample()) + m
sample = ht(wiener_curvature.draw_sample()) + m
sample_variance += sample**2
sample_mean += sample
variance = sample_variance/n_samples - (sample_mean/n_samples)**2
......
......@@ -62,9 +62,12 @@ if __name__ == "__main__":
D = ift.InversionEnabler(D, inverter)
m = D(j)
# Plotting
d_field = ift.Field(s_space, val=d.val)
zmax = max(ht(sh).max(), d_field.max(), ht(m).max())
zmin = min(ht(sh).min(), d_field.min(), ht(m).min())
plotdict = {"xlabel": "Pixel index", "ylabel": "Pixel index",
"colormap": "Planck-like"}
"colormap": "Planck-like", "zmax": zmax, "zmin": zmin}
ift.plot(ht(sh), name="mock_signal.png", **plotdict)
ift.plot(ift.Field(s_space, val=d.val),
name="data.png", **plotdict)
ift.plot(ht(m), name="map.png", **plotdict)
ift.plot(d_field, name="data.png", **plotdict)
ift.plot(ht(m), name="reconstruction.png", **plotdict)
......@@ -65,7 +65,6 @@ if __name__ == "__main__":
domain=data_domain, random_type='normal',
std=noise_amplitude, mean=0)
data = noiseless_data + noise
# Wiener filter
j = R.adjoint_times(N.inverse_times(data))
ctrl = ift.GradientNormController(
......@@ -79,10 +78,10 @@ if __name__ == "__main__":
sspace2 = ift.RGSpace(shape, distances=L/N_pixels/nu.m)
ift.plot(ift.Field(sspace2, ht(mock_signal).val)/nu.K, name="mock_signal.png")
#data = ift.dobj.to_global_data(data.val).reshape(sspace2.shape)
#data = ift.Field(sspace2, val=ift.dobj.from_global_data(data))
zmax = max(m_s.max(), ht(mock_signal).max())
zmin = min(m_s.min(), ht(mock_signal).min())
plotdict = {"zmax": zmax/nu.K, "zmin": zmin/nu.K}
ift.plot(ift.Field(sspace2, ht(mock_signal).val)/nu.K, name="mock_signal.png", **plotdict)
ift.plot(ift.Field(sspace2, val=data.val), name="data.png")
print "msig",np.min(ht(mock_signal).val)/nu.K, np.max(ht(mock_signal).val)/nu.K
print "map",np.min(m_s.val)/nu.K, np.max(m_s.val)/nu.K
ift.plot(ift.Field(sspace2, m_s.val)/nu.K, name="map.png")
ift.plot(ift.Field(sspace2, m_s.val)/nu.K, name="reconstruction.png", **plotdict)
......@@ -6,33 +6,38 @@ np.random.seed(42)
if __name__ == "__main__":
# Set up position space
s_space = ift.RGSpace([128, 128])
# s_space = ift.HPSpace(32)
# s_space = ift.RGSpace([128, 128])
s_space = ift.HPSpace(32)
# Define associated harmonic space and harmonic transformation
h_space = s_space.get_default_codomain()
ht = ift.HarmonicTransformOperator(h_space, s_space)
# Setting up power space
# Set up power space
p_space = ift.PowerSpace(h_space)
# Choosing the prior correlation structure and defining
# correlation operator
# Choose prior correlation structure and define correlation operator
p_spec = (lambda k: (42/(k+1)**3))
S = ift.create_power_operator(h_space, power_spectrum=p_spec)
# Drawing a sample sh from the prior distribution in harmonic space
# Draw sample sh from the prior distribution in harmonic space
sp = ift.PS_field(p_space, p_spec)
sh = ift.power_synthesize(sp, real_signal=True)
# Choosing the measurement instrument
# Choose measurement instrument
diag = np.ones(s_space.shape)
diag[20:80, 20:80] = 0
if len(s_space.shape) == 1:
diag[3000:7000] = 0
elif len(s_space.shape) == 2:
diag[20:80, 20:80] = 0
else:
raise NotImplementedError
diag = ift.Field(s_space, ift.dobj.from_global_data(diag))
Instrument = ift.DiagonalOperator(diag)
# Adding a harmonic transformation to the instrument
# Add harmonic transformation to the instrument
R = Instrument*ht
noiseless_data = R(sh)
signal_to_noise = 1.
......@@ -43,31 +48,36 @@ if __name__ == "__main__":
std=noise_amplitude,
mean=0)
# Creating the mock data
# Create mock data