Commit bf947d1b authored by Martin Reinecke's avatar Martin Reinecke
Browse files

Merge branch 'cf_wo_asperity' into 'NIFTy_7'

Allow for opting out of asperity and flexibility

See merge request !556
parents 2e872c5d 9dc11964
Pipeline #77882 passed with stages
in 26 minutes and 6 seconds
Changes since NIFTy 6 Changes since NIFTy 6
===================== =====================
CorrelatedFieldMaker interface change
-------------------------------------
The interface of `ift.CorrelatedFieldMaker.make` and
`ift.CorrelatedFieldMaker.add_fluctuations` changed; it now expects the mean
and the standard deviation of their various parameters not as separate
arguments but as a tuple.
Furthermore, it is now possible to disable the asperity and the flexibility
together with the asperity in the correlated field model. Note that disabling
only the flexibility is not possible.
SimpleCorrelatedField
---------------------
A simplified version of the correlated field model was introduced which does not
allow for multiple power spectra, the presence of a degree of freedom parameter
`dofdex`, or `total_N` larger than zero. Except for the above mentioned
limitations, it is equivalent to `ift.CorrelatedFieldMaker`. Hence, if one
wants to understand the implementation idea behind the model, it is easier to
grasp from reading `ift.SimpleCorrelatedField` than from going through
`ift.CorrelatedFieldMaker`.
Change in external dependencies Change in external dependencies
------------------------------- -------------------------------
...@@ -20,7 +43,7 @@ The implementation tests for nonlinear operators are now available in ...@@ -20,7 +43,7 @@ The implementation tests for nonlinear operators are now available in
MetricGaussianKL interface MetricGaussianKL interface
-------------------------- --------------------------
Users do not instanciate `MetricGaussianKL` by its constructor anymore. Rather Users do not instantiate `MetricGaussianKL` by its constructor anymore. Rather
`MetricGaussianKL.make()` shall be used. `MetricGaussianKL.make()` shall be used.
...@@ -91,10 +114,10 @@ New approach for sampling complex numbers ...@@ -91,10 +114,10 @@ New approach for sampling complex numbers
When calling draw_sample_with_dtype with a complex dtype, When calling draw_sample_with_dtype with a complex dtype,
the variance is now used for the imaginary part and real part separately. the variance is now used for the imaginary part and real part separately.
This is done in order to be consistent with the Hamiltonian. This is done in order to be consistent with the Hamiltonian.
Note that by this, Note that by this,
``` ```
np.std(ift.from_random(domain, 'normal', dtype=np.complex128).val) np.std(ift.from_random(domain, 'normal', dtype=np.complex128).val)
```` ````
does not give 1, but sqrt(2) as a result. does not give 1, but sqrt(2) as a result.
......
...@@ -135,7 +135,7 @@ ...@@ -135,7 +135,7 @@
"## Before the Action: The Moment-Matched Log-Normal Distribution\n", "## Before the Action: The Moment-Matched Log-Normal Distribution\n",
"Many properties of the correlated field are modelled as being lognormally distributed.\n", "Many properties of the correlated field are modelled as being lognormally distributed.\n",
"\n", "\n",
"The distribution models are parametrized via their means `_mean` and standard-deviations `_stddev`.\n", "The distribution models are parametrized via their means and standard-deviations (first and second position in tuple).\n",
"\n", "\n",
"To get a feeling of how the ratio of the `mean` and `stddev` parameters influences the distribution shape,\n", "To get a feeling of how the ratio of the `mean` and `stddev` parameters influences the distribution shape,\n",
"here are a few example histograms: (observe the x-axis!)" "here are a few example histograms: (observe the x-axis!)"
...@@ -189,21 +189,16 @@ ...@@ -189,21 +189,16 @@
"# Neutral model parameters yielding a quasi-constant field\n", "# Neutral model parameters yielding a quasi-constant field\n",
"cf_make_pars = {\n", "cf_make_pars = {\n",
" 'offset_mean': 0.,\n", " 'offset_mean': 0.,\n",
" 'offset_std_mean': 1e-3,\n", " 'offset_std': (1e-3, 1e-16),\n",
" 'offset_std_std': 1e-16,\n",
" 'prefix': ''\n", " 'prefix': ''\n",
"}\n", "}\n",
"\n", "\n",
"cf_x_fluct_pars = {\n", "cf_x_fluct_pars = {\n",
" 'target_subdomain': x_space,\n", " 'target_subdomain': x_space,\n",
" 'fluctuations_mean': 1e-3,\n", " 'fluctuations': (1e-3, 1e-16),\n",
" 'fluctuations_stddev': 1e-16,\n", " 'flexibility': (1e-3, 1e-16),\n",
" 'flexibility_mean': 1e-3,\n", " 'asperity': (1e-3, 1e-16),\n",
" 'flexibility_stddev': 1e-16,\n", " 'loglogavgslope': (0., 1e-16)\n",
" 'asperity_mean': 1e-3,\n",
" 'asperity_stddev': 1e-16,\n",
" 'loglogavgslope_mean': 0.,\n",
" 'loglogavgslope_stddev': 1e-16\n",
"}\n", "}\n",
"\n", "\n",
"init_model(cf_make_pars, cf_x_fluct_pars)" "init_model(cf_make_pars, cf_x_fluct_pars)"
...@@ -225,18 +220,18 @@ ...@@ -225,18 +220,18 @@
"source": [ "source": [
"# Parameter Showcases\n", "# Parameter Showcases\n",
"\n", "\n",
"## The `fluctuations_` parameters of `add_fluctuations()`\n", "## The `fluctuations` parameters of `add_fluctuations()`\n",
"\n", "\n",
"determine the **amplitude of variations along the field dimension**\n", "determine the **amplitude of variations along the field dimension**\n",
"for which `add_fluctuations` is called.\n", "for which `add_fluctuations` is called.\n",
"\n", "\n",
"`fluctuations_mean` set the _average_ amplitude of the fields fluctuations along the given dimension,\\\n", "`fluctuations[0]` set the _average_ amplitude of the fields fluctuations along the given dimension,\\\n",
"`fluctuations_stddev` sets the width and shape of the amplitude distribution.\n", "`fluctuations[1]` sets the width and shape of the amplitude distribution.\n",
"\n", "\n",
"The amplitude is modelled as being log-normally distributed,\n", "The amplitude is modelled as being log-normally distributed,\n",
"see `The Moment-Matched Log-Normal Distribution` above for details.\n", "see `The Moment-Matched Log-Normal Distribution` above for details.\n",
"\n", "\n",
"#### `fluctuations_mean`:" "#### `fluctuations` mean:"
] ]
}, },
{ {
...@@ -245,14 +240,14 @@ ...@@ -245,14 +240,14 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"vary_parameter('fluctuations_mean', [0.05, 0.5, 2.], samples_vary_in='xi')" "vary_parameter('fluctuations', [(0.05, 1e-16), (0.5, 1e-16), (2., 1e-16)], samples_vary_in='xi')"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"#### `fluctuations_stddev`:" "#### `fluctuations` std:"
] ]
}, },
{ {
...@@ -261,21 +256,21 @@ ...@@ -261,21 +256,21 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"cf_x_fluct_pars['fluctuations_mean'] = 1.0\n", "vary_parameter('fluctuations', [(1., 0.01), (1., 0.1), (1., 1.)], samples_vary_in='fluctuations')\n",
"vary_parameter('fluctuations_stddev', [0.01, 0.1, 1.0], samples_vary_in='fluctuations')" "cf_x_fluct_pars['fluctuations'] = (1., 1e-16)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## The `loglogavgslope_` parameters of `add_fluctuations()`\n", "## The `loglogavgslope` parameters of `add_fluctuations()`\n",
"\n", "\n",
"determine **the slope of the loglog-linear (power law) component of the power spectrum**.\n", "determine **the slope of the loglog-linear (power law) component of the power spectrum**.\n",
"\n", "\n",
"The slope is modelled to be normally distributed.\n", "The slope is modelled to be normally distributed.\n",
"\n", "\n",
"#### `loglogavgslope_mean`:" "#### `loglogavgslope` mean:"
] ]
}, },
{ {
...@@ -284,14 +279,14 @@ ...@@ -284,14 +279,14 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"vary_parameter('loglogavgslope_mean', [-3., -1., 1.], samples_vary_in='xi')" "vary_parameter('loglogavgslope', [(-3., 1e-16), (-1., 1e-16), (1., 1e-16)], samples_vary_in='xi')"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"#### `loglogavgslope_stddev`:" "#### `loglogavgslope` std:"
] ]
}, },
{ {
...@@ -300,25 +295,25 @@ ...@@ -300,25 +295,25 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"cf_x_fluct_pars['loglogavgslope_mean'] = -1.0\n", "vary_parameter('loglogavgslope', [(-1., 0.01), (-1., 0.1), (-1., 1.0)], samples_vary_in='loglogavgslope')\n",
"vary_parameter('loglogavgslope_stddev', [0.01, 0.1, 1.0], samples_vary_in='loglogavgslope')" "cf_x_fluct_pars['loglogavgslope'] = (-1., 1e-16)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## The `flexibility_` parameters of `add_fluctuations()`\n", "## The `flexibility` parameters of `add_fluctuations()`\n",
"\n", "\n",
"determine **the amplitude of the integrated Wiener process component of the power spectrum**\n", "determine **the amplitude of the integrated Wiener process component of the power spectrum**\n",
"(how strong the power spectrum varies besides the power-law).\n", "(how strong the power spectrum varies besides the power-law).\n",
"\n", "\n",
"`flexibility_mean` sets the _average_ amplitude of the i.g.p. component,\n", "`flexibility[0]` sets the _average_ amplitude of the i.g.p. component,\\\n",
"`flexibility_stddev` sets how much the amplitude can vary.\\\n", "`flexibility[1]` sets how much the amplitude can vary.\\\n",
"These two parameters feed into a moment-matched log-normal distribution model,\n", "These two parameters feed into a moment-matched log-normal distribution model,\n",
"see above for a demo of its behavior.\n", "see above for a demo of its behavior.\n",
"\n", "\n",
"#### `flexibility_mean`:" "#### `flexibility` mean:"
] ]
}, },
{ {
...@@ -327,14 +322,14 @@ ...@@ -327,14 +322,14 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"vary_parameter('flexibility_mean', [0.1, 1.0, 3.0], samples_vary_in='spectrum')" "vary_parameter('flexibility', [(0.1, 1e-16), (1.0, 1e-16), (3.0, 1e-16)], samples_vary_in='spectrum')"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"#### `flexibility_stddev`:" "#### `flexibility` std:"
] ]
}, },
{ {
...@@ -343,23 +338,23 @@ ...@@ -343,23 +338,23 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"cf_x_fluct_pars['flexibility_mean'] = 2.0\n", "vary_parameter('flexibility', [(2., 0.01), (2., 0.1), (2., 1.)], samples_vary_in='flexibility')\n",
"vary_parameter('flexibility_stddev', [0.01, 0.1, 1.0], samples_vary_in='flexibility')" "cf_x_fluct_pars['flexibility'] = (2., 1e-16)"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## The `asperity_` parameters of `add_fluctuations()`\n", "## The `asperity` parameters of `add_fluctuations()`\n",
"\n", "\n",
"`asperity_` determines **how rough the integrated Wiener process component of the power spectrum is**.\n", "`asperity` determines **how rough the integrated Wiener process component of the power spectrum is**.\n",
"\n", "\n",
"`asperity_mean` sets the average roughness, `asperity_stddev` sets how much the roughness can vary.\\\n", "`asperity[0]` sets the average roughness, `asperity[1]` sets how much the roughness can vary.\\\n",
"These two parameters feed into a moment-matched log-normal distribution model,\n", "These two parameters feed into a moment-matched log-normal distribution model,\n",
"see above for a demo of its behavior.\n", "see above for a demo of its behavior.\n",
"\n", "\n",
"#### `asperity_mean`:" "#### `asperity` mean:"
] ]
}, },
{ {
...@@ -368,14 +363,14 @@ ...@@ -368,14 +363,14 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"vary_parameter('asperity_mean', [0.001, 1.0, 5.0], samples_vary_in='spectrum')" "vary_parameter('asperity', [(0.001, 1e-16), (1.0, 1e-16), (5., 1e-16)], samples_vary_in='spectrum')"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"#### `asperity_stddev`:" "#### `asperity` std:"
] ]
}, },
{ {
...@@ -384,8 +379,8 @@ ...@@ -384,8 +379,8 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"cf_x_fluct_pars['asperity_mean'] = 1.0\n", "vary_parameter('asperity', [(1., 0.01), (1., 0.1), (1., 1.)], samples_vary_in='asperity')\n",
"vary_parameter('asperity_stddev', [0.01, 0.1, 1.0], samples_vary_in='asperity')" "cf_x_fluct_pars['asperity'] = (1., 1e-16)"
] ]
}, },
{ {
...@@ -406,10 +401,10 @@ ...@@ -406,10 +401,10 @@
"outputs": [], "outputs": [],
"source": [ "source": [
"# Reset model to neutral\n", "# Reset model to neutral\n",
"cf_x_fluct_pars['fluctuations_mean'] = 1e-3\n", "cf_x_fluct_pars['fluctuations'] = (1e-3, 1e-16)\n",
"cf_x_fluct_pars['flexibility_mean'] = 1e-3\n", "cf_x_fluct_pars['flexibility'] = (1e-3, 1e-16)\n",
"cf_x_fluct_pars['asperity_mean'] = 1e-3\n", "cf_x_fluct_pars['asperity'] = (1e-3, 1e-16)\n",
"cf_x_fluct_pars['loglogavgslope_mean'] = 1e-3" "cf_x_fluct_pars['loglogavgslope'] = (1e-3, 1e-16)"
] ]
}, },
{ {
...@@ -425,15 +420,15 @@ ...@@ -425,15 +420,15 @@
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"## The `offset_std_` parameters of `CorrelatedFieldMaker.make()`\n", "## The `offset_std` parameters of `CorrelatedFieldMaker.make()`\n",
"\n", "\n",
"Variation of the global offset of the field are modelled as being log-normally distributed.\n", "Variation of the global offset of the field are modelled as being log-normally distributed.\n",
"See `The Moment-Matched Log-Normal Distribution` above for details.\n", "See `The Moment-Matched Log-Normal Distribution` above for details.\n",
"\n", "\n",
"The `offset_std_mean` parameter sets how much NIFTy will vary the offset *on average*.\\\n", "The `offset_std[0]` parameter sets how much NIFTy will vary the offset *on average*.\\\n",
"The `offset_std_std` parameters defines the with and shape of the offset variation distribution.\n", "The `offset_std[1]` parameters defines the with and shape of the offset variation distribution.\n",
"\n", "\n",
"#### `offset_std_mean`:" "#### `offset_std` mean:"
] ]
}, },
{ {
...@@ -442,14 +437,14 @@ ...@@ -442,14 +437,14 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"vary_parameter('offset_std_mean', [1e-16, 0.5, 2.], samples_vary_in='xi')" "vary_parameter('offset_std', [(1e-16, 1e-16), (0.5, 1e-16), (2., 1e-16)], samples_vary_in='xi')"
] ]
}, },
{ {
"cell_type": "markdown", "cell_type": "markdown",
"metadata": {}, "metadata": {},
"source": [ "source": [
"#### `offset_std_std`:" "#### `offset_std` std:"
] ]
}, },
{ {
...@@ -458,8 +453,7 @@ ...@@ -458,8 +453,7 @@
"metadata": {}, "metadata": {},
"outputs": [], "outputs": [],
"source": [ "source": [
"cf_make_pars['offset_std_mean'] = 1.0\n", "vary_parameter('offset_std', [(1., 0.01), (1., 0.1), (1., 1.)], samples_vary_in='zeromode')"
"vary_parameter('offset_std_std', [0.01, 0.1, 1.0], samples_vary_in='zeromode')"
] ]
} }
], ],
...@@ -479,7 +473,7 @@ ...@@ -479,7 +473,7 @@
"name": "python", "name": "python",
"nbconvert_exporter": "python", "nbconvert_exporter": "python",
"pygments_lexer": "ipython3", "pygments_lexer": "ipython3",
"version": "3.8.2" "version": "3.8.3"
} }
}, },
"nbformat": 4, "nbformat": 4,
......
...@@ -73,10 +73,10 @@ def main(): ...@@ -73,10 +73,10 @@ def main():
sp2 = ift.RGSpace(npix2) sp2 = ift.RGSpace(npix2)
# Set up signal model # Set up signal model
cfmaker = ift.CorrelatedFieldMaker.make(0., 1e-2, 1e-6, '') cfmaker = ift.CorrelatedFieldMaker.make(0., (1e-2, 1e-6), '')
cfmaker.add_fluctuations(sp1, 0.1, 1e-2, 1, .1, .01, .5, -2, 1., 'amp1') cfmaker.add_fluctuations(sp1, (0.1, 1e-2), (1, .1), (.01, .5), (-2, 1.), 'amp1')
cfmaker.add_fluctuations(sp2, 0.1, 1e-2, 1, .1, .01, .5, cfmaker.add_fluctuations(sp2, (0.1, 1e-2), (1, .1), (.01, .5),
-1.5, .5, 'amp2') (-1.5, .5), 'amp2')
correlated_field = cfmaker.finalize() correlated_field = cfmaker.finalize()
A1 = cfmaker.normalized_amplitudes[0] A1 = cfmaker.normalized_amplitudes[0]
......
...@@ -92,7 +92,7 @@ from .library.correlated_fields_simple import SimpleCorrelatedField ...@@ -92,7 +92,7 @@ from .library.correlated_fields_simple import SimpleCorrelatedField
from . import extra from . import extra
from .utilities import memo, frozendict from .utilities import memo, frozendict, myassert
from .logger import logger from .logger import logger
......
...@@ -18,7 +18,6 @@ ...@@ -18,7 +18,6 @@
from itertools import combinations from itertools import combinations
import numpy as np import numpy as np
from numpy.testing import assert_
from .domain_tuple import DomainTuple from .domain_tuple import DomainTuple
from .field import Field from .field import Field
...@@ -29,6 +28,7 @@ from .operators.energy_operators import EnergyOperator ...@@ -29,6 +28,7 @@ from .operators.energy_operators import EnergyOperator
from .operators.linear_operator import LinearOperator from .operators.linear_operator import LinearOperator
from .operators.operator import Operator from .operators.operator import Operator
from .sugar import from_random from .sugar import from_random
from .utilities import myassert
__all__ = ["check_linear_operator", "check_operator", __all__ = ["check_linear_operator", "check_operator",
"assert_allclose"] "assert_allclose"]
...@@ -137,20 +137,6 @@ def assert_equal(f1, f2): ...@@ -137,20 +137,6 @@ def assert_equal(f1, f2):
assert_equal(val, f2[key]) assert_equal(val, f2[key])
def _nozero(fld):
if isinstance(fld, Field):
return np.testing.assert_((fld != 0).s_all())
for val in fld.values():
_nozero(val)
def _allzero(fld):
if isinstance(fld, Field):
return np.testing.assert_((fld == 0.).s_all())
for val in fld.values():
_allzero(val)
def _adjoint_implementation(op, domain_dtype, target_dtype, atol, rtol, def _adjoint_implementation(op, domain_dtype, target_dtype, atol, rtol,
only_r_linear): only_r_linear):
needed_cap = op.TIMES | op.ADJOINT_TIMES needed_cap = op.TIMES | op.ADJOINT_TIMES
...@@ -206,32 +192,32 @@ def _domain_check_linear(op, domain_dtype=None, inp=None): ...@@ -206,32 +192,32 @@ def _domain_check_linear(op, domain_dtype=None, inp=None):
inp = from_random(op.domain, "normal", dtype=domain_dtype) inp = from_random(op.domain, "normal", dtype=domain_dtype)
elif inp is None: elif inp is None:
raise ValueError('Need to specify either dtype or inp') raise ValueError('Need to specify either dtype or inp')
assert_(inp.domain is op.domain) myassert(inp.domain is op.domain)
assert_(op(inp).domain is op.target) myassert(op(inp).domain is op.target)
def _domain_check_nonlinear(op, loc): def _domain_check_nonlinear(op, loc):
_domain_check(op) _domain_check(op)
assert_(isinstance(loc, (Field, MultiField))) myassert(isinstance(loc, (Field, MultiField)))
assert_(loc.domain is op.domain) myassert(loc.domain is op.domain)
for wm in [False, True]: for wm in [False, True]:
lin = Linearization.make_var(loc, wm) lin = Linearization.make_var(loc, wm)
reslin = op(lin) reslin = op(lin)
assert_(lin.domain is op.domain) myassert(lin.domain is op.domain)
assert_(lin.target is op.domain) myassert(lin.target is op.domain)
assert_(lin.val.domain is lin.domain) myassert(lin.val.domain is lin.domain)
assert_(reslin.domain is op.domain) myassert(reslin.domain is op.domain)
assert_(reslin.target is op.target) myassert(reslin.target is op.target)
assert_(reslin.val.domain is reslin.target) myassert(reslin.val.domain is reslin.target)
assert_(reslin.target is op.target) myassert(reslin.target is op.target)
assert_(reslin.jac.domain is reslin.domain) myassert(reslin.jac.domain is reslin.domain)
assert_(reslin.jac.target is reslin.target) myassert(reslin.jac.target is reslin.target)
assert_(lin.want_metric == reslin.want_metric) myassert(lin.want_metric == reslin.want_metric)
_domain_check_linear(reslin.jac, inp=loc) _domain_check_linear(reslin.jac, inp=loc)
_domain_check_linear(reslin.jac.adjoint, inp=reslin.jac(loc)) _domain_check_linear(reslin.jac.adjoint,