ift issueshttps://gitlab.mpcdf.mpg.de/groups/ift/-/issues2017-07-06T05:50:22Zhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/15d2o: Contraction functions rely on non-degeneracy of distribution strategy2017-07-06T05:50:22ZTheo Steiningerd2o: Contraction functions rely on non-degeneracy of distribution strategySeveral methods of the distributed_data_object rely on the fact, that the distribution strategy behaves as if the local data was non-degenerate. Currently the non-distributor fixes this by returning trivial (local) results in the _allgat...Several methods of the distributed_data_object rely on the fact, that the distribution strategy behaves as if the local data was non-degenerate. Currently the non-distributor fixes this by returning trivial (local) results in the _allgather and the _Allreduce_sum method.
Affected d2o methods are at least: _contraction_helper, mean
Fix: Move the functionality for sum, prod, etc... into the distributor.Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/16Add out-array parameter to numerical d2o operations.2017-07-06T05:50:22ZTheo SteiningerAdd out-array parameter to numerical d2o operations.Numpy supports to specify an out array in order to avoid memory reallocation.
a = np.array([1,2,3,4])
b = np.array([5,6,7,8])
# slow:
a = a + b
# fast:
np.add(a,b,out=a)
Numpy supports to specify an out array in order to avoid memory reallocation.
a = np.array([1,2,3,4])
b = np.array([5,6,7,8])
# slow:
a = a + b
# fast:
np.add(a,b,out=a)
Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/17Add axis keyword to obj.vdot2017-07-06T05:50:22ZTheo SteiningerAdd axis keyword to obj.vdotTheo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/18Improve obj.searchsorted such that it supports scalar, numpy arrays and distr...2017-07-06T05:50:22ZTheo SteiningerImprove obj.searchsorted such that it supports scalar, numpy arrays and distributed_data_objects as input......and return a distributed_data_object if the input was distributed.
...and return a distributed_data_object if the input was distributed.
Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/19reshaping -> enfold/defold2017-07-06T05:50:22ZTheo Steiningerreshaping -> enfold/defoldenfold in the slicing distributor relies on the fact, that the global axes correspond to the local array axes, as it operates on local data from `get_local_data()`.
In general this is not true for generic distribution strategies. enfold in the slicing distributor relies on the fact, that the global axes correspond to the local array axes, as it operates on local data from `get_local_data()`.
In general this is not true for generic distribution strategies. Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/20Initialization: prefer global_shape/local_shape over shape of data2017-07-06T05:50:22ZTheo SteiningerInitialization: prefer global_shape/local_shape over shape of dataRight now, durring initialization if some init-data is provided, the shape of this data is prefered over an explicitly given shape.
-> Change the behavior in Distributor-Factory.
-> Make a disperse_data instead of a distribute_data a...Right now, durring initialization if some init-data is provided, the shape of this data is prefered over an explicitly given shape.
-> Change the behavior in Distributor-Factory.
-> Make a disperse_data instead of a distribute_data at the end of init.
Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/21Create dedicated object for 'distribution_strategy'2017-07-06T05:50:22ZTheo SteiningerCreate dedicated object for 'distribution_strategy'Only global-type distribution strategies are comparable by their name directly. In order to compare local-type distribution strategies as well -> implement an object which represents the distribution strategy. For 'freeform' this include...Only global-type distribution strategies are comparable by their name directly. In order to compare local-type distribution strategies as well -> implement an object which represents the distribution strategy. For 'freeform' this includes the individual slices lengths. Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/22`imag` and `real` break memory view2017-07-06T05:50:22ZTheo Steininger`imag` and `real` break memory viewfrom d2o import *
a = np.array([1,2,3,4], dtype=np.complex)
obj = distributed_data_object(a)
obj.imag[0] = 1234
objfrom d2o import *
a = np.array([1,2,3,4], dtype=np.complex)
obj = distributed_data_object(a)
obj.imag[0] = 1234
objTheo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/108Docstring: FFTOperator2017-07-01T11:06:18ZTheo SteiningerDocstring: FFTOperatorMartin ReineckeMartin Reineckehttps://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/154RGSpace smoothing width2017-06-23T23:54:59ZTheo SteiningerRGSpace smoothing widthI've got the impression that the smoothing width for RGSpace is currently not correct in master.
Please try this code:
```
from nifty import *
x = RGSpace((256,), distances=1.)
f = Field(x, val=0)
f.val[100] = 1
sm = SmoothingOperator(...I've got the impression that the smoothing width for RGSpace is currently not correct in master.
Please try this code:
```
from nifty import *
x = RGSpace((256,), distances=1.)
f = Field(x, val=0)
f.val[100] = 1
sm = SmoothingOperator(x, sigma=50.)
plo = plotting.RG1DPlotter(title='test.html')
plo(sm(f))
g = sm(f)
g.sum()
```
The Gaussian bell is only half as broad as it should be. I fix is given in dc16a255d1b6d63db837278dc35d1cc8409b70b6
@dpumpe Could you please check, if this makes sense?
@mtr Could you please check, if the formula for smoothing on the sphere is correct? https://gitlab.mpcdf.mpg.de/ift/NIFTy/blob/master/nifty/spaces/lm_space/lm_space.py#L169Pumpe, Daniel (dpumpe)Pumpe, Daniel (dpumpe)https://gitlab.mpcdf.mpg.de/ift/D2O/-/issues/23"float128"-error on Windows 10 64bit2017-06-22T13:16:50ZChristoph Lienhard"float128"-error on Windows 10 64bitI get an error when using D20 related functions in NIFTy on my Windows machine:
site-packages\d2o-1.1.0-py2.7.egg\d2o\dtype_converter.py", line 54, in __init__
[np.dtype('float128'), MPI.LONG_DOUBLE],
TypeError: data type "float...I get an error when using D20 related functions in NIFTy on my Windows machine:
site-packages\d2o-1.1.0-py2.7.egg\d2o\dtype_converter.py", line 54, in __init__
[np.dtype('float128'), MPI.LONG_DOUBLE],
TypeError: data type "float128" not understood
As far as I understand numpy.float128 does not exist on every system (for some reason).
Edit:
same Problem with "complex256":
\site-packages\d2o-1.1.0-py2.7.egg\d2o\distributed_data_object.py", line 1898, in _to_hdf5
if self.dtype is np.dtype(np.complex256):
AttributeError: 'module' object has no attribute 'complex256'https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/132DiagonalOperator: trace_log() and log_determinant()2017-06-21T08:02:59ZMartin ReineckeDiagonalOperator: trace_log() and log_determinant()trace_log() and log_determinant() should throw exceptions if trace/determinant is <=0.trace_log() and log_determinant() should throw exceptions if trace/determinant is <=0.https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/133SmoothingOperator causes log(0.) for logarithmic distances2017-06-21T07:58:05ZTheo SteiningerSmoothingOperator causes log(0.) for logarithmic distanceshttps://gitlab.mpcdf.mpg.de/ift/NIFTy/blob/master/nifty/operators/smoothing_operator/fft_smoothing_operator.py#L27
https://gitlab.mpcdf.mpg.de/ift/NIFTy/blob/master/nifty/operators/smoothing_operator/fft_smoothing_operator.py#L27
Smooth...https://gitlab.mpcdf.mpg.de/ift/NIFTy/blob/master/nifty/operators/smoothing_operator/fft_smoothing_operator.py#L27
https://gitlab.mpcdf.mpg.de/ift/NIFTy/blob/master/nifty/operators/smoothing_operator/fft_smoothing_operator.py#L27
Smoothing should be translationally invariant. Couldn't we just add 1.0 to the distance array?Martin ReineckeMartin Reineckehttps://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/142ProductSpaces drawing random fields and checking their spectra2017-06-21T07:57:40ZPumpe, Daniel (dpumpe)ProductSpaces drawing random fields and checking their spectraHi, currently I am testing ProductSpace reconstructions (PLN over RGSpace (x) RGSpace). I am facing a problem concerning the spectra. For a simple test scenario I would like to check the power of field drawn from two spectra, as the foll...Hi, currently I am testing ProductSpace reconstructions (PLN over RGSpace (x) RGSpace). I am facing a problem concerning the spectra. For a simple test scenario I would like to check the power of field drawn from two spectra, as the following
```
from nifty import *
np.random.seed(42)
#########################################################################
# setting space one
#########################################################################
npix_0 = np.array([200]) # number of pixels
total_volume_0 = 1. # total length
# setting signal parameters
lambda_s_0 = .5 # signal correlation length
sigma_s_0 = 1. # signal variance
#setting length smoothing kernel
kernel_0 = 0.
# calculating parameters
k0_0 = 4./(2*np.pi*lambda_s_0)
a_s_0 = sigma_s_0**2.*lambda_s_0*total_volume_0
# creation of spaces
x_0 = RGSpace(npix_0, distances=total_volume_0/npix_0,
zerocenter=False)
k_0 = RGRGTransformation.get_codomain(x_0)
p_0 = PowerSpace(harmonic_partner=k_0, logarithmic=False)
# creating Power Operator with given spectrum
spec_0 = (lambda k: a_s_0/(1+(k/k0_0)**2)**2)
p_field_0 = Field(p_0, val=spec_0, dtype=np.float64)
# creating FFT-Operator and Response-Operator with Gaussian convolution
FFTOp_0 = FFTOperator(domain=x_0, target=k_0)
#########################################################################
# setting space two
#########################################################################
npix_1 = np.array([100]) # number of pixels
total_volume_1 = 1. # total length
# setting signal parameters
lambda_s_1 = .05 # signal correlation length
sigma_s_1 = 2. # signal variance
#setting length smoothing kernel
kernel_1 = 0.
# calculating parameters
k0_1 = 4./(2*np.pi*lambda_s_1)
a_s_1 = sigma_s_1**2.*lambda_s_1*total_volume_1
# creation of spaces
x_1 = RGSpace(npix_1, distances=total_volume_1/npix_1,
zerocenter=False)
k_1 = RGRGTransformation.get_codomain(x_1)
p_1 = PowerSpace(harmonic_partner=k_1, logarithmic=False)
# creating Power Operator with given spectrum
spec_1 = (lambda k: a_s_0/(1+(k/k0_1)**2)**2)
p_field_1 = Field(p_1, val=spec_1, dtype=np.float64)
# creating FFT-Operator and Response-Operator with Gaussian convolution
FFTOp_1 = FFTOperator(domain=x_1, target=k_1)
#########################################################################
# drawing a random field with above defined correlation structure
#########################################################################
fp_0 = Field(p_0, val=(lambda k: spec_0(k)**1/2.))
fp_1 = Field(p_1, val=(lambda k: spec_1(k)**1/2.))
outer = np.outer(fp_0.val.data, fp_1.val.data)
fp = Field((p_0, p_1), val=outer)
sk = fp.power_synthesize(spaces=(0,1), real_signal=True)
```
The question is, how do I recover fp_0 and fp_1 from sk (i.e. simply the power of the drawn field).
In my mind this should be:
```
def test_product_space_power(x):
samples = 100
ps_0 = Field(p_0, val=0.)
ps_1 = Field(p_1, val=1.)
for ii in xrange(samples):
sk = x.power_synthesize(spaces=(0,1), real_signal=True)
p0, p1 = analyze_sk(sk)
ps_1 += p1
ps_0 += p0
p_0_test_result = ps_0/samples
p_1_test_result = ps_1/samples
return p_0_test_result, p_1_test_result
def analyze_sk(x):
sp = x.power_analyze(spaces=0, decompose_power=False)\
.power_analyze(spaces=1, decompose_power=False)
p0= sp.sum(spaces=1)/fp_1.sum()
p1= sp.sum(spaces=0)/fp_0.sum()
return p0, p1
def analyze_fp(x):
p0= x.sum(spaces=1)/fp_1.sum()
p1= x.sum(spaces=0)/fp_0.sum()
return p0, p1
```
However it seems to be not correct. Analysing the power of sk with ```test_product_space_power(fp) ``` in direction of space 0 seems to have a constant offset, while the power in direction of space 1 seems to have the wrong shape. ```analyze_fp(fp) ``` only serves as a double check to see if the covariance from which sk is actually drawn is the appropriate one for fp_0 and fp_1.
Am I using ```.power_analyze``` wrong or are we facing an issue when we draw the random fields.
Thanks a lot for your help!https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/12Remove np support in point_space2017-06-21T04:52:39ZTheo SteiningerRemove np support in point_spacehttps://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/31Spaces & Operators: Protect attributes that should not be altered after init2017-06-21T04:52:39ZTheo SteiningerSpaces & Operators: Protect attributes that should not be altered after initThe user is able to change attributes that produce inconsistent results:
from nifty import *
x = rg_space((100, 100))
x.dtype = np.complex
This conflicts with
x.complexity
Solution: Protect all critical att...The user is able to change attributes that produce inconsistent results:
from nifty import *
x = rg_space((100, 100))
x.dtype = np.complex
This conflicts with
x.complexity
Solution: Protect all critical attributes with a `property` decorator. https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/37Define NIFTy-wide comm in nifty_configuration.2017-06-13T07:44:57ZTheo SteiningerDefine NIFTy-wide comm in nifty_configuration.Use this communicator globally for all NIFTy objects. Inherit to all D2O arrays. Pass forward to pyfftw.Use this communicator globally for all NIFTy objects. Inherit to all D2O arrays. Pass forward to pyfftw.Theo SteiningerTheo Steiningerhttps://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/73creation of distributed_data_object without dtype2017-06-13T07:44:11ZMartin Reineckecreation of distributed_data_object without dtypeIn rg_space.py and power_space.py there are calls to the constructor of "distributed_data_object", which do not specify a "dtype" argument.
This is accepted by D2O, but an annoying message is printed to the console.
I would suggest to pr...In rg_space.py and power_space.py there are calls to the constructor of "distributed_data_object", which do not specify a "dtype" argument.
This is accepted by D2O, but an annoying message is printed to the console.
I would suggest to produce a hard error whenever the user tries to create a D2O object without specifying a type. However, for the time being: is it correct to specify "dtype=np.float64" in the two cases I mentioned?
A related question: inside the distance_array-related functions of RGSpace and LMSpace, the data type np.float128 is used. Is that deliberate or an oversight?https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/150The dot product of a field with itself2017-06-12T23:31:34ZSebastian HutschenreuterThe dot product of a field with itselfThe dot product only considers the volume factors of the first field in case bare=False. This leads to the (in my mind) odd result noted below.
In [2]: np.random.seed(123)
In [3]: rgf = Field(domain=RGSpace(4),val=np.random.randn(4))
...The dot product only considers the volume factors of the first field in case bare=False. This leads to the (in my mind) odd result noted below.
In [2]: np.random.seed(123)
In [3]: rgf = Field(domain=RGSpace(4),val=np.random.randn(4))
In [4]: rgf.dot(rgf)
Out[4]: 1.1305730855452751
This is equal to
In [5]: (rgf.weight(power=1) * rgf).sum()
Out[5]: 1.1305730855452751
Wouldn't the two following cases be more appropriate, depending on the context?
In [6]: (rgf.weight(power=1) * rgf.weight(power=1)).sum()
Out[6]: 0.28264327138631878
In [7]: (rgf*rgf).sum()
Out[7]: 4.5222923421811005https://gitlab.mpcdf.mpg.de/ift/nifty/-/issues/149Behaviour of the Field class on empty domains2017-06-12T23:24:23ZSebastian HutschenreuterBehaviour of the Field class on empty domainsField seems to do odd things when the domain is set empty under variation of the dtype keyword.
I know that he example is rather artificial, non the less I thought I should mention it.
In [2]: ff = Field(domain=(),dtype=np.float)
In [...Field seems to do odd things when the domain is set empty under variation of the dtype keyword.
I know that he example is rather artificial, non the less I thought I should mention it.
In [2]: ff = Field(domain=(),dtype=np.float)
In [3]: ff.val.get_full_data()
[MainThread][WARNING ] _distributor_factory Distribution strategy was set to 'not' because of global_shape == ()
Out[3]: array(0.0)
In [4]: ff = Field(domain=(),dtype=np.complex)
In [5]: ff.val.get_full_data()
[MainThread][WARNING ] _distributor_factory Distribution strategy was set to 'not' because of global_shape == ()
Out[5]: array((8.772981689857213+0j))