Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
N
NIFTy
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
13
Issues
13
List
Boards
Labels
Service Desk
Milestones
Merge Requests
10
Merge Requests
10
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Incidents
Environments
Packages & Registries
Packages & Registries
Container Registry
Analytics
Analytics
CI / CD
Repository
Value Stream
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ift
NIFTy
Commits
3c388314
Commit
3c388314
authored
May 23, 2017
by
Theo Steininger
Browse files
Options
Browse Files
Download
Plain Diff
Merge branch 'misc_tweaks' into 'master'
# Conflicts: # nifty/sugar.py
parents
aa21eae3
95248edc
Pipeline
#12814
failed with stages
in 1 minute and 27 seconds
Changes
22
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
Showing
22 changed files
with
38 additions
and
111 deletions
+38
-111
.gitlab-ci.yml
.gitlab-ci.yml
+5
-22
README.md
README.md
+6
-13
ci/install_basics.sh
ci/install_basics.sh
+1
-1
ci/install_h5py.sh
ci/install_h5py.sh
+1
-6
ci/install_mpi4py.sh
ci/install_mpi4py.sh
+1
-1
ci/install_pyHealpix.sh
ci/install_pyHealpix.sh
+1
-3
ci/install_pyfftw.sh
ci/install_pyfftw.sh
+2
-4
nifty/field_types/field_array.py
nifty/field_types/field_array.py
+3
-4
nifty/operators/fft_operator/transformations/gllmtransformation.py
...rators/fft_operator/transformations/gllmtransformation.py
+0
-5
nifty/operators/fft_operator/transformations/hplmtransformation.py
...rators/fft_operator/transformations/hplmtransformation.py
+0
-5
nifty/operators/fft_operator/transformations/lmgltransformation.py
...rators/fft_operator/transformations/lmgltransformation.py
+0
-5
nifty/operators/fft_operator/transformations/lmhptransformation.py
...rators/fft_operator/transformations/lmhptransformation.py
+0
-5
nifty/operators/fft_operator/transformations/rg_transforms.py
...y/operators/fft_operator/transformations/rg_transforms.py
+0
-10
nifty/operators/invertible_operator_mixin/invertible_operator_mixin.py
...rs/invertible_operator_mixin/invertible_operator_mixin.py
+1
-0
nifty/operators/propagator_operator/harmonic_propagator_operator.py
...ators/propagator_operator/harmonic_propagator_operator.py
+1
-2
nifty/operators/smoothing_operator/direct_smoothing_operator.py
...operators/smoothing_operator/direct_smoothing_operator.py
+1
-0
nifty/operators/smoothing_operator/fft_smoothing_operator.py
nifty/operators/smoothing_operator/fft_smoothing_operator.py
+2
-0
nifty/spaces/power_space/power_space.py
nifty/spaces/power_space/power_space.py
+1
-0
nifty/spaces/space/space.py
nifty/spaces/space/space.py
+1
-3
nifty/sugar.py
nifty/sugar.py
+8
-18
test/test_operators/test_diagonal_operator.py
test/test_operators/test_diagonal_operator.py
+3
-1
test/test_operators/test_smoothing_operator.py
test/test_operators/test_smoothing_operator.py
+0
-3
No files found.
.gitlab-ci.yml
View file @
3c388314
...
...
@@ -16,38 +16,21 @@ before_script:
-
pip install --upgrade -r ci/requirements.txt
test_
min
:
test_
all
:
stage
:
test
script
:
-
python setup.py build_ext --inplace
-
nosetests -vv
test_mpi
:
stage
:
test
script
:
-
nosetests
-
ci/install_pyHealpix.sh
-
ci/install_mpi4py.sh
-
python setup.py build_ext --inplace
-
nosetests -vv
test_mpi_fftw
:
stage
:
test
script
:
-
ci/install_pyHealpix.sh
-
ci/install_mpi4py.sh
-
nosetests
-
ci/install_pyfftw.sh
-
python setup.py build_ext --inplace
-
nosetests -vv
test_mpi_fftw_hdf5
:
stage
:
test
script
:
-
ci/install_pyHealpix.sh
-
ci/install_mpi4py.sh
-
ci/install_pyfftw.sh
-
nosetests
-
ci/install_h5py.sh
-
python setup.py build_ext --inplace
-
nosetests -
vv -
-with-coverage --cover-package=nifty --cover-branches
-
nosetests --with-coverage --cover-package=nifty --cover-branches
-
>
coverage report | grep TOTAL | awk '{ print "TOTAL: "$6; }'
...
...
README.md
View file @
3c388314
...
...
@@ -11,7 +11,7 @@ Summary
### Description
**NIFTY**
, "
**N**
umerical
**I**
nformation
**F**
ield
**T**
heor
**y**
", is
**NIFTY**
, "
**N**
umerical
**I**
nformation
**F**
ield
**T**
heor
<strong>
y
</strong>
", is
a versatile library designed to enable the development of signal
inference algorithms that operate regardless of the underlying spatial
grid and its resolution. Its object-oriented framework is written in
...
...
@@ -87,21 +87,17 @@ Starting with a fresh Ubuntu installation move to a folder like
-
Install basic packages like python, python-dev, gsl and others:
sudo apt-get install curl git autoconf python-dev python-pip python-numpy
sudo apt-get install curl git autoconf
libtool
python-dev python-pip python-numpy
-
Install pyHealpix:
git clone https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git
cd pyHealpix
autoreconf -i && ./configure --prefix=$HOME/.local --enable-openmp --enable-native-optimizations && make -j4 && make install
cd ..
(cd pyHealpix && autoreconf -i && ./configure --prefix=$HOME/.local --enable-openmp --enable-native-optimizations && make -j4 install)
-
Finally, NIFTy:
git clone https://gitlab.mpcdf.mpg.de/ift/NIFTy.git
cd NIFTy
python setup.py install --user
cd ..
(cd NIFTy && python setup.py install --user)
### Installation on Linux systems in general
...
...
@@ -139,16 +135,13 @@ may cause trouble.
-
Install NIFTy:
git clone https://gitlab.mpcdf.mpg.de/ift/NIFTy.git
cd NIFTy
python setup.py install --user
cd ..
(cd NIFTy && python setup.py install --user)
### Running the tests
In oder to run the tests one needs two additional packages:
pip install nose
pip install parameterized
pip install nose parameterized
Afterwards the tests (including a coverage report) are run using the following
command in the repository root:
...
...
ci/install_basics.sh
View file @
3c388314
#!/bin/bash
apt-get
install
-y
build-essential python python-pip python-dev git autoconf
gsl-bin libgsl-dev wget
apt-get
install
-y
build-essential python python-pip python-dev git autoconf
libtool gsl-bin libgsl-dev wget
ci/install_h5py.sh
View file @
3c388314
#!/bin/bash
apt-get
install
-y
libhdf5-10 libhdf5-dev libhdf5-openmpi-10 libhdf5-openmpi-dev hdf5-tools
export
CC
=
mpicc
export
HDF5_DIR
=
/usr/lib/x86_64-linux-gnu/hdf5/openmpi
export
HDF5_MPI
=
"ON"
pip
install
--no-binary
=
h5py h5py
CC
=
mpicc
HDF5_DIR
=
/usr/lib/x86_64-linux-gnu/hdf5/openmpi
HDF5_MPI
=
"ON"
pip
install
--no-binary
=
h5py h5py
ci/install_mpi4py.sh
View file @
3c388314
#!/bin/bash
apt-get
install
-y
openmpi-bin libopenmpi-dev
pip
install
mpi4py
\ No newline at end of file
pip
install
mpi4py
ci/install_pyHealpix.sh
View file @
3c388314
#!/bin/bash
git clone https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git
cd
pyHealpix
autoreconf
-i
&&
./configure
&&
make
-j4
install
cd
..
(
cd
pyHealpix
&&
autoreconf
-i
&&
./configure
--enable-openmp
--enable-native-optimizations
&&
make
-j4
install
)
rm
-rf
pyHealpix
ci/install_pyfftw.sh
View file @
3c388314
...
...
@@ -3,7 +3,5 @@
apt-get
install
-y
libatlas-base-dev libfftw3-bin libfftw3-dev libfftw3-double3 libfftw3-long3 libfftw3-mpi-dev libfftw3-mpi3 libfftw3-quad3 libfftw3-single3
git clone
-b
mpi https://github.com/fredros/pyFFTW.git
cd
pyFFTW/
CC
=
mpicc python setup.py build_ext
install
cd
..
rm
-r
pyFFTW
(
cd
pyFFTW
&&
CC
=
mpicc python setup.py build_ext
install
)
rm
-rf
pyFFTW
nifty/field_types/field_array.py
View file @
3c388314
...
...
@@ -22,12 +22,11 @@ from field_type import FieldType
class
FieldArray
(
FieldType
):
def
__init__
(
self
,
shape
):
super
(
FieldArray
,
self
).
__init__
()
try
:
new
_shape
=
tuple
([
int
(
i
)
for
i
in
shape
])
self
.
_shape
=
tuple
([
int
(
i
)
for
i
in
shape
])
except
TypeError
:
new_shape
=
(
int
(
shape
),
)
self
.
_shape
=
new_shape
super
(
FieldArray
,
self
).
__init__
()
self
.
_shape
=
(
int
(
shape
),
)
@
property
def
shape
(
self
):
...
...
nifty/operators/fft_operator/transformations/gllmtransformation.py
View file @
3c388314
...
...
@@ -100,11 +100,6 @@ class GLLMTransformation(SlicingTransformation):
super
(
GLLMTransformation
,
cls
).
check_codomain
(
domain
,
codomain
)
def
_transformation_of_slice
(
self
,
inp
,
**
kwargs
):
if
inp
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
inp
.
dtype
))
nlat
=
self
.
domain
.
nlat
nlon
=
self
.
domain
.
nlon
lmax
=
self
.
codomain
.
lmax
...
...
nifty/operators/fft_operator/transformations/hplmtransformation.py
View file @
3c388314
...
...
@@ -92,11 +92,6 @@ class HPLMTransformation(SlicingTransformation):
super
(
HPLMTransformation
,
cls
).
check_codomain
(
domain
,
codomain
)
def
_transformation_of_slice
(
self
,
inp
,
**
kwargs
):
if
inp
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
inp
.
dtype
))
lmax
=
self
.
codomain
.
lmax
mmax
=
lmax
...
...
nifty/operators/fft_operator/transformations/lmgltransformation.py
View file @
3c388314
...
...
@@ -106,11 +106,6 @@ class LMGLTransformation(SlicingTransformation):
super
(
LMGLTransformation
,
cls
).
check_codomain
(
domain
,
codomain
)
def
_transformation_of_slice
(
self
,
inp
,
**
kwargs
):
if
inp
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
inp
.
dtype
))
nlat
=
self
.
codomain
.
nlat
nlon
=
self
.
codomain
.
nlon
lmax
=
self
.
domain
.
lmax
...
...
nifty/operators/fft_operator/transformations/lmhptransformation.py
View file @
3c388314
...
...
@@ -94,11 +94,6 @@ class LMHPTransformation(SlicingTransformation):
super
(
LMHPTransformation
,
cls
).
check_codomain
(
domain
,
codomain
)
def
_transformation_of_slice
(
self
,
inp
,
**
kwargs
):
if
inp
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
inp
.
dtype
))
nside
=
self
.
codomain
.
nside
lmax
=
self
.
domain
.
lmax
mmax
=
lmax
...
...
nifty/operators/fft_operator/transformations/rg_transforms.py
View file @
3c388314
...
...
@@ -433,11 +433,6 @@ class FFTW(Transform):
not
all
(
axis
in
range
(
len
(
val
.
shape
))
for
axis
in
axes
):
raise
ValueError
(
"Provided axes does not match array shape"
)
if
val
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
val
.
dtype
))
# If the input is a numpy array we transform it locally
if
not
isinstance
(
val
,
distributed_data_object
):
# Cast to a np.ndarray
...
...
@@ -582,11 +577,6 @@ class NUMPYFFT(Transform):
not
all
(
axis
in
range
(
len
(
val
.
shape
))
for
axis
in
axes
):
raise
ValueError
(
"Provided axes does not match array shape"
)
if
val
.
dtype
not
in
(
np
.
float
,
np
.
complex
):
self
.
logger
.
warn
(
"The input array has dtype: %s. The FFT will "
"be performed at double precision."
%
str
(
val
.
dtype
))
return_val
=
val
.
copy_empty
(
global_shape
=
val
.
shape
,
dtype
=
np
.
complex
)
...
...
nifty/operators/invertible_operator_mixin/invertible_operator_mixin.py
View file @
3c388314
...
...
@@ -104,6 +104,7 @@ class InvertibleOperatorMixin(object):
x0
=
x0
)
return
result
#MR FIXME: why? shouldn't this be equivalent to the adjoint inverse?
def
_inverse_adjoint_times
(
self
,
x
,
spaces
):
raise
NotImplementedError
(
"no generic instance method 'inverse_adjoint_times'."
)
nifty/operators/propagator_operator/harmonic_propagator_operator.py
View file @
3c388314
...
...
@@ -83,7 +83,7 @@ class HarmonicPropagatorOperator(InvertibleOperatorMixin, EndomorphicOperator):
# ---Overwritten properties and methods---
def
__init__
(
self
,
S
=
None
,
M
=
None
,
R
=
None
,
N
=
None
,
inverter
=
None
,
def
__init__
(
self
,
S
,
M
=
None
,
R
=
None
,
N
=
None
,
inverter
=
None
,
preconditioner
=
None
):
"""
Sets the standard operator properties and `codomain`, `_A1`, `_A2`,
...
...
@@ -102,7 +102,6 @@ class HarmonicPropagatorOperator(InvertibleOperatorMixin, EndomorphicOperator):
"""
# infer domain, and target
# infer domain, and target
if
M
is
not
None
:
self
.
_codomain
=
M
.
domain
self
.
_likelihood
=
M
.
times
...
...
nifty/operators/smoothing_operator/direct_smoothing_operator.py
View file @
3c388314
...
...
@@ -144,6 +144,7 @@ class DirectSmoothingOperator(SmoothingOperator):
distribution_strategy
=
'not'
)
distance_array
=
distance_array
.
get_local_data
(
copy
=
False
)
#MR FIXME: this causes calls of log(0.) which should probably be avoided
if
self
.
log_distances
:
np
.
log
(
distance_array
,
out
=
distance_array
)
...
...
nifty/operators/smoothing_operator/fft_smoothing_operator.py
View file @
3c388314
...
...
@@ -24,6 +24,7 @@ class FFTSmoothingOperator(SmoothingOperator):
kernel
=
codomain
.
get_distance_array
(
distribution_strategy
=
axes_local_distribution_strategy
)
#MR FIXME: this causes calls of log(0.) which should probably be avoided
if
self
.
log_distances
:
kernel
.
apply_scalar_function
(
np
.
log
,
inplace
=
True
)
...
...
@@ -43,6 +44,7 @@ class FFTSmoothingOperator(SmoothingOperator):
# apply the kernel
if
inverse
:
#MR FIXME: danger of having division by zero or overflows
local_transformed_x
/=
local_kernel
else
:
local_transformed_x
*=
local_kernel
...
...
nifty/spaces/power_space/power_space.py
View file @
3c388314
...
...
@@ -261,6 +261,7 @@ class PowerSpace(Space):
hdf5_group
.
attrs
[
'nbin'
]
=
str
(
self
.
config
[
"nbin"
])
hdf5_group
.
attrs
[
'binbounds'
]
=
str
(
self
.
config
[
"binbounds"
])
#MR FIXME: why not "return None" as happens everywhere else?
return
{
'harmonic_partner'
:
self
.
harmonic_partner
,
'pindex'
:
self
.
pindex
,
...
...
nifty/spaces/space/space.py
View file @
3c388314
...
...
@@ -202,6 +202,4 @@ class Space(DomainObject):
raise
NotImplementedError
def
__repr__
(
self
):
string
=
""
string
+=
str
(
type
(
self
))
+
"
\n
"
return
string
return
str
(
type
(
self
))
+
"
\n
"
nifty/sugar.py
View file @
3c388314
...
...
@@ -18,25 +18,22 @@
from
nifty
import
PowerSpace
,
\
Field
,
\
DiagonalOperator
,
\
FFTOperator
DiagonalOperator
__all__
=
[
'create_power_operator'
]
def
create_power_operator
(
domain
,
power_spectrum
,
dtype
=
None
,
distribution_strategy
=
'not'
):
<<<<<<<
nifty
/
sugar
.
py
""" Creates a diagonal operator with the given power spectrum.
Constructs a diagonal operator that lives over the specified domain, or
its default harmonic codomain in case it is not harmonic.
Constructs a diagonal operator that lives over the specified domain.
Parameters
----------
domain : DomainObject
Domain over which the power operator shall live. If this is not a
harmonic domain, it will return an operator for its harmonic codomain
instead.
Domain over which the power operator shall live.
power_spectrum : (array-like, method)
An array-like object, or a method that implements the square root
of a power spectrum as a function of k.
...
...
@@ -53,21 +50,14 @@ def create_power_operator(domain, power_spectrum, dtype=None,
DiagonalOperator : An operator that implements the given power spectrum.
"""
if
not
domain
.
harmonic
:
fft
=
FFTOperator
(
domain
)
domain
=
fft
.
target
[
0
]
=======
>>>>>>>
nifty
/
sugar
.
py
power_domain
=
PowerSpace
(
domain
,
distribution_strategy
=
distribution_strategy
)
fp
=
Field
(
power_domain
,
val
=
power_spectrum
,
dtype
=
dtype
,
fp
=
Field
(
power_domain
,
val
=
power_spectrum
,
dtype
=
dtype
,
distribution_strategy
=
distribution_strategy
)
fp
**=
2
f
=
fp
.
power_synthesize
(
mean
=
1
,
std
=
0
,
real_signal
=
False
)
power_operator
=
DiagonalOperator
(
domain
,
diagonal
=
f
,
bare
=
True
)
return
power_operator
return
DiagonalOperator
(
domain
,
diagonal
=
f
,
bare
=
True
)
test/test_operators/test_diagonal_operator.py
View file @
3c388314
...
...
@@ -106,6 +106,7 @@ class DiagonalOperator_Tests(unittest.TestCase):
assert_allclose
(
trace_op
,
np
.
sum
(
1.
/
diag
.
val
.
get_full_data
()))
@
expand
(
product
(
spaces
,
[
True
,
False
]))
#MR FIXME: what if any diagonal element <=0?
def
test_trace_log
(
self
,
space
,
copy
):
diag
=
Field
.
from_random
(
'normal'
,
domain
=
space
)
D
=
DiagonalOperator
(
space
,
diagonal
=
diag
,
copy
=
copy
)
...
...
@@ -127,8 +128,9 @@ class DiagonalOperator_Tests(unittest.TestCase):
assert_allclose
(
inv_det
,
1.
/
D
.
determinant
())
@
expand
(
product
(
spaces
,
[
True
,
False
],
[
True
,
False
]))
#MR FIXME: what if determinant <=0?
def
test_log_determinant
(
self
,
space
,
bare
,
copy
):
diag
=
Field
.
from_random
(
'normal'
,
domain
=
space
)
D
=
DiagonalOperator
(
space
,
diagonal
=
diag
,
bare
=
bare
,
copy
=
copy
)
log_det
=
D
.
log_determinant
()
assert_allclose
(
log_det
,
np
.
log
(
D
.
determinant
()))
\ No newline at end of file
assert_allclose
(
log_det
,
np
.
log
(
D
.
determinant
()))
test/test_operators/test_smoothing_operator.py
View file @
3c388314
...
...
@@ -36,9 +36,6 @@ def _get_rtol(tp):
else
:
return
1e-5
from
itertools
import
product
from
test.common
import
expand
class
SmoothingOperator_Tests
(
unittest
.
TestCase
):
spaces
=
[
RGSpace
(
100
)]
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment