Merge branch 'cleanup2' into 'NIFTy_4'

Cleanup2

See merge request ift/NIFTy!233

.gitlab-ci.yml

@@ -18,7 +18,6 @@ before_script:
 test_min:
   stage: test
   script:
-    - nosetests -q
     - nosetests3 -q
     - OMP_NUM_THREADS=1 mpiexec --allow-run-as-root -n 4 nosetests -q 2>/dev/null
     - OMP_NUM_THREADS=1 mpiexec --allow-run-as-root -n 4 nosetests3 -q 2>/dev/null

ci/requirements.txt

 parameterized
 coverage
 git+https://gitlab.mpcdf.mpg.de/ift/pyHealpix.git
-sphinx==1.6.7
+sphinx
 sphinx_rtd_theme
 numpydoc

cleanup.sh

-find . -name "*.pyc" -delete
-find . -name "__pycache__" -exec rm -rf {} \; 2> /dev/null
-rm -rf build dist oprofile_data
-rm -rf *.egg-info .eggs
-rm -f log.log*
-rm -f *.pdf *.png
-rm -rf nifty2go
-find . -type d -empty -delete
-rm -rf docs/build docs/source/mod

differences

-Significant differences between NIFTy3 and nifty4
-=================================================
-
-1) Field domains in nifty4 are stored in DomainTuple objects, which makes
-   comparisons between domains and computation of axis indices etc. much simpler
-   and more efficient.
-   No impact on the user ... these objects are generated whenever needed and
-   have all necessary functions to make them look like tuples of spaces.
-
-2) In nifty4 an operator's domain and target refer to the _full_ domains of
-   the input and output fields read/written by times(), adjoint_times() etc.
-   In NIFTy nightly, domain and target only refer to the (sub-)domain on
-   which the operator actually acts. This leads to complications like the need
-   for the "default_spaces" argument in the operator constructor and the
-   "spaces" keywords in all operator calls.
-   Advantages of the nifty4 approach:
-    - less error-prone and easier to understand; less code overall
-    - operators have more knowledge and may tune themselves better
-    - difficulties with the design of ComposedOperator (issue 152) resolve
-      themselves automatically
-   Disadvantages:
-    - operators cannot be used as flexibly as before; in a few circumstances
-      it will be necessary to create more operator objects than with the current
-      approach.
-      However, I have not found any such situation in the current code base, so
-      it appears to be rare.
-
-3) nifty4 uses one of two different "data_object" modules for array
-   storage instead of D2O.
-   A "data_object" module consists of a class called "data_object" which
-   provides a subset of the numpy.ndarray interface, plus a few additional
-   functions for manipulating these data objects.
-   If no MPI support is found on the system, or if a computation is run on a
-   single task, nifty4 automatically loads a minimalistic "data_object"
-   module where the data_object class is simply identical to numpy.ndarray.
-   The support functions are mostly trivial as well.
-   If MPI is required, another module is loaded, which supports parallel
-   array operations; this module is in a working state, but not polished and
-   tuned yet.
-
-4) Spaces no longer have a weight() method; it has been replaced by
-   scalar_dvol() and dvol() methods, which return the scalar volume element,
-   if available, or a numpy array with all volume elements for the respective
-   space.
-   By using scalar_dvol() whenever possible, one can save quite some
-   time when computing Field.weight().
-
-5) renamings:
-   get_distance_array -> get_k_length_array
-   get_unique_distances -> get_unique_k_lengths
-   (to be specific which "distances" are meant and to make more clear why this
-   only exists for harmonic spaces)
-   kindex -> k_lengths
-   (because this is not an index)
-
-6) In nifty4, PowerSpace is not a harmonic space.
-
-7) In nifty4, parallel probing should work (needs systematic testing)
-
-9) Many default arguments have been removed in nifty4, wherever there is no
-   sensible default (in my opinion). My personal impression is that this has
-   actually made the demos more readable, but I'm sure not everyone will agree
-   :)
-
-10) Plotting has been replaced by something very minimalistic but usable.
-   Currently supported output formats are PDF and PNG.
-
-11) Co-domains are now obtained directly from the corresponding Space
-   objects via the method "get_default_codomain()". This is implemented for
-   RGSpace, LMSpace, HPSpace and GLSpace.
-
-12) Instead of inheriting from "InvertibleOperatorMixin", support for numerical
-   inversion is now added via the "InversionEnabler" class, which takes the
-   original operator as a constructor argument.
-
-13) External dependencies are only loaded when they are really needed: e.g.
-   pyHealpix is only imported within the spherical harmonic transform functions,
-   and pyfftw is only loaded within the RG transforms.
-   So there are no mandatory dependencies besides numpy (well, pyfftw is
-   more or less always needed).
-
-14) A new approach is used for FFTs along axes that are distributed among
-   MPI tasks. As a consequence, nifty4 works well with the standard version
-   of pyfftw and does not need the MPI-enabled fork.
-
-15) Arithmetic functions working on Fields have been moved from
-   basic_arithmetics.py to field.py.
-
-16) Operators can be comined via "*", "+" and "-", resulting in new combined
-   operators.
-
-17) Every operator has the properties ".adjoint" and ".inverse", which return
-   its adjoint and inverse, respectively.
-
-18) Handling of volume factors has been changed completely.

docs/generate.sh

 rm -rf docs/build docs/source/mod
 sphinx-apidoc -l -e -d 2 -o docs/source/mod nifty4
-#python docs/better_apidoc.py -l -e -d 3 -t docs/generation-templates -o docs/source/mod nifty4
 sphinx-build -b html docs/source/ docs/build/

docs/generation-templates/module.rst

-{% if name %}
-{{ name }}
-{% for item in range(8 + name|length) -%}={%- endfor %}
-{% else %}
-{{ fullname }}
-{% for item in range(8 + fullname|length) -%}={%- endfor %}
-{% endif %}
-({{ fullname }} module)
-
-.. currentmodule:: {{ fullname }}
-
-.. automodule:: {{ fullname }}
-    {% if members -%}
-    :members: {{ members|join(", ") }}
-    :undoc-members:
-    :show-inheritance:
-    :member-order: bysource
-
-    Summary
-    -------
-
-    {%- if exceptions %}
-
-    Exceptions:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in exceptions %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-    {%- if classes %}
-
-    Classes:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in classes %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-    {%- if functions %}
-
-    Functions:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in functions %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-    {%- endif %}
-
-    {%- if data %}
-
-    Data:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in data %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-{% if all_refs %}
-    ``__all__``: {{ all_refs|join(", ") }}
-{%- endif %}
-
-
-{% if members %}
-    Reference
-    ---------
-
-{%- endif %}

docs/generation-templates/package.rst

-{% if name %}
-{{ name }}
-{% for item in range(8 + name|length) -%}={%- endfor %}
-{% else %}
-{{ fullname }}
-{% for item in range(8 + fullname|length) -%}={%- endfor %}
-{% endif %}
-({{ fullname }} package)
-
-.. automodule:: {{ fullname }}
-    {% if members -%}
-    :members: {{ members|join(", ") }}
-    :undoc-members:
-    :show-inheritance:
-    {%- endif %}
-
-
-    {% if submodules %}
-    Submodules
-    ----------
-
-    .. toctree::
-        :maxdepth: 1
-        {% for item in submodules %}
-        {{ fullname }}.{{ item }}
-        {%- endfor %}
-    {%- endif -%}
-
-    {% if subpackages %}
-
-
-    Subpackages
-    -----------
-
-    .. toctree::
-        :maxdepth: 1
-        {% for item in subpackages %}
-        {{ fullname }}.{{ item }}
-        {%- endfor %}
-
-
-    {%- endif %}
-
-    {% if members %}
-    Summary
-    -------
-
-    {%- if exceptions %}
-
-    Exceptions:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in exceptions %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-    {%- if classes %}
-
-    Classes:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in classes %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-    {%- if functions %}
-
-    Functions:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in functions %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-    {%- endif %}
-
-    {%- if data %}
-
-    Data:
-
-    .. autosummary::
-        :nosignatures:
-        {% for item in data %}
-        {{ item }}
-        {%- endfor %}
-    {%- endif %}
-
-{% if all_refs %}
-    ``__all__``: {{ all_refs|join(", ") }}
-{%- endif %}
-
-
-{% if members %}
-    Reference
-    ---------
-
-{%- endif %}

test/common.py

@@ -18,13 +18,12 @@
 
 from builtins import str
 from parameterized import parameterized
-import nifty4 as ift
 import numpy as np
 
 np.seterr(all='raise', under='ignore')
 
 
-def custom_name_func(testcase_func, param_num, param):
+def _custom_name_func(testcase_func, param_num, param):
     return "%s_%s" % (
         testcase_func.__name__,
         parameterized.to_safe_name("_".join(str(x) for x in param.args)),
@@ -32,17 +31,5 @@ def custom_name_func(testcase_func, param_num, param):
 
 
 def expand(*args, **kwargs):
-    return parameterized.expand(*args, testcase_func_name=custom_name_func,
+    return parameterized.expand(*args, testcase_func_name=_custom_name_func,
                                 **kwargs)
-
-
-def generate_spaces():
-    spaces = [ift.RGSpace(4),
-              ift.PowerSpace(ift.RGSpace((4, 4), harmonic=True)),
-              ift.LMSpace(5), ift.HPSpace(4), ift.GLSpace(4)]
-    return spaces
-
-
-def generate_harmonic_spaces():
-    spaces = [ift.RGSpace(4, harmonic=True), ift.LMSpace(5)]
-    return spaces

test/test_operators/test_composed_operator.py

@@ -19,13 +19,14 @@
 import unittest
 from numpy.testing import assert_allclose, assert_equal
 import nifty4 as ift
-from test.common import generate_spaces
 from itertools import product
 from test.common import expand
 
 
 class ComposedOperator_Tests(unittest.TestCase):
-    spaces = generate_spaces()
+    spaces = [ift.RGSpace(4),
+              ift.PowerSpace(ift.RGSpace((4, 4), harmonic=True)),
+              ift.LMSpace(5), ift.HPSpace(4), ift.GLSpace(4)]
 
     @expand(product(spaces, spaces))
     def test_times_adjoint_times(self, space1, space2):

test/test_operators/test_diagonal_operator.py

@@ -20,13 +20,14 @@ from __future__ import division
 import unittest
 from numpy.testing import assert_equal, assert_allclose
 import nifty4 as ift
-from test.common import generate_spaces
 from itertools import product
 from test.common import expand
 
 
 class DiagonalOperator_Tests(unittest.TestCase):
-    spaces = generate_spaces()
+    spaces = [ift.RGSpace(4),
+              ift.PowerSpace(ift.RGSpace((4, 4), harmonic=True)),
+              ift.LMSpace(5), ift.HPSpace(4), ift.GLSpace(4)]
 
     @expand(product(spaces))
     def test_property(self, space):

test/test_spaces/test_interface.py

@@ -20,12 +20,16 @@ import unittest
 from numpy.testing import assert_
 from itertools import product
 from types import LambdaType
-from test.common import expand, generate_spaces, generate_harmonic_spaces
+from test.common import expand
 import nifty4 as ift
 
+spaces = [ift.RGSpace(4),
+          ift.PowerSpace(ift.RGSpace((4, 4), harmonic=True)),
+          ift.LMSpace(5), ift.HPSpace(4), ift.GLSpace(4)]
+
 
 class SpaceInterfaceTests(unittest.TestCase):
-    @expand(product(generate_spaces(), [
+    @expand(product(spaces, [
                     ['harmonic', bool],
                     ['shape', tuple],
                     ['size', int]]))
@@ -33,7 +37,7 @@ class SpaceInterfaceTests(unittest.TestCase):
         assert_(isinstance(getattr(space, attr_expected_type[0]),
                            attr_expected_type[1]))
 
-    @expand(product(generate_harmonic_spaces(), [
+    @expand(product([ift.RGSpace(4, harmonic=True), ift.LMSpace(5)], [
         ['get_k_length_array', ift.Field],
         ['get_fft_smoothing_kernel_function', 2.0, LambdaType],
         ]))