Planned maintenance on Wednesday, 2021-01-20, 17:00-18:00. Expect some interruptions during that time

Commit 80ff8fdc authored by Theo Steininger's avatar Theo Steininger

Updated energy and minimization docstrings.

parent 50beeb97
Pipeline #12608 failed with stage
in 3 minutes and 16 seconds
*.html # custom
setup.cfg
.idea
.DS_Store
# from https://github.com/github/gitignore/blob/master/Python.gitignore
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.c *.c
*.o *.o
*.so *.so
*.pyc
*.log # Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg *.egg
*.egg-info
*~
.git/ # PyInstaller
.svn/ # Usually these files are written by a python script from a template
.spyderproject # before PyInstaller builds the exe, so as to inject date/other infos into it.
.document *.manifest
build *.spec
.idea # Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage .coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# IPython Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# dotenv
.env
# virtualenv
venv/
ENV/
# Spyder project settings
.spyderproject
# Rope project settings
.ropeproject
\ No newline at end of file
...@@ -22,44 +22,44 @@ from keepers import Loggable ...@@ -22,44 +22,44 @@ from keepers import Loggable
class Energy(Loggable, object): class Energy(Loggable, object):
""" The Energy object provides the structure required for minimization schemes. """ Provides the functional used by minimization schemes.
The implementation of a scalar function with its gradient and curvature at some position. The Energy object is an implementation of a scalar function including its
gradient and curvature at some position.
Parameters Parameters
---------- ----------
position : Field, float position : Field
The parameter of the scalar function and its first and second derivative. The input parameter of the scalar function.
Attributes Attributes
---------- ----------
position : Field, float position : Field
The Field location in parameter space where value, gradient and curvature is evaluated. The Field location in parameter space where value, gradient and
value : float curvature are evaluated.
The evaluation of the energy functional at given position. value : np.float
gradient : Field, float The value of the energy functional at given `position`.
The gradient at given position in parameter direction. gradient : Field
curvature : callable The gradient at given `position` in parameter direction.
A positive semi-definite operator or function describing the curvature of the potential curvature : LinearOperator, callable
at given position. A positive semi-definite operator or function describing the curvature
of the potential at the given `position`.
Raises
------
NotImplementedError
Raised if
* value, gradient or curvature is called
AttributeError
Raised if
* copying of the position fails
Notes Notes
----- -----
The Energy object gives the blueprint how to formulate the model in order to apply An instance of the Energy class is defined at a certain location. If one
various inference schemes. The functions value, gradient and curvature have to be is interested in the value, gradient or curvature of the abstract energy
implemented according to the concrete inference problem. functional one has to 'jump' to the new position using the `at` method.
This method returns a new energy instance residing at the new position. By
this approach, intermediate results from computing e.g. the gradient can
safely be reused for e.g. the value or the curvature.
Memorizing the evaluations of some quantities minimizes the computational effort Memorizing the evaluations of some quantities (using the memo decorator)
for multiple calls. minimizes the computational effort for multiple calls.
See also
--------
memo
""" """
...@@ -74,7 +74,7 @@ class Energy(Loggable, object): ...@@ -74,7 +74,7 @@ class Energy(Loggable, object):
self._position = position self._position = position
def at(self, position): def at(self, position):
""" Initializes and returns new Energy object at new position. """ Initializes and returns a new Energy object at the new position.
Parameters Parameters
---------- ----------
...@@ -87,20 +87,43 @@ class Energy(Loggable, object): ...@@ -87,20 +87,43 @@ class Energy(Loggable, object):
Energy object at new position. Energy object at new position.
""" """
return self.__class__(position) return self.__class__(position)
@property @property
def position(self): def position(self):
"""
The Field location in parameter space where value, gradient and
curvature are evaluated.
"""
return self._position return self._position
@property @property
def value(self): def value(self):
"""
The value of the energy functional at given `position`.
"""
raise NotImplementedError raise NotImplementedError
@property @property
def gradient(self): def gradient(self):
"""
The gradient at given `position` in parameter direction.
"""
raise NotImplementedError raise NotImplementedError
@property @property
def curvature(self): def curvature(self):
"""
A positive semi-definite operator or function describing the curvature
of the potential at the given `position`.
"""
raise NotImplementedError raise NotImplementedError
...@@ -20,52 +20,58 @@ from .energy import Energy ...@@ -20,52 +20,58 @@ from .energy import Energy
class LineEnergy(Energy): class LineEnergy(Energy):
"""A Energy object restricting an underlying Energy along only some line direction. """ Evaluates an underlying Energy along a certain line direction.
Given some Energy and line direction, its position is parametrized by a scalar
step size along the descent direction. Given an Energy class and a line direction, its position is parametrized by
a scalar step size along the descent direction relative to a zero point.
Parameters Parameters
---------- ----------
position : float position : float
The step length parameter along the given line direction. The step length parameter along the given line direction.
energy : Energy energy : Energy
The Energy object which will be restricted along the given line direction The Energy object which will be evaluated along the given direction.
line_direction : Field, float line_direction : Field
Line direction restricting the Energy. Direction used for line evaluation.
zero_point : Field, float zero_point : Field *optional*
Fixing the zero point of the line restriction. Used to memorize this position in new Fixing the zero point of the line restriction. Used to memorize this
initializations (default : None) position in new initializations. By the default the current position
of the supplied `energy` instance is used (default : None).
Attributes Attributes
---------- ----------
position : float position : float
The step length along the given line direction. The position along the given line direction relative to the zero point.
value : float value : float
The evaluation of the energy functional at given position. The value of the energy functional at given `position`.
gradient : float gradient : float
The gradient along the line direction projected on the current line position. The gradient of the underlying energy instance along the line direction
projected on the line direction.
curvature : callable curvature : callable
A positive semi-definite operator or function describing the curvature of the potential A positive semi-definite operator or function describing the curvature
at given position. of the potential at given `position`.
line_direction : Field line_direction : Field
Direction along which the movement is restricted. Does not have to be normalized. Direction along which the movement is restricted. Does not have to be
normalized.
energy : Energy energy : Energy
The underlying Energy at the resulting position along the line according to the step length. The underlying Energy at the `position` along the line direction.
Raises Raises
------ ------
NotImplementedError NotImplementedError
Raised if Raised if
* value, gradient or curvature of the attribute energy is not implemented. * value, gradient or curvature of the attribute energy are not
implemented.
Notes Notes
----- -----
The LineEnergy is used in minimization schemes in order to determine the step size along The LineEnergy is used in minimization schemes in order perform line
some descent direction using a line search. It describes an underlying Energy which is restricted searches. It describes an underlying Energy which is restricted along one
along one direction, only requiring the step size parameter to determine a new position. direction, only requiring the step size parameter to determine a new
position.
""" """
def __init__(self, position, energy, line_direction, zero_point=None): def __init__(self, position, energy, line_direction, zero_point=None):
super(LineEnergy, self).__init__(position=position) super(LineEnergy, self).__init__(position=position)
self.line_direction = line_direction self.line_direction = line_direction
...@@ -78,19 +84,20 @@ class LineEnergy(Energy): ...@@ -78,19 +84,20 @@ class LineEnergy(Energy):
self.energy = energy.at(position=position_on_line) self.energy = energy.at(position=position_on_line)
def at(self, position): def at(self, position):
""" Initializes and returns new LineEnergy object at new position, memorizing the zero point. """ Returns LineEnergy at new position, memorizing the zero point.
Parameters Parameters
---------- ----------
position : float position : float
Parameter for the new position. Parameter for the new position on the line direction.
Returns Returns
------- -------
out : LineEnergy out : LineEnergy
LineEnergy object at new position with same zero point. LineEnergy object at new position with same zero point as `self`.
""" """
return self.__class__(position, return self.__class__(position,
self.energy, self.energy,
self.line_direction, self.line_direction,
......
...@@ -23,56 +23,58 @@ from keepers import Loggable ...@@ -23,56 +23,58 @@ from keepers import Loggable
class ConjugateGradient(Loggable, object): class ConjugateGradient(Loggable, object):
"""Implementation of the Conjugate Gradient scheme. """ Implementation of the Conjugate Gradient scheme.
It is an iterative method for solving a linear system of equations: It is an iterative method for solving a linear system of equations:
Ax = b Ax = b
SUGGESTED LITERATURE:
Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, Springer-Verlag New York
Parameters Parameters
---------- ----------
convergence_tolerance : scalar convergence_tolerance : float *optional*
Tolerance specifying convergence. (default: 1E-4) Tolerance specifying the case of convergence. (default: 1E-4)
convergence_level : integer convergence_level : integer *optional*
Number of times the tolerance should be undershot before exiting. Number of times the tolerance must be undershot before convergence
(default: 3) is reached. (default: 3)
iteration_limit : integer *optional* iteration_limit : integer *optional*
Maximum number of iterations performed. (default: None) Maximum number of iterations performed (default: None).
reset_count : integer, *optional* reset_count : integer *optional*
Number of iterations after which to restart; i.e., forget previous Number of iterations after which to restart; i.e., forget previous
conjugated directions. (default: None) conjugated directions (default: None).
preconditioner : function *optional* preconditioner : Operator *optional*
The user can provide a function which transforms the variables of the This operator can be provided which transforms the variables of the
system to make the converge more favorable.(default: None) system to improve the conditioning (default: None).
callback : function, *optional* callback : callable *optional*
Function f(energy, iteration_number) specified by the user to print Function f(energy, iteration_number) supplied by the user to perform
iteration number and energy value at every iteration step. It accepts in-situ analysis at every iteration step. When being called the
an Energy object(energy) and integer(iteration_number). (default: None) current energy and iteration_number are passed. (default: None)
Attributes Attributes
---------- ----------
convergence_tolerance : float convergence_tolerance : float
Tolerance specifying convergence. Tolerance specifying the case of convergence.
convergence_level : float convergence_level : integer
Number of times the tolerance should be undershot before exiting. Number of times the tolerance must be undershot before convergence
is reached. (default: 3)
iteration_limit : integer iteration_limit : integer
Maximum number of iterations performed. Maximum number of iterations performed.
reset_count : integer reset_count : integer
Number of iterations after which to restart; i.e., forget previous Number of iterations after which to restart; i.e., forget previous
conjugated directions. conjugated directions.
preconditioner : function preconditioner : function
The user can provide a function which transforms the variables of the This operator can be provided which transforms the variables of the
system to make the converge more favorable. system to improve the conditioning (default: None).
callback : function callback : callable
Function f(energy, iteration_number) specified by the user to print Function f(energy, iteration_number) supplied by the user to perform
iteration number and energy value at every iteration step. It accepts in-situ analysis at every iteration step. When being called the
an Energy object(energy) and integer(iteration_number). current energy and iteration_number are passed. (default: None)
""" References
----------
Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, Springer-Verlag New York
"""
def __init__(self, convergence_tolerance=1E-4, convergence_level=3, def __init__(self, convergence_tolerance=1E-4, convergence_level=3,
iteration_limit=None, reset_count=None, iteration_limit=None, reset_count=None,
preconditioner=None, callback=None): preconditioner=None, callback=None):
...@@ -95,14 +97,15 @@ class ConjugateGradient(Loggable, object): ...@@ -95,14 +97,15 @@ class ConjugateGradient(Loggable, object):
self.callback = callback self.callback = callback
def __call__(self, A, b, x0): def __call__(self, A, b, x0):
"""Runs the conjugate gradient minimization. """ Runs the conjugate gradient minimization.
For `Ax = b` the variable `x` is infered.
Parameters Parameters
---------- ----------
A : Operator A : Operator
Operator `A` applicable to a Field. Operator `A` applicable to a Field.
b : Field b : Field
Resulting Field of the operation `A(x)`. Result of the operation `A(x)`.
x0 : Field x0 : Field
Starting guess for the minimization. Starting guess for the minimization.
...@@ -115,6 +118,7 @@ class ConjugateGradient(Loggable, object): ...@@ -115,6 +118,7 @@ class ConjugateGradient(Loggable, object):
has converged or not. has converged or not.
""" """
r = b - A(x0) r = b - A(x0)
d = self.preconditioner(r) d = self.preconditioner(r)
previous_gamma = r.dot(d) previous_gamma = r.dot(d)
......
...@@ -27,53 +27,56 @@ from .line_searching import LineSearchStrongWolfe ...@@ -27,53 +27,56 @@ from .line_searching import LineSearchStrongWolfe
class DescentMinimizer(Loggable, object): class DescentMinimizer(Loggable, object):
"""A class used by other minimization methods to find a local minimum. """ A base class used by gradient methods to find a local minimum.
Descent minimization methods are used to find a local minimum of a scalar function Descent minimization methods are used to find a local minimum of a scalar
by following a descent direction. This class implements the minimization procedure, function by following a descent direction. This class implements the
the descent direction has to be implemented separately. minimization procedure once a descent direction is known. The descent
direction has to be implemented separately.
Parameters Parameters
---------- ----------
line_searcher : callable line_searcher : callable *optional*
Function which finds the step size in descent direction. (default: Function which infers the step size in the descent direction
LineSearchStrongWolfe()) (default : LineSearchStrongWolfe()).
callback : function, *optional* callback : callable *optional*
Function f(energy, iteration_number) specified by the user to print Function f(energy, iteration_number) supplied by the user to perform
iteration number and energy value at every iteration step. It accepts in-situ analysis at every iteration step. When being called the
an Energy object(energy) and integer(iteration_number). (default: None) current energy and iteration_number are passed. (default: None)
convergence_tolerance : scalar convergence_tolerance : float *optional*
Tolerance specifying convergence. (default: 1E-4) Tolerance specifying the case of convergence. (default: 1E-4)
convergence_level : integer convergence_level : integer *optional*
Number of times the tolerance should be undershot before Number of times the tolerance must be undershot before convergence
exiting. (default: 3) is reached. (default: 3)
iteration_limit : integer *optional* iteration_limit : integer *optional*
Maximum number of iterations performed. (default: None) Maximum number of iterations performed (default: None).
Attributes Attributes
---------- ----------
convergence_tolerance : float convergence_tolerance : float
Tolerance specifying convergence. Tolerance specifying the case of convergence.
convergence_level : float convergence_level : integer
Number of times the tolerance should be undershot before Number of times the tolerance must be undershot before convergence
exiting. is reached. (default: 3)
iteration_limit : integer iteration_limit : integer
Maximum number of iterations performed. Maximum number of iterations performed.
line_searcher : callable line_searcher : LineSearch
Function which finds the step size into the descent direction Function which infers the optimal step size for functional minization
given a descent direction.
callback : function callback : function
Function f(energy, iteration_number) specified by the user to print Function f(energy, iteration_number) supplied by the user to perform
iteration number and energy value at every iteration step. It accepts in-situ analysis at every iteration step. When being called the
an Energy object(energy) and integer(iteration_number). current energy and iteration_number are passed.
Raises Notes
------ ------
StopIteration The callback function can be used to externally stop the minimization by
Raised if raising a `StopIteration` exception.
*callback function does not match the specified form. Check `get_descent_direction` of a derived class for information on the
concrete minization scheme.
"""
"""
__metaclass__ = NiftyMeta __metaclass__ = NiftyMeta
def __init__(self, line_searcher=LineSearchStrongWolfe(), callback=None, def __init__(self, line_searcher=LineSearchStrongWolfe(), callback=None,
...@@ -81,7 +84,7 @@ class DescentMinimizer(Loggable, object): ...@@ -81,7 +84,7 @@ class DescentMinimizer(Loggable, object):
iteration_limit=None): iteration_limit=None):
self.convergence_tolerance = np.float(convergence_tolerance) self.convergence_tolerance = np.float(convergence_tolerance)
self.convergence_level = np.float(convergence_level) self.convergence_level = np.int(convergence_level)
if iteration_limit is not None: if iteration_limit is not None:
iteration_limit = int(iteration_limit) iteration_limit = int(iteration_limit)
...@@ -91,16 +94,13 @@ class DescentMinimizer(Loggable, object): ...@@ -91,16 +94,13 @@ class DescentMinimizer(Loggable, object):
self.callback = callback