Skip to content
GitLab
Menu
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
ift
NIFTy
Commits
80ff8fdc
Commit
80ff8fdc
authored
May 18, 2017
by
Theo Steininger
Browse files
Updated energy and minimization docstrings.
parent
50beeb97
Pipeline
#12608
failed with stage
in 3 minutes and 16 seconds
Changes
9
Pipelines
1
Hide whitespace changes
Inline
Sidebyside
.gitignore
View file @
80ff8fdc
*.html
# custom
setup.cfg
.idea
.DS_Store
# from https://github.com/github/gitignore/blob/master/Python.gitignore
# Bytecompiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.c
*.c
*.o
*.o
*.so
*.so
*.pyc
*.log
# Distribution / packaging
.Python
env/
build/
developeggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egginfo/
.installed.cfg
*.egg
*.egg
*.egginfo
*~
.git/
# PyInstaller
.svn/
# Usually these files are written by a python script from a template
.spyderproject
# before PyInstaller builds the exe, so as to inject date/other infos into it.
.documen
t
*.manifes
t
build
*.spec
.idea
# Installer logs
piplog.txt
pipdeletethisdirectory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassetscache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# IPython Notebook
.ipynb_checkpoints
# pyenv
.pythonversion
# celery beat schedule file
celerybeatschedule
# dotenv
.env
# virtualenv
venv/
ENV/
# Spyder project settings
.spyderproject
# Rope project settings
.ropeproject
\ No newline at end of file
nifty/energies/energy.py
View file @
80ff8fdc
...
@@ 22,44 +22,44 @@ from keepers import Loggable
...
@@ 22,44 +22,44 @@ from keepers import Loggable
class
Energy
(
Loggable
,
object
):
class
Energy
(
Loggable
,
object
):
"""
The Energy object provides the structure required for
minimization schemes.
"""
Provides the functional used by
minimization schemes.
The implementation of a scalar function with its gradient and curvature at some position.
The Energy object is an implementation of a scalar function including its
gradient and curvature at some position.
Parameters
Parameters


position : Field
, float
position : Field
The parameter of the scalar function
and its first and second derivative
.
The
input
parameter of the scalar function.
Attributes
Attributes


position : Field, float
position : Field
The Field location in parameter space where value, gradient and curvature is evaluated.
The Field location in parameter space where value, gradient and
value : float
curvature are evaluated.
The evaluation of the energy functional at given position.
value : np.float
gradient : Field, float
The value of the energy functional at given `position`.
The gradient at given position in parameter direction.
gradient : Field
curvature : callable
The gradient at given `position` in parameter direction.
A positive semidefinite operator or function describing the curvature of the potential
curvature : LinearOperator, callable
at given position.
A positive semidefinite operator or function describing the curvature
of the potential at the given `position`.
Raises

NotImplementedError
Raised if
* value, gradient or curvature is called
AttributeError
Raised if
* copying of the position fails
Notes
Notes


The Energy object gives the blueprint how to formulate the model in order to apply
An instance of the Energy class is defined at a certain location. If one
various inference schemes. The functions value, gradient and curvature have to be
is interested in the value, gradient or curvature of the abstract energy
implemented according to the concrete inference problem.
functional one has to 'jump' to the new position using the `at` method.
This method returns a new energy instance residing at the new position. By
this approach, intermediate results from computing e.g. the gradient can
safely be reused for e.g. the value or the curvature.
Memorizing the evaluations of some quantities minimizes the computational effort
Memorizing the evaluations of some quantities (using the memo decorator)
for multiple calls.
minimizes the computational effort for multiple calls.
See also

memo
"""
"""
...
@@ 74,7 +74,7 @@ class Energy(Loggable, object):
...
@@ 74,7 +74,7 @@ class Energy(Loggable, object):
self
.
_position
=
position
self
.
_position
=
position
def
at
(
self
,
position
):
def
at
(
self
,
position
):
""" Initializes and returns new Energy object at new position.
""" Initializes and returns
a
new Energy object at
the
new position.
Parameters
Parameters


...
@@ 87,20 +87,43 @@ class Energy(Loggable, object):
...
@@ 87,20 +87,43 @@ class Energy(Loggable, object):
Energy object at new position.
Energy object at new position.
"""
"""
return
self
.
__class__
(
position
)
return
self
.
__class__
(
position
)
@
property
@
property
def
position
(
self
):
def
position
(
self
):
"""
The Field location in parameter space where value, gradient and
curvature are evaluated.
"""
return
self
.
_position
return
self
.
_position
@
property
@
property
def
value
(
self
):
def
value
(
self
):
"""
The value of the energy functional at given `position`.
"""
raise
NotImplementedError
raise
NotImplementedError
@
property
@
property
def
gradient
(
self
):
def
gradient
(
self
):
"""
The gradient at given `position` in parameter direction.
"""
raise
NotImplementedError
raise
NotImplementedError
@
property
@
property
def
curvature
(
self
):
def
curvature
(
self
):
"""
A positive semidefinite operator or function describing the curvature
of the potential at the given `position`.
"""
raise
NotImplementedError
raise
NotImplementedError
nifty/energies/line_energy.py
View file @
80ff8fdc
...
@@ 20,52 +20,58 @@ from .energy import Energy
...
@@ 20,52 +20,58 @@ from .energy import Energy
class
LineEnergy
(
Energy
):
class
LineEnergy
(
Energy
):
"""A Energy object restricting an underlying Energy along only some line direction.
""" Evaluates an underlying Energy along a certain line direction.
Given some Energy and line direction, its position is parametrized by a scalar
step size along the descent direction.
Given an Energy class and a line direction, its position is parametrized by
a scalar step size along the descent direction relative to a zero point.
Parameters
Parameters


position : float
position : float
The step length parameter along the given line direction.
The step length parameter along the given line direction.
energy : Energy
energy : Energy
The Energy object which will be restricted along the given line direction
The Energy object which will be evaluated along the given direction.
line_direction : Field, float
line_direction : Field
Line direction restricting the Energy.
Direction used for line evaluation.
zero_point : Field, float
zero_point : Field *optional*
Fixing the zero point of the line restriction. Used to memorize this position in new
Fixing the zero point of the line restriction. Used to memorize this
initializations (default : None)
position in new initializations. By the default the current position
of the supplied `energy` instance is used (default : None).
Attributes
Attributes


position :
float
position : float
The
step length
along the given line direction.
The
position
along the given line direction
relative to the zero point
.
value : float
value : float
The
e
valu
ation
of the energy functional at given position.
The valu
e
of the energy functional at given
`
position
`
.
gradient : float
gradient : float
The gradient along the line direction projected on the current line position.
The gradient of the underlying energy instance along the line direction
projected on the line direction.
curvature : callable
curvature : callable
A positive semidefinite operator or function describing the curvature
of the potential
A positive semidefinite operator or function describing the curvature
at given position.
of the potential
at given
`
position
`
.
line_direction : Field
line_direction : Field
Direction along which the movement is restricted. Does not have to be normalized.
Direction along which the movement is restricted. Does not have to be
normalized.
energy : Energy
energy : Energy
The underlying Energy at the
resulting
position along the line
according to the step length
.
The underlying Energy at the
`
position
`
along the line
direction
.
Raises
Raises


NotImplementedError
NotImplementedError
Raised if
Raised if
* value, gradient or curvature of the attribute energy is not implemented.
* value, gradient or curvature of the attribute energy are not
implemented.
Notes
Notes


The LineEnergy is used in minimization schemes in order
to determine the step size along
The LineEnergy is used in minimization schemes in order
perform line
some descent direction using a line
search. It describes an underlying Energy which is restricted
search
es
. It describes an underlying Energy which is restricted
along one
along one
direction, only requiring the step size parameter to determine a new
position.
direction, only requiring the step size parameter to determine a new
position.
"""
"""
def
__init__
(
self
,
position
,
energy
,
line_direction
,
zero_point
=
None
):
def
__init__
(
self
,
position
,
energy
,
line_direction
,
zero_point
=
None
):
super
(
LineEnergy
,
self
).
__init__
(
position
=
position
)
super
(
LineEnergy
,
self
).
__init__
(
position
=
position
)
self
.
line_direction
=
line_direction
self
.
line_direction
=
line_direction
...
@@ 78,19 +84,20 @@ class LineEnergy(Energy):
...
@@ 78,19 +84,20 @@ class LineEnergy(Energy):
self
.
energy
=
energy
.
at
(
position
=
position_on_line
)
self
.
energy
=
energy
.
at
(
position
=
position_on_line
)
def
at
(
self
,
position
):
def
at
(
self
,
position
):
"""
Initializes and r
eturns
new
LineEnergy
object
at new position, memorizing the zero point.
"""
R
eturns LineEnergy at new position, memorizing the zero point.
Parameters
Parameters


position : float
position : float
Parameter for the new position.
Parameter for the new position
on the line direction
.
Returns
Returns


out : LineEnergy
out : LineEnergy
LineEnergy object at new position with same zero point.
LineEnergy object at new position with same zero point
as `self`
.
"""
"""
return
self
.
__class__
(
position
,
return
self
.
__class__
(
position
,
self
.
energy
,
self
.
energy
,
self
.
line_direction
,
self
.
line_direction
,
...
...
nifty/minimization/.DS_Store
deleted
100644 → 0
View file @
50beeb97
File deleted
nifty/minimization/conjugate_gradient.py
View file @
80ff8fdc
...
@@ 23,56 +23,58 @@ from keepers import Loggable
...
@@ 23,56 +23,58 @@ from keepers import Loggable
class
ConjugateGradient
(
Loggable
,
object
):
class
ConjugateGradient
(
Loggable
,
object
):
"""Implementation of the Conjugate Gradient scheme.
"""
Implementation of the Conjugate Gradient scheme.
It is an iterative method for solving a linear system of equations:
It is an iterative method for solving a linear system of equations:
Ax = b
Ax = b
SUGGESTED LITERATURE:
Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, SpringerVerlag New York
Parameters
Parameters


convergence_tolerance :
scalar
convergence_tolerance :
float *optional*
Tolerance specifying convergence. (default: 1E4)
Tolerance specifying
the case of
convergence. (default: 1E4)
convergence_level : integer
convergence_level : integer
*optional*
Number of times the tolerance
should
be undershot before
exiting.
Number of times the tolerance
must
be undershot before
convergence
(default: 3)
is reached.
(default: 3)
iteration_limit : integer *optional*
iteration_limit : integer *optional*
Maximum number of iterations performed
.
(default: None)
Maximum number of iterations performed (default: None)
.
reset_count : integer
,
*optional*
reset_count : integer *optional*
Number of iterations after which to restart; i.e., forget previous
Number of iterations after which to restart; i.e., forget previous
conjugated directions
.
(default: None)
conjugated directions (default: None)
.
preconditioner :
function
*optional*
preconditioner :
Operator
*optional*
Th
e use
r can provide
a function
which transforms the variables of the
Th
is operato
r can
be
provide
d
which transforms the variables of the
system to
mak
e the con
verge more favorable.
(default: None)
system to
improv
e the con
ditioning
(default: None)
.
callback :
function,
*optional*
callback :
callable
*optional*
Function f(energy, iteration_number) s
pecif
ied by the user to p
rint
Function f(energy, iteration_number) s
uppl
ied by the user to p
erform
i
teration number and energy value
at every iteration step.
It accepts
i
nsitu analysis
at every iteration step.
When being called the
an Energy object(
energy
)
and
integer(
iteration_number
)
. (default: None)
current
energy and iteration_number
are passed
. (default: None)
Attributes
Attributes


convergence_tolerance : float
convergence_tolerance : float
Tolerance specifying convergence.
Tolerance specifying the case of convergence.
convergence_level : float
convergence_level : integer
Number of times the tolerance should be undershot before exiting.
Number of times the tolerance must be undershot before convergence
is reached. (default: 3)
iteration_limit : integer
iteration_limit : integer
Maximum number of iterations performed.
Maximum number of iterations performed.
reset_count : integer
reset_count : integer
Number of iterations after which to restart; i.e., forget previous
Number of iterations after which to restart; i.e., forget previous
conjugated directions.
conjugated directions.
preconditioner : function
preconditioner : function
The user can provide a function which transforms the variables of the
This operator can be provided which transforms the variables of the
system to make the converge more favorable.
system to improve the conditioning (default: None).
callback : function
callback : callable
Function f(energy, iteration_number) specified by the user to print
Function f(energy, iteration_number) supplied by the user to perform
iteration number and energy value at every iteration step. It accepts
insitu analysis at every iteration step. When being called the
an Energy object(energy) and integer(iteration_number).
current energy and iteration_number are passed. (default: None)
"""
References

Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, SpringerVerlag New York
"""
def
__init__
(
self
,
convergence_tolerance
=
1E4
,
convergence_level
=
3
,
def
__init__
(
self
,
convergence_tolerance
=
1E4
,
convergence_level
=
3
,
iteration_limit
=
None
,
reset_count
=
None
,
iteration_limit
=
None
,
reset_count
=
None
,
preconditioner
=
None
,
callback
=
None
):
preconditioner
=
None
,
callback
=
None
):
...
@@ 95,14 +97,15 @@ class ConjugateGradient(Loggable, object):
...
@@ 95,14 +97,15 @@ class ConjugateGradient(Loggable, object):
self
.
callback
=
callback
self
.
callback
=
callback
def
__call__
(
self
,
A
,
b
,
x0
):
def
__call__
(
self
,
A
,
b
,
x0
):
"""Runs the conjugate gradient minimization.
""" Runs the conjugate gradient minimization.
For `Ax = b` the variable `x` is infered.
Parameters
Parameters


A : Operator
A : Operator
Operator `A` applicable to a Field.
Operator `A` applicable to a Field.
b : Field
b : Field
Result
ing Field
of the operation `A(x)`.
Result of the operation `A(x)`.
x0 : Field
x0 : Field
Starting guess for the minimization.
Starting guess for the minimization.
...
@@ 115,6 +118,7 @@ class ConjugateGradient(Loggable, object):
...
@@ 115,6 +118,7 @@ class ConjugateGradient(Loggable, object):
has converged or not.
has converged or not.
"""
"""
r
=
b

A
(
x0
)
r
=
b

A
(
x0
)
d
=
self
.
preconditioner
(
r
)
d
=
self
.
preconditioner
(
r
)
previous_gamma
=
r
.
dot
(
d
)
previous_gamma
=
r
.
dot
(
d
)
...
...
nifty/minimization/descent_minimizer.py
View file @
80ff8fdc
...
@@ 27,53 +27,56 @@ from .line_searching import LineSearchStrongWolfe
...
@@ 27,53 +27,56 @@ from .line_searching import LineSearchStrongWolfe
class
DescentMinimizer
(
Loggable
,
object
):
class
DescentMinimizer
(
Loggable
,
object
):
"""A class used by other minimization methods to find a local minimum.
""" A base class used by gradient methods to find a local minimum.
Descent minimization methods are used to find a local minimum of a scalar function
Descent minimization methods are used to find a local minimum of a scalar
by following a descent direction. This class implements the minimization procedure,
function by following a descent direction. This class implements the
the descent direction has to be implemented separately.
minimization procedure once a descent direction is known. The descent
direction has to be implemented separately.
Parameters
Parameters


line_searcher : callable
line_searcher : callable
*optional*
Function which
f
in
d
s the step size in descent direction
. (default:
Function which in
fer
s the step size in
the
descent direction
LineSearchStrongWolfe())
(default :
LineSearchStrongWolfe())
.
callback :
function,
*optional*
callback :
callable
*optional*
Function f(energy, iteration_number) s
pecif
ied by the user to p
rint
Function f(energy, iteration_number) s
uppl
ied by the user to p
erform
i
teration number and energy value
at every iteration step.
It accepts
i
nsitu analysis
at every iteration step.
When being called the
an Energy object(
energy
)
and
integer(
iteration_number
)
. (default: None)
current
energy and iteration_number
are passed
. (default: None)
convergence_tolerance :
scalar
convergence_tolerance :
float *optional*
Tolerance specifying convergence. (default: 1E4)
Tolerance specifying
the case of
convergence. (default: 1E4)
convergence_level : integer
convergence_level : integer
*optional*
Number of times the tolerance
should
be undershot before
Number of times the tolerance
must
be undershot before
convergence
exiting
. (default: 3)
is reached
. (default: 3)
iteration_limit : integer *optional*
iteration_limit : integer *optional*
Maximum number of iterations performed
.
(default: None)
Maximum number of iterations performed (default: None)
.
Attributes
Attributes


convergence_tolerance : float
convergence_tolerance : float
Tolerance specifying convergence.
Tolerance specifying
the case of
convergence.
convergence_level :
float
convergence_level :
integer
Number of times the tolerance
should
be undershot before
Number of times the tolerance
must
be undershot before
convergence
exiting.
is reached. (default: 3)
iteration_limit : integer
iteration_limit : integer
Maximum number of iterations performed.
Maximum number of iterations performed.
line_searcher : callable
line_searcher : LineSearch
Function which finds the step size into the descent direction
Function which infers the optimal step size for functional minization
given a descent direction.
callback : function
callback : function
Function f(energy, iteration_number) s
pecif
ied by the user to p
rint
Function f(energy, iteration_number) s
uppl
ied by the user to p
erform
i
teration number and energy value
at every iteration step.
It accepts
i
nsitu analysis
at every iteration step.
When being called the
an Energy object(
energy
)
and
integer(
iteration_number
)
.
current
energy and iteration_number
are passed
.
Rais
es
Not
es


StopIteration
The callback function can be used to externally stop the minimization by
Raised if
raising a `StopIteration` exception.
*callback function does not match the specified form.
Check `get_descent_direction` of a derived class for information on the