Skip to content
GitLab
Projects
Groups
Snippets
Help
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
N
NIFTy
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
14
Issues
14
List
Boards
Labels
Service Desk
Milestones
Merge Requests
8
Merge Requests
8
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Operations
Operations
Environments
Packages & Registries
Packages & Registries
Container Registry
Analytics
Analytics
CI / CD
Repository
Value Stream
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ift
NIFTy
Commits
80ff8fdc
Commit
80ff8fdc
authored
May 18, 2017
by
Theo Steininger
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Updated energy and minimization docstrings.
parent
50beeb97
Pipeline
#12608
failed with stage
in 3 minutes and 16 seconds
Changes
9
Pipelines
1
Expand all
Hide whitespace changes
Inline
Sidebyside
Showing
9 changed files
with
363 additions
and
306 deletions
+363
306
.gitignore
.gitignore
+91
11
nifty/energies/energy.py
nifty/energies/energy.py
+51
28
nifty/energies/line_energy.py
nifty/energies/line_energy.py
+32
25
nifty/minimization/.DS_Store
nifty/minimization/.DS_Store
+0
0
nifty/minimization/conjugate_gradient.py
nifty/minimization/conjugate_gradient.py
+40
36
nifty/minimization/descent_minimizer.py
nifty/minimization/descent_minimizer.py
+53
53
nifty/minimization/relaxed_newton.py
nifty/minimization/relaxed_newton.py
+10
28
nifty/minimization/steepest_descent.py
nifty/minimization/steepest_descent.py
+19
17
nifty/minimization/vl_bfgs.py
nifty/minimization/vl_bfgs.py
+67
108
No files found.
.gitignore
View file @
80ff8fdc
*.html
# custom
setup.cfg
.idea
.DS_Store
# from https://github.com/github/gitignore/blob/master/Python.gitignore
# Bytecompiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.c
*.o
*.so
*.pyc
*.log
# Distribution / packaging
.Python
env/
build/
developeggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
*.egginfo/
.installed.cfg
*.egg
*.egginfo
*~
.git/
.svn/
.spyderproject
.documen
t
build
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifes
t
*.spec
.idea
# Installer logs
piplog.txt
pipdeletethisdirectory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*,cover
.hypothesis/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassetscache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# IPython Notebook
.ipynb_checkpoints
# pyenv
.pythonversion
# celery beat schedule file
celerybeatschedule
# dotenv
.env
# virtualenv
venv/
ENV/
# Spyder project settings
.spyderproject
# Rope project settings
.ropeproject
\ No newline at end of file
nifty/energies/energy.py
View file @
80ff8fdc
...
...
@@ 22,44 +22,44 @@ from keepers import Loggable
class
Energy
(
Loggable
,
object
):
"""
The Energy object provides the structure required for
minimization schemes.
"""
Provides the functional used by
minimization schemes.
The implementation of a scalar function with its gradient and curvature at some position.
The Energy object is an implementation of a scalar function including its
gradient and curvature at some position.
Parameters

position : Field
, float
The
parameter of the scalar function and its first and second derivative
.
position : Field
The
input parameter of the scalar function
.
Attributes

position : Field, float
The Field location in parameter space where value, gradient and curvature is evaluated.
value : float
The evaluation of the energy functional at given position.
gradient : Field, float
The gradient at given position in parameter direction.
curvature : callable
A positive semidefinite operator or function describing the curvature of the potential
at given position.
Raises

NotImplementedError
Raised if
* value, gradient or curvature is called
AttributeError
Raised if
* copying of the position fails
position : Field
The Field location in parameter space where value, gradient and
curvature are evaluated.
value : np.float
The value of the energy functional at given `position`.
gradient : Field
The gradient at given `position` in parameter direction.
curvature : LinearOperator, callable
A positive semidefinite operator or function describing the curvature
of the potential at the given `position`.
Notes

The Energy object gives the blueprint how to formulate the model in order to apply
various inference schemes. The functions value, gradient and curvature have to be
implemented according to the concrete inference problem.
An instance of the Energy class is defined at a certain location. If one
is interested in the value, gradient or curvature of the abstract energy
functional one has to 'jump' to the new position using the `at` method.
This method returns a new energy instance residing at the new position. By
this approach, intermediate results from computing e.g. the gradient can
safely be reused for e.g. the value or the curvature.
Memorizing the evaluations of some quantities minimizes the computational effort
for multiple calls.
Memorizing the evaluations of some quantities (using the memo decorator)
minimizes the computational effort for multiple calls.
See also

memo
"""
...
...
@@ 74,7 +74,7 @@ class Energy(Loggable, object):
self
.
_position
=
position
def
at
(
self
,
position
):
""" Initializes and returns
new Energy object at
new position.
""" Initializes and returns
a new Energy object at the
new position.
Parameters

...
...
@@ 87,20 +87,43 @@ class Energy(Loggable, object):
Energy object at new position.
"""
return
self
.
__class__
(
position
)
@
property
def
position
(
self
):
"""
The Field location in parameter space where value, gradient and
curvature are evaluated.
"""
return
self
.
_position
@
property
def
value
(
self
):
"""
The value of the energy functional at given `position`.
"""
raise
NotImplementedError
@
property
def
gradient
(
self
):
"""
The gradient at given `position` in parameter direction.
"""
raise
NotImplementedError
@
property
def
curvature
(
self
):
"""
A positive semidefinite operator or function describing the curvature
of the potential at the given `position`.
"""
raise
NotImplementedError
nifty/energies/line_energy.py
View file @
80ff8fdc
...
...
@@ 20,52 +20,58 @@ from .energy import Energy
class
LineEnergy
(
Energy
):
"""A Energy object restricting an underlying Energy along only some line direction.
Given some Energy and line direction, its position is parametrized by a scalar
step size along the descent direction.
""" Evaluates an underlying Energy along a certain line direction.
Given an Energy class and a line direction, its position is parametrized by
a scalar step size along the descent direction relative to a zero point.
Parameters

position : float
The step length parameter along the given line direction.
energy : Energy
The Energy object which will be restricted along the given line direction
line_direction : Field, float
Line direction restricting the Energy.
zero_point : Field, float
Fixing the zero point of the line restriction. Used to memorize this position in new
initializations (default : None)
The Energy object which will be evaluated along the given direction.
line_direction : Field
Direction used for line evaluation.
zero_point : Field *optional*
Fixing the zero point of the line restriction. Used to memorize this
position in new initializations. By the default the current position
of the supplied `energy` instance is used (default : None).
Attributes

position :
float
The
step length along the given line direction
.
position : float
The
position along the given line direction relative to the zero point
.
value : float
The
evaluation of the energy functional at given position
.
The
value of the energy functional at given `position`
.
gradient : float
The gradient along the line direction projected on the current line position.
The gradient of the underlying energy instance along the line direction
projected on the line direction.
curvature : callable
A positive semidefinite operator or function describing the curvature
of the potential
at given position
.
A positive semidefinite operator or function describing the curvature
of the potential at given `position`
.
line_direction : Field
Direction along which the movement is restricted. Does not have to be normalized.
Direction along which the movement is restricted. Does not have to be
normalized.
energy : Energy
The underlying Energy at the
resulting position along the line according to the step length
.
The underlying Energy at the
`position` along the line direction
.
Raises

NotImplementedError
Raised if
* value, gradient or curvature of the attribute energy is not implemented.
* value, gradient or curvature of the attribute energy are not
implemented.
Notes

The LineEnergy is used in minimization schemes in order
to determine the step size along
s
ome descent direction using a line search. It describes an underlying Energy which is restricted
along one direction, only requiring the step size parameter to determine a new position.
The LineEnergy is used in minimization schemes in order
perform line
s
earches. It describes an underlying Energy which is restricted along one
direction, only requiring the step size parameter to determine a new
position.
"""
def
__init__
(
self
,
position
,
energy
,
line_direction
,
zero_point
=
None
):
super
(
LineEnergy
,
self
).
__init__
(
position
=
position
)
self
.
line_direction
=
line_direction
...
...
@@ 78,19 +84,20 @@ class LineEnergy(Energy):
self
.
energy
=
energy
.
at
(
position
=
position_on_line
)
def
at
(
self
,
position
):
"""
Initializes and returns new LineEnergy object
at new position, memorizing the zero point.
"""
Returns LineEnergy
at new position, memorizing the zero point.
Parameters

position : float
Parameter for the new position.
Parameter for the new position
on the line direction
.
Returns

out : LineEnergy
LineEnergy object at new position with same zero point.
LineEnergy object at new position with same zero point
as `self`
.
"""
return
self
.
__class__
(
position
,
self
.
energy
,
self
.
line_direction
,
...
...
nifty/minimization/.DS_Store
deleted
100644 → 0
View file @
50beeb97
File deleted
nifty/minimization/conjugate_gradient.py
View file @
80ff8fdc
...
...
@@ 23,56 +23,58 @@ from keepers import Loggable
class
ConjugateGradient
(
Loggable
,
object
):
"""Implementation of the Conjugate Gradient scheme.
"""
Implementation of the Conjugate Gradient scheme.
It is an iterative method for solving a linear system of equations:
Ax = b
SUGGESTED LITERATURE:
Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, SpringerVerlag New York
Parameters

convergence_tolerance :
scalar
Tolerance specifying convergence. (default: 1E4)
convergence_level : integer
Number of times the tolerance
should be undershot before exiting.
(default: 3)
convergence_tolerance :
float *optional*
Tolerance specifying
the case of
convergence. (default: 1E4)
convergence_level : integer
*optional*
Number of times the tolerance
must be undershot before convergence
is reached.
(default: 3)
iteration_limit : integer *optional*
Maximum number of iterations performed
. (default: None)
reset_count : integer
,
*optional*
Maximum number of iterations performed
(default: None).
reset_count : integer *optional*
Number of iterations after which to restart; i.e., forget previous
conjugated directions
. (default: None)
preconditioner :
function
*optional*
Th
e user can provide a function which transforms the variables of the
system to
make the converge more favorable.(default: None)
callback :
function,
*optional*
Function f(energy, iteration_number) s
pecified by the user to print
i
teration number and energy value at every iteration step. It accepts
an Energy object(energy) and integer(iteration_number)
. (default: None)
conjugated directions
(default: None).
preconditioner :
Operator
*optional*
Th
is operator can be provided which transforms the variables of the
system to
improve the conditioning (default: None).
callback :
callable
*optional*
Function f(energy, iteration_number) s
upplied by the user to perform
i
nsitu analysis at every iteration step. When being called the
current energy and iteration_number are passed
. (default: None)
Attributes

convergence_tolerance : float
Tolerance specifying convergence.
convergence_level : float
Number of times the tolerance should be undershot before exiting.
Tolerance specifying the case of convergence.
convergence_level : integer
Number of times the tolerance must be undershot before convergence
is reached. (default: 3)
iteration_limit : integer
Maximum number of iterations performed.
reset_count : integer
Number of iterations after which to restart; i.e., forget previous
conjugated directions.
preconditioner : function
The user can provide a function which transforms the variables of the
system to make the converge more favorable.
callback : function
Function f(energy, iteration_number) specified by the user to print
iteration number and energy value at every iteration step. It accepts
an Energy object(energy) and integer(iteration_number).
"""
This operator can be provided which transforms the variables of the
system to improve the conditioning (default: None).
callback : callable
Function f(energy, iteration_number) supplied by the user to perform
insitu analysis at every iteration step. When being called the
current energy and iteration_number are passed. (default: None)
References

Thomas V. Mikosch et al., "Numerical Optimization", Second Edition,
2006, SpringerVerlag New York
"""
def
__init__
(
self
,
convergence_tolerance
=
1E4
,
convergence_level
=
3
,
iteration_limit
=
None
,
reset_count
=
None
,
preconditioner
=
None
,
callback
=
None
):
...
...
@@ 95,14 +97,15 @@ class ConjugateGradient(Loggable, object):
self
.
callback
=
callback
def
__call__
(
self
,
A
,
b
,
x0
):
"""Runs the conjugate gradient minimization.
""" Runs the conjugate gradient minimization.
For `Ax = b` the variable `x` is infered.
Parameters

A : Operator
Operator `A` applicable to a Field.
b : Field
Result
ing Field
of the operation `A(x)`.
Result of the operation `A(x)`.
x0 : Field
Starting guess for the minimization.
...
...
@@ 115,6 +118,7 @@ class ConjugateGradient(Loggable, object):
has converged or not.
"""
r
=
b

A
(
x0
)
d
=
self
.
preconditioner
(
r
)
previous_gamma
=
r
.
dot
(
d
)
...
...
nifty/minimization/descent_minimizer.py
View file @
80ff8fdc
...
...
@@ 27,53 +27,56 @@ from .line_searching import LineSearchStrongWolfe
class
DescentMinimizer
(
Loggable
,
object
):
"""A class used by other minimization methods to find a local minimum.
Descent minimization methods are used to find a local minimum of a scalar function
by following a descent direction. This class implements the minimization procedure,
the descent direction has to be implemented separately.
""" A base class used by gradient methods to find a local minimum.
Descent minimization methods are used to find a local minimum of a scalar
function by following a descent direction. This class implements the
minimization procedure once a descent direction is known. The descent
direction has to be implemented separately.
Parameters

line_searcher : callable
Function which
finds the step size in descent direction. (default:
LineSearchStrongWolfe())
callback :
function,
*optional*
Function f(energy, iteration_number) s
pecified by the user to print
i
teration number and energy value at every iteration step. It accepts
an Energy object(energy) and integer(iteration_number)
. (default: None)
convergence_tolerance :
scalar
Tolerance specifying convergence. (default: 1E4)
convergence_level : integer
Number of times the tolerance
should be undershot befor
e
exiting
. (default: 3)
line_searcher : callable
*optional*
Function which
infers the step size in the descent direction
(default : LineSearchStrongWolfe()).
callback :
callable
*optional*
Function f(energy, iteration_number) s
upplied by the user to perform
i
nsitu analysis at every iteration step. When being called the
current energy and iteration_number are passed
. (default: None)
convergence_tolerance :
float *optional*
Tolerance specifying
the case of
convergence. (default: 1E4)
convergence_level : integer
*optional*
Number of times the tolerance
must be undershot before convergenc
e
is reached
. (default: 3)
iteration_limit : integer *optional*
Maximum number of iterations performed
. (default: None)
Maximum number of iterations performed
(default: None).
Attributes

convergence_tolerance : float
Tolerance specifying convergence.
convergence_level :
float
Number of times the tolerance
should be undershot befor
e
exiting.
Tolerance specifying
the case of
convergence.
convergence_level :
integer
Number of times the tolerance
must be undershot before convergenc
e
is reached. (default: 3)
iteration_limit : integer
Maximum number of iterations performed.
line_searcher : callable
Function which finds the step size into the descent direction
line_searcher : LineSearch
Function which infers the optimal step size for functional minization
given a descent direction.
callback : function
Function f(energy, iteration_number) s
pecified by the user to print
i
teration number and energy value at every iteration step. It accepts
an Energy object(energy) and integer(iteration_number)
.
Rais
es
Function f(energy, iteration_number) s
upplied by the user to perform
i
nsitu analysis at every iteration step. When being called the
current energy and iteration_number are passed
.
Not
es

StopIteration
Raised if
*callback function does not match the specified form.
The callback function can be used to externally stop the minimization by
raising a `StopIteration` exception.
Check `get_descent_direction` of a derived class for information on the
concrete minization scheme.
"""
"""
__metaclass__
=
NiftyMeta
def
__init__
(
self
,
line_searcher
=
LineSearchStrongWolfe
(),
callback
=
None
,
...
...
@@ 81,7 +84,7 @@ class DescentMinimizer(Loggable, object):
iteration_limit
=
None
):
self
.
convergence_tolerance
=
np
.
float
(
convergence_tolerance
)
self
.
convergence_level
=
np
.
floa
t
(
convergence_level
)
self
.
convergence_level
=
np
.
in
t
(
convergence_level
)
if
iteration_limit
is
not
None
:
iteration_limit
=
int
(
iteration_limit
)
...
...
@@ 91,16 +94,13 @@ class DescentMinimizer(Loggable, object):
self
.
callback
=
callback
def
__call__
(
self
,
energy
):
"""
Runs the minimization on the provided Energy class
.
"""
Performs the minimization of the provided Energy functional
.
Accepts the NIFTY Energy class which describes our system and it runs
the minimization to find the minimum of the system.
Parameters

energy : Energy object
Energy object
provided by the user from which we can calculate the
energy, gradient and curvature at a specific point
.
Energy object
which provides value, gradient and curvature at a
specific position in parameter space
.
Returns

...
...
@@ 109,16 +109,16 @@ class DescentMinimizer(Loggable, object):
convergence : integer
Latest convergence level indicating whether the minimization
has converged or not.
Note

It stops the minimization if:
*
callback function does not match the specified form.
*
a perfectly flat point is reached.
*
according to linesearch the minimum is found.
*
target convergence level is reached.
*iteration limit is reached.
The minimization is stopped if
*
the callback function raises a `StopIteration` exception,
*
a perfectly flat point is reached,
*
according to the linesearch the minimum is found,
*
the target convergence level is reached,
*
the
iteration limit is reached.
"""
convergence
=
0
...
...
@@ 146,7 +146,7 @@ class DescentMinimizer(Loggable, object):
break
# current position is encoded in energy object
descend_direction
=
self
.
_
get_descend_direction
(
energy
)
descend_direction
=
self
.
get_descend_direction
(
energy
)
# compute the step length, which minimizes energy.value along the
# search direction
...
...
@@ 188,5 +188,5 @@ class DescentMinimizer(Loggable, object):
return
energy
,
convergence
@
abc
.
abstractmethod
def
_
get_descend_direction
(
self
,
energy
):
def
get_descend_direction
(
self
,
energy
):
raise
NotImplementedError
nifty/minimization/relaxed_newton.py
View file @
80ff8fdc
...
...
@@ 21,26 +21,7 @@ from .line_searching import LineSearchStrongWolfe
class
RelaxedNewton
(
DescentMinimizer
):
""" A implementation of the relaxed Newton minimization scheme.
The relaxed Newton minimization exploits gradient and curvature information to
propose a step. A linesearch optimizes along this direction.
Parameter

line_searcher : LineSearch,
An implementation of a linesearch algorithm.
callback : function, *optional*
Function f(energy, iteration_number) specified by the user to print
iteration number and energy value at every iteration step. It accepts
an Energy object(energy) and integer(iteration_number). (default: None)
convergence_tolerance : float,
Specifies the required accuracy for convergence. (default : 10e4)
convergence_level : integer
Specifies the demanded level of convergence. (default : 3)
iteration_limit : integer
Limiting the maximum number of steps. (default : None)
"""
def
__init__
(
self
,
line_searcher
=
LineSearchStrongWolfe
(),
callback
=
None
,
convergence_tolerance
=
1E4
,
convergence_level
=
3
,
iteration_limit
=
None
):
...
...
@@ 53,27 +34,28 @@ class RelaxedNewton(DescentMinimizer):
self
.
line_searcher
.
prefered_initial_step_size
=
1.
def
_
get_descend_direction
(
self
,
energy
):
def
get_descend_direction
(
self
,
energy
):
""" Calculates the descent direction according to a Newton scheme.
The descent direction is determined by weighting the gradient at the
current parameter position with the inverse local curvature, provided by the
Energy object.
current parameter position with the inverse local curvature, provided
by the Energy object.
Parameters

energy : Energy
The energy object providing implementations of the to be minimized function,
its gradient and curvature
.
An instance of the Energy class which shall be minized. The
position of `energy` is used as the starting point of minization
.