Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
ift
NIFTy
Commits
c312b236
Commit
c312b236
authored
Jun 10, 2021
by
Jakob Knollmüller
Browse files
docs for parametric vi
parent
3f1212d2
Pipeline
#103253
passed with stages
in 15 minutes and 51 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
src/library/variational_models.py
View file @
c312b236
...
...
@@ -37,22 +37,41 @@ class MeanFieldVI:
"""Collect the operators required for Gaussian meanfield variational
inference.
Gaussian meanfield variational inference approximates some target
distribution with a Gaussian distribution with a diagonal covariance
matrix. The parameters of the approximation, in this case the mean and
standard deviation, are obtained by minimizing a stochastic estimate
of the Kullback-Leibler divergence between the target and the approximation.
In order to obtain gradients w.r.t the parameters, the reparametrization
trick is employed, which separates the stochastic part of the approximation
from a deterministic function, the generator. Samples from the approximation
are drawn by processing samples from a standard Gaussian through this generator.
Parameters
----------
position :
FIXME
hamiltonian :
FIXME
n_samples :
FIXME
mirror_samples :
FIXME
initial_sig :
FIXME
comm :
FIXME
position : Field
The initial estimate of the approximate mean parameter.
hamiltonian : Energy
Hamiltonian of the approximated probability distribution.
n_samples : int
Number of samples used to stochastically estimate the KL.
mirror_samples : bool
Whether the negative of the drawn samples are also used, as they are
equally legitimate samples. If true, the number of used samples
doubles. Mirroring samples stabilizes the KL estimate as extreme
sample variation is counterbalanced. Since it improves stability in
many cases, it is recommended to set `mirror_samples` to `True`.
initial_sig : positive Field or positive float
The initial estimate of the standard deviation.
comm : MPI communicator or None
If not None, samples will be distributed as evenly as possible
across this communicator. If `mirror_samples` is set, then a sample and
its mirror image will always reside on the same task.
nanisinf :
FIXME
If true, nan energies which can happen due to overflows in the forward
model are interpreted as inf. Thereby, the code does not crash on
these occasions but rather the minimizer is told that the position it
has tried is not sensible.
"""
def
__init__
(
self
,
position
,
hamiltonian
,
n_samples
,
mirror_samples
,
initial_sig
=
1
,
comm
=
None
,
nanisinf
=
False
):
...
...
@@ -98,24 +117,47 @@ class MeanFieldVI:
class
FullCovarianceVI
:
"""Collect the operators required for Gaussian full-covariance variational
inference.
Gaussian meanfield variational inference approximates some target
distribution with a Gaussian distribution with a diagonal covariance
matrix. The parameters of the approximation, in this case the mean and
a lower triangular matrix corresponding to a Cholesky decomposition of the covariance,
are obtained by minimizing a stochastic estimate of the Kullback-Leibler divergence
between the target and the approximation.
In order to obtain gradients w.r.t the parameters, the reparametrization
trick is employed, which separates the stochastic part of the approximation
from a deterministic function, the generator. Samples from the approximation
are drawn by processing samples from a standard Gaussian through this generator.
Note that the size of the covariance scales quadratically with the number of model
parameters.
Parameters
----------
position :
FIXME
hamiltonian :
FIXME
n_samples :
FIXME
mirror_samples :
FIXME
initial_sig :
FIXME
comm :
FIXME
position : Field
The initial estimate of the approximate mean parameter.
hamiltonian : Energy
Hamiltonian of the approximated probability distribution.
n_samples : int
Number of samples used to stochastically estimate the KL.
mirror_samples : bool
Whether the negative of the drawn samples are also used, as they are
equally legitimate samples. If true, the number of used samples
doubles. Mirroring samples stabilizes the KL estimate as extreme
sample variation is counterbalanced. Since it improves stability in
many cases, it is recommended to set `mirror_samples` to `True`.
initial_sig : positive float
The initial estimate for the standard deviation. Initially no correlation
between the parameters is assumed.
comm : MPI communicator or None
If not None, samples will be distributed as evenly as possible
across this communicator. If `mirror_samples` is set, then a sample and
its mirror image will always reside on the same task.
nanisinf :
FIXME
If true, nan energies which can happen due to overflows in the forward
model are interpreted as inf. Thereby, the code does not crash on
these occasions but rather the minimizer is told that the position it
has tried is not sensible.
"""
def
__init__
(
self
,
position
,
hamiltonian
,
n_samples
,
mirror_samples
,
initial_sig
=
1
,
comm
=
None
,
nanisinf
=
False
):
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment