Tars.distributions package¶
Submodules¶
Tars.distributions.distribution_models module¶
-
class
Tars.distributions.distribution_models.
Bernoulli
(mean_network, given, temp=0.1, seed=1)[source]¶
-
class
Tars.distributions.distribution_models.
Beta
(alpha_network, beta_network, given, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.distribution_models.
Categorical
(mean_network, given, temp=0.1, n_dim=1, seed=1)[source]¶
-
class
Tars.distributions.distribution_models.
Dirichlet
(alpha_network, given, k, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶
-
class
Tars.distributions.distribution_models.
Distribution
(distribution, mean_network, given, seed=1, set_log_likelihood=True)[source]¶ Bases:
object
- mean_network : lasagne.layers.Layer
- The network whose outputs express the paramater of this distribution.
- given : list
This contains instances of lasagne.layers.InputLayer, which mean the conditioning variables. e.g. if given = [x,y], then the corresponding log-likehood is
-
fprop
(x, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables, which must to correspond to ‘given’.
- mean : Theano variable
- The output of this distribution.
-
log_likelihood_given_x
(samples, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test sample.
- Theano variable, shape (n_samples,)
- A log-likelihood, p(sample|x).
-
sample_given_x
(x, repeat=1, **kwargs)[source]¶ - x : list
- This contains Theano variables, which must to correspond to ‘given’.
repeat : int or thenao variable
-
class
Tars.distributions.distribution_models.
DistributionDouble
(distribution, mean_network, var_network, given, seed=1)[source]¶
-
class
Tars.distributions.distribution_models.
Gamma
(alpha_network, beta_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.distribution_models.
Gaussian
(mean_network, var_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.distribution_models.
GaussianConstantVar
(mean_network, given, var=1, seed=1)[source]¶
-
class
Tars.distributions.distribution_models.
Kumaraswamy
(a_network, b_network, given, stick_breaking=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
[Naelisnick+ 2016] Deep Generative Models with Stick-Breaking Priors
-
class
Tars.distributions.distribution_models.
Laplace
(mean_network, var_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
Tars.distributions.distribution_samples module¶
-
class
Tars.distributions.distribution_samples.
DeterministicSample
(**kwargs)[source]¶ Bases:
Tars.distributions.distribution_samples.DistributionSample
Deterministic function p(x) = f(x)
-
class
Tars.distributions.distribution_samples.
BernoulliSample
(temp=0.1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GumbelSample
Bernoulli distribution p(x) = mean^x * (1-mean)^(1-x)
-
log_likelihood
(sample, mean)[source]¶ - sample : Theano variable
- This variable means test samples which you use to estimate a test log-likelihood.
- mean : Theano variable, the output of a fully connected layer (Sigmoid)
- This variable is a reconstruction of test samples. This must have the same shape as ‘sample’.
- Theano variable, shape (n_samples,)
- A log-likelihood, which is the same meaning as a negative binary cross-entropy error.
-
-
class
Tars.distributions.distribution_samples.
CategoricalSample
(temp=0.1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.ConcreteSample
Categorical distribution p(x) = prod mean^x
-
log_likelihood
(samples, mean)[source]¶ - sample : Theano variable
- This variable means test samples which you use to estimate a test log-likelihood.
- mean : Theano variable, the output of a fully connected layer (Softmax)
- This variable is a reconstruction of test samples. This must have the same shape as ‘sample’.
- Theano variable, shape (n_samples,)
- A log-likelihood, which is the same meaning as a negative categorical cross-entropy error.
-
-
class
Tars.distributions.distribution_samples.
GaussianSample
(seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.DistributionSample
Gaussian distribution p(x) =rac{1}{sqrt{2*pi*var}} * exp{- rac{{x-mean}^2}{2*var}}
-
class
Tars.distributions.distribution_samples.
GaussianConstantVarSample
(constant_var=1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GaussianSample
-
class
Tars.distributions.distribution_samples.
LaplaceSample
(seed=1, **kwargs)[source]¶ Bases:
Tars.distributions.distribution_samples.DistributionSample
Laplace distribution p(x) =rac{1}{sqrt{2*phi}} * exp{- rac{|x-mean|}{phi}}
-
class
Tars.distributions.distribution_samples.
GumbelSample
(seed=1, **kwargs)[source]¶ Bases:
Tars.distributions.distribution_samples.DistributionSample
Gumbel distribution
-
class
Tars.distributions.distribution_samples.
ConcreteSample
(temp=0.1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GumbelSample
- Concrete distribution (Gumbel-softmax)
- https://arxiv.org/abs/1611.01144 https://arxiv.org/abs/1611.00712
-
class
Tars.distributions.distribution_samples.
BetaSample
(iter_sampling=6, rejection_sampling=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GammaSample
Beta distribution x^(alpha-1) * (1-x)^(beta-1) / B(alpha, beta)
-
class
Tars.distributions.distribution_samples.
DirichletSample
(k, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GammaSample
Dirichlet distribution x^(alpha-1) * (1-x)^(beta-1) / B(alpha, beta)
-
class
Tars.distributions.distribution_samples.
KumaraswamySample
(seed=1, **kwargs)[source]¶ Bases:
Tars.distributions.distribution_samples.DistributionSample
[Naelisnick+ 2016] Deep Generative Models with Stick-Breaking Priors Kumaraswamy distribution p(x) = a*b*x^(a-1)(1-x^a)^(b-1)
-
class
Tars.distributions.distribution_samples.
UnitGaussianSample
(seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.GaussianSample
Standard normal gaussian distribution p(x) =rac{1}{sqrt{2*pi}} * exp{- rac{x^2}{2}}
-
class
Tars.distributions.distribution_samples.
UnitBernoulliSample
(temp=0.1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.BernoulliSample
Unit bernoulli distribution
-
log_likelihood
(samples)[source]¶ - sample : Theano variable
- This variable means test samples which you use to estimate a test log-likelihood.
- mean : Theano variable, the output of a fully connected layer (Sigmoid)
- This variable is a reconstruction of test samples. This must have the same shape as ‘sample’.
- Theano variable, shape (n_samples,)
- A log-likelihood, which is the same meaning as a negative binary cross-entropy error.
-
-
class
Tars.distributions.distribution_samples.
UnitCategoricalSample
(k=1, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.CategoricalSample
Unit Categorical distribution
-
log_likelihood
(samples)[source]¶ - sample : Theano variable
- This variable means test samples which you use to estimate a test log-likelihood.
- mean : Theano variable, the output of a fully connected layer (Softmax)
- This variable is a reconstruction of test samples. This must have the same shape as ‘sample’.
- Theano variable, shape (n_samples,)
- A log-likelihood, which is the same meaning as a negative categorical cross-entropy error.
-
-
class
Tars.distributions.distribution_samples.
UnitBetaSample
(alpha=1.0, beta=1.0, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_samples.BetaSample
Unit Beta distribution
Tars.distributions.estimate_kl module¶
Tars.distributions.multiple module¶
-
class
Tars.distributions.multiple.
Concatenate
(distributions)[source]¶ Bases:
object
This distribution is used to concatenate different distributions in their feature axis. Therefore, we can handle multiple distributions as one distribution when sampling from them or estimating their log-likelihood. However, every distribution must have same given variables.
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import Concatenate, Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[x]) >>> bernoulli = Bernoulli(mean, given=[x]) >>> concat = Concatenate([gauss, bernoulli])
-
fprop
(x, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables. The number of them must be same as ‘distributions’.
-
class
Tars.distributions.multiple.
MultiDistributions
(distributions, approximate=True)[source]¶ Bases:
object
This distribution is used to stack multiple distiributions, that is p(x|z) = p(x|z1)p(z1|z2)…p(zn-1|zn). If the distributions are approximate distributions, then a corresponding stacked distribution becomes like q(z|x) = q(zn|zn-1)…q(z2|z1)q(z1|x). If the stacked distribution is conditioned on y, then the corresponding mean field approximation becomes like p(x|z,y) = p(x|z1)p(z1|z2)…p(zn-1|zn,y), or q(z|x,y) = q(zn|zn-1)…q(z2|z1)q(z1|x,y). So far, each distribution except first layer cannot have conditioned variables more than two.
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import MultiDistributions >>> from Tars.distribution import Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[x]) >>> bernoulli = Bernoulli(mean, given=[z]) >>> multi = MultiDistributions([gauss, bernoulli])
-
fprop
(x, layer_id=-1, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables.
- mean : Theano variable
- The output of this distribution.
-
log_likelihood_given_x
(samples, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test samples, such as z1, z2,…,zn.
- Theano variable, shape (n_samples,)
- log_likelihood (q) : log_q(z1|[x,y,…])+…+log_q(zn|zn-1) log_likelihood (p) : log_p(zn-1|[zn,y,…])+…+log_p(x|z1)
-
class
Tars.distributions.multiple.
MultiPriorDistributions
(distributions, prior=None)[source]¶ Bases:
Tars.distributions.multiple.MultiDistributions
p(z) = p(zn,z’n)p(zn-1|zn,z’n)…p(z1|z2).
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import MultiPriorDistributions >>> from Tars.distribution import Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[z2]) >>> bernoulli = Bernoulli(mean, given=[z1]) >>> multi = MultiPriorDistributions([gauss, bernoulli])
-
log_likelihood_given_x
(samples, add_prior=True, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test samples, such as z1, z2,…,zn.
- Theano variable, shape (n_samples,)
- log_likelihood :
- add_prior=True : log_p(zn,z’n)+log_p(zn-1|zn,z’n)+…+log_p(z2|z1) add_prior=False : log_p(zn-1|zn,z’n)+…+log_p(z2|z1)
Module contents¶
-
class
Tars.distributions.
Distribution
(distribution, mean_network, given, seed=1, set_log_likelihood=True)[source]¶ Bases:
object
- mean_network : lasagne.layers.Layer
- The network whose outputs express the paramater of this distribution.
- given : list
This contains instances of lasagne.layers.InputLayer, which mean the conditioning variables. e.g. if given = [x,y], then the corresponding log-likehood is
-
fprop
(x, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables, which must to correspond to ‘given’.
- mean : Theano variable
- The output of this distribution.
-
log_likelihood_given_x
(samples, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test sample.
- Theano variable, shape (n_samples,)
- A log-likelihood, p(sample|x).
-
sample_given_x
(x, repeat=1, **kwargs)[source]¶ - x : list
- This contains Theano variables, which must to correspond to ‘given’.
repeat : int or thenao variable
-
class
Tars.distributions.
Categorical
(mean_network, given, temp=0.1, n_dim=1, seed=1)[source]¶
-
class
Tars.distributions.
Gaussian
(mean_network, var_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.
Laplace
(mean_network, var_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.
Kumaraswamy
(a_network, b_network, given, stick_breaking=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
[Naelisnick+ 2016] Deep Generative Models with Stick-Breaking Priors
-
class
Tars.distributions.
Gamma
(alpha_network, beta_network, given, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.
Beta
(alpha_network, beta_network, given, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶ Bases:
Tars.distributions.distribution_models.DistributionDouble
-
class
Tars.distributions.
Dirichlet
(alpha_network, given, k, iter_sampling=6, rejection_sampling=True, seed=1)[source]¶
-
class
Tars.distributions.
Concatenate
(distributions)[source]¶ Bases:
object
This distribution is used to concatenate different distributions in their feature axis. Therefore, we can handle multiple distributions as one distribution when sampling from them or estimating their log-likelihood. However, every distribution must have same given variables.
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import Concatenate, Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[x]) >>> bernoulli = Bernoulli(mean, given=[x]) >>> concat = Concatenate([gauss, bernoulli])
-
fprop
(x, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables. The number of them must be same as ‘distributions’.
-
class
Tars.distributions.
MultiDistributions
(distributions, approximate=True)[source]¶ Bases:
object
This distribution is used to stack multiple distiributions, that is p(x|z) = p(x|z1)p(z1|z2)…p(zn-1|zn). If the distributions are approximate distributions, then a corresponding stacked distribution becomes like q(z|x) = q(zn|zn-1)…q(z2|z1)q(z1|x). If the stacked distribution is conditioned on y, then the corresponding mean field approximation becomes like p(x|z,y) = p(x|z1)p(z1|z2)…p(zn-1|zn,y), or q(z|x,y) = q(zn|zn-1)…q(z2|z1)q(z1|x,y). So far, each distribution except first layer cannot have conditioned variables more than two.
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import MultiDistributions >>> from Tars.distribution import Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[x]) >>> bernoulli = Bernoulli(mean, given=[z]) >>> multi = MultiDistributions([gauss, bernoulli])
-
fprop
(x, layer_id=-1, *args, **kwargs)[source]¶ - x : list
- This contains Theano variables.
- mean : Theano variable
- The output of this distribution.
-
log_likelihood_given_x
(samples, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test samples, such as z1, z2,…,zn.
- Theano variable, shape (n_samples,)
- log_likelihood (q) : log_q(z1|[x,y,…])+…+log_q(zn|zn-1) log_likelihood (p) : log_p(zn-1|[zn,y,…])+…+log_p(x|z1)
-
class
Tars.distributions.
MultiPriorDistributions
(distributions, prior=None)[source]¶ Bases:
Tars.distributions.multiple.MultiDistributions
p(z) = p(zn,z’n)p(zn-1|zn,z’n)…p(z1|z2).
- distributions : list
- Contain multiple distributions.
>>> from Tars.distribution import MultiPriorDistributions >>> from Tars.distribution import Gaussian, Bernoulli >>> gauss = Gaussian(mean, var, given=[z2]) >>> bernoulli = Bernoulli(mean, given=[z1]) >>> multi = MultiPriorDistributions([gauss, bernoulli])
-
log_likelihood_given_x
(samples, add_prior=True, **kwargs)[source]¶ - samples : list
- This contains ‘x’, which has Theano variables, and test samples, such as z1, z2,…,zn.
- Theano variable, shape (n_samples,)
- log_likelihood :
- add_prior=True : log_p(zn,z’n)+log_p(zn-1|zn,z’n)+…+log_p(z2|z1) add_prior=False : log_p(zn-1|zn,z’n)+…+log_p(z2|z1)