Variational Calculus in VAE

Given a probability distribution , we want to find an encoder that approximates the true posterior . And we resort to minimize the Kullback-Leibler divergence between them:

And can be computed as:

We introduce a Lagrange multipler , and we can write the constrained optimization problem as:

$$ $$

Use the Euler-Lagrange Equation w.r.t.

Reduce to since is not explicitly dependent on .

Therefore

We can solve the above equations and get And with normalizing constraint, we know that .

And the next step is to decompose

which is equivalent to maximizing the ELBO:

And ELBO can be expressed as:

And represent the decoder.