Realization of Variational Divergence Minimization (VDM)

Understand the idea

What is VDM?

Variational Divergence Minimization (VDM) is a framework for training generative models by minimizing a divergence between the real-data distribution and the model-generated distribution.

The central idea is to convert a divergence, which may be difficult to compute directly, into a variational optimization objective that can be estimated from samples.

Goal: Train a generator so that its output distribution resembles the distribution of real training data.

Understand the idea

Dataset and real-data distribution

Consider a dataset:

D=\{x_1,x_2,\ldots,x_n\}

Assume the observations are independently and identically distributed:

x_i\overset{iid}{\sim}P_X

Here:

$D$ : training dataset
$x_i$ : an individual training sample
$n$ : number of samples
$P_X$ : real-data probability distribution
i.i.d.: independent and identically distributed

For example, if the dataset contains handwritten digits, $P_X$ describes the underlying distribution of real digit images.

We observe samples from $P_X$ , but generally do not know its exact probability density.

Real data follow P_X; generated data follow P_θ. The goal is to bring the generated distribution closer to the real one.

Understand the idea

Latent noise and the generator

First, sample a random latent vector:

z\sim\mathcal N(0,I)

where $0$ is the zero mean vector and $I$ is the identity covariance matrix.

A generator neural network transforms the latent vector into a synthetic observation:

\hat{x}=g_\theta(z)

Here:

$z$ : latent random vector
$g_\theta$ : generator function
$\theta$ : generator parameters, including weights and biases
$\hat{x}$ : generated sample
$P_\theta$ : distribution induced by the generator

Thus:

z\sim\mathcal N(0,I) \quad\longrightarrow\quad g_\theta(z)=\hat{x} \quad\longrightarrow\quad \hat{x}\sim P_\theta

Changing $\theta$ changes the generated samples and therefore the model distribution $P_\theta$ .

The generator maps latent noise to a sample. The scoring network receives a real or generated sample and returns a score.

Understand the idea

Original divergence minimization objective

We want the generated distribution $P_\theta$ to resemble the real distribution $P_X$ .

The mathematical objective is:

\boxed{\theta^*=\arg\min_\theta D_f(P_X\parallel P_\theta)}

Here, $D_f$ is an $f$ -divergence.

For distributions with densities $p_X$ and $p_\theta$ , under appropriate conditions:

D_f(P_X\parallel P_\theta) = \int p_\theta(x) f\left(\frac{p_X(x)}{p_\theta(x)}\right)dx

The function $f$ is convex, conventionally normalized by

f(1)=0

The notation $\arg\min_\theta$ means finding the generator parameters that minimize the divergence.

Important: The generator minimizes divergence; it does not necessarily minimize the individual output values of the scoring network.

Understand the idea

Why direct divergence minimization is difficult

The divergence formula involves the density ratio:

\frac{p_X(x)}{p_\theta(x)}

In many generative modeling problems, we can sample from the real and generated distributions but cannot evaluate both densities directly.

This makes the original divergence difficult to calculate and optimize.

The solution is to use the variational representation studied in Week 1.

Understand the idea

Variational representation of an f-divergence

Recall the convex conjugate:

f^*(t)=\sup_u\{ut-f(u)\}

Fenchel–Young inequality gives:

f(u)\ge ut-f^*(t)

Using the density ratio and taking expectations leads to:

D_f(P_X\parallel P_\theta) \ge \mathbb E_{P_X}[T(X)] - \mathbb E_{P_\theta}[f^*(T(X))]

Define the variational objective:

\boxed{ J(\theta,T)= \mathbb E_{P_X}[T(X)] - \mathbb E_{P_\theta}[f^*(T(X))] }

Here:

$T$ : an admissible scoring function
$f^*$ : convex conjugate of $f$
$\mathbb E_{P_X}$ : expectation over real samples
$\mathbb E_{P_\theta}$ : expectation over generated samples

Under appropriate conditions, maximizing over a sufficiently rich class of admissible functions recovers the divergence:

D_f(P_X\parallel P_\theta)=\sup_T J(\theta,T)

This formulation is useful because expectations can be approximated using samples.

Replace a difficult density-based divergence with a variational objective, then represent the scoring function by a neural network.

Understand the idea

Parameterizing the scoring function

Instead of searching over all possible functions $T$ , represent the scoring function with a neural network:

T_w(x)

where $w$ contains the scoring network's trainable parameters.

The practical objective becomes:

\boxed{ J(\theta,w)= \mathbb E_{P_X}[T_w(X)] - \mathbb E_{P_\theta}[f^*(T_w(X))] }

The scoring network learns a function that maximizes the variational objective for a fixed generator.

Its output is a score, not necessarily a probability.

The function must also satisfy the domain requirements of $f^*$ . The teacher introduces a suitable output activation for this in the next topic.

Understand the idea

The min–max objective

Combining generator optimization with scoring-network optimization gives:

\boxed{ \min_\theta\max_w \left[ \mathbb E_{P_X}[T_w(X)] - \mathbb E_{P_\theta}[f^*(T_w(X))] \right] }

The two networks have opposing optimization objectives.

Scoring network:

\max_w J(\theta,w)

For fixed generator parameters, find the scoring function that gives the strongest available variational lower bound.

Generator:

\min_\theta J(\theta,w)

For a fixed scoring function, update the generator to reduce the objective.

The intended overall result is to reduce the divergence between real and generated distributions.

When the scoring-function family is restricted, the maximized variational objective may be only a lower bound on the true divergence. Exact equivalence requires suitable assumptions about the function class and optimization.

The scoring network maximizes the variational objective; the generator minimizes it. The displayed objective is not a rule to minimize each individual score.

Worked example

Estimate the objective from samples

Expectations are population averages. With n real samples and m independent noise samples, replace each expectation by its own sample average. The real and generated batch sizes need not be equal.

\begin{aligned}\widehat J(\theta,w)&=\frac1n\sum_{i=1}^{n}T_w(x_i)\\&\quad-\frac1m\sum_{j=1}^{m}f^*\!\left(T_w(g_\theta(z_j))\right).\end{aligned}

Suppose the mean score on real samples is 1.5, and the mean conjugate-transformed score on generated samples is 0.6. The estimated objective is 1.5 − 0.6 = 0.9. These are illustrative evaluated averages, not raw probabilities.

Keep the scoring network fixed. If a generator update changes the second mean to 0.8, the estimate becomes 1.5 − 0.8 = 0.7. The objective decreased. Notice that the generated conjugate-transformed term increased: minimizing the difference does not mean pushing every scoring output down.

A finite-sample estimate fluctuates and need not itself be a certified lower bound on the population divergence. The variational lower-bound statement concerns the population expectations under its assumptions.

Understand the idea

Why is this adversarial optimization?

The problem is called adversarial because the two networks have opposing objectives.

The scoring network tries to expose discrepancies between real and generated distributions.
The generator tries to reduce those discrepancies.

The networks do not need to output opposite numerical scores. Their opposition comes from maximizing versus minimizing the same objective.

This leads to a min–max or saddle-point optimization problem.

The networks optimize the same objective in opposite directions: the critic exposes discrepancies and the generator reduces them.

Understand the idea

Understanding a saddle point

Consider the simple mathematical example:

J(\theta,w)=\theta^2-w^2

For $w=0$ :

J(\theta,0)=\theta^2

This is minimized at:

\theta=0

For $\theta=0$ :

J(0,w)=-w^2

This is maximized at:

w=0

Therefore:

\boxed{(\theta^*,w^*)=(0,0)}

The point is a minimum in the $\theta$ direction and a maximum in the $w$ direction.

This illustrates the mathematical meaning of a saddle point. Real neural-network objectives are more complicated and are not guaranteed to converge to such a point.

For J(θ,w) = θ² − w², the origin is a minimum in θ and a maximum in w. This toy example does not guarantee convergence for neural networks.

Understand the idea

Complete mathematical flow

There are two sampling streams, not a transformation from real observations into latent noise. Real observations come from the dataset; latent noise enters the generator. The scoring network evaluates samples from both streams.

\begin{aligned}x_i&\sim P_X &&\text{(real-data stream)}\\z_j&\sim\mathcal N(0,I),\quad \hat x_j=g_\theta(z_j)\sim P_\theta &&\text{(generated-data stream)}\end{aligned}

The original goal is to choose generator parameters that minimize the divergence. Its variational realization replaces the density calculation with expectations and a trainable scoring network.

\begin{aligned}\theta^*&\in\operatorname*{arg\,min}_\theta D_f(P_X\parallel P_\theta),\\\min_\theta\sup_w J(\theta,w)&=\min_\theta\sup_w\left\{\mathbb E_{P_X}[T_w(X)]-\mathbb E_{P_\theta}[f^*(T_w(X))]\right\}.\end{aligned}

We write sup when a best score may be approached without being attained. The usual min–max notation assumes the relevant extrema are attained. With a restricted neural-network family, the maximized objective can remain a lower bound rather than the full divergence.

Watch out

Common mistakes

Confusing $P_X$ and $P_\theta$ : $P_X$ is the real-data distribution; $P_\theta$ is the generated distribution.
Confusing $\theta$ and $w$ : $\theta$ belongs to the generator; $w$ belongs to the scoring network.
Assuming the critic minimizes: The scoring network maximizes the variational objective.
Assuming the scoring output must be zero: The goal is not to force every $T_w(x)$ to zero.
Treating a restricted variational lower bound as automatically equal to the divergence: Equality requires suitable conditions.
Confusing $\min$ and $\arg\min$ : The former returns the minimum value; the latter identifies parameters achieving it.

Understand the idea

Quick revision

VDM: Variational Divergence Minimization.

Real distribution: $P_X$ .

Noise: $z\sim\mathcal N(0,I)$ .

Generator: $\hat{x}=g_\theta(z)$ .

Generated distribution: $P_\theta$ .

Scoring network: $T_w(x)$ .

Generator objective: Minimize discrepancy.

Scoring network objective: Maximize the variational lower bound.

Overall objective:

\boxed{\min_\theta\max_w J(\theta,w)}

One-line memory anchor:

The scoring network finds the strongest available measure of the difference, while the generator learns to reduce that difference.

Next topic: Implementing VDM with neural networks, including the output activation needed to keep $T_w(x)$ in the domain of the convex conjugate.

Try it yourself

Check your understanding

What changes when the generator parameters change?

Show solution

The mapping from noise to generated observations changes, so the induced distribution $P_\theta$ may change. The real-data distribution $P_{X}$ is the target and is not trained by those updates.

Which network maximizes the objective?

Show solution

The scoring network, with parameters w, maximizes it while the generator is fixed. The generator, with parameters $\theta$ , minimizes it while the scoring network is fixed.

Does the scoring network output a probability by definition?

Show solution

No. It outputs an admissible score. Its permitted output range depends on the effective domain of the chosen convex conjugate.

Is every neural scoring network guaranteed to recover the exact divergence?

Show solution

No. A restricted function family and imperfect optimization may leave a gap. Equality requires a sufficiently rich admissible class and suitable regularity assumptions.

Real mean score = 2.0 and generated mean conjugate-transformed score = 1.3. What is the estimated objective?

Show solution

2.0 − 1.3 = 0.7. The second term is the average after applying the convex conjugate, not necessarily the average raw critic score.

In the saddle example, what happens along $\theta$ when w = 0?

Show solution

The objective is $\theta$ ², minimized at $\theta$ = 0. Along w with $\theta$ = 0, it is −w², maximized at w = 0.

Are real samples generated by passing them through the noise distribution?

Show solution

No. Real samples and latent noise are separate inputs. Noise passes through the generator to produce synthetic samples; the scoring network can evaluate both real and synthetic samples.

What is the difference between min and argmin?

Show solution

min gives the smallest objective value; argmin gives the parameter value or set of values that achieves that value. An optimum need not always be attained.