SECTION 05 / 8
Why the optimizing scalar becomes a function T(x)
In f-divergence the density ratio depends on the location x. Therefore the optimizing scalar t can also depend on x. Write , where T maps the data space X to the real numbers. , , and may be different, but they need not be.
The optimization variable is now an entire function T, not one fixed number. A function class is a collection of candidate functions. Restricting this class may make optimization more practical, but the supremum over a restricted class cannot exceed that over the full admissible class.
