1.3 Preview of infinite-dimensional optimization

In Section 1.2 we considered the problem of minimizing a function $f:\mathbb{R}^n\to\mathbb{R}$ . Now, instead of $\mathbb{R}^n$ we want to allow a general vector space $V$ , and in fact we are interested in the case when this vector space $V$ is infinite-dimensional. Specifically, $V$ will itself be a space of functions. Let us denote a generic function in $V$ by $y$ , reserving the letter $x$ for the argument of $y$ . (This $x$ will typically be a scalar, and has no relation with $x\in\mathbb{R}^n$ from the previous section.) The function to be minimized is a real-valued function on $V$ , which we now denote by $J$ . Since $J$ is a function on a space of functions, it is called a functional. To summarize, we are minimizing a functional $J: V\to \mathbb{R}$ .

Unlike in the case of $\mathbb{R}^n$ , there does not exist a ``universal" function space. Many different choices for $V$ are possible, and specifying the desired space $V$ is part of the problem formulation. Another issue is that in order to define local minima of $J$ over $V$ , we need to specify what it means for two functions in $V$ to be close to each other. Recall that in the definition of a local minimum in Section 1.2, a ball of radius $\varepsilon$ with respect to the standard Euclidean norm on $\mathbb{R}^n$ was used to define the notion of closeness. In the present case we will again employ $\varepsilon$ -balls, but we need to specify which norm we are going to use. While in $\mathbb{R}^n$ all norms are equivalent (i.e., are within a constant multiple of one another), in function spaces different choices of a norm lead to drastically different notions of closeness. Thus, the first thing we need to do is become more familiar with function spaces and norms on them.