4.2.6 Key topological lemma

Up until now, we have not yet used the fact that $u^*$ is an optimal control and $y^*$ is an optimal trajectory. As discussed in Section 4.2.1 and demonstrated in Figure 4.2, optimality means that no other trajectory $y$ corresponding to another control $u$ can reach the line $S'$ (the vertical line through $\Big({\textstyle{0}\atop \textstyle{x_1}}\Big)$ in the $y$ -space) at a point below $y^*(t^*)$ . Since the terminal cone $C_{t^*}$ is a linear approximation of the set of points that we can reach by applying perturbed controls, we expect that the terminal cone should face ``upward."

To formalize this observation, consider the vector

$$\mu:=\begin{pmatrix}-1&0&\cdots&0\end{pmatrix}^T\in\mathbb{R}^{n+1}$$ (4.26)

and let $\vec\mu$ be the ray generated by this vector (which points downward) originating at $y^*(t^*)$ . Optimality suggests that $\vec\mu$ should be directed outside of $C_{t^*}$ , a situation illustrated in Figure 4.9. Since $C_{t^*}$ is only an approximation, the correct claim is actually slightly weaker.

Figure: Illustrating the statement of Lemma 4.1

In other words, $\vec\mu$ can in principle touch $C_{t^*}$ along the boundary, but it cannot lie inside it. We note that since $C_{t^*}$ is a cone, $\vec\mu$ intersects its interior if and only if all points of $\vec\mu$ except $y^*(t^*)$ are interior points of $C_{t^*}$ .

Let us see what would happen if the statement of the lemma were false and $\vec\mu$ were inside $C_{t^*}$ . By construction of the terminal cone, as explained at the end of Section 4.2.5, there would exist a (spatial plus temporal) perturbation of $u^*$ such that the terminal point of the perturbed trajectory $y$ would be given by

$$y(t_f)=y^*(t^*)+\varepsilon \beta\mu+o(\varepsilon )$$

for some (arbitrary) $\beta>0$ . Writing this out in terms of the components $(x^0,x)$ of $y$ and recalling the definition (4.26) of $\mu$ and the relation (4.6) between $x^0$ and the cost, we obtain

$$J( u) =J(u^*)-\varepsilon \beta+o(\varepsilon )$$

$$x(t_f) =x_1+o(\varepsilon )$$

where $u$ is the perturbed control that generates $y$ . Presently there is no direct contradiction with optimality of $u^*$ yet, because the terminal point $x(t_f)$ of the perturbed trajectory $x$ is different from the prescribed terminal point $x_1$ , i.e., $x$ need not hit the target set. Thus we see that although Lemma 4.1 certainly seems plausible, it is not obvious.

Let us try to build a more convincing argument in support of Lemma 4.1. If the statement of the lemma is false, then we can pick a point $\hat y$ on the ray $\vec\mu$ below $y^*(t^*)$ such that $\hat y$ is contained in $C_{t^*}$ together with a ball of some positive radius $\varepsilon$ around it; let us denote this ball by $B_\varepsilon$ . For a suitable value of $\beta>0$ , we have $\hat y=y^*(t^*)+\varepsilon \beta\mu$ . Since the points in $B_\varepsilon$ belong to $C_{t^*}$ , they are of the form (4.25) and can be written as $y^*(t^*)+\varepsilon \nu$ where the vectors $\varepsilon \nu$ are first-order perturbations of the terminal point arising from control perturbations constructed earlier. We know that the actual terminal points of trajectories corresponding to these control perturbations are given by

$$y^*(t^*)+\varepsilon \nu+o(\varepsilon ).$$ (4.27)

We denote the set of these terminal points by $\widetilde B_\varepsilon$ ; we can think of it as a ``warped" version of $B_\varepsilon$ , since it is $o(\varepsilon )$ away from $B_\varepsilon$ .

In the above discussion, $\varepsilon >0$ was fixed; we now make it tend to 0. The point $y^*(t^*)+\varepsilon \beta\mu$ , which we relabel as $\hat y_\varepsilon$ to emphasize its dependence on $\varepsilon$ , will approach $y^*(t^*)$ along the ray $\vec\mu$ as $\varepsilon \to 0$ (here $\beta$ is the same fixed positive number as in the original expression for $\hat y$ ). The ball $B_\varepsilon$ , which now stands for the ball of radius $\varepsilon$ around $\hat y_\varepsilon$ , will still belong to $C_{t^*}$ and consist of the points $y^*(t^*)+\varepsilon \nu$ for each value of $\varepsilon$ . Terminal points of perturbed state trajectories (the perturbations being parameterized by $\varepsilon$ ) will still generate a ``warped ball" $\widetilde B_\varepsilon$ consisting of points of the form (4.27). Figure 4.10 should help visualize this construction.

Figure: Proving Lemma 4.1

Since the center of $B_\varepsilon$ is on $\vec\mu$ below $y^*(t^*)$ , the radius of $B_\varepsilon$ is $\varepsilon$ , and the ``warping" is of order $o(\varepsilon )$ , for sufficiently small $\varepsilon$ the set $\widetilde B_\varepsilon$ will still intersect the ray $\vec\mu$ below $y^*(t^*)$ . But this means that there exists a perturbed trajectory $x$ which hits the desired terminal point $x_1$ with a lower value of the cost. The resulting contradiction proves the lemma.

The above claim about a nonempty intersection between $\widetilde B_\varepsilon$ and $\vec\mu$ seems intuitively obvious. The original proof of the maximum principle in [PBGM62] states that this fact is obvious, but then adds a lengthy footnote explaining that a rigorous proof can be given using topological arguments. A conceivable scenario that must be ruled out is one in which the set $\widetilde B_\varepsilon$ has a hole (or dent) in it and the ray $\vec\mu$ goes through this hole. It turns out that this is indeed impossible, thanks to continuity of the ``warping" map that transforms $B_\varepsilon$ to $\widetilde B_\varepsilon$ . In fact, it can be shown that $\widetilde B_\varepsilon$ contains, for $\varepsilon$ small enough, a ball centered at $\hat y_\varepsilon$ whose radius is of order $\varepsilon -o(\varepsilon )$ . One quick way to prove this is by applying Brouwer's fixed point theorem (which states that a continuous map from a ball to itself must have a fixed point).