Up until now, we have not yet used the fact that is an optimal control and is an optimal trajectory. As discussed in Section 4.2.1 and demonstrated in Figure 4.2, optimality means that no other trajectory corresponding to another control can reach the line (the vertical line through in the -space) at a point below . Since the terminal cone is a linear approximation of the set of points that we can reach by applying perturbed controls, we expect that the terminal cone should face ``upward."
To formalize this observation, consider the vector
In other words, can in principle touch along the boundary, but it cannot lie inside it. We note that since is a cone, intersects its interior if and only if all points of except are interior points of .
Let us see what would happen if the statement of the lemma were false and were inside . By construction of the terminal cone, as explained at the end of Section 4.2.5, there would exist a (spatial plus temporal) perturbation of such that the terminal point of the perturbed trajectory would be given by
for some (arbitrary) . Writing this out in terms of the components of and recalling the definition (4.26) of and the relation (4.6) between and the cost, we obtain
Let us try to build a more convincing argument in support of Lemma 4.1. If the statement of the lemma is false, then we can pick a point on the ray below such that is contained in together with a ball of some positive radius around it; let us denote this ball by . For a suitable value of , we have . Since the points in belong to , they are of the form (4.25) and can be written as where the vectors are first-order perturbations of the terminal point arising from control perturbations constructed earlier. We know that the actual terminal points of trajectories corresponding to these control perturbations are given by
In the above discussion, was fixed; we now make it tend to 0. The point , which we relabel as to emphasize its dependence on , will approach along the ray as (here is the same fixed positive number as in the original expression for ). The ball , which now stands for the ball of radius around , will still belong to and consist of the points for each value of . Terminal points of perturbed state trajectories (the perturbations being parameterized by ) will still generate a ``warped ball" consisting of points of the form (4.27). Figure 4.10 should help visualize this construction.
Since the center of is on below , the radius of is , and the ``warping" is of order , for sufficiently small the set will still intersect the ray below . But this means that there exists a perturbed trajectory which hits the desired terminal point with a lower value of the cost. The resulting contradiction proves the lemma.
The above claim about a nonempty intersection between and seems intuitively obvious. The original proof of the maximum principle in [PBGM62] states that this fact is obvious, but then adds a lengthy footnote explaining that a rigorous proof can be given using topological arguments. A conceivable scenario that must be ruled out is one in which the set has a hole (or dent) in it and the ray goes through this hole. It turns out that this is indeed impossible, thanks to continuity of the ``warping" map that transforms to . In fact, it can be shown that contains, for small enough, a ball centered at whose radius is of order . One quick way to prove this is by applying Brouwer's fixed point theorem (which states that a continuous map from a ball to itself must have a fixed point).