4.2.9.2 Statement 3: $\left .H\right \vert _{*}\equiv 0$

The separation property (4.29) applies, in particular, to $\delta=\delta(\tau)\in C_{t^*}$ , the terminal state perturbation vector corresponding to a temporal perturbation of the control. We know from (4.9) and (4.7) that this vector is given by

$$\delta(\tau)= \begin{pmatrix} L(x^*(t^*),u^*(t^*))\\ f(x^*(t^*),u^*(t^*)) \end{pmatrix}\tau.$$

Since $\tau$ can be either positive or negative, the inequality (4.29) can be satisfied only if

$$\left\langle\begin{pmatrix} p_0^* \\ p^*(t^*) \end{pmatrix},\... ... L(x^*(t^*),u^*(t^*))\\ f(x^*(t^*),u^*(t^*)) \end{pmatrix}\right\rangle= 0.$$

By virtue of (4.8), this is equivalent to

$$H(x^*(t^*),u^*(t^*),p^*(t^*),p_0^*)=0.$$

In other words, $\left.H\right\vert _{*}$ equals 0 at the terminal time.^4.4 We need to prove that it is 0 everywhere.

Let us show that $\left.H\right\vert _{*}(\cdot)=H(x^*(\cdot),u^*(\cdot),p^*(\cdot),p_0^*)$ is a continuous function of time, even though the optimal control $u^*$ need not be continuous. The argument that follows is very similar to the one that the reader presumably used a while ago to solve Exercise 3.3 on page . Let $t$ be a point of discontinuity of $u^*$ . Of course, $x^*$ and $p^*$ are continuous everywhere. Applying the Hamiltonian maximization condition (4.35) with $b<t$ and $w=u^*(t^+)$ and making $b$ approach $t$ from the left, we have

$$H(x^*(t),u^*(t^+),p^*(t),p_0^*)\le H(x^*(t),u^*(t^-),p^*(t),p_0^*).$$

Similarly, applying (4.35) with $b>t$ and $w=u^*(t^-)$ , in the limit as $b$ approaches $t$ from the right we obtain

$$H(x^*(t),u^*(t^+),p^*(t),p_0^*)\ge H(x^*(t),u^*(t^-),p^*(t),p_0^*).$$

Thus the two quantities must actually be equal, and the continuity claim is established.

Next, let us show that the function $\left.H\right\vert _{*}$ is constant. In Section 3.4.4, in the context of the variational approach, we established this property by simply differentiating the Hamiltonian with respect to time, but here we need to be more careful because the existence of ${H}_{u}$ has not been assumed. In view of the Hamiltonian maximization condition, we can write

$$H(x^*(t),u^*(t),p^*(t),p_0^*)=m(x^*(t),p^*(t))$$

where

$$m(x,p):=\max_{u\in U}H(x,u,p,p_0^*).$$

We just saw that $m(x^*(\cdot),p^*(\cdot))$ is a continuous function of time. For an arbitrary pair of times $t$ , $t'$ , we have the inequalities

$$\begin{split}H(x^*(t'),u^*(t),p^*(t'),p_0^*)&-H(x^*(t),u^*(t)... ...t'),p^*(t'),p_0^*)&-H(x^*(t),u^*(t'),p^*(t),p_0^*). \end{split}$$ (4.36)

In view of the standing assumptions made at the beginning of Section 4.1.1 and the canonical equations (4.1), it is straightforward to show that the function $H(x^*(\cdot),u^*(\bar t),p^*(\cdot),p_0^*)$ is continuously differentiable for each fixed $\bar t\in[t_0,t^*]$ , with an upper bound on the magnitude of its derivative independent of $\bar t$ . From this fact and (4.36) we easily conclude that the function $m(x^*(\cdot),p^*(\cdot))$ is locally Lipschitz. Therefore, it is absolutely continuous, and hence differentiable for almost all $t$ (see page ). We can now study its derivative.

We have shown that the function $H(x^*(\cdot),u^*(\cdot),p^*(\cdot),p_0^*)$ equals 0 at $t=t^*$ , is continuous everywhere, and has zero derivative almost everywhere. Thus it is identically 0, as claimed. Our proof of the maximum principle for the Basic Fixed-Endpoint Control Problem is now complete. At this point the reader should also finish Exercise 4.2.