4.3.1.3 Terminal cost

Let us now consider a situation where a terminal cost of the form $K=K(x_f)$ is present. To illustrate just one simple case, we suppose that we are dealing with a free-time, free-endpoint problem in the Mayer form, i.e., there is no running cost ($L\equiv 0$ ). We assume the function $K$ to be differentiable as many times as desired; everything else is as in the Basic Variable-Endpoint Control Problem.

Since $L\equiv 0$ , there is no need to consider the $x^0$ -coordinate. Temporal and spatial control perturbations can be used to construct the terminal cone $C_{t^*}$ as before, except that it now lives in the original $x$ -space: $C_{t^*}\subset \mathbb{R}^n$ . By optimality, and since the terminal state is free, no perturbed state trajectory can have a terminal cost lower than $K(x^*(t^*))$ , the terminal cost of the candidate optimal trajectory. Thus we expect that $K$ should not decrease along any direction in $C_{t^*}$ , a property that we can write as

$$\left\langle -{K}_{x}(x^*(t^*)),\delta\right\rangle\le 0\qquad \forall\,\delta \ \, x^*(t^*)+\delta\in C_{t^*}.$$ (4.42)

Geometrically, this means that $C_{t^*}$ lies on one side of the hyperplane passing through $x^*(t^*)$ with normal $-{K}_{x}(x^*(t^*))$ , which we henceforth assume to be a nonzero vector. A comparison of (4.42) with (4.29) unmistakably suggests that we should define

$$p^*(t^*):=-{K}_{x}(x^*(t^*)).$$ (4.43)

Indeed, we know that in a free-endpoint problem the final value of the costate should be completely constrained (to give the correct total number of boundary conditions for the system of canonical equations). With the above definition of $p^*(t^*)$ the inequality (4.42) matches (4.29), following which the adjoint equation and the analysis of the Hamiltonian

$$H(x,u,p)=\langle p,f(x,u)\rangle$$ (4.44)

are developed in the same fashion as before. The companion inequality (4.30) in our earlier proof of the maximum principle was only needed to show that $p_0^*\le 0$ ; here we do not have such an inequality, and we do not need it because there is no $p_0^*$ .

The above argument is of course not rigorous, so let us again validate our conjecture by formally reducing the present scenario to the Basic Variable-Endpoint Control Problem. A transformation that accomplishes this goal is provided by the formula

$$K(x_f)=K(x_0)+\int_{t_0}^{t_f}\left\langle {K}_{x}(x(t)),f(x(t),u(t))\right\rangle dt.$$

Ignoring $K(x_0)$ which is a known constant, we arrive at an equivalent problem in the Lagrange form with $L=\left\langle {K}_{x}, f\right\rangle$ . For this modified problem, the Hamiltonian is $\overline H=\langle \bar p,f\rangle+p_0\left\langle {K}_{x}, f\right\rangle=\left\langle \bar p+p_0{K}_{x}, f\right\rangle$ where $\bar p$ is the costate. Applying the maximum principle, we obtain the differential equation $\dot{\bar p}^*=\left.-{\overline H}_{x}\right\vert _{*}=-\big.{\left({f}_{x}\r... ...{*}-p_0^*\big.{\left({f}_{x}\right)^T}\big\vert _*\left.{K}_{x}\right\vert _{*}$ with the boundary condition $\bar p^*(t_f)=0$ . The latter tells us, by the way, that $p_0^*\ne 0$ , hence from now on we assume $p_0^*$ and $\bar p^*$ to have been normalized so that $p_0^*=-1$ . Now, matching the expression for $\overline H$ with (4.44), let us define the costate for our original problem to be

$$p^*(t):=\bar p^*(t)-{K}_{x}(x^*(t)).$$ (4.45)

Its final value is consistent with (4.43). We can check that $p^*$ defined in this way satisfies the correct canonical equation.

By construction, $H$ is maximized and equals 0 along the optimal trajectory because these properties hold for $\overline H$ . We thus conclude that the statement of the maximum principle for the present case, with respect to the Hamiltonian (4.44), is obtained from that for the Basic Variable-Endpoint Control Problem by eliminating $p_0^*$ and changing the transversality condition to $p^*(t_f)=-{K}_{x}(x^*(t_f))$ . Recall that we already saw such a boundary condition for the costate in (3.37).