4.1.1 Basic fixed-endpoint control problem

We will label as the Basic Fixed-Endpoint Control Problem the optimal control problem from Section 3.3 with the following additional specifications: $f=f(x,u)$ and $L=L(x,u)$ , with no $t$ -argument (the control system and the running cost are time-independent); $f$ , ${f}_{x}$ , $L$ , and ${L}_{x}$ are continuous (in other words, both $f$ and $L$ satisfy the stronger set of regularity conditions from Section 3.3.1); the target set is $S=[t_0,\infty)\times\{x_1\}$ (this is a free-time, fixed-endpoint problem); and $K\equiv0$ (the terminal cost is absent). For this special problem, the maximum principle takes the following form.

Maximum Principle for the Basic Fixed-Endpoint Control Problem Let $u^*:[t_0,t_f]\to U$ be an optimal control (in the global sense) and let $x^*:[t_0,t_f]\to\mathbb{R}^n$ be the corresponding optimal state trajectory. Then there exist a function $p^*:[t_0,t_f]\to\mathbb{R}^n$ and a constant $p_0^*\le 0$ satisfying $(p_0^*,p^*(t))\ne (0,0)$ for all $t\in[t_0,t_f]$ and having the following properties:

1. $x^*$ and $p^*$ satisfy the canonical equations

   $$\begin{split}\dot x^*&={H}_{p}(x^*,u^*,p^*,p_0^*)\\ \dot p^*&=-{H}_{x}(x^*,u^*,p^*,p_0^*) \end{split}$$ (4.1)

   
   
   with the boundary conditions $x^*(t_0)=x_0$ and $x^*(t_f)=x_1$ , where the Hamiltonian $H:\mathbb{R}^n\times U\times \mathbb{R}^n\times\mathbb{R}\to\mathbb{R}$ is defined as

   $$H(x,u,p,p_0):=\langle p,f(x,u)\rangle+p_0L(x,u).$$ (4.2)

   
   

2. For each fixed $t$ , the function $u\mapsto H(x^*(t),u,p^*(t),p_0^*)$ has a global maximum at $u=u^*(t)$ , i.e.,
   $$H(x^*(t),u^*(t),p^*(t),p_0^*)\ge H(x^*(t),u,p^*(t),p_0^*)\qquad\forall\, t\in[t_0,t_f], \ \forall\,u\in U.$$

3. $H(x^*(t),u^*(t),p^*(t),p_0^*)= 0$ for all $t\in[t_0,t_f]$ .

A few clarifications are in order. First, the maximum principle, as stated here, describes necessary conditions for global optimality. However, we announced in Section 3.4.5 that one of our goals is to capture an appropriate notion of local optimality. The proof of the maximum principle will make it clear that the same conditions are indeed necessary for local optimality in the sense outlined in Section 3.4.5. We thus postpone further discussion of this issue until after the proof (see Section 4.3). Second, while the adjoint vector, or costate, $p^*$ is already familiar from Section 3.4, one difference with the necessary conditions derived using the variational approach is the presence of $p_0^*$ . This nonpositive scalar is called the abnormal multiplier. Similarly to the abnormal multiplier $\lambda_0^*$ from Section 2.5, it equals 0 in degenerate cases; otherwise $p_0^*\ne 0$ and we can recover our earlier definition (3.29) of the Hamiltonian by normalizing $(p_0^*, p^*(t))$ so that $p_0^*=-1$ (note that such scaling does not affect any of the properties listed in statement of the maximum principle). In the future, whenever the abnormal multiplier is not explicitly written, it is assumed to be equal to $-1$ . Finally, in Section 3.4.4 we saw another scenario where $H$ was constant, but the claim that $H\equiv 0$ may seem surprising. We will see later (in Section 4.3.1) that this is a special feature of free-time problems. The next exercise provides an early illustration of the usefulness of the above result.

Let us now ask ourselves how restricted the Basic Fixed-Endpoint Control Problem really is. Time-independence of $f$ and $L$ and the absence of the terminal cost do not really introduce a loss of generality. Indeed, we know from Section 3.3 that we can eliminate $t$ and $K$ from the problem formulation by introducing the extra state variable $x_{n+1}:=t$ (although this entails stronger regularity assumptions on the original right-hand side as a function of $t$ ) and passing to the new running cost $\hat L:=L+{K}_{t}+{K}_{x}\cdot f$ . On the other hand, the target set $S=[t_0,\infty)\times\{x_1\}$ is not very general, as it does not allow any flexibility in choosing the final state. This motivates us to consider the following refined problem formulation.