4.1.2 Basic variable-endpoint control problem

The Basic Variable-Endpoint Control Problem is the same as the Basic Fixed-Endpoint Control Problem except the target set is now of the form $S=[t_0,\infty)\times S_1$ , where $S_1$ is a $k$ -dimensional surface in $\mathbb{R}^n$ for some nonnegative integer $k\le n$ . As in Section 1.2.2^4.1 we define such a surface via equality constraints:

$$S_1=\{x\in\mathbb{R}^n: h_1(x)=h_2(x)=\dots=h_{n-k}(x)=0\}$$

where $h_i$ , $i=1, \dots, n-k$ are $\mathcal C^1$ functions from $\mathbb{R}^n$ to $\mathbb{R}$ . We also assume that every $x\in S_1$ is a regular point. As two extreme special cases, for $k=n$ we obtain $S_1=\mathbb{R}^n$ (which gives a free-time, free-endpoint problem) while for $k=0$ the surface $S_1$ reduces either to a single point (as in the Basic Fixed-Endpoint Control Problem) or to a set consisting of isolated points. The difference between the maximum principle for this problem and the previous one lies only in the boundary conditions for the system of canonical equations.

Maximum Principle for the Basic Variable-Endpoint Control Problem Let $u^*:[t_0,t_f]\to U$ be an optimal control and let $x^*:[t_0,t_f]\to\mathbb{R}^n$ be the corresponding optimal state trajectory. Then there exist a function $p^*:[t_0,t_f]\to\mathbb{R}^n$ and a constant $p_0^*\le 0$ satisfying $(p_0^*,p^*(t))\ne (0,0)$ for all $t\in[t_0,t_f]$ and having the following properties:

1. $x^*$ and $p^*$ satisfy the canonical equations (4.1) with respect to the Hamiltonian (4.2) with the boundary conditions $x^*(t_0)=x_0$ and $x^*(t_f)\in S_1$ .

2. $H(x^*(t),u^*(t),p^*(t),p_0^*)\ge H(x^*(t),u,p^*(t),p_0^*)$ for all $t\in[t_0,t_f]$ and all $u\in U$ .

3. $H(x^*(t),u^*(t),p^*(t),p_0^*)= 0$ for all $t\in[t_0,t_f]$ .

4. The vector $p^*(t_f)$ is orthogonal to the tangent space to $S_1$ at $x^*(t_f)$ :

   $$\langle p^*(t_f),d\rangle=0 \qquad \forall\,d\in T_{x^*(t_f)}S_1.$$ (4.3)

   
   

The additional necessary condition (4.3) is called the transversality condition (we encountered its loose analog in Example 2.4 and Exercise 2.6 in Section 2.3.5). We know from Section 1.2.2 that the tangent space can be characterized as

$$T_{x^*(t_f)}S_1=\left\{d\in\mathbb{R}^n:\left\langle \nabla h_i (x^*(t_f)),d\right\rangle=0,\ i=1,\dots,n-k\right\}$$ (4.4)

and that (4.3) is equivalent to saying that $p^*(t_f)$ is a linear combination of the gradient vectors $\nabla {h_i} (x^*(t_f))$ , $i=1, \dots, n-k$ . Note that when $k=n$ and hence $S_1=\mathbb{R}^n$ , the transversality condition reduces to $p^*(t_f)=0$ (because the tangent space is the entire $\mathbb{R}^n$ ). On the other hand, the previous version of the maximum principle is a special case of the present result: when $S_1=\{x_1\}$ , its tangent space is 0 and (4.3) is true for all $p^*(t_f)$ . In general, here as well as in the Basic Fixed-Endpoint Control Problem, we have $n$ boundary conditions imposed on $(x^*,p^*)$ at $t=t_0$ and $n$ more at $t=t_f$ . This gives the correct total number of boundary conditions to specify a solution of the $2n$ -dimensional system (4.1). However, in the Basic Fixed-Endpoint Control Problem $x^*(t_f)$ was fixed and $p^*(t_f)$ was free, while here we have $k$ degrees of freedom for $x^*(t_f)\in S_1$ but only $n-k$ degrees of freedom for $p^*(t_f)\perp T_{x^*(t_f)} S_1$ . We see that the freer the state, the less free the costate: each additional degree of freedom for $x^*(t_f)$ eliminates one degree of freedom for $p^*(t_f)$ .

The maximum principle was developed by the Pontryagin school in the Soviet Union in the late 1950s. It was presented to the wider research community at the first IFAC World Congress in Moscow in 1960 and in the celebrated book [PBGM62]. It is worth reflecting that the developments we have covered so far in this book--starting from the Euler-Lagrange equation, continuing to the Hamiltonian formulation, and culminating in the maximum principle--span more than 200 years. The progress made during this time period is quite remarkable, yet the origins of the maximum principle are clearly traceable to the early work in calculus of variations.