4.4.3 Nonlinear systems, singular controls, and Lie brackets

Let us now investigate whether the preceding results can be extended beyond the class of linear control systems. Regarding the bang-bang principle cited in the previous paragraph, the hope that it might be true for general nonlinear systems is quickly shattered by the following example.

A distinguishing feature of the above example is that the function $p_2^*$ , whose sign determines the value of the optimal control $u^*$ , identically vanishes. Consequently, the Hamiltonian maximization condition alone does not give us enough information to find $u^*$ . In problems where this situation occurs on some interval of time, the optimal control on that interval is called singular, and the corresponding piece of the optimal state trajectory is called a singular arc.

Example 4.1 should not be taken to suggest, however, that we must give up hope of formulating a bang-bang principle for nonlinear systems. After all, we saw in Section 4.4.2 that even for linear systems, to be able to prove that all time-optimal controls are bang-bang we need the normality assumption. It is conceivable that the bang-bang property of time-optimal controls for certain nonlinear systems can be guaranteed under an appropriate nonlinear counterpart of that assumption.

Motivated by these remarks, our goal now is to better formalize the phenomenon of singularity--and reach a deeper understanding of its reasons--for a class of systems that includes the linear systems considered in Section 4.4.2 as well as the nonlinear system (4.58). This class is composed of nonlinear systems affine in controls, defined as

$$\dot x=f(x)+G(x)u=f(x)+\sum_{i=1}^m g_i(x)u_i$$ (4.58)

where $x\in\mathbb{R}^n$ , $u\in U\subset \mathbb{R}^m$ , $G(x)$ is an $n\times m$ matrix with columns $g_1(x),\dots, g_m(x)$ , and for the control set $U$ we again take the hypercube (4.52). The Hamiltonian for the time-optimal control problem is

$$H(x,u,p,p_0)=\Big\langle p,f(x)+\sum_{i=1}^m g_i(x)u_i\Big\rangle +p_0.$$

From the Hamiltonian maximization condition we obtain, completely analogously to Section 4.4.2, that the components $u_i^*(t)$ of the optimal control are determined by the signs of the functions $\varphi_i(t):=\langle p^*(t),g_i(x^*(t))\rangle$ . These functions of time (always associated with a specific optimal trajectory) are called the switching functions. To investigate the bang-bang property, we need to study zeros of the switching functions.

In order to simplify calculations, from this point on we assume that $m=1$ , so that the input $u$ is scalar and we have only one switching function

$$\varphi(t)=\langle p^*(t),g(x^*(t))\rangle .$$ (4.59)

The optimal control satisfies

$$u^*(t)= (\varphi(t))=\begin{cases}1 \quad&\text{ if }\ \varphi(t)>0\\ -1\quad&\text{ if }\ \varphi(t)<0\\ ?\quad&\text{ if }\ \varphi(t)=0 \end{cases}$$ (4.60)

The canonical equations are $\dot x^*=f(x^*)+g(x^*)u^*$ and

$$\dot p^*=\left.-{H}_{x}\right\vert _{*}=-\big.{\left({f}_{x}\right)^T}\big\vert _*p^*-\big.{\left({g}_{x}\right)^T}\big\vert _*p^*u^*$$

where ${f}_{x}$ and ${g}_{x}$ are the Jacobian matrices of $f$ and $g$ . Let us now compute the derivative of $\varphi$ :

$$\begin{split}\dot\varphi&=\langle \dot p^*,g(x^*)\rangle +\la... ...^*,{g}_{x}f-{f}_{x}g\right\rangle \right\vert _{*}. \end{split}$$ (4.61)

We see that $\dot \varphi$ is the inner product of $p^*$ with the vector $\left.({g}_{x}f-{f}_{x}g)\right\vert _{*}$ . Perhaps the vector field ${g}_{x}f-{f}_{x}g$ , which we have not encountered up to now, has some significant meaning?

In general, the Lie bracket of two differentiable vector fields $f$ and $g$ is defined as

$$[f,g](x):={g}_{x}(x)f(x)-{f}_{x}(x)g(x).$$

Note that the definitions of the Lie bracket for matrices (in linear algebra) and for vector fields (in differential geometry) usually follow the opposite sign conventions. The geometric meaning of the Lie bracket--which justifies its alternative name ``commutator"--is as follows (see Figure 4.16). Suppose that, starting at some point $x_0$ , we move along the vector field $f$ for $\varepsilon$ units of time, then along the vector field $g$ for $\varepsilon$ units of time, after that along $-f$ (backward along $f$ ) for $\varepsilon$ units of time, and finally along $-g$ for $\varepsilon$ units of time. It is straightforward (although quite tedious) to check that for small $\varepsilon$ the resulting motion is approximated, up to terms of higher order in $\varepsilon$ , by $\varepsilon ^2 [f,g](x_0)$ . In particular, we will return to $x_0$ if $[f,g]\equiv0$ in a neighborhood of $x_0$ , in which case we say that $f$ and $g$ commute.

Figure: Geometric interpretation of the Lie bracket

We can now write the result of the calculation (4.62) more informatively as

$$\dot \varphi(t)=\langle p^*(t),[f,g](x^*(t))\rangle .$$ (4.62)

Coupled with the law (4.61), this equation reveals a fundamental connection between Lie brackets and optimal control.

Lie brackets can help us shed light on the bang-bang property. For a singular optimal control to exist, $\varphi$ must identically vanish on some time interval. In view of (4.60) and (4.63), this can happen only if $p^*(t)$ stays orthogonal to both $g(x^*(t))$ and $[f,g](x^*(t))$ . We have seen that in time-optimal problems $p^*(t)\ne 0$ for all $t$ . Thus for planar systems ($n=2$ ) we can rule out singularity if $g$ and $[f,g]$ are linearly independent along the optimal trajectory.

If $n>2$ or $g$ and $[f,g]$ are not linearly independent, then we have to look at higher derivatives of $\varphi$ and see what it takes for them to vanish as well. Rather than differentiating $\dot \varphi$ again, let us revisit our derivation of $\dot \varphi$ and try to see a general pattern in it. Consider an arbitrary differentiable function $h:\mathbb{R}^n \to \mathbb{R}^n$ . Following the same calculation steps as in (4.62) and using the definition of the Lie bracket, we easily arrive at

$$\dfrac d{dt}\langle p^*(t),h(x^*(t))\rangle =\langle p^*(t),[f,h](x^*(t))\rangle +\langle p^*(t),[g,h](x^*(t))\rangle u^*.$$ (4.63)

The formula (4.63) for $\dot \varphi$ is recovered from this result as a special case by setting $h:=g$ which gives $[g,h]=[g,g]=0$ . Now, if we want to compute $\ddot\varphi$ , we only need to set $h:=[f,g]$ to obtain the following expression in terms of iterated Lie brackets of $f$ and $g$ :

$$\ddot \varphi(t)=\langle p^*(t),[f,[f,g]](x^*(t))\rangle +\langle p^*(t),[g,[f,g]](x^*(t))\rangle u^*.$$ (4.64)

A singular optimal control must make $\ddot\varphi$ vanish. The control

$$u^*(t)=-\dfrac{\langle p^*(t),[f,[f,g]](x^*(t))\rangle }{\langle p^*(t),[g,[f,g]](x^*(t))\rangle }$$ (4.65)

can potentially be singular if $\langle p^*,g(x^*)\rangle =\langle p^*,[f,g](x^*)\rangle \equiv 0$ . However, it should meet the magnitude constraint $\vert u^*(t)\vert\le 1$ . If we assume, for example, that the relation

$$[g,[f,g]](x)=\alpha(x)g(x)+\beta(x)[f,g](x)+\gamma(x)[f,[f,g]](x)$$

holds with $\vert\gamma(x)\vert<1$ for all $x$ , then (4.66) would not be an admissible control unless $\langle p^*,[f,[f,g]](x^*)\rangle \equiv 0$ . To investigate the possibility that this last function does vanish, we need to consider its derivative given by (4.64) with $h:=[f,[f,g]]$ , and so on.

We are now in a position to gain a better insight into our earlier observations by using the language of Lie brackets.

LINEAR SYSTEMS (SECTION ). In the single-input case, we have $f(x)=Ax$ and $g(x)=b$ . Calculating the relevant Lie brackets, we obtain $[f,g]=-Ab$ , $[f,[f,g]]=A^2b$ , $[g,[f,g]]=0$ , $[f,[f,[f,g]]]=-A^3b$ , $[g,[f,[f,g]]]=0$ , etc. A crucial consequence of linearity is that iterated Lie brackets containing two $g$ 's are 0, which makes the derivatives of the switching function $\varphi$ independent of $u$ . It is easy to see that $\varphi$ cannot vanish if the vectors $b,Ab,\dots,A^{n-1}b$ span $\mathbb{R}^n$ , which is precisely the controllability condition.

EXAMPLE  REVISITED. For the system (4.58), we have $f=\Big({\textstyle{x_2^2-1}\atop \textstyle{0}}\Big)$ , $g=\Big({\textstyle{0}\atop \textstyle{1}}\Big)$ . The first Lie bracket is

$$[f,g](x)=-\begin{pmatrix} 0 & 2x_2\\ 0 & 0 \end{pmatrix}\begin{pmatrix} 0\\ 1 \end{pmatrix}=\begin{pmatrix} -2x_2\\ 0 \end{pmatrix}.$$

On the $x_1$ -axis, where the singular optimal trajectory lives, $[f,g]$ vanishes and so $g$ and $[f,g]$ do not span $\mathbb{R}^2$ . In fact, $\langle p^*,g(x^*)\rangle =\langle p^*,[f,g](x^*)\rangle = 0$ when $p_2^*= 0$ . The next Lie bracket that we should then calculate is

$$[f,[f,g]](x)=\begin{pmatrix} 0 & -2\\ 0 & 0 \end{pmatrix}\beg... ...pmatrix} -2x_2\\ 0 \end{pmatrix}=\begin{pmatrix} 0\\ 0 \end{pmatrix}.$$

Since $u\equiv 0$ , (4.65) gives $\ddot\varphi\equiv 0$ . All higher-order Lie brackets are obviously 0, hence $\varphi\equiv 0$ . We see that all information about the singularity is indeed encoded in the Lie brackets.

It is worth noting that singular controls are not necessarily complex. The optimal control $u^*\equiv 0$ in Example 4.1 is actually quite simple. For single-input planar systems $\dot x=f(x)+g(x)u$ , $x\in\mathbb{R}^2$ , $u\in[-1,1]$ , with $f$ and $g$ real analytic, it can be shown that all time-optimal trajectories are concatenations of a finite number of ``bang" pieces (each corresponding to either $u= 1$ or $u= -1$ ) and real analytic singular arcs. It is natural to ask whether a similar claim holds for other optimal control problems in $\mathbb{R}^2$ or for time-optimal control problems in $\mathbb{R}^3$ . We are about to see that these two questions are related and that the answer to both is negative.