4.6 Notes and references for Chapter 4

The Basic Fixed-Endpoint Control Problem and Basic Variable-Endpoint Control Problem match the Special Problem 1 and Special Problem 2 in [AF66], respectively, and the statements of the maximum principle for these two problems correspond to Theorem 5-5P and Theorem 5-6P in [AF66, Section 5-15]. Our proof of the maximum principle is also heavily based on the one presented in [AF66], but there are significant differences between the two proofs. First, we substantially reorganized the proof structure of [AF66], changing the main steps and the order in which they are given. Second, we filled in some details of proving Lemmas 4.1 and 4.2 not included in [AF66]; these are taken from the proofs of Lemmas 3 and 10 in the original book [PBGM62]. Finally, as we already explained earlier, our sign convention (maximum versus minimum) is the opposite of that in [AF66]. Among other expositions of the proof of the maximum principle built around similar ideas, we note the one in [LM67] and the more modern approach of [Sus00]. Our derivation of the variational equation (step 4) proceeded similarly to the argument in [Kha02, Section 3.3].

A precise definition of the topology mentioned in Section 4.3 (uniform convergence for $x$ and $\mathcal L_1$ convergence for $u$ ), which leads to a notion of local optimality suitable for the maximum principle, can be found in [MO98, p. 23]; that book uses the terminology ``convergence in Pontryagin's sense" and ``Pontryagin minimum." The book [Vin00] works with the somewhat different concept of a $W^{1,1}$ local minimizer (defined there on pp. 287-288). Changes of variables for deriving the maximum principle for other classes of problems, including those discussed in Section 4.3.1, are thoroughly covered in [AF66]. For the reader wishing to reconstruct a proof of the maximum principle for the Mayer problem in more detail, Section 6.1 of [BP07] should be of help. In the majority of the literature the initial state is assumed to be fixed; references that do deal with variable initial states and resulting transversality conditions include [LM67] and [Vin00].

The example of Section 4.4.1 is standard and appears in many places, including [PBGM62], [BP07, Section 7.3], [AF66, Section 7-2], [Kno81], and [Son98, Chapter 10]. The last three references also discuss the bang-bang principle for linear systems derived in Section 4.4.2. When all eigenvalues of the matrix $A$ are real, a more precise result can be obtained, namely, each component of the optimal control can switch at most $n-1$ times (see, e.g., [AF66, Theorem 6-8]). For further information on synthesizing optimal (in particular, time-optimal) controls in state feedback form, see [Sus83,PS00,BP04] and the references therein. The weaker bang-bang principle mentioned at the end of Section 4.4.2 remains true for linear time-varying systems and for arbitrary compact and convex control sets $U$ , provided that by bang-bang controls one understands controls taking values in the set of extreme points of $U$ ; see, e.g., [LM67,Ces83,Kno81] (the last of which only addresses the time-varying aspect). However, the proofs become less elementary. Another advantage of $U$ being a convex polyhedron is that a bound on the number of switches (which in general depends on the length of the time interval) can be established, as explained in [Sus83, Section 8.1]. Our treatment of singular optimal controls and their connection with Lie brackets in Section 4.4.3 was inspired by the nice expositions in [Sus83] and [Sus00, Handout 5]. The characterization of time-optimal controls in the plane mentioned at the end of Section 4.4.3 is stated in [Sus83] as Theorem 8.4.1. Fuller's problem is studied in [Ful85] and in the earlier work by the same author cited in that paper. The relevant results are conveniently summarized in [Rya87], while a detailed analysis can be found in [Jur96, Chapter 10]. Fuller's phenomenon is observed in other problems too, such as controlling a Dubins car (see [AS04, Section 20.6]).

Perron's paradox and its ramifications are carefully examined in [You80]. Examples 4.3 and 4.4 are contained in [Son98, Remark 10.1.2]. Filippov's theorem is commonly attributed to [Fil88, §7, Theorem 3], although that book cites earlier works. An extra step (the so-called Filippov's Selection Lemma) is needed to pass from the differential inclusion setting of [Fil88] to that of control systems; see, e.g., [Vin00, Section 2.3]. Filippov's theorem is also discussed, among many other sources, in [BP07, Section 3.5]. Boundedness of reachable sets without the convexity assumption is shown in [LSW96, Proposition 5.1]. For linear systems, compactness of reachable sets is addressed in a more direct fashion in [Son98, Section 10.1]. Existence of optimal controls for the Mayer problem with more general target sets, as well as for the Bolza problem via a reduction to the Mayer problem, is investigated in [BP07, Chapter 5]. Our proof of Theorem 4.3 follows [Son98] and [Kno81]. This argument can be extended to nonlinear systems affine in controls; the essential details are in [Son98, Lemma 10.1.2 and Remark 10.1.10] and [Sus79, pp. 633-634]. The assumption of convexity of $U$ in Theorem 4.3 can actually be dropped, as a consequence of available bang-bang theorems for linear systems (see [BP07, Theorem 5.1.2] or [Sus83, Theorem 8.1.1]). Further results on existence of optimal controls can be found in [Kno81] and [LM67]. For an in-depth treatment of this issue we recommend the book [Ces83] which dedicates several chapters to it. The issue of controllability of linear systems with bounded controls, which is relevant to the material of Sections 4.4.2 and 4.5, is considered in [Jur96, Chapter 5] (a necessary condition for controllability is that the matrix $A$ has purely imaginary eigenvalues).

We must stress that the basic setting of this chapter and the maximum principle that we developed are far from being the most general possible. As we said earlier, we work with piecewise continuous controls rather than the larger class of measurable controls considered in [LM67], [Vin00], or [Sus00]. An analogous maximum principle is valid for Lipschitz and not necessarily $\mathcal C^1$ systems, but its derivation requires additional technical tools; see [Cla89], [Vin00], [Sus07], and the references therein. Although the classical formulation of the maximum principle relies on first-order analysis, high-order versions are also available; see [Kre77], [Bre85], and [AS04, Chapter 20]. The applicability of the maximum principle has been extended to other problems; some of these--namely, control problems on manifolds and hybrid control problems--will be touched upon in Chapter 7, while others are completely beyond the scope of this book (for example, see [YZ99] for a stochastic maximum principle). Versions of the maximum principle for discrete-time systems also exist and are surveyed in [NRV84].