5.4 Notes and references for Chapter 5

The treatment of the discrete problem in Section 5.1.1 is based on [Son98, Section 8.1]. The material of Sections 5.1.2-5.1.4 is quite standard; it is assembled from the texts [AF66,BP07,LM67,Son98,YZ99] which further elaborate on some aspects of this theory. The brief historical remarks of Section 5.1.5 are compiled largely from [YZ99]; for additional reading on how the subject has evolved we recommend [Bry96] and [PB94] (the latter paper is more technical and explains how Carathéodory's work was, along with subsequent work of Hestenes, also a precursor of the maximum principle). A derivation of the Hamilton-Jacobi PDE as a necessary condition for optimality in the context of calculus of variations (via the general formula for the variation of a functional), as well as an explanation of its connection to Hamilton's canonical equations, can be found in [GF63, Section 23]; see also [YZ99, pp. 222-223].

Our discussion of the relationship between the HJB equation and the maximum principle in Section 5.2 follows [YZ99, pp. 229-230] and [Vin00, Section 1.6]. The economic interpretation of the adjoint vector and the value function is developed in more detail in [YZ99, pp. 231-232]; similar ideas appear in [Lue84, Section 4.4]. The example of Section 5.2.1 is Example 2.3 in [YZ99, Chapter 4], where it is proved that the corresponding HJB equation admits no solution. The same example is also studied in [Vin00, Section 1.7], and a different example with similar features is Example 7.2 in [BP07]. For results showing--under different assumptions--that the value function is locally Lipschitz, see Theorem 2.5 in [YZ99, Chapter 4], Lemma 8.6.2 in [BP07], or Proposition 12.3.5 in [Vin00].

Section 5.3 is heavily based on the exposition in [BP07, Chapter 8]. The original reference on viscosity solutions is [CL83]. The convergence result sketched at the end of Section 5.3.2 is Theorem 8.4.2 in [BP07]; for background on the motion of viscous fluids the reader can consult [FLS63, Chapter II-41]. A complete proof of the main result in Section 5.3.3, again under different technical assumptions, can be found in [BP07] and [YZ99]. Other generalized solution concepts for HJB equations have been proposed; see the notes and references in [YZ99, Chapter 4] and [Vin00, Chapter 12]. For an in-depth study of HJB equations and their viscosity solutions, including numerical methods, see [BCD97].