Our primary reference on manifolds and related mathematical facts was [Arn89] (see in particular Sections 18, 34, and 37 of that book). Most of this material is also presented in [Jur96] where the maximum principle on manifolds is discussed too. Two other good sources of basic information about manifolds, on which we occasionally relied in Section 7.1, are [Arn92, Chapter 5] and [Isi95, Appendix A]; for further reading on this subject we recommend [Boo86] or [War83]. An in-depth treatment of the maximum principle on manifolds can be found in [Sus97] as well as in [AS04, Chapter 12].
A general source of information on PDEs and the method of characteristics is [Gar86]; in our presentation in Section 7.2 we drew upon Chapter 2 of that book (which also has a section on the Hamilton-Jacobi theory). References dealing specifically with the connection between the HJB equation and the canonical equations via characteristics (without going into the details of the general PDE theory) are [YZ99, Chapter 5] and [BP07, Chapter 7].
All the ingredients of our treatment of the gain are contained in [Bro70, Sections 23 and 25] and [Kha02, Section 5.3]. Our derivation of the state feedback controller follows the paper [Pet87], whereas the general output feedback controller is presented in the paper [DGKF89] as well as the book [ZDG96]. A good reference on LMIs, methods for solving them, and their role in system and control theory is [BGFB94], while [HJ85] supplies relevant technical details on Schur complements. Although we only considered linear systems for simplicity, most of the concepts described in Section 7.3 can be as naturally developed for nonlinear systems (with Hamilton-Jacobi partial differential inequalities replacing Riccati matrix inequalities); besides the already mentioned text [Kha02], this topic is discussed in much greater detail in [vdS96].
Section 7.4 is based on the paper [Sus99]. Related work is reported in the papers [GP05] and [DK08], the latter of which contains the ``stacking" argument to which we alluded at the end of Section 7.4.2. A general reference on hybrid systems is [vdSS00]. Optimal control of hybrid systems is an active research area; see, e.g., [BWEV05] and the references therein.