A Concise Introduction

**Daniel Liberzon**
*University of Illinois at Urbana-Champaign*

This book grew out of my lecture notes for a graduate course on optimal control theory which I taught at the University of Illinois at Urbana-Champaign during the period from 2005 to 2010. While preparing the lectures, I have accumulated an entire shelf of textbooks on calculus of variations and optimal control systems. Although some of them are excellent, navigating and combining the material from so many sources was a daunting task, and I was unable to find a single text with all the features that I wanted:

APPROPRIATE PRESENTATION LEVEL. I needed a friendly introductory text accessible to graduate students who have not had extensive training in control theory or mathematical analysis; a text that is rigorous and builds a solid understanding of the subject without getting bogged down in technicalities.

LOGICAL AND NOTATIONAL CONSISTENCY AMONG TOPICS. There are intimate connections between the classical calculus of variations, the maximum principle, and the Hamilton-Jacobi-Bellman theory, which I wanted to emphasize throughout the course. Due to differences in notation and presentation style in the existing sources, it is very difficult for students (and even for instructors) to piece this material together.

PROOF OF THE MAXIMUM PRINCIPLE. The maximum principle is a centerpiece of optimal control theory which also elucidates and puts in perspective earlier developments in calculus of variations, and I felt it was important to cover its proof. This is a challenging task because a complete proof is rather long and, without adequate preparation and guidance, the effort required to understand it can seem prohibitive to most students.

HISTORICAL PERSPECTIVE. A course on optimal control gives students a unique chance to get exposed to the work of some of the greatest mathematical minds and trace the beautiful historical development of the subject, from early
calculus of variations results
to modern optimal control theory. A good text, while of course focusing on the mathematical developments, should also give students a sense of the history behind them.^{1}

MANAGEABLE SIZE. It is very useful to have a text compact enough so that all of the material in it can be covered in one semester, since very few students will take more than one course specifically on this subject.

The present book is an attempt to meet all of the above challenges (inevitably, with varying degrees of success). The comment about proving the maximum principle in class deserves some elaboration. I believe that to build the understanding necessary to correctly and effectively use the maximum principle, it is very helpful--if not essential--to master its proof. In my experience, the proof given in this book is not too difficult for students to follow. This has to do not just with the clarity and structure of the proof itself, but also with the fact that special attention is paid to preparing students for this proof earlier on in the course (as well as helping them digest it through subsequent discussions). On the other hand, due to the central role that the proof of the maximum principle plays in this book, instructors not planning to cover this proof in class will not fully benefit from adopting this text.

While some prior exposure to control theory, mathematical analysis, and optimization is certainly helpful, this book is sufficiently self-contained so that any motivated graduate student specializing in a technical subject (such as engineering or mathematics) should be able to follow it. Depending on the student's background, some supplementary reading may be necessary from time to time; the notes and references located at the end of each chapter should facilitate this process.

Chapters 1-6 form the main part of the book and can serve as the basis for a one-semester course. Depending on the pace, the instructor may also have time for some or all of the advanced topics discussed in Chapter 7. I included only topics that in my opinion directly extend and enhance the understanding of the core material. The instructor may give preference to some other areas instead, such as the important issue of state constraints in optimal control, the classical subject of stochastic optimal control and estimation, the very popular model predictive control, or the numerous applications of optimal control theory. Fortunately, good references covering these topics are readily available. It is also possible that the instructor will want to further elaborate on some aspects of the theory presented in Chapters 1-6; in this regard, the end-of-chapter notes and references may be a useful resource.

About 60 exercises are interspersed throughout the first six chapters and represent an integral part of the book. My intention is for students to start working on each exercise as soon as the corresponding material is covered. A solutions manual is available upon request for instructors.

I am grateful to my colleagues Tamer Basar, Yi Ma, and Bill Perkins who shared their valuable experiences and lecture notes from their offerings of the optimal control course at UIUC. I am also indebted to students and friends who suffered through earlier drafts of this book and provided numerous corrections and suggestions, along with stimulating discussions and much-needed encouragement; I would like to especially thank Sairaj Dhople, Michael Margaliot, Nathan Shemonski, Hyungbo Shim, Guosong Yang, and Jingjin Yu. I appreciated the helpful and thoughtful comments of the reviewers and the support and professionalism of Vickie Kearn and other editorial staff at Princeton University Press. On the artistic side, I thank Polina Ben-Sira for the very cool cover drawing, and my wife Olga not only for moral support but also for substantial help with all the figures inside the book. Finally, I am thankful to my daughter Ada who told me that I should write the book and gave me a toy tangerine which helped in the process, as well as to my little son Eduard whose arrival definitely boosted my efforts to finish the book.

I decided not to eliminate all errors from the book, and instead left several of them on purpose in undisclosed locations as a way to provide additional educational experience. To report success stories of correcting errors, please contact me at my current email address (which is easy to find on the web).

Daniel Liberzon

Champaign-Chicago-Paris