     Next: 6.2.2 Infinite-horizon problem and Up: 6.2 Infinite-horizon LQR problem Previous: 6.2 Infinite-horizon LQR problem   Contents   Index

## 6.2.1 Existence and properties of the limit

We begin by making a series of observations about the behavior of as a function of , with the goal of establishing that exists (under the additional assumption of controllability) and has some interesting properties. This path will eventually lead us to a complete solution of the infinite-horizon LQR problem.

MONOTONICITY. It is not hard to see that the finite-horizon optimal cost is a monotonically nondecreasing function of the final time . Indeed, let . Using (6.25), the definition of the value function, and the standing assumptions that and , we have BOUNDEDNESS. It is not true in general that the optimal cost remains bounded as . For example, if the system is (no control) then its solutions are growing exponentials and the infinite-horizon cost is clearly unbounded. However, we now show that the finite-horizon optimal cost remains bounded as assuming that is a controllable pair. Indeed, controllability guarantees the existence of a time and a control that steers the state from at time to 0 at time . After time , set equal to 0. This control yields a state trajectory satisfying for all , and we have Since the above integral does not depend on , it provides a uniform bound for the optimal cost--a single bound that is valid for all sufficiently large , as desired. We leave the controllability assumption in force for the rest of this chapter (except for Exercise 6.5 on page where its necessity will be re-examined).

EXISTENCE OF THE LIMIT. From the previous two claims it immediately follows that has a limit as . It turns out that more is true, namely, the matrix is well defined. To see why, let us consider some specific initial conditions (we can do this because all the facts established so far are valid for arbitrary ). First, let with as defined on page for some . Then , implying that each diagonal entry of has a limit as . Next, let for some . Recalling that is symmetric (Exercise 6.2), we have , from which we can deduce that the off-diagonal entries of converge as well. We can think of as the solution of the RDE (6.14) that, starting from the zero matrix, has flown backward for infinite time and reached steady state; Figure 6.1 should help visualize this situation. PROPERTIES OF THE LIMIT. Since the RDE (6.14) is now a time-invariant differential equation, its solution actually depends only on the difference . Thus it is clear that the steady-state solution , whose existence we just established, does not depend on , i.e., it is a constant matrix. Denoting it simply by , we have (6.26)

Next, passing to the limit as on both sides of the RDE (6.14), we see that must also exist and be a constant matrix, which must then necessarily be the zero matrix. We thus conclude that is a solution of the algebraic Riccati equation (ARE) (6.27)

Conceptually, the step of passing from the RDE (6.14), which is a matrix differential equation, to the ARE (6.28), which is a static matrix equation, mirrors our earlier step of passing from the general HJB equation (5.10) to its infinite-horizon counterpart (5.19). In both cases, the time derivative is eliminated; in the present case, we are left with no derivatives at all! The ARE (6.28) can be solved analytically or numerically without difficulties. It may happen that the ARE has spurious" extra solutions other than (6.27). In this regard, it is useful to note that the matrix given by (6.27) must be symmetric positive semidefinite (because so is for each ). We can hope that the ARE has only one solution with this additional property. This is the case in the next exercise, and we will show in Section 6.2.4 that this is always the case under appropriate assumptions.      Next: 6.2.2 Infinite-horizon problem and Up: 6.2 Infinite-horizon LQR problem Previous: 6.2 Infinite-horizon LQR problem   Contents   Index
Daniel 2010-12-20