6.2 Infinite-horizon LQR problem

The infinite-horizon version of the LQR problem is a special case of the general infinite-horizon problem constructed in Section 5.1.3. Starting with the finite-horizon LQR problem defined in Section 6.1, we first assume that both the control system and the cost functional are time-invariant and that there is no terminal cost; this simply means that $A$ , $B$ , $Q$ , and $R$ are now constant matrices and $M=0$ . Then, we want to consider the limit as the final time $t_1$ approaches $\infty$ . The resulting problem does not directly fit into the basic problem formulation of Section 3.3 and its well-posedness needs to be proved; in particular, we do not know a priori whether the optimal cost is finite (cf. footnote 2 on page ). We did not try to settle this issue in the general context of Section 5.1.3, but we will do it here for the LQR problem. In this section we assume that the reader is familiar with basic concepts of linear system theory such as controllability and observability.

In preparation for studying the limit as $t_1\to\infty$ , let us treat the final time $t_1$ as a parameter which, unlike in Section 6.1, is no longer fixed. We want to make our notation more explicit by displaying the dependence of relevant quantities on this parameter. Specifically, from now on let us write $V^{t_1}$ for the value function and denote by $P(t,t_1)$ the solution at time $t$ of the RDE (6.14) with the boundary condition $P(t_1)=0$ . For each $t_1$ , let us also relabel the optimal control and the optimal state trajectory (passing through the given initial condition $x_0$ at time $t_0$ ) for the corresponding finite-horizon LQR problem as $u^*_{t_1}$ and $x^*_{t_1}$ , respectively. In this notation, the results of Section 6.1.3 say that

$$u^*_{t_1}(t)=-R^{-1}B^TP(t,t_1)x^*_{t_1}(t)$$ (6.23)

and

$$V^{t_1}(t,x)=x^TP(t,t_1)x.$$ (6.24)

In particular,

$$V^{t_1}(t_0,x_0)=x_0^TP(t_0,t_1)x_0$$ (6.25)

is the finite-horizon optimal cost.