6.2.3 Closed-loop stability

In the previous subsection we were able to obtain a complete solution to the infinite-horizon LQR problem, under the assumption (enforced since Section 6.2.1) that the system is controllable. Here we investigate an important property of the optimal closed-loop system which will motivate us to introduce one more assumption.

An optimal control that causes the state to grow unbounded is hardly acceptable. The reason for this undesirable situation in the above example is that the cost only takes into account the control effort and does not penalize instability. It is natural to ask under what additional assumption(s) the optimal control (6.31) automatically ensures that the state converges to 0. One option is to require $Q$ to be strictly positive definite, but we will see in a moment that this would be an overkill. It is well known and easy to show (for example, via diagonalization) that every symmetric positive semidefinite matrix $Q$ can be written as

$$Q=C^TC$$

where $C$ is a $p\times n$ matrix with $p\le n$ . Introducing the auxiliary output

$$y:=Cx$$

we can rewrite the cost (6.29) as

$$J(u)=\int_{t_0}^\infty \left(y^T(t)y(t)+u^T(t)Ru(t)\right)dt.$$

Let us now assume that our system is observable through this output, i.e., assume that $(A,C)$ is an observable pair. Note that if $Q>0$ then $C=Q^{1/2}$ (matrix square root) which is $n\times n$ and nonsingular, and the observability assumption automatically holds. On the other hand, in Example 6.2 we had $Q=0$ hence $C= 0$ and the observability assumption fails.

It is not difficult to see why observability guarantees that the optimal closed-loop system is asymptotically stable. We know from (6.32) that the optimal control $u^*$ gives a bounded cost (for arbitrary $x_0$ ). This cost is

$$\begin{split}J(u^*) &=\int_{t_0}^\infty \left((y^*)^T (t)y^*(t)+ (u^* )^T(t)Ru^* (t)\right)dt \end{split}$$ (6.32)

where $y^*(t)=Cx^*(t)$ is the output along the optimal trajectory. Next, recall that $u^*$ is generated by the linear feedback law (6.31), which means that the optimal closed-loop system is a linear time-invariant system. Its solution $x^*$ is thus given by a linear combination of (complex) exponential functions of time, and the same is true about $y^*$ and $u^*$ . This observation and the fact that $R>0$ make it clear that in order for the integral in (6.33) to be bounded we must have $y^*(t)\to 0$ and $u^*(t)\to 0$ as $t\to\infty$ . It is well known that if the output and the input of an observable linear time-invariant system converge to 0, then so does the state. (One way to see this is as follows: by observability, there exists a matrix $L$ such that $A-LC$ has arbitrary desired eigenvalues with negative real parts; rewriting the system dynamics as $\dot x=(A-LC)x+Ly+Bu$ , it is easy to show that $x\to 0$ when $y,u\to 0$ .) Therefore, the optimal closed-loop system must be asymptotically--in fact, exponentially--stable.