In the previous subsection we were able to obtain a complete solution to the infinite-horizon LQR problem, under the assumption (enforced since Section 6.2.1) that the system is controllable. Here we investigate an important property of the optimal closed-loop system which will motivate us to introduce one more assumption.
An optimal control that causes the state to grow unbounded is hardly acceptable. The reason for this undesirable situation in the above example is that the cost only takes into account the control effort and does not penalize instability. It is natural to ask under what additional assumption(s) the optimal control (6.31) automatically ensures that the state converges to 0. One option is to require to be strictly positive definite, but we will see in a moment that this would be an overkill. It is well known and easy to show (for example, via diagonalization) that every symmetric positive semidefinite matrix can be written as
where is a matrix with . Introducing the auxiliary output
we can rewrite the cost (6.29) as
Let us now assume that our system is observable through this output, i.e., assume that is an observable pair. Note that if then (matrix square root) which is and nonsingular, and the observability assumption automatically holds. On the other hand, in Example 6.2 we had hence and the observability assumption fails.
It is not difficult to see why observability guarantees that the optimal closed-loop system is asymptotically stable. We know from (6.32) that the optimal control gives a bounded cost (for arbitrary ). This cost is