Suppose that \(L_{f} V(x) \leq 0\) for all \(x\) and that \(\dot{x}=f(x)+G(x) u\) is globally stabilized by \(u=-(\nabla V(x) \cdot G(x))^{\prime}\), as in Proposition 5.9.1. Show that \(u=k(x)\) is an optimal feedback, and \(V\) is the value function, for some suitably chosen cost. (Hint: Let $Q(x):=-L_{f} V(x)+\frac{1}{2} L_{G} V(x)\left(L_{G} V(x)\right)^{\prime}$, which gives (8.64) for which \(R\) ? Use Exercise 8.5.5.)

Show that, if \((A, C)\) is observable, then $\Pi:=\lim _{t \rightarrow \infty} \Pi(t)$ exists and that it satisfies the dual algebraic Riccati equation $$ A \Pi+\Pi A^{\prime}-\Pi C^{\prime} Q C \Pi+B R^{-1} B^{\prime}=0 . $$

Consider the case when \(B=0, Q=I\), and \(S\) approaches zero. Show that the formulas for least-squares observation in Section \(6.3\) can be recovered from the results in this section. (Hint: The equation for \(\widetilde{P}\) can be solved with final condition zero, and its solution at the initial time can be expressed directly in terms of the Wronskian \(W\).)

If \(\Sigma\) is a controllable time-invariant linear continuous-time system over \(\mathbb{K}=\mathbb{R}\), then there exists an \(m \times n\) real matrix \(F\) such that \(A+B F\) is a Hurwitz matrix. The above Corollary is, of course, also a consequence of the Pole-Shifting Theorem, which establishes a far stronger result. Next we consider a discretetime analogue; its proof follows the same steps as in the continuous- time case.

Consider the following system (with \(n=m=1\) ): \(\dot{x}=x u\), and take the problem of minimizing $$ \int_{0}^{\infty} \frac{1}{8} x^{4}+\frac{1}{2} u^{2} d t $$ among all controls making \(x(t) \rightarrow 0\) as \(t \rightarrow \infty\). Show that there is a solution, given in feedback form by a quadratic feedback \(k(x)=-c x^{2}\), for some \(c>0\).

(Infinite gain margin of LQ feedback.) As above, suppose that \(\Pi\) is a positive definite solution of the ARE and that \(Q\) is also positive definite. Pick any \(\rho \in[1 / 2, \infty)\) and let \(F:=-\rho R^{-1} B^{\prime} \Pi\). Show that the closed-loop matrix \(A_{c l}=A+B F\) is Hurwitz. The result in Lemma 5.7.18 is needed in the next proof. This states that the operator $$ \mathcal{L}: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n \times n}, \quad \mathcal{L}(X):=M X+X N $$ is invertible if both \(M\) and \(N\) are Hurwitz.