Consider the following system (with \(n=m=1\) ): \(\dot{x}=x u\), and take the problem of minimizing $$ \int_{0}^{\infty} \frac{1}{8} x^{4}+\frac{1}{2} u^{2} d t $$ among all controls making \(x(t) \rightarrow 0\) as \(t \rightarrow \infty\). Show that there is a solution, given in feedback form by a quadratic feedback \(k(x)=-c x^{2}\), for some \(c>0\).

(Infinite gain margin of LQ feedback.) As above, suppose that \(\Pi\) is a positive definite solution of the ARE and that \(Q\) is also positive definite. Pick any \(\rho \in[1 / 2, \infty)\) and let \(F:=-\rho R^{-1} B^{\prime} \Pi\). Show that the closed-loop matrix \(A_{c l}=A+B F\) is Hurwitz. The result in Lemma 5.7.18 is needed in the next proof. This states that the operator $$ \mathcal{L}: \mathbb{R}^{n \times n} \rightarrow \mathbb{R}^{n \times n}, \quad \mathcal{L}(X):=M X+X N $$ is invertible if both \(M\) and \(N\) are Hurwitz.

Suppose that \(L_{f} V(x) \leq 0\) for all \(x\) and that \(\dot{x}=f(x)+G(x) u\) is globally stabilized by \(u=-(\nabla V(x) \cdot G(x))^{\prime}\), as in Proposition 5.9.1. Show that \(u=k(x)\) is an optimal feedback, and \(V\) is the value function, for some suitably chosen cost. (Hint: Let $Q(x):=-L_{f} V(x)+\frac{1}{2} L_{G} V(x)\left(L_{G} V(x)\right)^{\prime}$, which gives (8.64) for which \(R\) ? Use Exercise 8.5.5.)

Consider the discrete-time time-invariant one dimensional system with \(X=\mathbb{R}\), transitions $$ x^{+}=x+u, $$ and control value space the nonnegative reals: $$ \mathcal{U}=\mathbb{R}_{+} . $$ With \(q(t, x, u):=u^{2}, p(x):=x^{2}, \sigma=0\), and \(\tau=3\), find \(V(t, x)\) for all \(x\) and \(t=0,1,2\) using the dynamic programming technique. Next guess a general formula for arbitrary \(t, \sigma, \tau\) and establish its validity.

If \(\Sigma\) is a controllable time-invariant linear continuous-time system over \(\mathbb{K}=\mathbb{R}\), then there exists an \(m \times n\) real matrix \(F\) such that \(A+B F\) is a Hurwitz matrix. The above Corollary is, of course, also a consequence of the Pole-Shifting Theorem, which establishes a far stronger result. Next we consider a discretetime analogue; its proof follows the same steps as in the continuous- time case.

Find the value function \(V\) and the optimal feedback solution for the problem of minimizing \(\int_{0}^{\infty} x^{2}+u^{2} d t\) for the scalar system \(\dot{x}=x^{2}+u\). (Hint: The HJB equation is now an ordinary differential equation, in fact, a quadratic equation on the derivative of \(V\).)