Suggested languages for you:

Americas

Europe

Problem 1

Consider the discrete-time time-invariant one dimensional system with \(X=\mathbb{R}\), transitions $$ x^{+}=x+u, $$ and control value space the nonnegative reals: $$ \mathcal{U}=\mathbb{R}_{+} . $$ With \(q(t, x, u):=u^{2}, p(x):=x^{2}, \sigma=0\), and \(\tau=3\), find \(V(t, x)\) for all \(x\) and \(t=0,1,2\) using the dynamic programming technique. Next guess a general formula for arbitrary \(t, \sigma, \tau\) and establish its validity.

Expert verified

In summary, using the dynamic programming technique for the given one-dimensional discrete-time linear system, we derived the cost-to-go function \(V(t, x)\) for \(t = 0, 1, 2\). The general formula for arbitrary values of \(t, \sigma\), and \(\tau\) is given by:
\(V(t, x) = (1 + \frac{t}{\tau - \sigma})x^2\) for \(x \ge 0\), and
\(V(t,x) = (1 - \frac{t}{\tau - \sigma})x^2\) for \(x < 0\)

What do you think about this solution?

We value your feedback to improve our textbook solutions.

- Access over 3 million high quality textbook solutions
- Access our popular flashcard, quiz, mock-exam and notes features
- Access our smart AI features to upgrade your learning

Chapter 8

Suppose that \(L_{f} V(x) \leq 0\) for all \(x\) and that \(\dot{x}=f(x)+G(x) u\) is globally stabilized by \(u=-(\nabla V(x) \cdot G(x))^{\prime}\), as in Proposition 5.9.1. Show that \(u=k(x)\) is an optimal feedback, and \(V\) is the value function, for some suitably chosen cost. (Hint: Let $Q(x):=-L_{f} V(x)+\frac{1}{2} L_{G} V(x)\left(L_{G} V(x)\right)^{\prime}$, which gives (8.64) for which \(R\) ? Use Exercise 8.5.5.)

Chapter 8

Find the value function \(V\) and the optimal feedback solution for the problem of minimizing \(\int_{0}^{\infty} x^{2}+u^{2} d t\) for the scalar system \(\dot{x}=x^{2}+u\). (Hint: The HJB equation is now an ordinary differential equation, in fact, a quadratic equation on the derivative of \(V\).)

Chapter 8

Prove, without using the Pole-Shifting Theorem: If \(\Sigma\) is a controllable time-invariant linear discrete-time system over \(\mathbb{K}=\mathbb{R}\), then there exists an \(m \times n\) real matrix \(F\) such that \(A+B F\) is a convergent matrix.

Chapter 8

Consider the case when \(B=0, Q=I\), and \(S\) approaches zero. Show that the formulas for least-squares observation in Section \(6.3\) can be recovered from the results in this section. (Hint: The equation for \(\widetilde{P}\) can be solved with final condition zero, and its solution at the initial time can be expressed directly in terms of the Wronskian \(W\).)

Chapter 8

Consider the problem of minimizing the same cost (8.42) over the set of all trajectories of the system $$ \dot{\xi}=A \xi+G \bar{u}+B \omega, $$ where \(\bar{u}\) is a known control on \([\sigma, \tau]\). Show that the minimum is achieved through the solution of $$ \dot{z}=A z+L[C z-\bar{y}]+G \bar{u}, \quad z(\sigma)=0, $$ where \(L\) is the same as before. (Hint: Simply convert the original problem into an estimation problem for \(\dot{\widetilde{\xi}}=A \xi+B \omega\), where \(\tilde{\xi}:=\xi-\bar{x}\) and \(\bar{x}\) satisfies $\dot{\bar{x}}=A \bar{x}+G \bar{u} .)$

The first learning app that truly has everything you need to ace your exams in one place.

- Flashcards & Quizzes
- AI Study Assistant
- Smart Note-Taking
- Mock-Exams
- Study Planner