1 Linear Quadratic Regulator

Consider a linear system modeled by

$\displaystyle \boldsymbol{x}(k + 1) = A\boldsymbol{x}(k) + B\boldsymbol{u}(k), \quad \boldsymbol{x}({k_0}) = {\boldsymbol{x}_0}$

where $\boldsymbol{x}(k) \in {R^n}$ and $\boldsymbol{u}(k) \in {R^m}$ . The pair $(A,\; B)$ is controllable.

The objective is to design a stabilizing linear state feedback controller $\boldsymbol{u}(k) = - K\boldsymbol{x}(k)$ which will minimize the quadratic performance index, given by,

$\displaystyle J = \sum\limits_{k = 0}^\infty ({\boldsymbol{x}^T}(k)Q\boldsymbol{x}(k) + {\boldsymbol{u}^T}(k)R\boldsymbol{u}(k))$

where, $Q = {Q^T} \ge 0$ and $R = {R^T} > 0$ . Such a controller is denoted by u*.

We first assume that a linear state feedback optimal controller exists such that the closed loop system

$\displaystyle \boldsymbol{x}(k + 1) = (A-BK)\boldsymbol{x}(k)$

is asymptotically stable.

This assumption implies that there exists a Lyapunov function $V(\boldsymbol{x}(k))= \boldsymbol{x}(k)^TP\boldsymbol{x}(k)$ for the closed loop system, for which the forward difference

$\displaystyle \Delta V(\boldsymbol{x}(k))=V(\boldsymbol{x}(k+1))-V(\boldsymbol{x}(k))$

is negative definite.

We will now use the theorem as discussed in the previous lecture which says if the controller u* is optimal, then

$\displaystyle \min_{\boldsymbol{u}}(\Delta V(\boldsymbol{x}(k)) + \boldsymbol{x}^T(k)Q \boldsymbol{x}(k) + \boldsymbol{u}^T(k)R\boldsymbol{u}(k))=0$

Now, finding an optimal controller implies that we have to find an appropriate Lyapunov function which is then used to construct the optimal controller.

Let us first find the u* that minimizes the function

$\displaystyle f = f(\boldsymbol{u}(k)) = \Delta V(\boldsymbol{x}(k)) + \boldsymbol{x}^T(k)Q \boldsymbol{x}(k) + \boldsymbol{u}^T(k)R\boldsymbol{u}(k)$

If we substitute $\Delta V$ in the above expression, we get