H-Infinity

In my graduate work, I had a fair amount of experience tuning various PID (proportional-integral-derivative) feedback controllers in the lab. Usually this just amounted to using a heuristic like the Ziegler-Nichols method or making small tweaks based on a power-spectral density for some single-input-single-output system.

I then came to learn about linear-quadratic gaussian (LQG) control and thought it was pretty cool that you can find an optimal controller for fairly complicated multi-input problems. The LQG results and especially the optimal observer (the Kalman filter) are pretty well known, but the extension of these methods to what is known as \(H_{\infty}\) control are less so. I happened to come across the book by Green and Limebeer ¹ and learned that of course one could optimize over something other than a quadratic cost. Rather than considering the square error, one can instead minimize the effect of the worst-case scenario. However, as the field of \(H_{\infty}\) control is newer than LQG control, there are fewer references with a more simplified expositions for the lay-person.

The upshot is similar to that of LQG control, where the problem can be broken into two problems of finding an optimal observer and an optimal controller (as opposed to finding both at once). And again, just as in LQG control, each optimal solution require solving a matrix Ricatti equation. However, there appear to be some caveats with this, where the pair may not exist for the specified maximum errors.

This is a collection of equations from various authors partly simplified and reduced to a similar notation so that I could possibly apply them. However, I am by no means an expert in control theory, so I can’t claim to understand the difference in formulas between the sources. In the future this may contain some toy examples with comparisons to LQG control, but for now, it’s just a collection of the same methods from different authors.

Dan Simon \(H_{\infty}\) Filter

$$ \begin{aligned} x_{k+1} &= A_k x_k + B_k w_k \cr y_k &= C_k x_k + v_k \cr z_k &= L_k x_k \end{aligned} $$ Here we consider the cost $$ J_1 = \frac{\sum_{k=1}^{K}|z_K-\hat{z}_k|_{S_k}^2}{|x_0-\hat{x}_0|_{P_0}^2 + \sum_{k=1}^{K}(|w_k|_{Q^{-1}_k}^2 + |v_k|_{R^{-1}_k}^2)} $$ Which we attempt to find the optimal estimator \(\hat{z}\) to minimize the cost given the worst case disturbances and starting position \(w_k, v_k, x_0\). However, this cannot be optimized directly, so instead we choose the performance bound $$ J_1 < \gamma^{2} $$ and optimize.

We will assume a LTI state-space so that \((A, B, C, L)\) are constant. Then the optimal observer is found by solving a discrete-algebraic-Ricatti-equation (DARE)

$$ \overline{Q} = L^T Q L $$

$$ \begin{aligned} M_k &= (I - \gamma^{-2} \overline{Q} P_k + C^T V^{-1}C P_k)^{-1} \cr K_k &= A P_k M_k C^T V^{-1} \cr P_{k+1} &= A P_k M_k A^T + B^T W B \end{aligned} $$

so that the a posteriori estimate is

$$ \hat{x}_{k+1} = A \hat{x}_k + B u_k + K_k (y_{k+1} - C \hat{x}_k) $$

$$ \hat{z}_k = L \hat{x}_k $$

Shen and Li point out²:

“It is important to note that the optimal \(H_{\infty}\) filter depends on the weighting on the estimation error in the performance criterion, i.e., the designer chooses the weighting matrices based on the performance requirements, whereas both Wiener and Kalman filters are dependent on the variance of the noises.”

This is largely material referenced from this paper³ (which had some formatting issues) and the references², ⁴.

Basar \(H_{\infty}\) Design (simplified)

\(H_{\infty}\) Filter

Compare this to the results of Basar⁵. We need to make fairly heavy use of the Woodbury Matrix Identity (WMI):

$$ (A + UCV)^{-1} = A^{-1} - A^{-1}U(C^{-1}+VA^{-1}U)^{-1}VA^{-1}~. $$ We will also need these fairly simple identities: $$ (S^{-1}+B)^{-1} = S(I+BS)^{-1} $$ and $$ (S^{-1}+B)^{-1} = (I+SB)^{-1}S~.$$

They assume a state-space model as before with a couple simplifications: $$ \begin{aligned} x_{k+1} = A_k x_k &+ B_k u_k + D_k w_k \cr y_k = C_k x_k &+ E w_k \cr z_k = H_k x_k &+ G_k u_k \cr H^{\prime}_kH_k = Q_k, ~~G^{\prime}_kG_k &= I, ~~H^{\prime}_kG_k = 0 \cr E_kE^{\prime}_k = N_k>0 &, ~~ D_kE^{\prime}_k = 0 \end{aligned} $$

And again we will consider the simplest case of an LTI-system and even further to just the infinite-horizon case. Basar gives 6.48b which they describe as a game-algebraic-Ricatti-equation (GARE): $$ S = A(I + S(C^{\prime}N^{-1}C - \gamma^{-2}Q))^{-1}SA^{\prime} + DD^{\prime} $$ Applying WMI with \(A=I, U=S, V=I, C=C^{\prime}N^{-1}C-\gamma^{-2}Q \) results in a more familiar DARE: $$ \rightarrow 0 = ASA^{\prime}-S-AS((C^{\prime}N^{-1}C-\gamma^{-2}Q)^{-1}+S)^{-1}SA^{\prime}+DD^{\prime} $$ We can easily plug this into a discrete algebraic-Ricatti equation solver to find \(S\) and the optimal observer is (6.49b) $$ \hat{x}_{k+1} = A \hat{x}_k + B u_k + A(I + S(C^{\prime}N^{-1}C-\gamma^{-2}Q))^{-1}S \left[\gamma^{-2}Q \hat{x}_k + C^{\prime}N^{-1}(y_{k+1}-C\hat{x}_k)\right] $$

The DARE is identical to that of Simon, but the optimal observer is different.

\(H_{\infty}\) Controller

The controller is found by optimizing the cost $$ J_{\gamma}(u, w) = x_{K+1}^T Q_f x_{K+1} + \sum_{k=1}^{K}(|x_k|_Q^2 + |u_k|^2 - \gamma^2|w_k|^2)~. $$

Basar⁵ tells us that the full-information controller (infinite horizon) relies on solving another GARE (equation 6.45) $$ M = A^{\prime}(M^{-1}+BB^{\prime}-\gamma^{-2}DD^{\prime})^{-1}A + Q $$ Again, we apply the WMI to put this into a more familiar DARE, with \(A=M^{-1}, U=I, V=I, C=BB^{\prime}-\gamma^{-2}DD^{\prime}\): $$ \rightarrow 0 = A^{\prime}MA-M-A^{\prime}M((BB^{\prime}-\gamma^{-2}DD^{\prime})^{-1}+M)^{-1}MA + Q $$

And the resulting controller is (6.9): $$ u_k = -B^{\prime}(M^{-1}+BB^{\prime}-\gamma^{-2}DD^{\prime})^{-1}Ax_k $$ Which can be simplified using identity 1: $$ u_k = -B^{\prime}M(I+(BB^{\prime}-\gamma^{-2}DD^{\prime})M)^{-1}Ax_k ~.$$

I don’t understand this step (because I haven’t read it), but in assuming an observer with imperfect information, Basar optimizes for something (6.41) so that $$ x^* = (I-\gamma^{-2}SM)^{-1}\hat{x} $$ and the optimal controller is then (6.53b) $$ u_k = -B^{\prime}M(I+(BB^{\prime}-\gamma^{-2}DD^{\prime})M)^{-1}A(I-\gamma^{-2}SM)^{-1}\hat{x}_k ~.$$

Basar \(H_{\infty}\) Design (full)

Basar also does us a favor and writes out the solutions for a state-space model without having performed any transformations to simplify things. They state-space model is: $$ \begin{aligned} x_{k+1} = A_k x_k &+ B_k u_k + D_k w_k \cr y_k = C_k x_k &+ E w_k \cr z_k = H_k x_k &+ G_k u_k \cr H^{\prime}_k H_k = Q_k, ~~G^{\prime}_kG_k &= R_k, ~~H^{\prime}_kG_k = P_k \cr E_kE^{\prime}_k = N_k>0 &, ~~ D_kE^{\prime}_k = L_k \end{aligned} $$

The resulting Riccati equation for the controller is (6.56) $$ \begin{aligned} M_k &= (A_k-B_kR^{-1}_kP^{\prime}_k)^{\prime}(M_{k+1}^{-1}+B_kR^{-1}_kB^{\prime}_k-\gamma^{-2}D_kD^{\prime}_k)(A_k-B_kR^{-1}_kP^{\prime}_k)^{\prime}) \cr &\qquad\qquad\qquad\qquad\qquad\qquad~~~+ Q_k-P_kR^{-1}_kP^{\prime}_k \cr \text{with} & ~~M_{K+1} = Q_f \end{aligned} $$

where the optimal state-feedback control strategy is given in eqn. 6.58 as $$ u_k = -R^{-1}_k[B^{\prime}_k(M_{k+1}^{-1}+B_kR^{-1}_kB^{\prime}_k-\gamma^{-2}D_kD^{\prime}_k)^{-1}(A_k-B_kR^{-1}_kP^{\prime}_k)^{\prime}) + P^{\prime}_k]x_k $$

Again, we’ll consider the simpler case of an infinite-horizon LTI system. We apply the WMI to the DARE $$ \begin{aligned} \rightarrow 0 &= (A-BR^{-1}P^{\prime})^{\prime}M(A-BR^{-1}P^{\prime}) - M + Q-PR^{-1}P^{\prime} \cr &- (A-BR^{-1}P^{\prime})^{\prime}M((BR^{-1}B^{\prime}-\gamma^{-2}DD^{\prime})^{-1}+M)^{-1}M(A-BR^{-1}P^{\prime})^{\prime} \end{aligned} $$ and identity 1 to the control strategy $$ u_k = -R^{-1}[B^{\prime}M(I+(BR^{-1}B^{\prime}-\gamma^{-2}DD^{\prime})M)^{-1}(A-BR^{-1}P^{\prime})^{\prime}) + P^{\prime}]x_k $$

Similarly, the Riccati equation given for the filter is (6.61) $$ \begin{aligned} S_{k+1} &= (A_k-L_kN^{-1}_kC_k)(S_k^{-1}+C^{\prime}_kN^{-1}_kC_k-\gamma^{-2}Q_k)^{-1}(A_k-L_kN^{-1}_kC_k)^{\prime} \cr &\qquad\qquad\qquad\qquad\qquad\qquad~~~+ (D_kD^{\prime}_k-L_kN^{-1}_kL^{\prime}_k) \cr \text{with} & ~~S_{1} = Q_0^{-1}~. \end{aligned} $$ We again consider the infinite-horizon LTI case and apply the WMI with \(A=S^{-1} U=I, V=I, C=C^{\prime}N^{-1}C-\gamma^{-2}Q\) to convert this to a more familiar form $$ \begin{aligned} \rightarrow 0 &= (A-LN^{-1}C)S(A-LN^{-1}C)^{\prime} - S + (DD^{\prime}-LN^{-1}L^{\prime}) \cr &- (A-LN^{-1}C)S((C^{\prime}N^{-1}C-\gamma^{-2}Q)^{-1}+S)^{-1}S(A-LN^{-1}C)^{\prime} \end{aligned} $$

The observer is given in equation (6.62) (I believe with a typo \( g_k \rightarrow G_k\)): $$ \begin{aligned} \hat{x}_{k+1} &= A \hat{x}_k + B u_k + \gamma^{-2}(A-LN^{-1}C)(I + S(C^{\prime}N^{-1}C-\gamma^{-2}Q)^{-1}SH^{\prime}\hat{z}_k \cr &+ ((A-LN^{-1}C)(I + S(C^{\prime}N^{-1}C-\gamma^{-2}Q)^{-1}SC^{\prime} + G)N^{-1}(y_{k} - C\hat{x}_k) \end{aligned} $$ $$\hat{x}_1 = 0 $$ $$ \hat{z}_k = H_k \hat{x}_k + G_k u_k $$

And the partial-information controller has the same relation $$ u_k = -R^{-1}[B^{\prime}M(I+(BR^{-1}B^{\prime}-\gamma^{-2}DD^{\prime})M)^{-1}(A-BR^{-1}P^{\prime})^{\prime}) + P^{\prime}](I-\gamma^{-2}S M)^{-1} \hat{x}_k $$

Limebeer \(H_{\infty}\) Design

The book by Green and Limebeer ¹ delegates discrete \(H_{\infty}\) control design to an appendix, and again chooses another completely different choice of notation to define a discrete-algebraic-Ricatti-equation. These come in the form of block-matrix equations that are related to their formulations of linear-fractional transformations. However, it leaves a bit to discover for the reader in terms of how to implement with a traditional solver.

They give the state-space model as: $$ \begin{aligned} x_{k+1} = A_k x_k &+ B_{1, k} w_k + B_{2, k} u_k \cr z_k = C_{1, k} x_k &+ D_{11, k} w_k + D_{12, k} u_k \cr w_k = &~ I_{l} w_k \cr \end{aligned} $$ which they simplify as $$ \begin{bmatrix} A & B \cr \overline{C} & \overline{D} \end{bmatrix}= \begin{bmatrix} A & | & B_{1} & B_{2} \cr \hline C & | & D_{11} & D_{12} \cr 0 & | & I_l & 0 \end{bmatrix}~. $$ We are looking for a controller that satisfies $$ |z|^2 - \gamma^2|w|^2 \le \epsilon |w|^2 $$ at a specified level \(\gamma^2\) for some \(\epsilon > 0\) and for all \(w\). The controller for this equation requires solving the DARE $$ X_{ k+1} = A_k^{\prime} X_{k} A_k + \overline{C}^{\prime}_k J \overline{C}_k - L^{\prime}_k R^{-1}_k L_k $$ where $$ J = \begin{bmatrix} I_p & 0 \cr 0 & -\gamma^2 I_l \end{bmatrix} $$ $$ \begin{aligned} R_k &= \overline{D}^{\prime}_kJ\overline{D}_k + B^{\prime}_k X_{k+1}B_k \cr L_k &= \overline{D}^{\prime}_kJ\overline{C}_k + B^{\prime}_k X_{k+1}A_k ~. \end{aligned} $$ I struggled a fair bit with this equation, because I wrongly believed it needed to be expanded and rearranged into a form expected for a DARE solver. In particular it appears the \(B^{\prime}_k X_{k+1}B_k\) term in the definition of \(R\) has some dimensional issues, but this is not the case. That is, if \(X\) is an \(n\times n\) matrix and \(B_1\) and \(B_2\) are \(n\times l\) and \(n\times m\) dimensional matrices, respectively, then \(B\) is \(n\times (l+m)\) and the product results in a perfectly fine \((l+m)\times(l+m)\) matrix.

The optimal full-information controller is $$ u_k = -R^{-1}_{3, k} \left[L_{2, k}~~ R_{2, k}\right]{x_k \brack w_k} $$ However, Green and Limebeer point out that this is not “a state-feedback law” since it acts on the “noise” term \(w_k\) as well. A memory-less state-feedback controller is given instead as $$ u_k = -(R_3 - R_2R_1^{-1}R_2^{\prime})^{-1} (L_2 - R_2R_1^{-1}L_1)x_k~. $$

The discrete \(H_{\infty}\) observer is developed in the same manner. The state-space model to consider is $$ \begin{aligned} x_{k+1} = A_k x_k &+ B_{1, k} w_k \cr \hat{z}_k - z_k= C_{1, k} x_k &+ D_{11, k} w_k + \hat{z}_k \cr y_k = C_{2, k} x_k &+ D_{21, k} w_k \cr \end{aligned} $$ which they simplify as $$ \begin{bmatrix} A & \widetilde{B} \cr C & \widetilde{D} \end{bmatrix}= \begin{bmatrix} A & | & B_{1} & 0 \cr \hline C_{1} & | & D_{11} & I \cr C_{2} & | & D_{21} & 0 \end{bmatrix}~. $$ We are looking for an observer that satisfies $$ |\hat{z}-z|^2 - \gamma^2|w|^2 \le \epsilon |w|^2 $$ at a specified maximum level \(\gamma^2\) for some \(\epsilon > 0\) and for all \(w\). An observer exists if a matrix \(Y_k\) satifies the DARE $$ Y_{k+1} = A_k Y_{k} A^{\prime}_k + \widetilde{B}_k \widetilde{J} \widetilde{B}^{\prime}_k - M_k S^{-1}_k M^{\prime}_k $$ where $$ \widetilde{J} = \begin{bmatrix} I_l & 0 \cr 0 & -\gamma^2 I_p \end{bmatrix} $$ $$ \begin{aligned} S_k &= \widetilde{D}_k\widetilde{J}\widetilde{D}^{\prime}_k + C_k Y_{k}C^{\prime}_k \cr M_k &= \widetilde{B}_k\widetilde{J}\widetilde{D}^{\prime}_k + A_k Y_{k}C^{\prime}_k ~. \end{aligned} $$

The observer is then $$ \begin{aligned} \hat{x}_{k+1} &= A_k \hat{x}_k + M_{2, k}S^{-1}_{3, k}(y_k - C_{2, k}\hat{x}_k) \cr \hat{z}_k &= -C_{1, k}\hat{x}_k - S_{2, k}S^{-1}_{3, k}(y_k - C_{2, k}\hat{x}_k)~. \end{aligned} $$

If the current measurement \(y_k\) isn’t immediately available to generate \(\hat{z}_k\), the filter can be modified $$ \begin{aligned} \hat{x}_{k+1} &= A \hat{x}_k + (M_{2}-M_{1}S_{1}^{-1}S_2)(S_3-S_2^{\prime}S_1^{-1}S_2)^{-1}(y_k - C_{2}\hat{x}_k) \cr \hat{z}_k &= -C_{1}\hat{x}_k~. \end{aligned} $$

Finally, if we wish to find a solution to the full measurement feedback problem, the state-space model is $$ \begin{aligned} x_{k+1} = A_k x_k &+ B_{1, k} w_k + B_{2, k} u_k \cr z_k = C_{1, k} x_k &+ D_{11, k} w_k + D_{12, k} u_k \cr y_k = C_{2, k} &+ D_{21, k} w_k \cr \end{aligned} $$

Green and Limebeer gets a little convoluted here since they are attempting to describe the whole class of possible controllers. Somehow, this boils down to solving the system $$ \begin{aligned} \begin{bmatrix} x_k \cr r_k \cr y_k \end{bmatrix} &= \begin{bmatrix} A - B_1\nabla^{-1}L_{\nabla} & B_1 V_{2 1}^{-1} & B_2 \cr V_{1 2} R_3^{-1}(L_2 - R_2\nabla^{-1}L_{\nabla}) & V_{1 2} R_3^{-1}R_2 V_{2 1}^{-1} & V_{1 2} \cr C_2 - D_{2 1}\nabla^{-1}L_{\nabla} & D_{2 1} V_{2 1}^{-1} & 0 \end{bmatrix} \begin{bmatrix} x_k \cr s_k \cr u_k \end{bmatrix} \end{aligned} $$ where $$ \begin{aligned} \nabla &= R_1 - R_2^{\prime}R_3^{-1}R_2 \cr L_{\nabla} &= L_1 - R_2^{\prime}R_3^{-1}L_2~, \end{aligned} $$ with \(R, L\) are defined above. Also, \(V_{1 2}, V_{2 1}, r_k, s_k\) are defined by $$ \begin{aligned} \begin{bmatrix} s_k \cr r_k \end{bmatrix} &= \begin{bmatrix} V_{2 1} & 0 \cr 0 & V_{1 2} \end{bmatrix} \begin{bmatrix} w_k - w_k^* \cr u_k - u_k^* \end{bmatrix}~, \end{aligned} $$ $$ \begin{aligned} V_{1 2}^{\prime}V_{1 2} &= R_3 \cr V_{2 1}^{\prime}V_{2 1} &= -\gamma^{-2}\nabla~. \end{aligned} $$ The \(B_2\) term may be ignored in this matrix, since it can simply be added back to the observer equation, and apparently the \((2,3)\) component \(V_{1 2}\) can be replaced by \(I\) since \(V_{1 2}\) is non-singular and we simply scale the control filter \(\bf{K}\)\(=V_{1 2}[V_{1 2}^{-1}\bf{K}]\) for the controller \(u = \bf{K}\)\([C_2~~ D_{2 1}]{x \brack w}\).

Again we partition the matrix $$ \begin{bmatrix} \widehat{A} & \widehat{B} \cr \widehat{C} & \widehat{D} \end{bmatrix}= \begin{bmatrix} A - B_1\nabla^{-1}L_{\nabla} & | & B_1 V_{2 1}^{-1} & 0 \cr \hline V_{1 2} R_3^{-1}(L_2 - R_2\nabla^{-1}L_{\nabla}) & | & V_{1 2} R_3^{-1}R_2 V_{2 1}^{-1} & I \cr C_2 - D_{2 1}\nabla^{-1}L_{\nabla} & | & D_{2 1} V_{2 1}^{-1} & 0 \end{bmatrix} $$ and solve a DARE $$ Z_{k+1} = \widehat{B}\widehat{J}\widehat{B}^{\prime} + \widehat{A}Z_k\widehat{A}^{\prime} - \widehat{M}\widehat{S}^{-1}\widehat{M}^{\prime} $$ with $$ \begin{aligned} \widehat{S} &= \widehat{D}\widehat{J}\widehat{D}^{\prime} + \widehat{C}Z_k\widehat{C}^{\prime} \cr \widehat{M} &= \widehat{B}\widehat{J}\widehat{D}^{\prime} + \widehat{A}Z_k\widehat{C}^{\prime} \end{aligned} $$ $$ \widehat{J} = \begin{bmatrix} I_l & 0 \cr 0 & -\gamma^2 I_m \end{bmatrix}~. $$ A controller that achieves the objective is $$ \begin{aligned} \hat{x}_{k+1} &= \widehat{A}\hat{x}_k + B_2 u_k + \widehat{M}_2 \widehat{S}_3^{-1}(y_k-\widehat{C}_2 \hat{x}_k) \cr V_{1 2}u_k &= -\widehat{C}_1\hat{x}_k - \widehat{S}_2 \widehat{S}_3^{-1}(y_k - \widehat{C}_2\hat{x}_k)~. \end{aligned} $$ The inclusion of \(V_{1 2}\) multiplying \(u_k\) is a bit confusing for implementing. For now I guess I’ll stick with the Basar formulation.