3 GLS

3.1 Introduction

Since heteroscedasticity and serial correlation affect both linear and nonlinear regression models in the same way, there is no harm in limiting our attention to the simpler linear case.

$$ where $\bf \Omega$, the variance-covariance matrix of the error term, is a positive definite $n \times n$ matrix. If $\bf \Omega$ is equal to $\sigma^2 \bf I$, then it is just the linear regression model without heteroscedasticity and serial correlation. If $\bf \Omega$ is diagonal with nonconstant diagonal elements, then the error terms are still uncorrelated, but there is heteroscedasticity. If $\bf \Omega$ is not diagonal, then $u_i$ and $u_j$ are correlated. In next section, we obtain an efficient estimator for the vector $\beta$ by transforming the regression so that it satisfies the conditions of Gauss-Markov Theorem. This efficient estimator is called the Generalized Least Squares, or GLS, estimator.

3.2 The GLS Estimator

Since $\bf \Omega$, the variance-covariance matrix of the error term, is a positive definite $n \times n$ matrix, there always exist full-rank $n \times n$ matrices (usually triangular) $\bf \Psi$ such that \[ \bf \Omega^{-1} =\Psi \Psi'. \]

Premultiplying (the equation above) by $\bf \Psi'$ gives \[ \bf \Psi'y=\Psi'X\beta+\Psi'u. \]

The OLS estimator from regression (the equation above) is \[ \bf \hat \beta_{GLS}=(X'\Psi \Psi' X)^{-1}X'\Psi \Psi' y=(X'\Omega^{-1} X)^{-1}X'\Omega^{-1} y \]

The transformed error term has an identity variance-covariance matrix: \[ \rm E \bf (\Psi' u u' \Psi)=\Psi' {\rm E} (u u') \Psi= \Psi' \Omega \Psi=I \]

Since the transformed model satisfies OLS assumptions, we have

\[ \begin{aligned} {\rm Var} \bf (\hat \beta_{GLS}) &= \bf (X' \Psi \Psi'X)^{-1}\\ &= \bf (X' \Omega^{-1} X)^{-1} \end{aligned} \]

3.3 Weighted Least Squares

It is easy to obtain GLS estimates when the error terms are heteroscedastic but uncorrelated, which means $\bf \Omega$ is diagonal. Let $\omega_t^2$ denote the $t$th diagonal element of $\bf \Omega$. Then $\bf \Psi$ can be chosen as the diagonal matrix with $t$th diagonal element $\omega_t^{-1}$. For a typical observation, regression can be written as $$ _t^{-1}y_t=_t{-1} {}+ _t^{-1} u_t.

$$ This is called weighted least squares, or WLS. The weight given to each observation is $\omega_t^{-1}$. Observations of which variance of the error term is large are given low weights, and observations for which it is small are given high weights. ### Feasible Generalized Least Squares

In many cases it is reasonable to suppose that $\bf \Omega$ depends in a known way on a vector of unknown parameters $\bf \gamma$. If so, it may be possible to estimate $\mathbf{\gamma}$ consistently, so as to obtain $\bf \Omega (\hat \gamma)$. This type of procedure is called feasible generalized least squares, or feasible GLS. But we’ll have to specify the error term as some function of some known variables.

# GLS ## Introduction Since heteroscedasticity and serial correlation affect both linear and nonlinear regression models in the same way, there is no harm in limiting our attention to the simpler linear case. $$ \bf y=X\beta+u, \quad {\rm E} (u)=0, \quad {\rm E} (u u')=\Omega, $$ where $\bf \Omega$, the variance-covariance matrix of the error term, is a positive definite $n \times n$ matrix. If $\bf \Omega$ is equal to $\sigma^2 \bf I$, then it is just the linear regression model without heteroscedasticity and serial correlation. If $\bf \Omega$ is diagonal with nonconstant diagonal elements, then the error terms are still uncorrelated, but there is heteroscedasticity. If $\bf \Omega$ is not diagonal, then $u_i$ and $u_j$ are correlated. In next section, we obtain an efficient estimator for the vector $\beta$ by transforming the regression so that it satisfies the conditions of Gauss-Markov Theorem. This efficient estimator is called the Generalized Least Squares, or GLS, estimator. ## The GLS Estimator Since $\bf \Omega$, the variance-covariance matrix of the error term, is a positive definite $n \times n$ matrix, there always exist full-rank $n \times n$ matrices (usually triangular) $\bf \Psi$ such that $$ \bf \Omega^{-1} =\Psi \Psi'. $$ Premultiplying (the equation above) by $\bf \Psi'$ gives $$ \bf \Psi'y=\Psi'X\beta+\Psi'u. $$ The OLS estimator from regression (the equation above) is $$ \bf \hat \beta_{GLS}=(X'\Psi \Psi' X)^{-1}X'\Psi \Psi' y=(X'\Omega^{-1} X)^{-1}X'\Omega^{-1} y $$ The transformed error term has an identity variance-covariance matrix: $$ \rm E \bf (\Psi' u u' \Psi)=\Psi' {\rm E} (u u') \Psi= \Psi' \Omega \Psi=I $$ Since the transformed model satisfies OLS assumptions, we have $$ \begin{aligned} {\rm Var} \bf (\hat \beta_{GLS}) &= \bf (X' \Psi \Psi'X)^{-1}\\ &= \bf (X' \Omega^{-1} X)^{-1} \end{aligned} $$ ## Weighted Least Squares It is easy to obtain GLS estimates when the error terms are heteroscedastic but uncorrelated, which means $\bf \Omega$ is diagonal. Let $\omega_t^2$ denote the $t$th diagonal element of $\bf \Omega$. Then $\bf \Psi$ can be chosen as the diagonal matrix with $t$th diagonal element $\omega_t^{-1}$. For a typical observation, regression can be written as $$ \omega_t^{-1}y_t=\omega_t^{-1} {\bf X_t \beta}+ \omega_t^{-1} u_t. $$ This is called weighted least squares, or WLS. The weight given to each observation is $\omega_t^{-1}$. Observations of which variance of the error term is large are given low weights, and observations for which it is small are given high weights. ### Feasible Generalized Least Squares In many cases it is reasonable to suppose that $\bf \Omega$ depends in a known way on a vector of unknown parameters $\bf \gamma$. If so, it may be possible to estimate $\mathbf{\gamma}$ consistently, so as to obtain $\bf \Omega (\hat \gamma)$. This type of procedure is called feasible generalized least squares, or feasible GLS. But we'll have to specify the error term as some function of some known variables.