0

In one part of our notes (and some books), the following expression is used :

For the generalized linear model $y=X\beta + \epsilon$, it is : $$\text{cov}(y_i,\hat{y_i})=\sigma^2h_{ii}$$

Question : How does one proceed with proving that formally ?

Rebellos
  • 21,324
  • Could you define $\hat y_i, \sigma^2,h_{ii}$? – StubbornAtom Oct 30 '18 at 18:54
  • @StubbornAtom $\hat{y_i}$ is the estimator of $y_i$ of the expected value of the linear model, $\sigma^2$ is the variance of the model of the independent value $y_i$ and $h_{ii}$ the diagonal elements of the hat matrix $\mathbf{H}$. – Rebellos Oct 30 '18 at 18:57
  • Similar question: https://math.stackexchange.com/questions/266026/what-is-operatornamecov-widehaty-y?noredirect=1&lq=1. – StubbornAtom Oct 30 '18 at 19:00

1 Answers1

2

Use $y-E(y)=\epsilon$ and $\hat y = Hy$, so $\hat y - E(\hat y)=Hy - E(Hy)=H(y-E(y))=H\epsilon$. Then $$ \operatorname{Cov}(y,\hat y) = E\left[(y-E(y))(\hat y - E(\hat y))^T\right] =E(\epsilon\epsilon^TH)=\sigma^2H $$ The $i,i$ diagonal element of the LHS is $\operatorname{Cov}(y_i, \hat y_i)$, while the $i,i$ diagonal element of the RHS is $\sigma^2 h_{i,i}$.

grand_chat
  • 38,951
  • Nicely elaborated. I also proved that $$\text{Var}(\hat{y}i) = \text{Var}\left( \sum{j=1}^{n}h_{ij}y_j \right) = \sum_{j=1}^{n} h_{ij}^2 \text{Var}(y_j) = \sigma^2h_{ii}$$ but why would it be $\text{Var}(\hat{y_i}) = \text{cov}(y_i,\hat{y_i})$ ? – Rebellos Oct 30 '18 at 19:04
  • @Rebellos The statement $\operatorname{cov}(y_i, \hat y_i)=\operatorname{cov}(\hat y_i, \hat y_i)$ is the same as the statement $\operatorname{cov}(y_i-\hat y_i, \hat y_i)=0$, i.e., the $i$th residual is uncorrelated with the $i$th fitted value. – grand_chat Oct 30 '18 at 20:51