1

If $S'^2 = \dfrac{\sum_{i=1}^n (Y_i - \bar{Y})^2}{n}$ and $S^2 = \dfrac{\sum_{i=1}^n (Y_i - \bar{Y})^2}{n-1}$

then $S^{'2}$ is a biased estimator of $σ^2$, but $S^2$ is an unbiased estimator of the same parameter. we sample from a normal population.

$S^2$ and $S^{'2}$ are two estimators for $σ^2$ that are of the form $c\sum_{i=1}^n (Y_i − \bar{Y} )^2$. What value for $c$ yields the estimator for $\sigma^2$ with the smallest mean square error among all estimators of the form $c\sum_{i=1}^n (Y_i − \bar{Y} )^2$ ?

I first set $\hat{\sigma}^2 = c \sum_{i=1}^n (Y_i − \bar{Y} )^2$. The I get $E(\hat{\sigma}^2 ) = c(n-1)\sigma^2$. Then I am given $V(\hat{\sigma}^2 ) = 2c^2(n-1)\sigma^4$. I am wondering how to find $E[S^4]$ since $V(\hat{\sigma}^2 ) = E(S^4) - [E(S^2)]^2$

afsdf dfsaf
  • 1,687

1 Answers1

0

$$ \sum_{i=1}^n (Y_i-\bar Y)^2 \sim \sigma^2\chi^2_{n-1}. $$

"Everybody knows" that $\mathbb E(\chi^2_k)=k$. What is perhaps less widely known is that $\operatorname{var}(\chi^2_k)=2k$. Where that comes from I will say something about below.

That implies $$ \operatorname{var}\left( c\sum_{i=1}^k (Y_i-\bar Y)^2 \right) = c^2\sigma^4 2(n-1). $$ So \begin{align} & \mathbb E\left(\left( \sigma^2- c\sum_{i=1}^n(Y_i-\bar Y)^2 \right)^2\right) \\[10pt] = & \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2\mathbb E((\chi^2_{n-1})^2) \\[10pt] = & \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big( \operatorname{var}(\chi^2_{n-1}) + (n-1)^2 \Big). \tag 1 \end{align}

If I have the details right, the value of $c$ that makes $(1)$ as small as possible should be $1/(n+1)$ (thus an estimator with even more bias than the MLE is "best" in the mean-squared-error sense).

So where did we get $\operatorname{var}(\chi^2_k)= 2k\ {}$?

Since the chi-square random variable as distributed as $Z_1^2+\cdots+Z_k^2$ where the $Z$s are i.i.d. standard normals, the variance should be $k\operatorname{var}(Z_1^2)$. We have $$ \operatorname{var}(Z^2) = \mathbb E(Z^4) - \left(\mathbb E(Z^2)\right)^2 = \mathbb E(Z^4) - 1. \tag 2 $$ This is \begin{align} & \int_{-\infty}^\infty z^4 \varphi(z)\,dz = 2\int_0^\infty z^4 \varphi(z)\,dz \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty z^4 e^{-z^2/2} \, dz = 2\frac{1}{\sqrt{2\pi}} \int_0^\infty z^3 e^{-z^2/2} \Big( z \, dz \Big) \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \sqrt{2}^3 u^{3/2} e^{-u} \, du = 2\frac{1}{\sqrt{2\pi}} \sqrt{2}^3 \Gamma\left(\frac 5 2 \right) \\[10pt] = & 2\frac{1}{\sqrt{2\pi}} \cdot \sqrt{2}^3\cdot \frac 3 2 \cdot \frac 1 2 \Gamma\left(\frac 1 2 \right) \end{align}

Recalling that $\Gamma(1/2)=\sqrt{\pi}$, the above reduces to $3$, so in view of $(2)$, we have $\operatorname{var}(\chi^2_1)=2$.

A postscript prompted by comments below: We had $$ \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big( \operatorname{var}(\chi^2_{n-1}) + (n-1)^2 \Big). \tag 1 $$ Then we found that $\operatorname{var}(\chi^2_{n-1})=2(n-1)$, so $(1)$ becomes $$ \sigma^4 -2\sigma^2\Big(c(n-1)\sigma^2\Big) + c^2 \sigma^4\Big(2(n-1) + (n-1)^2 \Big) $$ This is $\sigma^4$ times $$ 1 -2 \Big(c(n-1) \Big) + c^2 \Big(2(n-1) + (n-1)^2 \Big). $$ By routine algebra this becomes $$ g(c) = (n-1)(n+1)c^2 -2(n-1)c + 1. $$ As a function of $c$, this is a parabola that opens upward, so the lowest point is the one value of $c$ at which $g'(c)=0$. So find $$ g'(c) = 2(n-1)(n+1) c - 2(n-1) = 2(n-1)\Big( (n+1)c - 1 \Big). $$ Clearly the expression in the $\Big(\text{big parentheses}\Big)$ is $0$ precisely when $c = \dfrac{1}{n+1}$.

(It can also be done by completing the square.)

  • @afsdfdfsaf : Correct --- I had $n+1$ where I should have had $1/(n+1)$. ${}\qquad{}$ – Michael Hardy Apr 03 '14 at 16:38
  • At this moment I suspect some details above are not quite right, but this line of reasoning should do it. – Michael Hardy Apr 03 '14 at 16:39
  • I don't know whether there's a quick way to do it that doesn't require recalling the value of $\Gamma(1/2)$. ${}\qquad{}$ – Michael Hardy Apr 03 '14 at 16:40
  • Is your (1) the MSE? – afsdf dfsaf Apr 03 '14 at 16:41
  • @afsdfdfsaf : I hope so. – Michael Hardy Apr 03 '14 at 16:44
  • Thanks will look into details several hours from now...in the meantime, I have two posts which are related to this in terms of $E(S^4)$ and $\Gamma (\frac{1}{2})$ If you have time, feel free to give a help on it. They are the following: http://math.stackexchange.com/questions/737807/if-s2-frac-sum-i-1n-y-i-bary2n-and-s2-frac-sum-i-1 and http://math.stackexchange.com/questions/737679/if-x-sqrty-1-y-2-then-find-a-multiple-of-x-that-is-an-unbiased – afsdf dfsaf Apr 03 '14 at 16:45
  • OK, I've fixed a detail and now I'm comfortable with this. – Michael Hardy Apr 03 '14 at 16:53
  • Can you elaborate on how you get your $V(\hat{\sigma}^2)$? – afsdf dfsaf Apr 04 '14 at 03:08
  • @afsdfdfsaf : I don't explicitly mention $\operatorname{var}(\hat\sigma^2)$. Could you be explicit about which part you want me to elaborate on? – Michael Hardy Apr 04 '14 at 03:18
  • $$ \operatorname{var}\left( c\sum_{i=1}^k (Y_i-\bar Y)^2 \right) = c^2\sigma^4 2(n-1). $$ – afsdf dfsaf Apr 04 '14 at 03:20
  • We've got $\displaystyle\sum_{i=1}^n(Y_i-\bar Y)^2\sim\sigma^2\chi^2_{n-1}$, so the variance of that expression is $\sigma^4\operatorname{var}(\chi^2_{n-1})$. And $\operatorname{var}(\chi^2_{n-1})=2(n-1)$. The reason why the variance of the chi-square distribution is twice the number of degrees of freedom is explained in the part where I find the integral whose value is $\mathbb E(Z^4)$. ${}\qquad{}$ – Michael Hardy Apr 04 '14 at 03:28
  • Can you elaborate how you got $\frac{1}{n+1}$ as I keep getting the wrong number? If you could elaborate how you got var$(\chi_{n-1}^2) = 2(n-1)$ will be great – afsdf dfsaf Apr 04 '14 at 05:05
  • The way I got $\operatorname{var}(\chi_{n-1}^2)$ is at the end above. I showed that $\operatorname{var}(\chi^2_1)=2$. Since the chi-square distribution with $k$ degrees of freedom is the distribution of the sum of $n$ independent random variables, each with a chi-square distribution with $1$ degree of freedom, that does it. The variance is $\operatorname{var}(\chi^2_1)=\mathbb E((\chi^2_1)^2)- (\mathbb E(\chi^2_1))^2$. Since $\mathbb E(\chi^2_1)=1$, it remains only to show that $\mathbb E((\chi^2_1)^2)=3$. That's the integral at the end. – Michael Hardy Apr 04 '14 at 16:15
  • Could you show how you got $\frac{1}{n+1}$? – afsdf dfsaf Apr 04 '14 at 16:58
  • Why you showed var$(\chi_1^2)$ instead of var$(\chi_{n-1}^2)$? – afsdf dfsaf Apr 04 '14 at 17:07
  • Because $\chi^2_k$ is the distribution of the sum of $k$ independent random variables each with a $\chi^2_1$ distribution, so you can find the variance of $\chi^2_1$ and then just multiply it by $k$ to get the variance of $\chi^2_k$. – Michael Hardy Apr 04 '14 at 22:27
  • I've added a postscript on how $c=1/(n+1)$ was found. – Michael Hardy Apr 04 '14 at 22:39