2

Imagine that we have a family of probability disributions with p.d.f $f_{\theta}(z)$ where $\theta \in \Theta$. We also know that there is a linear dependence between parameters. As a consequence we can restrict to a nested model with p.d.f $f_{\theta}(z)$, where $\theta \in \Theta_{0} \subseteq \Theta$.

Formally we have such a situation: \begin{align} \Theta \subseteq \mathbb{R}^p,~~~~ h:\mathbb{R}^p \to \mathbb{R}^{p-q},~~~~ \Theta_{0} = \{\theta \in \Theta : h(\theta)=0\}. \end{align} where h is a linear map onto $\mathbb{R}^{p-q}$ so we can say that: \begin{align} h(\theta) = A\theta = 0, \end{align} where $A$ is a $(p-q) \times p$ matrix of a linear map $h$.

As a result we can say that $\Theta_{0} \subseteq \mathbb{R}^q$. HERE BEGINS MY PROBLEM. I would be very grateful if someone could tell me why we can conclude now that \begin{align} \sup \limits_{\theta \in \Theta_{1}}f_{\theta}(z) \overset{\huge{?}}{=} \sup \limits_{\theta \in \Theta}f_{\theta}(z). \end{align} Consequently the test statistic of a likelihood ratio test is \begin{align} \lambda(z) = \frac{\sup_{\theta \in \Theta_{1}}f_{\theta}(z)}{\sup_{\theta \in \Theta_{0}}f_{\theta}(z)}\overset{\huge{?}}{=} \frac{\sup_{\theta \in \Theta}f_{\theta}(z)}{\sup_{\theta \in \Theta_{0}}f_{\theta}(z)}. \end{align}

John Snow
  • 51
  • Is $\Theta_1$ the parameter space under the alternative? Generally, with a two-sided hypothesis the LRT is the ratio of maximized likelihood in the entire parameter space (i.e. unrestricted maximization under the alternative) vs the restricted maximization under the null (sometimes the maximization is trivial because there's just one element/vector in the corresponding parameter space). – hejseb Sep 19 '13 at 06:30
  • That's right $\Theta_{1}$ is the parameter space under the alternative. We have \begin{align} H_{0}&: y \backsim f_{\theta}(z) \text{ for some } \theta \in \Theta_{0} \nonumber \ \text{versus} \nonumber \ H_{1}&: y \backsim f_{\theta}(z) \text{ for some } \theta \in \Theta_{1}=\Theta \setminus\Theta_{0} \nonumber \end{align} – John Snow Sep 19 '13 at 07:27
  • I don't think that this case should be a two-sided hypothesis. – John Snow Sep 19 '13 at 07:39

1 Answers1

0

After having thought about this a little and discussed with some peers, here's my take on it.

The equality you are asking about is not necessarily true. The likelihood function is unimodal, so if the maximum likelihood estimate $\hat{\theta}$ is in $\Theta_0$, then obviously $\sup_{\theta\in \Theta_1}L(\theta|z)\neq \sup_{\theta\in\Theta}L(\theta|z)= \sup_{\theta\in\Theta_0}L(\theta|z)$, where $\Theta_0$ and $\Theta_1$ are disjoint subsets of $\Theta$.

The issue here, I think, is that $H_1 : \theta \in \Theta_1$ does not mean that we should maximize the likelihood in $\Theta_1$. Instead, we conduct unrestricted maximization -- $\sup_{\theta\in\Theta_0\bigcup\Theta_1}L(\theta|z)=\sup_{\theta\in\Theta}L(\theta|z)$. See for example Definition 8.2.1 in Casella & Berger's Statistical Inference (p. 375).

It might seem unintuitive why it is this way, and I haven't found any explanation why it actually is this way. However, two reasons may be that a) unrestricted maximization ensures that the ratio is always between 0 and 1 (having $H_0$ in the numerator) which is convenient, and b) it is easier calculating one unrestricted and one restricted maximization than two restricted maximizations.

I hope this clears things up at least a little bit for you!

Edit: Here's the definition mentioned above.

enter image description here

hejseb
  • 4,745
  • 1
  • 17
  • 35