2

I am interested in how to express the Wishart distribution given here as a member of an exponential family. In Wikipedia, it says that Wishart is a member of the exponential family, but I am a bit confused about how to write it in the form \begin{align} f(x|\theta)=h(x) \exp[\theta' T(x)+F(\theta)]. \end{align} Specifically, what is confusing to me is that both $x$ and $\theta$ are matrices and I am not sure how to identify $h(x), \theta, T(x)$ and $F(\theta)$.

A related question was posted about multivariate gaussian here. Wikipedia suggests using vectorization operation, but I am not sure how to exactly do it.

Lisa
  • 2,941

1 Answers1

2

Consider the family of Wishart distributions on the space $V$ of real symmetric $p\times p$ matrices $\boldsymbol x=((x_{ij}))$. Then dimension of $V$ is $\frac12p(p+1)$ and a matrix $\boldsymbol x$ in this case can be considered as a vector $(x_{ij})_{1\le i\le j\le p}$. This is the 'vectorization' of $\boldsymbol x$.

If $V^+$ is the set of positive definite matrices of $V$, the Wishart distribution $W_p(n,\Sigma)$ (with $n\ge p$) has density function

$$f(\boldsymbol x\mid \Sigma)=\frac1{C_p(n)|\Sigma|^{n/2}}\exp\left\{-\frac12\operatorname{tr}(\Sigma^{-1}\boldsymbol x)\right\}|\boldsymbol x|^{(n-p-1)/2}\mathbf1_{V^{+}}(\boldsymbol x)\,, \tag{1}$$

where $C_p(n)$ is the normalizing constant and $\Sigma \in V^+$.

If you reparameterize $\boldsymbol\theta=-\frac12\Sigma^{-1}$, then keeping in mind the vectorization of $\boldsymbol\theta$,

$$-\frac12\operatorname{tr}(\Sigma^{-1}\boldsymbol x)=\sum_{1\le i\le j\le p}\theta_{ij}x_{ij}$$

And $(1)$ is now equivalent to

$$f(\boldsymbol x\mid \boldsymbol\theta)=h(\boldsymbol x)\exp\left\{\boldsymbol\theta^T \boldsymbol x - q(\boldsymbol\theta)\right\} \tag{2}\,,$$

with $$h(\boldsymbol x)=\frac1{C_p(n)}|\boldsymbol x|^{(n-p-1)/2}\mathbf1_{V^{+}}(\boldsymbol x) \quad, \quad q(\boldsymbol\theta)=\frac{n}2 \ln|\Sigma|=-\frac{n}2 \ln|-2\boldsymbol\theta|$$

So for fixed $n(\ge p)$, we can say that $f(\cdot\mid \Sigma)$ is a member of a natural exponential family with natural parameter $\boldsymbol\theta=-\frac12\Sigma^{-1}$ and sufficient statistic $T(\boldsymbol x)=\boldsymbol x$.

StubbornAtom
  • 17,052
  • So, $\theta=vec(-\frac{1}{2} \Sigma^{-1})$ and $T(x)=vec(x)$? Is this correct? – Lisa Apr 10 '21 at 18:55
  • In the density you can write $\operatorname{vec}(\boldsymbol\theta)$ and $\operatorname{vec}(\boldsymbol x)$ if you want, but $\boldsymbol\theta$ and $\boldsymbol x$ are both matrices. So writing $T(\boldsymbol x)=\operatorname{vec}(\boldsymbol x)$ is fine but not the other one. Here I haven't made such a distinction between a matrix and a vector to keep the notation simple. – StubbornAtom Apr 10 '21 at 19:19
  • 1
    I don't think what you have it is correct. There reason is that $n$ is also a parameter. Therefore, $\theta=[vec(-\frac{1}{2} \Sigma^{-1}, \frac{n-p+1}{2}]$ or something of this form. – Lisa Apr 15 '21 at 19:12
  • 1
    This answer is for fixed $n$ as I have mentioned in the last line. Usually $n$ is known in this context. – StubbornAtom Apr 15 '21 at 19:19