Can someone explain and provide example(s) of where we need to appeal/use conditional expectation to determine a conditional probability given a zero-probability event has occurred? I've seen books describe something like $E[X|Y]$ as allowing us to derive such conditional probabilities, which are not amenable to the usual $P(A|B)=\frac{P(A\cap B)}{P(B)}$ since $P(B)=0$....
Asked
Active
Viewed 2,230 times
6
-
Here's an example that was asked previously: http://math.stackexchange.com/questions/236998/conditional-probability-and-division-by-zero – David K Dec 30 '14 at 21:06
-
@DavidK thanks, but that thread doesn't show how we use conditional expectation to derive conditional probabilities when the conditioning event has $0$ probability. – Jan 01 '15 at 07:08
2 Answers
9
In such situations it is good to recall that division is often defined as an inverse to multiplication: $x = b/a$ if $ax =b$. Whenever you are stack with some tricky division, you can just write it down in terms of equation involving the multplication. For example, you can define conditional probability as a solution of the following equation: $$ x= P(A|B) \quad \text{ if } \quad P(A\cap B) = x\cdot P(B) $$ or passing to the formula of total probability, given a finite partition $(H_i)$ of the probability space, the corresponding conditional probabilities $x_i = P(A|H_i)$ must satisfy $$ P(A) = \sum_i x_i(A) P(H_i) \qquad \forall A $$ Now, passing to a general case seems easy: just put a more general integral instead of a sum. That is, $$ P(A) = \int x_\omega(A) P(\mathrm d\omega) \qquad \forall A \tag{1} $$ is the (implicit) definition of (regular) conditional probability. Note that solution of $(1)$ is only unique almost surely, hence for any given null event you may change the conditional probability function $x_\omega(\cdot)$ and it will still solve this equation. For this reason, often you just need to guess what is a good candidate for the conditional probability, and then show that it solves the desired equation. E.g. Did's example looks very natural, although we can arbitrarily change it on rational numbers, and still get a valid conditional probability. In this case of course $E[X|Y]$ may differ from $\frac12 Y$ for rational numbers.
Disclaimer: $(1)$ is not perfectly formal, but it shall give an intuition.
Edit: regarding the question in the first comment - this is exactly what you do. Once you have defined conditional expectation, it is natural to say that conditional probability is defined by $P(A|\mathcal F) = E[1_A|\mathcal F]$. Now, the tower rule gives us exactly $(1)$: $$ P(A) = E[1_A] = E[E[1_A|\mathcal F]] = \int E[1_A|\mathcal F](\omega) P(\mathrm d\omega) $$ so that $E[1_A|\mathcal F](\omega)$ is a good candidate for $x_\omega(A)$. The only issue is that for some fixed $\omega$ it may happen that $E[1_A|\mathcal F](\omega)$ is not a measure as a function of $A$ - recall that conditional expectation is only defined $P$-a.s. uniquely, hence on some null set it may behave weirdly enough. For this reason, we distinguish regular conditional probabilities - those versions of $E[1_A|\mathcal F](\omega)$ that are probability measure as a function of $A$ for each fixed $\omega$. There are some measurable spaces and probability measures on them that do not admit regular versions, however for Borel spaces it works fine - you may wanna google about this if interested in details.
SBF
- 36,041
-
1+1 Thanks! I am trying to go the other way around though...given that the conditioning event $A$ has probability $0$, how can I use conditional expectation to derive the conditional probability $P(X \in B|A)$? I think its as simple as calculating $E[\mathbf{1}_{B}(x)|A]$? Its just that so many textbooks discuss conditional expectation as a way to derive conditional probability in the most general way. I'd like to see an example were this route to deriving the conditional probability via conditional expectation is needed. – Jan 02 '15 at 17:05
-
1
-
1
4
Consider $(X,Y)$ uniformly distributed in a triangle with vertices $(0,0)$, $(1,0)$, and $(0,1)$. Then $P(Y=y)=0$ for every $y$ and yet one wants to be able to declare that "when $Y=y$", $X$ is uniformly distributed on the interval $(0,y)$ hence the conditional expectation of $X$ should be $\frac12y$. This is rigorously stated as the fact that $E(X\mid Y)=\frac12Y$ almost surely.
Did
- 279,727
-
+1 Thanks for the example. Pardon if I am being dense, but how does $E(X|Y)=\frac{Y}{2} \implies f_{X|Y=y}(x)=\frac{1}{y}\mathbf{1}_{0\leq x \leq y}(x)$? – Dec 31 '14 at 02:33
-
1This implication does not hold (and is not mentioned in my answer). The reverse one does, by definition. – Did Dec 31 '14 at 08:12
-
I didn't think it did..but how does it allow you to declare the conditional density of X, as you said in your post "one wants to be able to declare that...." I understand conditional expectation, per se, but how does it generalize the usual definition of conditional probability to zero probability events. – Dec 31 '14 at 18:30
-
My answer is based on the assumption that, faced with a picture of the triangle and knowing that (X,Y) is uniformly distributed in this triangle, you will find natural to think that the abscissa X of the points on the horizontal line Y=y is uniform on (0,y). Lots of students (mine and others') found this fact intuitive when faced with this picture. If you do not, this answer is useless to you. If your question is how to derive this result rigorously, you might have said so explicitely. – Did Dec 31 '14 at 18:35
-
1The first line of my post states "use conditional expectation to determine a conditional probability given a zero-probability event has occurred." Your example was very clear, but its seems to be going in the other direction. I guess one could state $E[\mathbf{1}_{[a,b]\subset [0,Y]}(X)|Y]=P(X\in[a,b]|Y)$? – Dec 31 '14 at 19:00
-
Here is what i was tring to get at: http://en.wikipedia.org/wiki/Conditional_expectation#Definition_of_conditional_probability. I think I approached it in my comment above, but I'd like your take as well, if you have time. – Dec 31 '14 at 20:02