Rigorous proof of the recursion method to compute expectations in probability.

Question

When solving expectation problems in probability, people sometime use recursive argument, but I have never seen a proof that this argument always works.

For example, what is the expected number of rolls of a fair six sided die until you roll "1"?

The recursive method says,

1) You have a $1/6$ probability of rolling "1" on the first roll and a $5/6$ chance of rolling the other five numbers.

2) If you roll any of the other five numbers you have to start over again, but now you start at roll 1 to try and get your "1".

If E is the expected number of rolls to get "1", the recursive method says,

$E = (1/6)*(1 roll) + (5/6)*[(E + 1) rolls]$

So,

$E = (1/6) + (5/6)(E + 1)$ and solving for $E$ we get $E = 6$.

There are also recursion methods for things like "What is the expected number of flips of 2 sided coin to get two consecutive heads", and other such problems involving questions like "how many rolls/flips/etc. to get somethings".

My question is:

Is there a rigorous mathematical proof that such recursion methods always work?

By rigorous I mean please start with the definition of expectation as the integral of the random var. over its sample space. From this definition, can you prove the recursion method? Even if you prove it for a specific example that would be OK too.

The issue I am having is that the recursion method seems "intuitive" (it could be rigorous, but no book has proved it that I've read!) and I do not see where the definition of expectation (as an integral) is used in the method. The recursion method seems to be "trick" books discuss, but never relate to the definition of expectation as the integral of the function.

Answer 1

In probability theory, conditional expectations are rigorously defined so that, for instance, for a pair of random variables $X$ and $Y$ defined on the same probability space,

$$ \mathbb{E}\left[X\right]{}={}\mathbb{E}\left[\mathbb{E}\left[X \big| Y\right]\right]\,. $$

This is also true if "$Y$" above is replaced by a suitable collection of probabilistic "events" $A$ (if you are interested, the technical term for this collection is a Sigma-algebra), so that

$$ \mathbb{E}\left[\mathbb{E}\left[X \big| A\right]\right] $$

also makes sense. In any probabilistic experiment, therefore, computing expectations recursively simply requires identifying those mutually exclusive events on which to compute these conditional expectations on.

Edit:

A simple application of this can be seen as follows. For a discrete random variable $X$ describing an experiment that can either terminate or repeat (with suitable independence assumptions), we have

$$ \begin{eqnarray*} \mathbb{E}\left[X\right]{}={}\mathbb{E}\left[X \big| X\mbox{ ends}\right]P(X\mbox{ ends}){}+{}\mathbb{E}\left[X \big| X\mbox{ starts again}\right]P(X\mbox{ starts again})\,. \end{eqnarray*} $$

But, with suitable independence assumptions,

$$ \mathbb{E}\left[X \big| X\mbox{ starts again}\right]{}={}\mathbb{E}\left[X \right]\,, $$

which is why the recursion works.

More explicitly, using the example in the question, consider the following Measurable space.

Define outcomes, all of which are contained in the sampe space $\Omega$, as follows:

1) $\omega_i$ := roll first 1 on ith roll, for all $i\in\mathbb{N}$;

2) $\omega_{\infty}$ := never roll a 1;

Use the power set of $\Omega$, $P^{\Omega}$, as the preferred sigma-algebra, and assume a suitable probability measure. Assume a random variable defined on this probability space whose realisations are either the number of tosses before observing the first $1$ or 0. That is, for $\omega\in\Omega$:

$$ X(\omega){}={}\left\{\begin{array}{c} i\,,\mbox{ if }\omega=\omega_i \\ 0\,,\mbox{ otherwise} \end{array}\right. $$

Further, assume that $X$ can be written as the conditional sum of independent, identically distributed random variables $X_i$, each defined on the same probability space as $X$ as follows. For $\omega\in\Omega$,

$$ X_i(\omega){}={}\left\{\begin{array}{c} 1\,,\mbox{ if }\omega=\omega_i \\ 0\,,\mbox{ otherwise} \end{array}\right. $$

so that

$$ X(\omega)=\sum\limits_{i=1}^{\infty}{\textbf{1}}_{\left\{\omega=\omega_i\right\}}\left(X_i{}+{}\sum\limits_{j=1}^{i-1}(1{}-{}X_{j-1})\right)\,. $$

Note that, due to the independence and identically distributed nature of the $X_i$, we must have

$$ \mathbb{E}\left[X\right]=\mathbb{E}\left[ \sum\limits_{i=j+1}^{\infty}{\textbf{1}}_{\left\{\omega=\omega_i\right\}}\left(X_i{}+{}\sum\limits_{r=j+1}^{i-1}(1{}-{}X_r)\right) \bigg|\, X_k=0 \mbox{ for all }k\leq j \right]\,. $$

We are done, for we must have

$$ \begin{eqnarray*} \mathbb{E}\left[X\right]&{}={}&\mathbb{E}\left[X{\textbf{1}}_{\left\{\omega=\omega_1\right\}}\right]{}+{}\mathbb{E}\left[X{\textbf{1}}_{\left\{\omega\neq\omega_1\right\}}\right]\newline &{}={}& 1.P\left({\left\{\omega=\omega_1\right\}}\right)\newline &&{}+{}\mathbb{E}\left[ \sum\limits_{i=2}^{\infty}{\textbf{1}}_{\left\{\omega=\omega_i\right\}}\left(X_i{}+{}\sum\limits_{r=2}^{i-1}(1{}-{}X_r)\right) \bigg|\, X_k=0 \mbox{ for all }k\leq 1 \right]P\left({\left\{\omega\neq\omega_1\right\}}\right)\newline &{}={}& P\left({\left\{\omega=\omega_1\right\}}\right){}+{}\mathbb{E}\left[X\right]P\left({\left\{\omega\neq\omega_1\right\}}\right) \end{eqnarray*} $$

and, therefore, recursion.

Answer 2

The question sounds very strange and reflected a poor understanding of measure theory. You should study the definition of a martingale.

Answer 3

We have a sample space, $\Omega$, consisting of all possible outcomes. Explicitly, $\Omega$ consists of all possible infinite strings of die rolls. We can write $\Omega = \Omega_1 \sqcup \Omega_2$, where $\Omega_1$ is the event that the first roll is a 1 and $\Omega_2$ is the event that the first roll is not a 1.

Let $X$ be the random variable which tells you how many rolls you need to get a 1. Then:

$$E(X) = \int_{\Omega}X\, d\mu = \int_{\Omega_1}X\, d\mu + \int_{\Omega_2}X\, d\mu$$

Now, on $\Omega_1$, $X$ is equal to the constant function $1$. So $\int_{\Omega_1}X = \mu(\Omega_1) = 1/6$.

Now for the recursion. We basically want to show:

$$\int_{\Omega_2}X\, d\mu = \frac{5}{6}\int_{\Omega}(X + 1)\, d\mu$$

To show this, notice there is a map $\phi:\Omega_2 \rightarrow \Omega$ which is just 'forget the first roll'. We have:

$$\int_{\Omega_2} (X \circ \phi) \, d\mu = \int_{\Omega} X\, d(\phi\mu)$$

Now, the pushforward measure $\phi\mu$ is just equal to $\frac{5}{6}\mu$, since $\mu(\phi^{-1}(A)) = \frac{5}{6}\mu(A)$ for any $A \subset \Omega$. And $(X \circ \phi) = X - 1$, since 'forget the first roll' subtracts one from how many rolls it takes. So we get:

$$\int_{\Omega_2}X - 1\, d\mu = \int_{\Omega} X \, \frac{5}{6}d\mu$$