1

For $E (X - EX)^2$ to exist (may be infinite),

  1. according to $E (X - EX)^2 = E X^2 - (EX)^2$, I think a necessary and sufficient condition is $EX$ exists and is finite, because $ E X^2 \geq (E X)^2$ by Jensen's inequality and $\infty - \infty$ is not defined.
  2. But as long as $EX$ exists, $(X - EX)^2$ is a nonnegative measurable function and $E (X - EX)^2$ should exists. So it seems a necessary and sufficient condition is $EX$ exists.

Why do the above two not agree with each other? Do I miss something? Thanks.

Note that Glen_b uses a different definition for variance (see comment), which might exists when the traditional definition doesn't. In the definition he uses, if $X$ has a Pareto distribution with $α=1$, then $EX = \infty$ and $Var X = \infty$, from https://stats.stackexchange.com/a/91515/1005.

Community
  • 1
Tim
  • 49,412
  • According to what I've learned, the variance of a Pareto distribution with parameter $1$ does not exist, since $EX = +\infty$. One could of course say that $(X - EX)^2 = (X-\infty)^2 \equiv \infty$ and hence $E(X-EX)^2 = \infty$, but the texts and lectures I've read/heard made the finiteness of $E\lvert X\rvert$ a precondition of talking about variance. – Daniel Fischer Nov 03 '14 at 21:04
  • @DanielFischer: Thanks. Do you think it is correct that for $E(X−EX)^2$ to exist (may be infinite), a necessary and sufficient condition is $EX$ exists and is finite? – Tim Nov 03 '14 at 21:41
  • According to what I've learned, a variance is necessarily finite if it exists at all. Of course, it can make sense to allow infinite variances, but probably one pays for that by losing some desirable properties. I don't know if the trade-off is worth it. Maybe some probability-theorist can give a better founded judgment. – Daniel Fischer Nov 03 '14 at 21:58
  • @DanielFischer: I assume that a variance exists if the lebesgue integral in its definition exists, no matter if it is finite or infinite. So a variance doesn't exist, when the Lebesgue integral of its positive part and the lebesgue integral of its negative part are both $\infty$ or $-\infty$. – Tim Nov 03 '14 at 22:07
  • In that case the question is whether you allow an integrand of the form $(x-\infty)^2$. If you do, you get the stats answer, if you don't, wikipedia. – Daniel Fischer Nov 03 '14 at 22:13
  • It's perfectly possible to define the variance without reference to the mean, though, by writing variance as half the average squared distance between pairs of observations (/ pairs of independent random variables drawn from the same distribution). It may be possible under that approach to have infinite variance yet an undefined expectation, for example. – Glen_b Nov 03 '14 at 22:19
  • @Glen_b: Thanks. do you mean $Var X := \frac{1}{2} \int (X_1-X_2)^2 d P$, where $X_1$ and $X_2$ are iid with the same distribution of $X$? That definition does make sense, and always exist (whether it is finite or infinite). Since we have more than one definitions, which definition of variance is used in rigorous probability theory? In what references do you see that definition? – Tim Nov 03 '14 at 22:31
  • Yes. References do what references do; I'm not sure what the relevance of that is, really. In different circumstances their authors might consider broader definitions. The question is what definitions you find useful for your problem, and whether you care to use a definition that extends the number of cases under which the quantity is defined (a common thing to do in mathematics, since it's often useful). Practically speaking, for this case there's probably not a lot to be gained from doing so, but it's worth keeping in mind that there's often more than one useful way to define things. – Glen_b Nov 03 '14 at 22:48
  • @Glen_b: (1) Can central moments of other orders, i.e. $E (X - EX)^n, n \in \mathbb N$, be defined similarly? Or is variance the only case (or one of a few cases) that can be defined without refering to $EX$? (2) About references, I haven't seen such definition before, although I admit I don't read much. So I would like to know more, for example, to answer the questions in (1) and many other questions I have and will have. – Tim Nov 03 '14 at 22:58
  • I don't know of other results; there may be some. (Even without that result you can make the argument that the even moments should always exist, though they may be infinite.) – Glen_b Nov 03 '14 at 23:44
  • @Glen_b: (1) do you mean that the even central moments exist iff $EX$ exists? (2) I understand you use a different definition in your stat.se reply, and I updated that in my question. Now I hope for some help with what I miss when comparing two other different ways. – Tim Nov 03 '14 at 23:50
  • (1) No, I was saying it's possible to argue they may exist (but be infinite) even if it doesn't. – Glen_b Nov 04 '14 at 00:50
  • @Glen_b: ok. Then is it right that even order central moments $E(X−EX)^n,n∈N$ exist iff EX exists? – Tim Nov 04 '14 at 00:51
  • 2
    Please don't turn it into a quiz session in comments. I've already indulged this behavior too long here. Either change your question to reflect what you want to know, or ask a new one. – Glen_b Nov 04 '14 at 01:14

3 Answers3

1

I'm going to answer a simpler but related question. You can work out the details by noting that your expression $E(X - E(X))^2$ expands to a expression in $E(X^2)$ and $E(X)$.

Yes, it is necessary for $E[X]$ to exist for $E[X^2]$ to exist.

This is because if

$$ \int x^m f(x) dx $$

is undefined (un-integrable), then so is ($n > m$)

$$ \int x^n f(x) dx $$

It can be meaningful to say that these integrals are $\infty$, with the usual caveats about talking about $\infty$ in the extended real numbers. But you're not going to get any meaningful information out of the higher order integrals, once you start hitting $\infty$.

On the other hand, the existence of $E[X]$ is not a sufficient condition for $E[X^2]$ to exist, though I am having a hard time thinking of an example at the moment. (I suspect that the square root of a Cauchy random variable might do it.)

nomen
  • 2,876
0

$E (X - EX)^2 = EX^2 - 2 EX EX + (EX)^2 = E X^2 - (EX)^2$ needs $EX$ to be finite besides its existence. Otherwise $\infty - \infty$ in the last equality.

But I still confirmation for the second part: is it true that $Var X := E (X - EX)^2$ exists iff $EX$ exists?

Tim
  • 49,412
  • Care to explain what I said wrong? – Tim Nov 04 '14 at 14:17
  • 1
    It seems like it's been explained to you sufficiently many times that both conventions are possible, while the one that makes a random variable with infinite expectation have undefined variance is more standard. So unless you have a specific reason to actually care about the distinction being made, there's nothing more to be said. (Not my downvote, by the way.) – Kevin Carlson Nov 04 '14 at 17:57
0

seems to me like one way this could make sense is by defining the limiting process - how each term gets to "infinity". for example suppose I define a sequence of random variables $X_n\sim Uniform(n,n+1)$. then we have $Var(X_n)=\frac{1}{12}$ and $E(X_n)=n+\frac{1}{2}$. Then as $n\to\infty$ the expectation diverges to infinity but the variance does not. you could define countless other examples like this. This example has finite central moments of all orders and infinite raw moments of all orders.

perhaps the other question is whether you consider how the parameters of the limiting process work, and whether or not they're linked. I think something like this is how you could generally define a variance...

$$\lim_{a,b,c,d\to\infty}\int_{-a}^{b}x^2 f(x)dx - \left[\int_{-c}^{d}x f(x)dx\right]^2$$

it may be that you need to the limits $a,b$ to relate somehow to the limits $c,d$ when the mean is infinite.

probabilityislogic
  • 1,055