Why are the random variables $X+Y$ and $X-Y$ independent when $X$ and $Y$ are i.i.d $N(0,1)$?

Question

$X,Y\sim N(0,1)$ and are independent, consider $X+Y$ and $X-Y$.

I can see why $X+Y$ and $X-Y$ are independent based on the fact that their joint distribution is equal to the product of their marginal distributions. Just, I'm having trouble understanding intuitively why this is so.

This is how I see it : When you look at $X+Y=u$, the set $\{(x,u-x)|x\in\mathbb{R}\}$ is the list of possibilities for $X$ and $Y$.

And intuitively, I understand independence of two random variables $A$ and $B$ as, the probability of the event $A=a$ being completely unaffected by the event $B=b$ happening.

But when you look at $X+Y=u$ given that $X-Y=v$, the set of possibilities has only one value $(\frac{u+v}{2},\frac{u-v}{2})$.

So, $\mathbb{P}(X+Y=u|X-Y=v)\neq \mathbb{P}(X+Y=u)$.

Doesn't this mean that $X+Y$ is affected by the occurrance of $X-Y$? So, they would have to be dependent? I'm sorry if this comes off as really stupid, it has been driving me crazy, even though I am sure that they are independent, it just doesn't feel right.

Thank you.

Answer 1

(1) The short, short answer is that it is wrong to say

$$\mathbb{P}(X+Y=u|X-Y=v)\neq \mathbb{P}(X+Y=u)\,\,\,\,\,\,\text{(this is wrong)}$$

because in fact, both sides $=0$, as these are continuous variables.

(2) The longer answer... Well first of all, the proper way to decide independence is to look at the joint PDF of $U = X+Y$ and $V=X-Y$, as you have already done. This is equivalent to checking:

$$f_U(U = u) \overset{?}= f_{U|V}(U = u \mid V = v) \equiv \frac{f_{U,V}(U = u \cap V = v)}{f_V(V = v)}$$

where you will find that both sides are non-zero and indeed equal.

(3) However, I wonder if your confusion comes from a more basic misunderstanding. It is of course true that $(U,V) = (u,v)$ defines exactly a single point in $(X,Y)$ space. However this does not automatically imply the conditional (prob or density) is $<$ the unconditional. After all, remember that all conditional prob (or density) are ratios. So if the numerator is very small but the denominator is proportionally small, then the ratio is unchanged and the conditional prob (or density) equals the unconditional version.

In your example, the unconditional asks for hitting a certain line $X+Y = u$ within the entire $2$-D $(X,Y)$ plane, while the conditional asks for hitting a point within a specific line $X-Y = v$. As mentioned, both probabilities are zero, but as you verified, both densities are non-zero and equal.

(4) Finally, you might like to know that multivariate Gaussians are the only variables with this property. So that might explain why your gut just keeps telling you that $X+Y, X-Y$ "cannot possibly be independent" when $X,Y$ are independent. :) I was confused about this in the recent past -- see this for a brief further discussion.

Answer 2

To understand a very intuitive brainstorming let's start with $X,Y$ iid $N(\theta;1)$ distribution.

You will probabily know that $X+Y$ is a "complete sufficient statistic" for $\theta$ while $X-Y\sim N(0;2)$ is independent of $\theta$ so it is "ancillary"

This is that $X+Y$ contains all the information about $\theta$ while $X-Y$ has no useful information...its distribution does not depend anymore from $\theta$

So they are independent

---

This intuitive brainstorming is, in poor words, Basu's Theorem

Answer 3

Intuitively, it's because the joint density of $X$ and $Y$ is rotation invariant, and the transformation from $(X,Y)$ to $((X+Y)/\sqrt{2},(X-Y)/\sqrt{2})$ is a rotation. Therefore $(X+Y,X-Y)$ has the same distribution as $(\sqrt{2}X, \sqrt{2}Y)$, and the random variables in this latter pair are independent.

Answer 4

Let $X$ and $Y$ be two random variables, with finite second moment. Consider the variables $Z_1=X-Y$ and $Z_2=X+Y$.

Their covariance is

$$\rm{Cov}(Z_1, Z_2) = E[(X-Y)(X+Y)] - E(X-Y)E(X+Y) = {\rm Var}(X) - {\rm Var}(Y).$$

So

$${\rm Var}(X) = {\rm Var}(Y) \implies \rm{Cov}(Z_1, Z_2) = 0,\;\;\; {\rm Var}(X) \neq {\rm Var}(Y) \implies \rm{Cov}(Z_1, Z_2) \neq 0.$$

So a necessary condition for independence of $Z_1$ and $Z_2$ is that ${\rm Var}(X) = {\rm Var}(Y)$. No matter what the marginal and joint distributions are of the variables involved, if the variances of the $X$ and $Y$ variables are not equal, the independence result cannot hold.

Given this, the second required condition for independence of $Z_1, Z_2$ is that their joint distribution is such that zero covariance implies independence. There are many such distribution families, not just the Normal. For example, if the joint distribution is of the Farlie-Gumbel-Morgenstern type.

PS: Now the interesting question becomes: assume that $X$ and $Y$ have no moments. Under which conditions $Z_1$ and $Z_2$ will be independent?

PS2: The above result does not make nor uses the assumption that $X,Y$ are independent random variables.