Statistical independence

From Wikinfo

In probability theory, when we assert that two events are independent, we intuitively mean that knowing whether or not one of them occurred makes it neither more probable nor less probable that the other occurred. For example, the events "today is Tuesday" and "it rains today" are independent.

Similarly, when we assert that two random variables are independent, we intuitively mean that knowing something about the value of one of them does not yield any information about the value of the other. For instance, the height of a person and their IQ are independent random variables. Another typical example of two independent variables is given by repeating an experiment: roll a die twice, let X be the number you get the first time, and Y the number you get the second time. These two variables are independent.

Contents 1 Independent events 2 Independent random variables 3 Conditionally independent random variables 3.1 References

Independent events

We define two events E₁ and E₂ of a probability space to be independent iff

P(E₁ ∩ E₂) = P(E₁) � P(E₂).

Here E₁ ∩ E₂ (the intersection of E₁ and E₂) is the event that E₁ and E₂ both occur; P denotes the probability of an event.

If P(E₂) ≠ 0, then the independence of E₁ and E₂ can also be expressed with conditional probabilities:

P(E₁ | E₂) = P(E₁)

which is closer to the intuition given above: the information that E₂ happened does not change our estimate of the probability of E₁.

If we have more than two events, then pairwise independence is insufficient to capture the intuitive sense of independence. So a set S of events is said to be independent if every finite nonempty subset {�E₁,�...,�E_n�} of S satisfies

P(E₁ ∩ ... ∩ E_n) = P(E₁) � ... � P(E_n).

This is called the multiplication rule for independent events.

Independent random variables

We define random variables X and Y to be independent if

P[(X in A) & (Y in B)] = P[X in A] � P[Y in B]

for any Borel subsets A and B of the real numbers.

If X and Y are independent, then the expectation operator E has the nice property

E[X� Y] = E[X] � E[Y]

and for the variance we have

Var(X + Y) = Var(X) + Var(Y) - 2Cov(X,Y).

Since X and Y are independent, the covariance Cov(X,Y) is zero; the converse is not true since even it is possible to have highly dependent random variables which are uncorrelated. This gives the simplified formula for independent random variables

Var(X + Y) = Var(X) + Var(Y).

Furthermore, if X and Y are independent and have probability densities f_X(x)and f_Y(y), then the combined random variable (X,Y) has a joint density

f_XY(x,y)dx dy = f_X(x)dx f_Y(y)dy.

Still need to deal with independence of sets of more than 2 random variables.

Conditionally independent random variables

We define random variables X and Y to be conditionally independent given random variable Z if

P[(X in A) & (Y in B) | Z in C] = P[X in A | Z in C] � P[Y in B | Z in C]

for any Borel subsets A, B and C of the real numbers.

If X and Y are conditionally independent given Z, then

P[(X in A) | (Y in B) & (Z in C)]

= P[(X in A) | (Z in C)]

for any Borel subsets A, B and C of the real numbers. That is, given Z, the value of Y does not add any additional information about the value of X.

Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.

References

• Adapted from the Wikipedia article, "Statistical_independence" http://en.wikipedia.org/wiki/Statistical_independence, used under the GNU Free Documentation License