Probability
1 Sample Space and Events
Deﬁnition 1
A sample space, denoted
S
, is the collection of all outcomes when anexperiment is performed.
Example 1
Toss a coin two times, the sample space is
S
=
{
HH,HT,TH,TT
}
Example 2
Toss a coin until a head appears, the sample space is
S
=
{
H,TH,TTH,TTTH,...
}
Deﬁnition 2
An event is a set outcomes form the sample space. The events areusually denoted
A,B,C,...
. If we regard
S
as a set, then an event is a subset of
S
.
Union and intersection of events
 The union of two events
A
and
B
, denoted
A
∪
B
, is the set of all outcomes that areeither in
A
or in
B
 The intersection of two events
A
and
B
, denoted
A
∩
B
, is the set of all outcomesthat are both in
A
and in
B
 The complement of an event
A
, denoted
A
, is the set of all outcomes that are not in
A
Most of elementary operations on sets can be expressed in terms of union, intersectionand complement.
Example 3
A
\
B
is the set of outcomes that are in
A
but not in
B
; it is left as anexercise to show that
A
\
B
=
A
∩
B
Mutually exclusive and exhaustive eventsDeﬁnition 3
A set of events
A
1
,A
2
,A
3
,...
is mutually exclusive if and only if
A
i
∩
A
j
=
∅
, i
=
j
(1)
Deﬁnition 4
A set of events
A
1
,A
2
,A
3
,...
is exhaustive if
A
1
∪
A
2
∪
A
3
...
=
S
(2)
Deﬁnition 5
A set of events
A
1
,A
2
,A
3
,...
is mutually exclusive and exhaustive (calledalso partition) if (1) and (2) are both satisﬁed.
1
2 Probability Basics
Axiom 1
A probability, denoted
P
is an application,
P
:
S
−→
[0
,
1], satisfying:a.
P
(
φ
) = 0,
P
(
S
) = 1, andb. If
A
1
,A
2
,A
3
,...
is a set of mutually exclusive events, then
P
(
A
1
∪
A
2
∪
A
3
...
) =
P
(
A
i
) =
P
(
A
1
) +
P
(
A
2
) +
P
(
A
3
) +
...
Property 1
(basic rules)
a.
(union rule)
P
(
A
∪
B
) =
P
(
A
) +
P
(
B
)
−
P
(
A
∩
B
)
b.
(complement rule)
P
(
A
) = 1
−
P
(
A
)
The proof of the above property is left as an exercise. Note that other basic rules of probability can be expressed in terms of the the union and the complement rules.
Example 4
P
(
A
\
B
) =
P
(
A
)
−
P
(
A
∩
B
)
Proof:
A
= (
A
\
B
)
∪
(
A
∩
B
) (mutually exclusive events)
P
(
A
) =
P
(
A
\
B
) +
P
(
A
∩
B
), hence
P
(
A
\
B
) =
P
(
A
)
−
P
(
A
∩
B
).
Exercise 1
Let
A,B
and
C
be three events, show that
P
(
A
∪
B
∪
C
) =
P
(
A
)+
P
(
B
)+
P
(
C
)
−
P
(
A
∩
B
)
−
P
(
A
∩
C
)
−
P
(
B
∩
C
)+
P
(
A
∩
B
∩
C
)
InclusionExclusion formula:
More generally, if
A
1
,A
2
,...,A
n
are
n
events, then
P
(
A
1
∪
A
2
∪
...
∪
A
n
) =
n
i
=1
P
(
A
i
)
−
n
i,j
=1
P
(
A
i
∩
A
j
) +
n
i,j,k
=1
P
(
A
i
∩
A
j
∩
A
k
)
...
+(
−
1)
n
−
1
P
(
A
1
∩
A
2
∩
...
∩
A
n
)
Proposition 1
(not required for STAT230)Let
A
n
,n
= 1
,
2
.
3
,...
be a increasing set of events (
A
1
⊆
A
2
⊆
A
3
...
), then
P
(
n
A
n
) = lim
n
→
+
∞
P
(
A
n
).
Proof:
Let
B
1
=
A
1
,
B
2
=
A
2
∩
A
1
,
B
3
=
A
3
∩
A
2
,
...
,
B
n
=
A
n
∩
A
n
−
1
. The events
B
n
,n
= 1
,
2
.
3
,...
are mutually exclusive, and
+
∞
n
=1
B
n
=
+
∞
n
=1
A
n
. By axiom 1,b),
P
(
+
∞
n
=1
A
n
) =
+
∞
n
=1
P
(
B
n
) = lim
k
→
+
∞
kn
=1
P
(
B
n
) = lim
k
→
+
∞
P
(
kn
=1
B
n
) = lim
k
→
+
∞
P
(
A
k
)
.
2
3 Counting
Deﬁnition 6
A selection of
k
objects out of
n
objects without replacement and without order is called combination, denoted
C
kn
, and deﬁned by:
C
kn
=
n
!
k
!(
n
−
k
)!
Deﬁnition 7
A selection of
k
objects out of
n
objects without replacement and withorder is called permutation, denoted
k
P
n
, and deﬁned by:
k
P
n
=
n
!(
n
−
k
)!
Example 5
a) In how many diﬀerent ways can a group of 2 persons be formed if 5persons are available ? b) In how many diﬀerent ways can a this be done if the groupmust contain a president and a vicepresident ?
Property 2
The number of permutations of
n
distinct objects is
n
!.
Property 3
(Basic counting rule)
If an experiment has
n
1
outcomes, and if anotherexperiment has
n
2
outcomes, then the total number of outcomes is
n
1
×
n
2
.
Exercise 2
In how many diﬀerent ways can a team of 2 men and 3 women be formedif 5 men and 7 women are available ? In how many ways can this be done if one manand one woman refuse to be together in the team ?
Exercise 3
In how many diﬀerent ways can a group of 10 persons be seated in a rowif 3 of them insist to be next to each other ?
Exercise 4
Find the probability of a fullhouse in a poker hand.
Permutations of indistinguishable objectsProperty 4
The number of permutations of
n
objects, from which
n
1
are the same,
n
2
are the same, ..., and
n
k
are the same (
n
1
+
n
2
+
...
+
n
k
=
n
) is
n
!
n
1
!
×
n
2
!
×
...
×
n
k
!
Proof:
There are
C
n
1
n
choices for the ﬁrst
n
1
objects, and
C
n
2
n
−
n
1
choices for the
n
2
objects, ..., and
C
n
k
n
−
n
1
−
n
2
−
...
−
n
k
−
1
choices for the
n
k
objects. By the basic countingrule, the total number of choices is
C
n
1
n
×
C
n
2
n
−
n
1
×
...
×
C
n
k
n
−
n
1
−
n
2
−
...
−
n
k
−
1
=
n
!
n
1
!
×
n
2
!
×
...
×
n
k
!
Example 6
How many diﬀerent words can be formed by using the letters of the wordMISSISSIPPI ?
Exercise 5
In how many diﬀerent ways can 15 students be seated in 3 classes of size5 each ? In how many ways can this be done if two of the students, Asma and Maya,want to be in the same class room ?
3
4 Conditional Probability
Deﬁnition 8
Let
A
and
B
be two events with
P
(
B
)
>
0. The conditional probabilityof
A
given
B
, denoted by
P
(
A

B
), is deﬁned by
P
(
A

B
) =
P
(
A
∩
B
)
P
(
B
)
Example 7
5 cards are drawn at random and without replacement from a deck of 52playing cards. Find the probability that all 5 cards are spades if at least 3 cards arespades.
Solution:
Let
A
=
{
event all 5 cards are spades
}
, and
B
=
{
at least 3 cards arespades
}
P
(
A

B
) =
P
(
A
∩
B
)
P
(
B
) =
P
(
A
)
P
(
B
) =
C
513
C
313
∗
C
239
+
C
413
∗
C
139
+
C
552
Proposition 2
The conditional probability
P
(
·
B
) is itself a probability.
Proof:a.
P
(
φ

B
) =
P
(
φ
∩
B
)
P
(
B
) =
P
(
φ
)
P
(
B
) = 0 ;
P
(
S

B
) =
P
(
S
∩
B
)
P
(
B
) =
P
(
B
)
P
(
B
) = 1
b.
Let
A
1
,A
2
,A
3
,...
be mutually exclusive events,
P
(
A
1
∪
A
2
∪
A
3
...

B
) =
P
((
A
1
∪
A
2
∪
A
3
...
)
∩
B
)
P
(
B
)=
P
((
A
1
∩
B
)
∪
(
A
2
∩
B
)
∪
(
A
3
∩
B
)
...
)
P
(
B
)=
P
(
A
1
∩
B
)
P
(
B
) +
P
(
A
2
∩
B
)
P
(
B
) +
P
(
A
3
∩
B
)
P
(
B
) +
...
=
P
(
A
1

B
) +
P
(
A
2

B
) +
P
(
A
3

B
) +
...
The previous proposition says that all rules of probability still apply to conditionalprobability; for example,
P
(
A

C
) = 1
−
P
(
A

C
)
Property 5
(Multiplication rule)
a.
P
(
A
∩
B
) =
P
(
A

B
)
×
P
(
B
) =
P
(
B

A
)
×
P
(
A
), and more generally
b.
P
(
A
1
∩
...
∩
A
n
) =
P
(
A
1
)
×
P
(
A
2

A
1
)
×
P
(
A
3

A
1
∩
A
2
)
...
×
P
(
A
n

A
1
∩
A
2
∩
...
∩
A
n
−
1
)
Example 8
Cards are drawn at random and without replacement from a deck of 52cards. Find the probability that the sixth spade is drawn at the tenth draw.
4