Week 1

Statistics for Data Science - 2
Week 1 Important formulas

Basic Probability
1. Experiment: Process or phenomenon that we wish to study statistically.

Example: Tossing a fair coin.
2. Outcome: Result of the experiment.

Example: head is an outcome on tossing a fair coin.
3. Sample space: A sample space is a set that contains all outcomes of an experiment.
• Sample space is a set, typically denoted S of an experiment.
• example: Toss a coin: S = { heads, tails }
4. Event: An event is a subset of the sample space.
• Toss a coin: S = { heads, tails }

– Events: empty set, {heads}, {tails}, { heads, tails }
– 4 events
• An event is said to have “occurred” if the actual outcome of the experiment
belongs to the event.
• One event can be contained in another, i.e. A ⊆ B
• Complement of an event A, denoted AC = { outcomes in S not in A } = (S \ A).
• Since events are subsets, one can do complements, unions, intersections.
5. Disjoint events: Two events with an empty intersection are said to be disjoint events.
• Throw a die: even number, odd number are disjoint.

• Multiple events: E1 , E2 , E3 , .... are disjoint if, for any i 6= j , Ei ∩ Ej = empty set.
6. De Morgan’s laws: For any two events A and B,

(A ∪ B)C = AC ∩ B C and (A ∩ B)C = AC ∪ B C .
7. Probability: “Probability” is a unction P that assigns to each event a real number

between 0 and 1 and satisfies the following two axioms:
(i) P (S) = 1 (probability of the entire sample space equals 1).

(ii) If E1 , E2 , E3 , ... are disjoint events ( Could be infinitely many),
P (E1 ∪ E2 ∪ E3 ∪ ...) = P (E1 ) + P (E2 ) + P (E3 ) + ...
• Probability function Assigns a value that represents chance of occurrence of the

event.
• Higher value of the probability of an event means higher chance of occurring that
event.
• 0 means event cannot occur and 1 means event always occurs.
8. Probability of the empty set (denoted φ) equals 0. that is
P (φ) = 0
9. Let E C be the complement of Event E. Then,
P (E C ) = 1 − P (E)
10. If event E is the subset of event F , that is E ⊆ F , then
P (F ) = P (E) + P (F \ E)
⇒ P (E) ≤ P (F )
11. If E and F are events, then
P (E) = P (E ∩ F ) + P (E \ F )
P (F ) = P (E ∩ F ) + P (F \ E)
12. If E and F are events, then
P (E ∪ F ) = P (E) + P (F ) − P (E ∩ F )
13. Equally likely events: assign the same probability to each outcome.
14. If sample space S contains the equally likely outcomes, then

1
• P (one outcome) =
Number of outcomes in S
Number of outcomes in event
• P (event) =
Number of outcomes in S
Page 2
15. Conditional probability space: Consider a probability space (S, E, P ), where S
represents the sample space, E represents the collection of events, and P represents
the probability function.
• Let B be an event in S with P (B) > 0. Now, conditional probability space given
B is defined as
For any event A in the original probability space (P, S, E), the conditional prob-
P (A ∩ B)
ability of A given B is .
P (B)
• It is denoted by P (A | B). And
P (A ∩ B) = P (B)P (A | B)
16. Law of total probability:
• If the events B and B c partitioned the sample space S such that P (B1 ), P (B2 ) 6= 0,
then for any event A of S,
P (A) = P (A | B)P (B) + P (A | B c )P (B c ).
• In general, if we have k events B1 , B2 , · · · , Bk that partition S, then for any event

A in S,
Xk Xk
P (A) = P (Bi ∩ A) = P (A | Bi )P (Bi ).
i=1 i=1
17. Bayes’ theorem: Let A and B are two events such that P (A) > 0, P (B) > 0.
P (A ∩ B) = P (B)P (A | B) = P (A)P (B | A)
P (B)P (A | B)
⇒ P (B | A) =
P (A)
In general, if the events B1 , B2 , · · · , Bk partition S such that P (Bi ) 6= 0 for i =
1, 2, · · · , k, then for any event A in S such that P (A) 6= 0,
P (Br )P (A | Br )
P (Br | A) = k
P
P (Bi )P (A | Bi )
i=1
for r = 1, 2, · · · , k.
18. Independence of two events: Two events A and B are independent iff
P (A ∩ B) = P (A)P (B)
• A and B independent ⇒ P (A | B) = P (A) and (B | A) = P (B) for P (A), P (B) >

0.
Page 3
• Disjoint events are never independent.
• A and B independent ⇒ A and B c are independent.
• A and B independent ⇒ Ac and B c are independent.
19. Mutual independence of three events: Events A, B, and C are mutually indepen-
dent if
(a) P (A ∩ B) = P (A)P (B)
(b) P (A ∩ C) = P (A)P (C)
(c) P (A ∩ B) = P (A)P (B)
(d) P (A ∩ B ∩ C) = P (A)P (B)P (C)
20. Mutual independence of multiple events: Events A1 , A2 , · · · ,n are mutually in-
dependent if, ∀i1 , i2 , · · · , ik ,
P (Ai1 ∩ Ai2 ∩ · · · Aik ∩) = P (Ai1 )P (Ai2 ) · · · P (Aik )
n events are mutually independent ⇒ any subset with or without complementing are
independent as well.
21. Occurrence of event A in a sample space is considered as success.
22. Non - occurrence of event A in a sample space is considered as failure.
23. Repeated independent trials:
(a) Bernoulli trials
• Single Bernoulli trial:
– Sample space is {success, failure} with P(success) = p.
– We can also write the sample space S as {0, 1}, where 0 denotes the
failure and 1 denotes the success with P (1) = p, P (0) = 1 − p.
This kind of distribution is denoted by Bernoulli(p).
• Repeated Bernoulli trials:
– Repeat a Bernoulli trial multiple times independently.
– For each of the trial, the outcome will be either 0 or 1.
(b) Binomial distribution: Perform n independent Bernoulli(p) trials.
• It models the number of success in n independent Bernoulli trials.
• Denoted by B(n, p).
• Sample space is {0, 1, · · · , n}.
• Probability distribution is given by
P (B(n, p) = k) = nCk pk (1 − p)n−k
where n represents the total number trials and k represent the number of
success in n trials.
Page 4
• P (B = 0) + P (B = 1) + · · · + P (B = n) = 1
⇒ (1 − p)n + nC2 p2 (1 − p)n−2 + · · · + pn = 1.
(c) Geometric distribution: It models the number of failures the first success.
• Outcomes: Number of trials needed for first success and is denoted by G(p).
• Sample space: {1, 2, 3, 4, · · · }
• P (G = k) = P (first k − 1 trials result in 0 and kth trial result in 1.) =
(1 − p)k−1 p.
• Identity: P (G ≤ k) = 1 − (1 − p)k .
Page 5

Week 1

Uploaded by

Copyright:

Available Formats

Week 1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Week 1

Uploaded by

Copyright:

Available Formats

Statistics for Data Science - 2

Week 1 Important formulas

1. Experiment: Process or phenomenon that we wish to study statistically.

2. Outcome: Result of the experiment.

4. Event: An event is a subset of the sample space.

• Toss a coin: S = { heads, tails }

• Throw a die: even number, odd number are disjoint.

6. De Morgan’s laws: For any two events A and B,

7. Probability: “Probability” is a unction P that assigns to each event a real number

(i) P (S) = 1 (probability of the entire sample space equals 1).

P (E1 ∪ E2 ∪ E3 ∪ ...) = P (E1 ) + P (E2 ) + P (E3 ) + ...

• Probability function Assigns a value that represents chance of occurrence of the

8. Probability of the empty set (denoted φ) equals 0. that is

9. Let E C be the complement of Event E. Then,

10. If event E is the subset of event F , that is E ⊆ F , then

11. If E and F are events, then

12. If E and F are events, then

14. If sample space S contains the equally likely outcomes, then

16. Law of total probability:

P (A) = P (A | B)P (B) + P (A | B c )P (B c ).

• In general, if we have k events B1 , B2 , · · · , Bk that partition S, then for any event

• A and B independent ⇒ P (A | B) = P (A) and (B | A) = P (B) for P (A), P (B) >

You might also like