Overview of Probability Theory
Overview
- Events
- Discrete random variables
- Continuous random variables
- Compound events
- Axioms of probability, What defines a reasonable theory of uncertainty
- Independent events
- Conditional probabilities
- Bayes rule and beliefs
- Join probability distribution
- Expectations
- Independence, conditional independence
The Axioms of Probability
- \(0 \leq P(A) \leq 1\)
- \(P(True) = 1\)
- \(P(False) = 0\)
- \(P(A \vee B) = P(A) + P(B) - P(A \wedge B)\)
Elementary Probability
- \(P(A)+P(\neg A) = 1\)
- \(P(A)=P(A\wedge B) +P(A\wedge \neg B)\)
Multivalued Discrete Random Variables
- \(P(A=v_i \wedge A=v_j)=0\quad \text{if}\ i\neq j\)
- \(P(A=v_i \vee A=v_2 \dots \vee A=v_k)=1\
\Sigma_{j=1}^kP(A=v_j)=1\)
Independent Events
Definition: two events \(A\) and \(B\) are independent if \(P(A\ \text{and}\ B)=P(A)\cdot P(B)\)
Conditional Probability
$$ P(A|B)=\frac{P(A\wedge B)}{P(B)} \tag{1} $$
Bayes Rule
$$ P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)} \tag{2} $$
Corollary: The Chain Rule
Dari conditional probability \((1)\) dapat ditulis menjadi $$ P(A\wedge B)=P(A|B)\cdot P(B) \tag{3} $$ berdasarkan persamaan di atas, dapat kita tuliskan pula bentuk yang kita kenal dengan chain rule $$ P(C\wedge A\wedge B)=P(C|A\wedge B)\cdot P(A|B)\cdot P(B) $$
Other Forms of Bayes Rule
Bayes rule juga dapat dituliskan sebagai berikut
$$
P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)}
$$
dengan mengingat elementary probability
$$
P(B)=P(B\wedge A)+P(B\wedge \neg A)
$$
maka dapat kita tuliskan persamaan bayes rule menjadi
$$
P(A|B) = \frac{P(B|A)\cdot P(A)}{P(B\wedge A)+P(B\wedge \neg A)}\
$$
dengan persamaan dari conditional probability, maka dapat kita tulis
$$
P(A|B) = \frac{P(B|A)\cdot P(A)}{P(B|A)\cdot P(A)+P(B|\neg A)\cdot P(\neg A)}
\tag{4}
$$
berdasarkan prinsip tersebut, dapat pula kita tuliskan persamaan bayes rule dalam bentuk lainnya
$$
P(A|B\wedge X)=\frac{P(B|A\wedge X)\cdot P(A\wedge X)}{P(B\wedge X)}
$$