Skip to Content

Overview of Probability Theory


  • Events
    • Discrete random variables
    • Continuous random variables
    • Compound events
  • Axioms of probability, What defines a reasonable theory of uncertainty
  • Independent events
  • Conditional probabilities
  • Bayes rule and beliefs
  • Join probability distribution
  • Expectations
  • Independence, conditional independence

The Axioms of Probability

  • \(0 \leq P(A) \leq 1\)
  • \(P(True) = 1\)
  • \(P(False) = 0\)
  • \(P(A \vee B) = P(A) + P(B) - P(A \wedge B)\)

Elementary Probability

  • \(P(A)+P(\neg A) = 1\)
  • \(P(A)=P(A\wedge B) +P(A\wedge \neg B)\)
Multivalued Discrete Random Variables
  • \(P(A=v_i \wedge A=v_j)=0\quad \text{if}\ i\neq j\)
  • \(P(A=v_i \vee A=v_2 \dots \vee A=v_k)=1\

Independent Events

Definition: two events \(A\) and \(B\) are independent if \(P(A\ \text{and}\ B)=P(A)\cdot P(B)\)

Conditional Probability

$$ P(A|B)=\frac{P(A\wedge B)}{P(B)} \tag{1} $$

Bayes Rule

$$ P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)} \tag{2} $$

Corollary: The Chain Rule

Dari conditional probability \((1)\) dapat ditulis menjadi $$ P(A\wedge B)=P(A|B)\cdot P(B) \tag{3} $$ berdasarkan persamaan di atas, dapat kita tuliskan pula bentuk yang kita kenal dengan chain rule $$ P(C\wedge A\wedge B)=P(C|A\wedge B)\cdot P(A|B)\cdot P(B) $$

Other Forms of Bayes Rule

Bayes rule juga dapat dituliskan sebagai berikut

$$ P(A|B)=\frac{P(B|A)\cdot P(A)}{P(B)} $$ dengan mengingat elementary probability $$ P(B)=P(B\wedge A)+P(B\wedge \neg A) $$ maka dapat kita tuliskan persamaan bayes rule menjadi $$ P(A|B) = \frac{P(B|A)\cdot P(A)}{P(B\wedge A)+P(B\wedge \neg A)}\
$$ dengan persamaan dari conditional probability, maka dapat kita tulis $$ P(A|B) = \frac{P(B|A)\cdot P(A)}{P(B|A)\cdot P(A)+P(B|\neg A)\cdot P(\neg A)} \tag{4} $$ berdasarkan prinsip tersebut, dapat pula kita tuliskan persamaan bayes rule dalam bentuk lainnya $$ P(A|B\wedge X)=\frac{P(B|A\wedge X)\cdot P(A\wedge X)}{P(B\wedge X)} $$

The Joint Distribution