---

Basics of the Theory of Probability

---

---

• Probability (likelihood) of an event
  • The probability of an event A is the likelihood of the occurring of that event.

    Concretely:

    Probability of an event A:

    # outcomes A Ҏ [ event A ] = ------------------------ # posible outcomes

  ---

  ---

• Probability mass/density function and Probability distribution function
  • The probability distribution function a discrete random variable is known as a probability mass function.

    The probability distribution function a continuous random variable is known as a probability density function.

    ---

  • The probability mass/density function is:

    A mathematical function used to model the frequencies (probabilities) of occurrences of each event          Specifically: p(k) = Probab[ x = k ] p(k) is the probability that the outcome is equal to k (x is called a random variable)

    Example: the probability mass/density function for Binomial(p, n) is:

    p(k) = C(n, k) pk (1 - p)n-k n! = ---------- pk (1 - p)n-k k!(n-k)!

    ---

    ---

  • The (cumulative) probability distribution function is the cumulative sum of the values of the probability density function:

    Discrete p(x): Q(k) = Ҏ[ x ≤ k ] = &sum x ≤ k p(x) Continuous p(x): Q(k) = Ҏ[ x ≤ k ] = &int x ≤ k p(x) Note that: d Q(x) p(x) = ------ dx

    Property of every probability distribution function:

    lim (x → ∞) Q(x) = 1

    ---

    ---

  • Example: Binomial(0.5, 20)

  ---

  ---

  ---

  ---

• Expected value or the mean (average) number of a random variable
  • Definition: expected value

    The expected value of a random variable is the mean/average value of the random variable

  • Mathematical definition of expected value:

    x is a random variable with a density function Ҏ[x] (Ҏ[x] is a short hand for Ҏ[x = x]) The expected value E[x] of x is: E[x] = ∑ (all values k) k Ҏ[k]

  ---

  ---

• Conditional Probability
  • Definition:

    The conditional probability A for given B Ҏ[ A | B ] is the probability of the event A given that the event B is true

    ---

  • The conditional probability Ҏ[A | B] can be computed as follows (sometimes it is used as its definition):

    Conditional Probability Ҏ[A | B]:

    Ҏ[A &cap B] Ҏ[A | B] = ---------- Ҏ[B]

  ---

  ---

• Law of Total Probability (Law of Alternatives or Partitioning Theorem)
  • Law of Total Probability

    Let B1, B2, ... BN finite (or countably infinite) partition of a probability space Then: Ҏ[ A ] = Ҏ[ A | B1 ] × Ҏ[ B1 ] + Ҏ[ A | B2 ] × Ҏ[ B2 ] + ... + Ҏ[ A | BN ] × Ҏ[ BN ]

    Proof:

    Ҏ[ A ] = Ҏ[ A ∩ B1 ] + Ҏ[ A ∩ B2 ] + ... + Ҏ[ A ∩ BN ] = Ҏ[ A | B1 ] × Ҏ[ B1 ] + Ҏ[ A | B2 ] × Ҏ[ B2 ] + ... + Ҏ[ A | BN ] × Ҏ[ BN ]