PadhaikoAll subjects

Probability Distributions and Sampling Distribution (part 1: 2.1–2.4)

Probability and Statistics (ENSH 304)

Open in the Padhaiko app

2.1 Random variables: Discrete and continuous

A random variable (RV) is a real-valued function that assigns a unique numerical value to every outcome in the sample space of a random experiment. Random variables allow us to analyze random phenomena using algebra and calculus rather than raw set descriptions.

AspectDiscrete Random VariableContinuous Random Variable
Possible ValuesFinite or countably infinite set (e.g., )Uncountably infinite values over an interval
Probability FunctionProbability Mass Function (PMF) Probability Density Function (PDF)
Point Probability can be strictly positive for every single point
Interval ProbabilitySum: Integral:
Engineering ExamplesNumber of defective chips, packet arrivals per secondThermal noise voltage, shaft diameter, server latency

2.1.1 Probability Mass Function (PMF) and Probability Density Function (PDF)

Definition

PMF, PDF, and Cumulative Distribution Function (CDF)

  1. Probability Mass Function (PMF) of a discrete RV : must satisfy:
  1. Probability Density Function (PDF) of a continuous RV : must satisfy:

Because for continuous RVs, .

  1. Cumulative Distribution Function (CDF): Defined for both discrete and continuous RVs as:

Properties of :


2.2 Expectation and variance of discrete and continuous random variables

2.2.1 Mathematical Expectation

Definition

Expected Value and Moments

The expected value (or mean, ) of a random variable is the probability-weighted average of its possible values:

More generally, for any function :

2.2.2 Variance and Standard Deviation

Definition

Variance and Properties

The variance measures the expected squared deviation from the mean :

The standard deviation is .

Key Properties of Variance:

  1. for any constant .
  2. (shifting by does not change spread; scaling by scales variance by ).
  3. If and are independent, then:

(Notice that variances add even when independent RVs are subtracted!)

Example 2.1

2.1

A continuous random variable representing normalized voltage drift has the probability density function:

Find: (a) the constant , (b) the mean and variance , and (c) .

(a) Determine using total probability :

(b) Compute , , and :

(c) Compute :

Answer:

(a) ; (b) , ; (c) .


2.3 Discrete probability distributions: Binomial, Poisson, negative Binomial

2.3.1 Binomial Distribution

A Bernoulli trial has exactly two mutually exclusive outcomes: Success (probability ) and Failure (probability ). If independent identical Bernoulli trials are performed, the total number of successes follows a Binomial distribution, .

Definition

Binomial Distribution

(Note: For any Binomial distribution, whenever .)

2.3.2 Poisson Distribution

The Poisson distribution, , models the count of rare, independent random events occurring in a fixed interval of time, length, area, or volume (e.g., packet arrivals per second, flaws per meter of optical fiber), where is the average rate of occurrence.

Definition

Poisson Distribution and Limiting Approximation

2.3.3 Negative Binomial Distribution

Whereas the Binomial distribution fixes the number of trials and counts successes, the Negative Binomial distribution fixes the target number of successes and counts how many trials (or failures) occur until the -th success is achieved in independent Bernoulli trials with success probability .

Definition

Negative Binomial Distribution (Two Equivalent Formulations)

  1. Total-Trials Formulation ( trial number of the -th success, ):
  1. Failures-Before--th-Success Formulation ( number of failures, ):
Summary of Discrete Probability Distributions
DistributionParameter(s)SupportPMFMeanVariance
Binomial
Poisson
Negative Binomial (trials )
Example 2.2

2.2

Solve the following three reliability and networking problems:

  1. (Binomial) Each transmitted data frame has an independent corruption probability of . Out of transmitted frames, find the probability that at least are corrupted.
  2. (Poisson) Server error logs record an average of warnings per hour. Find the probability of observing at most warnings in a randomly chosen hour.
  3. (Negative Binomial) A circuit board inspection passes a board with probability . Find the probability that the 3rd passing board is found on the 5th inspection.

Part 1 — Binomial (): Using the complement rule :

Part 2 — Poisson ():

Part 3 — Negative Binomial ():

Answer:
  1. Binomial ; 2. Poisson ; 3. Negative Binomial .

2.4 Continuous probability distributions: Normal, Gamma, Chi-Square

2.4.1 Normal (Gaussian) Distribution

The Normal distribution, , is the cornerstone distribution of statistics because measurement errors and sums of many small independent effects tend toward normality.

Definition

Normal and Standard Normal Distributions

A continuous RV has a Normal distribution with mean and variance if its PDF is:

2.4.2 Gamma Distribution

The Gamma distribution, , models positive skewed continuous quantities such as waiting times until the -th Poisson arrival, component lifetimes, and rainfall depths.

Definition

Gamma Function and Gamma Distribution

The Gamma function is (), satisfying:

A continuous RV follows a Gamma distribution with shape and scale if:

2.4.3 Chi-Square () Distribution

Definition

Chi-Square Distribution

The Chi-Square distribution with degrees of freedom, denoted , is a special case of the Gamma distribution with shape and scale :

Example 2.3

2.3

  1. (Normal) The breaking strength of steel rods follows . Find using standard normal values and .
  2. (Gamma) The time (in hours) until independent system failures occur follows a Gamma distribution with shape and scale . Find its mean, variance, and .

Part 1 — Standardize the Normal RV (, ):

Using symmetry :

Part 2 — Gamma Moments and Probability (, ):

  • Mean and Variance: , and .
  • PDF: Since , for .

With :

Answer:
  1. ; 2. , , and .

Practice Exercises — Random Variables & Probability Distributions

Question 1

Compare a Probability Mass Function (PMF), a Probability Density Function (PDF), and a Cumulative Distribution Function (CDF). Explain why for any continuous random variable, yet can exceed .

PMF vs. PDF vs. CDF:

  • PMF (): Applies to discrete RVs; values are actual probabilities satisfying and .
  • PDF (): Applies to continuous RVs; is probability per unit length (density), satisfying and .
  • CDF (): Applies to all RVs; always lies in , is non-decreasing, and accumulates probability up to .

Why and is possible: For a continuous RV, because a single point has zero interval width. Meanwhile, is a density, not a probability: only its integral over an interval must be . For instance, a uniform distribution on has on , while .

Answer:

A PMF gives point probabilities for discrete RVs, a PDF gives probability density for continuous RVs (whose integrals give probabilities), and a CDF gives for both. A continuous point has zero width so , while can exceed on a narrow interval as long as the total area equals .

Question 2

A discrete random variable takes values with PMF .

  1. Find the constant .
  2. Compute and .
  3. Let . Find and .

Find from :

Thus , , , .

Compute and :

Compute and for :

Answer:
  1. ; 2. , ; 3. , .
Question 3

In a large batch of microcontrollers, fail final burn-in testing. If a quality engineer randomly samples units, use the Poisson approximation to the Binomial to find the probability that at least units fail.

Check approximation condition and find : Since is large and is small, use with:

Compute :

Answer:

Using , .

Question 4

An automated drilling rig strikes a hard rock layer independently on any bore with probability . Let be the number of bores required to achieve the -nd strike.

  1. Find .
  2. Find the expected number of bores and the variance .

Probability using Negative Binomial ():

Mean and Variance of :

Answer:
  1. (); 2. , .
Question 5
  1. A machining process produces shafts whose length follows . Find (given ) and find the 95th percentile such that (given ).
  2. If (Chi-Square with degrees of freedom), state its mean, variance, and .

Part 1 — Normal probabilities and percentile:

  • For : , so:
  • For the 95th percentile: .

Part 2 — Chi-Square moments (): For with :

Since :

Answer:
  1. , ; 2. , , .
← Descriptive Statistics and Basic ProbabilityProbability Distributions and Sampling Distribution (part 2: 2.5–2.7) →