Probability Distributions and Sampling Distribution (part 1: 2.1–2.4)
Open in the Padhaiko app2.1 Random variables: Discrete and continuous
A random variable (RV) is a real-valued function that assigns a unique numerical value to every outcome in the sample space of a random experiment. Random variables allow us to analyze random phenomena using algebra and calculus rather than raw set descriptions.
| Aspect | Discrete Random Variable | Continuous Random Variable |
|---|---|---|
| Possible Values | Finite or countably infinite set (e.g., ) | Uncountably infinite values over an interval |
| Probability Function | Probability Mass Function (PMF) | Probability Density Function (PDF) |
| Point Probability | can be strictly positive | for every single point |
| Interval Probability | Sum: | Integral: |
| Engineering Examples | Number of defective chips, packet arrivals per second | Thermal noise voltage, shaft diameter, server latency |
2.1.1 Probability Mass Function (PMF) and Probability Density Function (PDF)
PMF, PDF, and Cumulative Distribution Function (CDF)
- Probability Mass Function (PMF) of a discrete RV : must satisfy:
- Probability Density Function (PDF) of a continuous RV : must satisfy:
Because for continuous RVs, .
- Cumulative Distribution Function (CDF): Defined for both discrete and continuous RVs as:
Properties of :
- , , and .
- is non-decreasing and right-continuous.
- For a continuous RV, wherever is differentiable, and .
2.2 Expectation and variance of discrete and continuous random variables
2.2.1 Mathematical Expectation
Expected Value and Moments
The expected value (or mean, ) of a random variable is the probability-weighted average of its possible values:
More generally, for any function :
- Linearity of Expectation: For any constants and any RVs (even if dependent):
- If and are independent, then .
2.2.2 Variance and Standard Deviation
Variance and Properties
The variance measures the expected squared deviation from the mean :
The standard deviation is .
Key Properties of Variance:
- for any constant .
- (shifting by does not change spread; scaling by scales variance by ).
- If and are independent, then:
(Notice that variances add even when independent RVs are subtracted!)
2.1
A continuous random variable representing normalized voltage drift has the probability density function:
Find: (a) the constant , (b) the mean and variance , and (c) .
(a) Determine using total probability :
(b) Compute , , and :
(c) Compute :
(a) ; (b) , ; (c) .
2.3 Discrete probability distributions: Binomial, Poisson, negative Binomial
2.3.1 Binomial Distribution
A Bernoulli trial has exactly two mutually exclusive outcomes: Success (probability ) and Failure (probability ). If independent identical Bernoulli trials are performed, the total number of successes follows a Binomial distribution, .
Binomial Distribution
(Note: For any Binomial distribution, whenever .)
2.3.2 Poisson Distribution
The Poisson distribution, , models the count of rare, independent random events occurring in a fixed interval of time, length, area, or volume (e.g., packet arrivals per second, flaws per meter of optical fiber), where is the average rate of occurrence.
Poisson Distribution and Limiting Approximation
- Poisson Approximation to Binomial: When is large () and is very small () such that stays moderate, .
2.3.3 Negative Binomial Distribution
Whereas the Binomial distribution fixes the number of trials and counts successes, the Negative Binomial distribution fixes the target number of successes and counts how many trials (or failures) occur until the -th success is achieved in independent Bernoulli trials with success probability .
Negative Binomial Distribution (Two Equivalent Formulations)
- Total-Trials Formulation ( trial number of the -th success, ):
- Failures-Before--th-Success Formulation ( number of failures, ):
- Geometric Distribution (): Setting gives the number of trials until the first success: , with .
| Distribution | Parameter(s) | Support | PMF | Mean | Variance |
|---|---|---|---|---|---|
| Binomial | |||||
| Poisson | |||||
| Negative Binomial (trials ) |
2.2
Solve the following three reliability and networking problems:
- (Binomial) Each transmitted data frame has an independent corruption probability of . Out of transmitted frames, find the probability that at least are corrupted.
- (Poisson) Server error logs record an average of warnings per hour. Find the probability of observing at most warnings in a randomly chosen hour.
- (Negative Binomial) A circuit board inspection passes a board with probability . Find the probability that the 3rd passing board is found on the 5th inspection.
Part 1 — Binomial (): Using the complement rule :
Part 2 — Poisson ():
Part 3 — Negative Binomial ():
- Binomial ; 2. Poisson ; 3. Negative Binomial .
2.4 Continuous probability distributions: Normal, Gamma, Chi-Square
2.4.1 Normal (Gaussian) Distribution
The Normal distribution, , is the cornerstone distribution of statistics because measurement errors and sums of many small independent effects tend toward normality.
Normal and Standard Normal Distributions
A continuous RV has a Normal distribution with mean and variance if its PDF is:
- Standardization: Defining transforms any normal RV into the Standard Normal RV with PDF and CDF .
- Symmetry: , and .
- Empirical () Rule:
2.4.2 Gamma Distribution
The Gamma distribution, , models positive skewed continuous quantities such as waiting times until the -th Poisson arrival, component lifetimes, and rainfall depths.
Gamma Function and Gamma Distribution
The Gamma function is (), satisfying:
A continuous RV follows a Gamma distribution with shape and scale if:
- Exponential Distribution (): (, rate ), with , , and the memoryless property .
2.4.3 Chi-Square () Distribution
Chi-Square Distribution
The Chi-Square distribution with degrees of freedom, denoted , is a special case of the Gamma distribution with shape and scale :
- Connection to Standard Normal: If are independent random variables, then .
- Additivity: If and are independent, then .
2.3
- (Normal) The breaking strength of steel rods follows . Find using standard normal values and .
- (Gamma) The time (in hours) until independent system failures occur follows a Gamma distribution with shape and scale . Find its mean, variance, and .
Part 1 — Standardize the Normal RV (, ):
Using symmetry :
Part 2 — Gamma Moments and Probability (, ):
- Mean and Variance: , and .
- PDF: Since , for .
With :
- ; 2. , , and .
Practice Exercises — Random Variables & Probability Distributions
Compare a Probability Mass Function (PMF), a Probability Density Function (PDF), and a Cumulative Distribution Function (CDF). Explain why for any continuous random variable, yet can exceed .
PMF vs. PDF vs. CDF:
- PMF (): Applies to discrete RVs; values are actual probabilities satisfying and .
- PDF (): Applies to continuous RVs; is probability per unit length (density), satisfying and .
- CDF (): Applies to all RVs; always lies in , is non-decreasing, and accumulates probability up to .
Why and is possible: For a continuous RV, because a single point has zero interval width. Meanwhile, is a density, not a probability: only its integral over an interval must be . For instance, a uniform distribution on has on , while .
A PMF gives point probabilities for discrete RVs, a PDF gives probability density for continuous RVs (whose integrals give probabilities), and a CDF gives for both. A continuous point has zero width so , while can exceed on a narrow interval as long as the total area equals .
A discrete random variable takes values with PMF .
- Find the constant .
- Compute and .
- Let . Find and .
Find from :
Thus , , , .
Compute and :
Compute and for :
- ; 2. , ; 3. , .
In a large batch of microcontrollers, fail final burn-in testing. If a quality engineer randomly samples units, use the Poisson approximation to the Binomial to find the probability that at least units fail.
Check approximation condition and find : Since is large and is small, use with:
Compute :
Using , .
An automated drilling rig strikes a hard rock layer independently on any bore with probability . Let be the number of bores required to achieve the -nd strike.
- Find .
- Find the expected number of bores and the variance .
Probability using Negative Binomial ():
Mean and Variance of :
- (); 2. , .
- A machining process produces shafts whose length follows . Find (given ) and find the 95th percentile such that (given ).
- If (Chi-Square with degrees of freedom), state its mean, variance, and .
Part 1 — Normal probabilities and percentile:
- For : , so:
- For the 95th percentile: .
Part 2 — Chi-Square moments (): For with :
Since :
- , ; 2. , , .