Statistical Inference (part 2: 3.4–3.5)
Open in the Padhaiko app3.4 Hypothesis testing of mean (Single mean, two means, paired t-test and one-way)
Comparing sample means against a specified standard, comparing two competing designs or processes, and comparing three or more treatments are among the most common statistical tasks in engineering.
3.4.1 Single-Mean and Two-Mean Hypothesis Tests
| Test Scenario | Conditions | Test Statistic | Null Distribution / df |
|---|---|---|---|
| Single Mean () | Known , or large sample (use for ) | ||
| Single Mean () | Unknown , small sample (), Normal population | ||
| Two Independent Means () | Known or large samples () | ||
| Two Independent Means (Pooled -test) | Small samples, Normal populations, equal variances () | ||
| Two Independent Means (Welch's -test) | Small samples, Normal populations, unequal variances () | (Welch–Satterthwaite ) |
Pooled Sample Variance and Welch Degrees of Freedom
- Pooled Variance (): When two small independent samples come from populations with equal variance :
- Welch–Satterthwaite Degrees of Freedom: When :
3.4.2 Paired -Test (Dependent Samples)
When two measurements are recorded on the same experimental units (e.g., before vs. after firmware optimization on the same servers, or two sensors measuring the same specimens), observations are paired rather than independent.
Paired t-Test Procedure
Given matched pairs , form the difference for . Compute the sample mean difference and sample standard deviation :
To test (typically ), assuming , the test statistic is:
3.4
Six blade servers had their peak CPU temperature (in ) measured before and after applying a new thermal-paste compound under identical workload:
| Server | 1 | 2 | 3 | 4 | 5 | 6 |
|---|---|---|---|---|---|---|
| Before () | ||||||
| After () |
Test at whether the new thermal paste reduces peak temperature ().
State hypotheses: Let (temperature drop).
Compute paired differences , mean , and SD : The differences are :
Compute the paired -statistic and conclude:
For and one-tailed , the critical value is . Since , reject .
: reject . There is strong statistical evidence that the new thermal paste reduces peak CPU temperature.
3.4.3 One-Way Analysis of Variance (One-Way ANOVA)
When comparing the means of populations simultaneously ( vs. : at least two means differ), performing multiple pairwise -tests inflates the overall Type I error rate. One-Way ANOVA tests all means in a single -test by partitioning total data variability into between-group variation and within-group (random error) variation.
One-Way ANOVA Partition and F-Test
Suppose independent normal populations with common variance are sampled with sizes and total sample size . Let be the -th group mean and be the overall grand mean.
- Sum of Squares Identity:
where:
- Total Sum of Squares: ().
- Between-Groups (Treatment) SS: ().
- Within-Groups (Error) SS: ().
- Mean Squares and -Statistic:
Reject at significance level if (always right-tailed).
| Source of Variation | Sum of Squares () | Degrees of Freedom () | Mean Square () | -Ratio |
|---|---|---|---|---|
| Between Groups (Treatments) | ||||
| Within Groups (Error) | — | |||
| Total | — | — |
3.5
Four specimens from each of aluminum alloys were tested for fatigue life (in cycles):
- Alloy 1:
- Alloy 2:
- Alloy 3:
Perform a one-way ANOVA at to test whether mean fatigue life differs among the three alloys ().
Compute group means and grand mean ():
Compute and :
(Check: .)
Construct the ANOVA table and decide:
| Source | ||||
|---|---|---|---|---|
| Between Alloys | ||||
| Within Alloys (Error) | — | |||
| Total | — | — |
Since , reject .
: reject . There is significant evidence that mean fatigue life differs across the three alloys.
3.5 Goodness of fit tests and independence of attributes (Chi-square and Kolmogorov–Smirnov test)
3.5.1 Chi-Square () Goodness-of-Fit Test
The goodness-of-fit test checks whether observed categorical counts follow a hypothesized discrete or grouped distribution with probabilities ().
Chi-Square Goodness-of-Fit Test
Given total sample size and expected counts :
where is the number of population parameters estimated from the sample data.
- Validity condition: Every expected frequency should satisfy ; if any category has , merge adjacent categories before computing (and adjust accordingly).
- Reject (that data follow the hypothesized model) if .
3.5.2 Chi-Square () Test of Independence of Attributes
To test whether two categorical attributes ( rows) and ( columns) are statistically independent in a population, counts are cross-tabulated in an contingency table.
Chi-Square Test of Independence
Under : Attributes and are independent:
- Reject (independence) at level if .
- For a table () with small counts, Yates' continuity correction replaces with .
3.5.3 Kolmogorov–Smirnov (K–S) One-Sample Test
Unlike the test—which requires grouping continuous data into bins and needs large so —the Kolmogorov–Smirnov (K–S) test is an exact non-parametric goodness-of-fit test for continuous distributions that works directly on ungrouped sorted data even for small sample sizes.
Kolmogorov–Smirnov Test Statistic
Let be a completely specified continuous CDF (under ) and let be the empirical CDF. At the ordered sample points :
Reject at significance level if from the K–S critical table.
3.6
- ( Independence) A factory inspects soldered boards across two shifts and classifies defect severity:
| Shift | Minor | Major | Critical | Row Total |
|---|---|---|---|---|
| Day Shift | ||||
| Night Shift | ||||
| Column Total |
Test at whether defect severity is independent of shift ().
- (K–S Test) Five pseudorandom numbers are observed: . Use the Kolmogorov–Smirnov test at to test : the sample comes from ().
Part 1 — Test of Independence: Expected frequencies : since , both rows have expected counts:
Compute :
With , : reject (defect severity depends on shift).
Part 2 — K–S Test against ( for ):
Thus , , and . Since , fail to reject .
- : reject (defect severity and shift are dependent); 2. : fail to reject (consistent with ).
Practice Exercises — Mean Tests, ANOVA, Chi-Square & K–S
Compare the Chi-Square goodness-of-fit test and the Kolmogorov–Smirnov (K–S) test. Explain why one-way ANOVA is preferred over multiple two-sample -tests when comparing treatment means.
Goodness-of-Fit vs. K–S Test:
- test: Works on grouped/categorical counts ( vs. ), requires (large ), loses information through binning for continuous data, and naturally adjusts when parameters are estimated.
- K–S test: Evaluates directly on unbinned continuous data, is valid for very small , but requires to be continuous and completely specified (standard critical values change if parameters are estimated from data).
Why One-Way ANOVA Beats Multiple -Tests: Comparing means pairwise requires separate -tests. Even if each test uses , the family-wise Type I error probability grows rapidly (e.g., for , tests yield chance of at least one false positive). One-way ANOVA performs a single simultaneous -test that holds the Type I error strictly at .
The test requires binning and while the K–S test works unbinned on small continuous samples with a fully specified . One-way ANOVA tests all means simultaneously at level , avoiding the inflated Type I error of multiple pairwise -tests.
A structural engineer tests steel bolts and finds a mean yield torque of with sample standard deviation . Test at (two-tailed) whether the true mean differs from the specification ().
State hypotheses and test statistic ( unknown, ):
Compare with critical value (): Since , does not fall in the rejection region. Fail to reject .
: fail to reject . There is insufficient evidence at that the mean yield torque differs from .
Two sorting algorithms are benchmarked on independent workloads (normal populations with equal variances):
- Algorithm A: , , .
- Algorithm B: , , .
Test at (two-tailed) whether there is a significant difference between their mean runtimes ().
Compute pooled variance ():
Compute the two-sample -statistic ( vs. ):
Since , reject .
, : reject . Mean runtimes of Algorithms A and B differ significantly.
In a one-way ANOVA comparing network routing protocols with runs each (), the group mean latencies are , , and , and the within-group sum of squares is . Complete the ANOVA table and test at ().
Compute grand mean and :
Compute mean squares and -ratio:
Since , fail to reject .
, , , : fail to reject .
A six-sided die used in a Monte Carlo hardware simulator is rolled times with observed counts for faces through :
Use a Chi-Square goodness-of-fit test at to test whether the die is fair ().
Expected frequencies under ( for all faces):
Compute ():
Since , fail to reject .
: fail to reject . The data are consistent with a fair die at .