PadhaikoAll subjects

Statistical Inference (part 2: 3.4–3.5)

Probability and Statistics (ENSH 304)

Open in the Padhaiko app

3.4 Hypothesis testing of mean (Single mean, two means, paired t-test and one-way)

Comparing sample means against a specified standard, comparing two competing designs or processes, and comparing three or more treatments are among the most common statistical tasks in engineering.

3.4.1 Single-Mean and Two-Mean Hypothesis Tests

Summary of Parametric Tests for Population Means
Test ScenarioConditionsTest StatisticNull Distribution / df
Single Mean ()Known , or large sample (use for )
Single Mean ()Unknown , small sample (), Normal population
Two Independent Means ()Known or large samples ()
Two Independent Means (Pooled -test)Small samples, Normal populations, equal variances ()
Two Independent Means (Welch's -test)Small samples, Normal populations, unequal variances () (Welch–Satterthwaite )
Definition

Pooled Sample Variance and Welch Degrees of Freedom

  1. Pooled Variance (): When two small independent samples come from populations with equal variance :
  1. Welch–Satterthwaite Degrees of Freedom: When :

3.4.2 Paired -Test (Dependent Samples)

When two measurements are recorded on the same experimental units (e.g., before vs. after firmware optimization on the same servers, or two sensors measuring the same specimens), observations are paired rather than independent.

Definition

Paired t-Test Procedure

Given matched pairs , form the difference for . Compute the sample mean difference and sample standard deviation :

To test (typically ), assuming , the test statistic is:

Example 3.4

3.4

Six blade servers had their peak CPU temperature (in ) measured before and after applying a new thermal-paste compound under identical workload:

Server123456
Before ()
After ()

Test at whether the new thermal paste reduces peak temperature ().

State hypotheses: Let (temperature drop).

Compute paired differences , mean , and SD : The differences are :

Compute the paired -statistic and conclude:

For and one-tailed , the critical value is . Since , reject .

Answer:

: reject . There is strong statistical evidence that the new thermal paste reduces peak CPU temperature.

3.4.3 One-Way Analysis of Variance (One-Way ANOVA)

When comparing the means of populations simultaneously ( vs. : at least two means differ), performing multiple pairwise -tests inflates the overall Type I error rate. One-Way ANOVA tests all means in a single -test by partitioning total data variability into between-group variation and within-group (random error) variation.

Theorem

One-Way ANOVA Partition and F-Test

Suppose independent normal populations with common variance are sampled with sizes and total sample size . Let be the -th group mean and be the overall grand mean.

  1. Sum of Squares Identity:

where:

  1. Mean Squares and -Statistic:

Reject at significance level if (always right-tailed).

Standard One-Way ANOVA Table
Source of VariationSum of Squares ()Degrees of Freedom ()Mean Square ()-Ratio
Between Groups (Treatments)
Within Groups (Error)—
Total——
Example 3.5

3.5

Four specimens from each of aluminum alloys were tested for fatigue life (in cycles):

Perform a one-way ANOVA at to test whether mean fatigue life differs among the three alloys ().

Compute group means and grand mean ():

Compute and :

(Check: .)

Construct the ANOVA table and decide:

Source
Between Alloys
Within Alloys (Error)—
Total——

Since , reject .

Answer:

: reject . There is significant evidence that mean fatigue life differs across the three alloys.


3.5 Goodness of fit tests and independence of attributes (Chi-square and Kolmogorov–Smirnov test)

3.5.1 Chi-Square () Goodness-of-Fit Test

The goodness-of-fit test checks whether observed categorical counts follow a hypothesized discrete or grouped distribution with probabilities ().

Definition

Chi-Square Goodness-of-Fit Test

Given total sample size and expected counts :

where is the number of population parameters estimated from the sample data.

3.5.2 Chi-Square () Test of Independence of Attributes

To test whether two categorical attributes ( rows) and ( columns) are statistically independent in a population, counts are cross-tabulated in an contingency table.

Definition

Chi-Square Test of Independence

Under : Attributes and are independent:

3.5.3 Kolmogorov–Smirnov (K–S) One-Sample Test

Unlike the test—which requires grouping continuous data into bins and needs large so —the Kolmogorov–Smirnov (K–S) test is an exact non-parametric goodness-of-fit test for continuous distributions that works directly on ungrouped sorted data even for small sample sizes.

Definition

Kolmogorov–Smirnov Test Statistic

Let be a completely specified continuous CDF (under ) and let be the empirical CDF. At the ordered sample points :

Reject at significance level if from the K–S critical table.

Example 3.6

3.6

  1. ( Independence) A factory inspects soldered boards across two shifts and classifies defect severity:
ShiftMinorMajorCriticalRow Total
Day Shift
Night Shift
Column Total

Test at whether defect severity is independent of shift ().

  1. (K–S Test) Five pseudorandom numbers are observed: . Use the Kolmogorov–Smirnov test at to test : the sample comes from ().

Part 1 — Test of Independence: Expected frequencies : since , both rows have expected counts:

Compute :

With , : reject (defect severity depends on shift).

Part 2 — K–S Test against ( for ):

Thus , , and . Since , fail to reject .

Answer:
  1. : reject (defect severity and shift are dependent); 2. : fail to reject (consistent with ).

Practice Exercises — Mean Tests, ANOVA, Chi-Square & K–S

Question 1

Compare the Chi-Square goodness-of-fit test and the Kolmogorov–Smirnov (K–S) test. Explain why one-way ANOVA is preferred over multiple two-sample -tests when comparing treatment means.

Goodness-of-Fit vs. K–S Test:

  • test: Works on grouped/categorical counts ( vs. ), requires (large ), loses information through binning for continuous data, and naturally adjusts when parameters are estimated.
  • K–S test: Evaluates directly on unbinned continuous data, is valid for very small , but requires to be continuous and completely specified (standard critical values change if parameters are estimated from data).

Why One-Way ANOVA Beats Multiple -Tests: Comparing means pairwise requires separate -tests. Even if each test uses , the family-wise Type I error probability grows rapidly (e.g., for , tests yield chance of at least one false positive). One-way ANOVA performs a single simultaneous -test that holds the Type I error strictly at .

Answer:

The test requires binning and while the K–S test works unbinned on small continuous samples with a fully specified . One-way ANOVA tests all means simultaneously at level , avoiding the inflated Type I error of multiple pairwise -tests.

Question 2

A structural engineer tests steel bolts and finds a mean yield torque of with sample standard deviation . Test at (two-tailed) whether the true mean differs from the specification ().

State hypotheses and test statistic ( unknown, ):

Compare with critical value (): Since , does not fall in the rejection region. Fail to reject .

Answer:

: fail to reject . There is insufficient evidence at that the mean yield torque differs from .

Question 3

Two sorting algorithms are benchmarked on independent workloads (normal populations with equal variances):

Test at (two-tailed) whether there is a significant difference between their mean runtimes ().

Compute pooled variance ():

Compute the two-sample -statistic ( vs. ):

Since , reject .

Answer:

, : reject . Mean runtimes of Algorithms A and B differ significantly.

Question 4

In a one-way ANOVA comparing network routing protocols with runs each (), the group mean latencies are , , and , and the within-group sum of squares is . Complete the ANOVA table and test at ().

Compute grand mean and :

Compute mean squares and -ratio:

Since , fail to reject .

Answer:

, , , : fail to reject .

Question 5

A six-sided die used in a Monte Carlo hardware simulator is rolled times with observed counts for faces through :

Use a Chi-Square goodness-of-fit test at to test whether the die is fair ().

Expected frequencies under ( for all faces):

Compute ():

Since , fail to reject .

Answer:

: fail to reject . The data are consistent with a fair die at .

← Statistical Inference (part 1: 3.1–3.3)Correlation and Regression →