PadhaikoAll subjects

Statistical Inference (part 1: 3.1–3.3)

Probability and Statistics (ENSH 304)

Open in the Padhaiko app

3.1 Point estimations and properties of estimators

Statistical inference uses sample data to draw conclusions about unknown population parameters (, such as , , or ). An estimator is a statistic (random variable) used to approximate ; a point estimate is the single numerical value obtained when is evaluated on an observed sample .

3.1.1 Properties of Good Point Estimators

Definition

Unbiasedness, Consistency, Efficiency, and Sufficiency

Let be an estimator of an unknown parameter based on a random sample of size :

  1. Unbiasedness: is an unbiased estimator of if its expected value equals the true parameter for all possible values of :

The bias is . The overall accuracy of any estimator (biased or unbiased) is measured by its Mean Squared Error ():

(Examples: Sample mean , sample proportion , and sample variance are unbiased; is biased by .)

  1. Consistency: A sequence of estimators is consistent for if converges in probability to as sample size increases ( for every ).
    • Sufficient condition: If and (i.e., ), then is consistent.
  2. Efficiency: Between two unbiased estimators and , is more efficient if . The relative efficiency of with respect to is:

An unbiased estimator achieving the smallest possible variance among all unbiased estimators is a Minimum Variance Unbiased Estimator (MVUE) (attaining the Cramér–Rao Lower Bound under regularity conditions).

  1. Sufficiency: An estimator is sufficient for if it extracts all information in the sample relevant to —meaning the conditional distribution of the sample given does not depend on . By the Neyman–Fisher Factorization Theorem, is sufficient iff the joint likelihood factors as .

3.1.2 Common Methods of Point Estimation

Example 3.1

3.1

Let be an i.i.d. random sample from a population with unknown mean and finite variance . Consider two candidate point estimators for :

  1. Check whether and are unbiased estimators of .
  2. Compute their variances and determine which estimator is more efficient (and its relative efficiency).
  3. Explain whether is consistent as .

Check unbiasedness using linearity of expectation ():

Both and are unbiased estimators of .

Compute variances using independence ():

Since , is more efficient, with relative efficiency:

Check consistency of : For any sample size , and . Therefore, is a consistent estimator of .

Answer:

Both and are unbiased; is more efficient (, relative efficiency ); is consistent because .


3.2 Confidence intervals for mean and proportions

A point estimate alone gives no indication of its precision. An interval estimate or Confidence Interval (CI) takes the form:

where is the confidence level (commonly , , or , corresponding to and two-tailed ).

3.2.1 Confidence Interval Formulas

Confidence Intervals for Population Mean and Proportion
Target ParameterConditionsConfidence Interval FormulaStandard Error ()
Mean (known , or large )Normal population or CLT (); if unknown and , replace by
Mean (unknown , small )Normal population; degrees of freedom
Proportion (large sample) and
Example 3.2

3.2

  1. (Mean, small sample) Tensile tests on welded joints give sample mean and sample standard deviation . Assuming normality, construct a confidence interval for the true mean tensile strength ().
  2. (Proportion) Out of tested PCB assemblies, fail automated optical inspection. Construct a confidence interval for the true defect proportion ().
  3. (Sample size) What minimum sample size is needed to estimate a proportion within at confidence () if no prior estimate of is available?

Part 1 — -interval for (, ):

Part 2 — -interval for (): Since and :

Part 3 — Conservative sample size for proportion ():

Answer:
  1. CI for : ; 2. CI for : ; 3. Minimum sample size: .

3.3 Hypothesis testing, parametric and non-parametric tests, procedure of hypothesis

A statistical hypothesis is a testable claim about one or more population parameters or the shape of a probability distribution. Hypothesis testing evaluates whether sample evidence is strong enough to reject a baseline claim.

3.3.1 Null and Alternative Hypotheses, Errors, and -Value

Definition

Hypotheses, Type I and Type II Errors, Power, and p-Value

  1. Null Hypothesis (): The baseline status-quo statement of "no difference" or "no effect" (always contains equality: , , or ). Tested directly under the assumption that is true.
  2. Alternative Hypothesis ( or ): The research or engineering claim of change/difference ( for a two-tailed test; for right-tailed; for left-tailed).
  3. Decision Errors:
    • Type I Error (, Producer's Risk): Rejecting when is actually true. is the level of significance.
    • Type II Error (, Consumer's Risk): Failing to reject when is actually true. .
    • Power of the Test (): Probability of correctly rejecting a false : .
  4. -Value: The smallest significance level at which would be rejected for the observed data—equivalently, the probability (assuming is true) of obtaining a test statistic at least as extreme as the observed value. Decision rule: Reject if .
Decision Matrix in Hypothesis Testing
Test Decision is Actually True is Actually False ( True)
Fail to Reject Correct decision ()Type II Error ()
Reject Type I Error ()Correct decision (Power )

3.3.2 General Procedure of Hypothesis Testing

Every formal hypothesis test follows six standard steps:

  1. State and clearly in terms of population parameters (and identify whether the test is two-tailed, right-tailed, or left-tailed).
  2. Choose the significance level (typically or ).
  3. Select the appropriate test statistic (e.g., , , , ) and verify its assumptions (normality, independence, sample size).
  4. Determine the critical (rejection) region from statistical tables (or set the -value threshold ).
  5. Compute the test statistic (and optionally the -value) from the sample data.
  6. Make a statistical decision and engineering conclusion: Reject if the test statistic falls in the critical region (or ); otherwise fail to reject . State the conclusion in the physical context of the problem.

3.3.3 Parametric vs. Non-Parametric Tests

AspectParametric TestsNon-Parametric Tests
Distributional AssumptionAssumes specific underlying population distribution (usually Normal) and interval/ratio scaleMakes no strict assumption about population shape; works on nominal/ordinal/ranked data
Target QuantityParameters such as mean () or variance ()Medians, ranks, signs, empirical distributions (), or independence
Statistical PowerHigher power when assumptions holdSlightly lower power on normal data, but more robust to outliers and heavy skew
Typical Examples-test, -tests (1-sample, 2-sample, paired), one-way ANOVA (-test)Sign test, Wilcoxon tests, Mann–Whitney , Kruskal–Wallis, Chi-square (), Kolmogorov–Smirnov
Example 3.3

3.3

A chemical process has standard yield mean with known standard deviation . To test against using a sample of size , an engineer uses the decision rule: reject if .

  1. Find the significance level (Type I error probability).
  2. Find the Type II error probability and the power of the test when the true mean yield has shifted to (, ).

Standard error of :

Part 1 — Type I Error () under :

Part 2 — Type II Error () and Power () under : The test fails to reject when :

Using :

Answer:
  1. Significance level ; 2. Type II error and Power ().

Practice Exercises — Point Estimation, Confidence Intervals & Hypothesis Concepts

Question 1

Define unbiasedness, consistency, efficiency, and sufficiency of a point estimator. Give the relationship between , , and bias, and state why the denominator is used in sample variance .

Four Properties of Estimators:

  • Unbiasedness: for all .
  • Consistency: as (implied if bias and ).
  • Efficiency: Smallest variance among unbiased estimators; is more efficient than if .
  • Sufficiency: captures all information in the sample about .

MSE Identity and Sample Variance:

Because , dividing by yields , making unbiased.

Answer:

The four properties are unbiasedness (), consistency (), efficiency (minimum variance), and sufficiency (retains all sample information about ). , and divides by to achieve .

Question 2

Given two independent observations from a population with mean and variance , compare the estimators and . Verify both are unbiased and compute the relative efficiency of with respect to .

Check unbiasedness:

Both estimators are unbiased.

Compare variances and relative efficiency:

Answer:

Both are unbiased; has smaller variance () and is more efficient with relative efficiency .

Question 3

A sample of steel cables has mean breaking load . Assume the population standard deviation is .

  1. Construct a confidence interval for the true mean breaking load ().
  2. How large a sample is required if the margin of error must not exceed ?

Part 1 — Confidence Interval:

Part 2 — Required sample size for :

Answer:
  1. CI: ; 2. Minimum sample size: .
Question 4

In a usability survey of engineering students, reported experiencing compiler timeout issues during lab exams. Construct a confidence interval for the true population proportion ().

Compute and :

Compute CI:

Answer:

The confidence interval for is (or to ).

Question 5

A machine fills resin cartridges to a target mean with . To test against with , the quality team rejects whenever .

  1. Compute the Type I error probability ().
  2. Compute the Type II error probability and the power if the true mean drops to ().

Compute under ():

Compute and Power under :

Answer:
  1. ; 2. and Power ().
← Probability Distributions and Sampling Distribution (part 2: 2.5–2.7)Statistical Inference (part 2: 3.4–3.5) →