Descriptive Statistics and Basic Probability
Open in the Padhaiko app1.1 Introduction to statistics and its importance in engineering
Statistics is the science of collecting, organizing, summarizing, analyzing, and interpreting data in order to draw reliable conclusions and make rational decisions in the presence of uncertainty. In engineering, measurements and processes always exhibit variability—from resistor tolerances and concrete compressive strengths to server response latencies and traffic flow rates—making statistical thinking essential across every engineering discipline.
Branches of Statistics
- Descriptive Statistics: Methods for organizing, summarizing, and presenting collected data in an informative way using tables, graphs (histograms, box plots, scatter plots), and numerical summary measures (mean, median, variance, standard deviation).
- Inferential Statistics: Methods for drawing conclusions, estimating parameters, and testing hypotheses about a large population based on observations from a representative sample, while quantifying uncertainty using probability.
1.1.1 Types of Data and Scales of Measurement
Engineering data are classified by nature and by measurement scale, which determines which statistical summaries and plots are valid:
| Category / Scale | Defining Property | Permissible Operations | Engineering Examples |
|---|---|---|---|
| Qualitative (Categorical) | Non-numerical labels or categories | Frequency counts, mode, proportions | Defect type (solder bridge, crack), protocol (TCP, UDP) |
| Quantitative Discrete | Countable numeric values (usually integers) | Addition, counting, mean, variance | Number of defective chips in a lot, packets dropped per second |
| Quantitative Continuous | Measured on an unbroken continuum | All arithmetic and calculus operations | Voltage (), beam deflection (), CPU temperature () |
| Nominal Scale | Labels with no intrinsic order | Equality (), mode | Machine ID (M1, M2, M3), material grade code |
| Ordinal Scale | Ranked categories without fixed intervals | Order (), median, percentiles | Severity rating (Low, Medium, High), surface finish rank |
| Interval Scale | Equal intervals, arbitrary zero point | Differences (), mean, standard deviation | Temperature in or , calendar year |
| Ratio Scale | Equal intervals with a true absolute zero | Ratios (), geometric mean, CV | Mass (), Kelvin temperature (), latency (), resistance () |
1.1.2 Importance of Statistics in Engineering
- Quality Control and Manufacturing: Monitoring production tolerances with control charts (, , -charts) and Six Sigma methods to minimize defects.
- Reliability and Life Testing: Modeling component failure times and calculating system reliability for series and parallel architectures.
- Experimental Design and Process Optimization: Separating true engineering effects (e.g., alloy composition or curing temperature) from random measurement noise.
- Signal Processing, Networks, and Data Analysis: Characterizing noise distributions, packet arrival rates, queueing delays, and sensor calibration errors.
- Civil, Mechanical, and Environmental Planning: Analyzing flood return periods, wind loads, traffic volumes, and structural safety factors.
1.2 Measure of central tendency and measure of variation
Summarizing a dataset requires two complementary families of measures: central tendency (where the data cluster) and variation/dispersion (how widely the data spread around the center).
1.2.1 Measures of Central Tendency
Mean, Median, and Mode
For an ungrouped dataset and a grouped frequency distribution with class midpoints , frequencies , and total frequency :
- Arithmetic Mean:
- Ungrouped sample mean: (population mean denoted ).
- Grouped mean: .
- Median (): The middle value when observations are arranged in ascending order (insensitive to extreme outliers).
- Ungrouped: the -th value if is odd; the average of the -th and -th values if is even.
- Grouped: locate the median class where cumulative frequency , then
where is the lower boundary of the median class, is the cumulative frequency before the median class, is the median class frequency, and is the class width.
- Mode (): The most frequently occurring value (useful for categorical and peak-demand analysis).
- Grouped: locate the modal class with maximum frequency , preceded by and followed by :
- Empirical Relation (Moderately Skewed Data):
- Geometric Mean () and Harmonic Mean ():
- is used for average growth rates, ratios, and compounding indices ().
- is used for averaging rates over fixed distance/work (e.g., average speed over equal segments).
| Property | Arithmetic Mean () | Median () | Mode () |
|---|---|---|---|
| Uses All Observations | Yes | No (positional) | No (peak frequency) |
| Effect of Outliers | Sensitive (pulled toward tail) | Robust (resistant) | Insensitive |
| Open-Ended Classes | Cannot be computed directly | Can be computed | Can be computed |
| Best Used When | Data are roughly symmetric | Data are skewed or contain outliers | Identifying the most common class/size |
1.2.2 Measures of Variation (Dispersion)
Two datasets can have the exact same mean yet behave completely differently if one is tightly clustered and the other is widely scattered.
Range, IQR, Variance, Standard Deviation, and CV
- Range: . Simple to compute, but depends solely on the two extreme values.
- Interquartile Range () and Quartile Deviation ():
where (25th percentile) and (75th percentile) bound the middle of the data.
- Sample Variance () and Standard Deviation ():
For grouped data with class frequencies and midpoints ():
(For an entire population of size , divide by instead of to get and .)
- Coefficient of Variation (): A dimensionless relative measure of dispersion used to compare variability across datasets with different units or widely different means:
1.1
The lifetimes (in hundreds of hours) of electronic capacitors tested in a reliability lab are grouped as follows:
| Lifetime () | |||||
|---|---|---|---|---|---|
| Frequency () |
Compute the mean, median, mode, sample standard deviation, and coefficient of variation ().
Construct the computation table using midpoints :
| Class | ||||||
|---|---|---|---|---|---|---|
| Total | — | — | — |
Compute Mean, Median, and Mode:
- Arithmetic Mean:
- Median: Since , the cumulative frequency first occurs in class (, , , ):
- Mode: The highest frequency is in class (, ):
Compute Sample Variance, Standard Deviation, and Coefficient of Variation:
Mean , Median , Mode (in hundreds of hours), sample standard deviation (), and .
1.3 Graphical representation of data: Histograms, box plots and scatter plots
Visualizing data reveals distribution shape, skewness, multimodality, outliers, and variable relationships that single summary numbers can hide.
1.3.1 Histograms
A histogram displays the frequency distribution of a continuous quantitative variable using contiguous vertical rectangles erected over class intervals with no gaps between bars.
- Equal class widths: Rectangle height is proportional to class frequency (or relative frequency ).
- Unequal class widths: Rectangle area must represent frequency, so the vertical axis must plot frequency density:
- Shape diagnostics: A histogram immediately shows whether data are symmetric (bell-shaped), positively (right) skewed (long right tail; ), negatively (left) skewed (long left tail; ), or bimodal (two distinct peaks, often indicating two mixed production shifts or machines).
1.3.2 Box Plots (Box-and-Whisker Plots)
Five-Number Summary and Tukey Outlier Fences
A box plot summarizes a distribution using five numbers:
- The central box spans from to (length ), with a line inside marking the median .
- Outlier fences (Tukey's rule):
- Any observation outside is flagged as an outlier (plotted as an individual point). The whiskers extend from and to the smallest and largest non-outlier observations within the fences.
1.3.3 Scatter Plots
A scatter plot plots paired observations of two quantitative variables on Cartesian axes to inspect bivariate relationships prior to correlation or regression modeling:
- Direction: Positive association ( increases with ), negative association ( decreases as increases), or no apparent trend.
- Form: Linear, curvilinear (e.g., quadratic or exponential), or clustered.
- Strength and Outliers: Tight clustering along a curve indicates strong association; isolated points far from the main cloud identify bivariate outliers or faulty sensor readings.
| Plot Type | Variables Shown | Primary Strength | Key Limitation |
|---|---|---|---|
| Histogram | Single continuous variable | Shows exact shape, peaks (multimodality), and gaps | Appearance depends on chosen bin width |
| Box Plot | Single variable (great across multiple groups) | Compact five-number summary, side-by-side comparison, explicit outlier detection | Hides multimodality and detailed shape inside the box |
| Scatter Plot | Two quantitative variables | Reveals linear/nonlinear trends, clusters, and bivariate outliers | Can suffer overplotting for huge datasets |
1.2
Eleven API response latencies (in ) recorded from an engineering microservice are already sorted in ascending order:
Construct the five-number summary, compute the , determine the outlier fences, and identify any outliers and whisker endpoints.
Locate the median () and quartiles (): With ordered observations:
- , .
- Median (): The -th observation is .
- Lower half ( observations below ): . Its middle (3rd) value is .
- Upper half ( observations above ): . Its middle (3rd) value is .
Compute and Tukey fences:
Identify outliers and whisker endpoints:
- Every value is , so there are no lower outliers; the lower whisker extends to .
- Since , the observation is an upper outlier.
- The largest non-outlier observation within is , so the upper whisker extends to .
Five-number summary: ; ; outlier fences ; is an outlier, and the whiskers span from to .
1.4 Basic probability concepts, additive law, multiplicative law
1.4.1 Sample Space, Events, and Probability Definitions
Sample Space, Events, and Axioms of Probability
- Random Experiment: A process whose outcome cannot be predicted with certainty beforehand (e.g., testing a batch of LEDs, measuring packet delay).
- Sample Space (): The set of all possible outcomes of a random experiment.
- Event (): Any subset of the sample space. Events and are mutually exclusive (disjoint) if , and exhaustive if .
- Kolmogorov's Axioms of Probability: A probability measure assigns a real number to every event such that:
- Non-negativity: for all .
- Normalization: .
- Countable Additivity: For mutually exclusive events ( for ),
Three classical interpretations complement the axiomatic foundation:
- Classical (A Priori): If has equally likely outcomes and event has favorable outcomes, .
- Empirical (Relative Frequency): , where is the count of occurrences in repeated trials.
- Subjective: Degree of belief based on engineering judgment when experiments cannot be repeated identically.
From the axioms, immediate corollaries follow: , , , and if then .
1.4.2 Additive Law of Probability
Additive Law of Probability
For any two events and :
- Special case (mutually exclusive events, ): .
- Three events :
- De Morgan's probability identities: and .
1.4.3 Multiplicative Law and Independence
Multiplicative Law of Probability
For any two events and with and :
For three events: .
Two events and are statistically independent if the occurrence of one does not alter the probability of the other (), which is equivalent to:
(Note: Mutually exclusive events with and are never independent, because .)
1.5 Conditional probability and Bayes’ theorem
1.5.1 Conditional Probability
Conditional Probability
Given an event with , the conditional probability of given that has occurred is:
Restricting attention to shrinks the sample space from to , re-normalizing probabilities by dividing by .
1.5.2 Law of Total Probability and Bayes’ Theorem
Suppose events form a partition of the sample space : they are pairwise mutually exclusive ( for ), exhaustive (), and each has .
Law of Total Probability and Bayes' Theorem
For any event with and any partition of :
- Law of Total Probability:
- Bayes' Theorem (Posterior Probability):
Here is the prior probability, is the likelihood, and is the posterior probability updated after observing evidence .
1.3
In a semiconductor plant, three machines , , and manufacture , , and of the total daily output of resistors, respectively. Historical quality records show that , , and of the outputs from , , and are defective. A resistor is selected at random from the combined daily production and found to be defective.
- Find the probability that a randomly chosen resistor is defective.
- Find the probability that the defective resistor was produced by machine .
Define events and list priors and likelihoods: Let be the events that the resistor was manufactured by machine , and let be the event that it is defective:
Apply the Law of Total Probability to find :
Apply Bayes' Theorem to find :
- Probability that a randomly chosen resistor is defective: ().
- Probability that the defective resistor came from : ().
Practice Exercises — Descriptive Statistics & Basic Probability
Distinguish between descriptive and inferential statistics. State the four scales of measurement with one engineering example of each, and explain how a box plot identifies skewness and outliers.
Descriptive vs. Inferential Statistics:
- Descriptive statistics summarizes and visualizes observed data (mean, standard deviation, histograms, box plots) without generalizing beyond the dataset.
- Inferential statistics uses probability models to estimate population parameters, build confidence intervals, and test hypotheses from a sample.
Four Measurement Scales:
- Nominal: Categorical labels without order (e.g., network fault codes or machine IDs).
- Ordinal: Ordered categories with unequal/undefined intervals (e.g., weld quality grade: Poor, Fair, Good).
- Interval: Ordered numeric scale with equal units and an arbitrary zero (e.g., Celsius temperature).
- Ratio: Ordered numeric scale with equal units and a true zero (e.g., power in watts, latency in ms).
Skewness and Outliers in a Box Plot:
- Outliers are points falling outside , where .
- Right skewness is indicated when the median lies closer to than to and the upper whisker is longer; left skewness shows the opposite pattern.
Descriptive statistics summarizes observed data, whereas inferential statistics draws sample-based conclusions about a population. The four scales are Nominal, Ordinal, Interval, and Ratio. A box plot flags outliers outside and indicates skewness via asymmetry of the median inside the box and unequal whisker lengths.
Eight AC mains voltage readings (in ) measured at a substation are:
Calculate the sample mean (), sample variance (), sample standard deviation (), and coefficient of variation ().
Compute the sample mean ():
Compute squared deviations from :
Compute , , and :
, , , and .
Eleven concrete cube compressive strength tests (in ) yield the following values:
Determine the five-number summary, compute the , and test whether any reading is an outlier using Tukey's rule.
Sort the values in ascending order:
- , .
- Median (, 6th value) .
- Lower half .
- Upper half .
Compute and outlier fences:
Since and all other values lie inside , is an upper outlier.
Five-number summary: ; ; fences ; is an outlier.
A control subsystem consists of a primary controller connected in series with a redundant backup pair of sensors and connected in parallel. All three components fail independently with operating probabilities , , and .
- Using the additive law, find the probability that the parallel sensor stage works.
- Using the multiplicative law, find the overall reliability of the system.
Probability that the parallel sensor stage ( or ) works: Since and are independent, . By the additive law:
(Equivalently, .)
Overall series-parallel system reliability: The system operates if and only if works and works. By independence:
- Parallel sensor stage reliability: .
- Overall system reliability: ().
An industrial vibration sensor monitors bearing faults. In a plant, of bearings develop a fault (). When a fault is present, the monitoring alarm triggers with probability ; when no fault is present (), a false alarm triggers with probability . If the alarm triggers for a randomly inspected bearing, what is the probability that the bearing actually has a fault?
Identify prior and conditional probabilities:
where is the event that the alarm triggers.
Compute via the Law of Total Probability:
Compute via Bayes' Theorem:
The probability that the bearing actually has a fault given the alarm is ().