PadhaikoAll subjects

Machine Learning: Supervised, Unsupervised & Reinforcement

Foundation of Data Science

Open in the Padhaiko app

6.1 Introduction to Machine Learning

Traditional computing relies on deterministic programming: a human software engineer formulates precise mathematical rules, conditional branches, and algorithms to transform known inputs into deterministic outputs. However, in many real-world domains—such as speech recognition, autonomous navigation, credit fraud detection, and medical pathology diagnosis—the underlying decision boundaries and physical laws are far too intricate or high-dimensional to be specified by hand.

Machine Learning (ML) provides an alternative computational paradigm by enabling computers to discover empirical patterns, decision boundaries, and statistical relationships directly from observational data, improving their inferential performance over time without being explicitly programmed with hand-crafted rules.

Definition

Machine Learning (Operational and Formal Definitions)

Operational Definition: Machine Learning (ML) is a subfield of artificial intelligence that builds statistical and computational systems capable of automatically extracting patterns, learning generalized functional mappings, and making predictions or decisions from data rather than following static, hardcoded instructions.

Formal Definition (Tom M. Mitchell, 1997): A computer program is said to learn from experience with respect to some class of tasks and performance measure , if its performance at tasks in , as measured by , improves with experience :

Example (Autonomous Braking System):

Traditional Programming vs. Machine Learning

The fundamental distinction between traditional software engineering and machine learning lies in the direction of dataflow and the origin of program logic.

AspectTraditional ProgrammingMachine Learning Paradigm
Dataflow Architecture
Logic OriginHuman programmer explicitly designs algorithms and control flow (if-else statements, equations).Algorithm optimizes internal mathematical parameters (weights, splits, coefficients) from data.
AdaptabilityRigid: any shift in operating environment requires manual code refactoring and redeployment.Dynamic: model can be periodically retrained or fine-tuned on newly collected operational records.
Problem DomainProblems with deterministic rules, clear logic, and low complexity (e.g., payroll tax calculation, sorting).Problems with perceptual ambiguity, high dimensionality, and latent patterns (e.g., computer vision, speech).
Scalability to ComplexityRapidly degrades as edge cases and input feature dimensions increase.Scales effectively with large volumes of high-dimensional sensor data and feature spaces.
Traditional Programming:
  [Data]  --------\
                   +---> [ Computer / CPU ] ---> [ Output / Result ]
  [Rules] --------/

Machine Learning:
  [Data]   -------\
                   +---> [ Learning Algorithm ] ---> [ Learned Model / Rules ]
  [Output] -------/

Artificial Intelligence vs. Machine Learning

In technical literature, Artificial Intelligence and Machine Learning are often conflated, but they exist in a strict hierarchical relationship.

Definition

Hierarchical Relationship: AI vs. ML

Artificial Intelligence (AI): The overarching academic discipline and computational paradigm devoted to synthesizing artificial systems, agents, or software capable of performing tasks that typically demand human cognition. This includes symbolic logic, automated theorem proving, knowledge graphs, heuristic search (such as algorithm), expert systems, natural language processing, and perception.

Machine Learning (ML): A specific, data-driven subset of AI. Rather than encoding expert knowledge symbolically through hand-engineered heuristic trees, machine learning equips agents to deduce operational rules autonomously by fitting parametric or non-parametric statistical estimators to observed data distributions.

Rule-Based AI vs. Learning-Based Machine Learning
CharacteristicSymbolic / Rule-Based AI (Expert Systems)Learning-Based Machine Learning
Knowledge RepresentationExplicit propositional logic, production rules ( clauses), ontologies.Continuous numerical parameter vectors, matrices, weights, decision boundaries.
Handling Novel CasesFails or throws exceptions if an input violates programmed deductive assumptions.Generalizes smoothly across continuous feature space via statistical interpolation.
Development BottleneckKnowledge acquisition: interviewing domain human experts to hand-write rules.Data engineering: collecting, cleaning, labeling, and transforming training datasets.
Representative ExampleMYCIN medical diagnostic expert system, Deep Blue chess evaluation tree.Gradient boosted trees for loan default prediction, logistic regression fraud detectors.

Real-World Engineering Applications of Machine Learning

Machine learning algorithms power core operations across modern engineering, scientific, and economic sectors:

  1. Healthcare & Biomedical Diagnostics:
    • Automated screening of radiological scans (X-rays, CT, MRI) to segment tumors and lesions.
    • Predicting patient readmission probabilities and acute sepsis onset using intensive care telemetry streams.
    • Computational genomics and molecular docking for accelerated pharmaceutical drug discovery.
  2. Financial Technology & Banking:
    • Real-time transaction fraud scoring using anomaly detection algorithms under millisecond latency constraints.
    • Credit risk assessment: estimating probability of default () on loan and mortgage applications.
    • Quantitative algorithmic trading through statistical arbitrage and time-series volatility estimation.
  3. Autonomous Vehicles & Robotics:
    • Real-time environmental perception: semantic obstacle segmentation, lane tracking, and pedestrian trajectory forecasting.
    • Dynamic motion planning, kinematic trajectory optimization, and adaptive cruise control.
  4. Recommendation Systems & Industrial Retrieval:
    • Collaborative filtering and matrix factorization to personalize content feeds and e-commerce catalogs.
    • Search ranking engines matching multi-modal queries to relevant enterprise documentation.
  5. Industrial Internet of Things (IIoT) & Predictive Maintenance:
    • Monitoring vibration and thermal signatures of rotating turbomachinery to forecast bearing fatigue.
    • Dynamic energy grid load forecasting to balance renewable generation with municipal power demands.

6.2 Introduction to Supervised, Unsupervised & Reinforcement Learning

Machine learning methodologies are categorized into three primary foundational paradigms based on the presence, nature, and timing of the feedback signal provided to the learning algorithm:

  1. Supervised Learning (learning with an explicit teacher / ground-truth target)
  2. Unsupervised Learning (discovering internal structure without supervisory targets)
  3. Reinforcement Learning (learning optimal behavioral policies via trial-and-error environment rewards)
                            Machine Learning
                                   |
         +-------------------------+-------------------------+
         |                                                   |
Supervised Learning                                 Unsupervised Learning
         |                                                   |
   +-----+-----+                                       +-----+-----+
   |           |                                       |           |
Classification Regression                          Clustering  Dimensionality
(Discrete)    (Continuous)                         (Grouping)   Reduction
                                                             
                                                             
                         +-------------------------+
                         |
               Reinforcement Learning
                         |
                   +-----+-----+
                   |           |
               Model-Based Model-Free

1. Supervised Learning

In Supervised Learning, the algorithm is provided with a training dataset of input-output pairs. The system acts like a student guided by a teacher who provides the correct ground-truth target for every historical training observation.

Definition

Supervised Learning Mathematical Formulation

Given a labeled training dataset of independent and identically distributed () observation pairs:

where is a -dimensional input feature vector, and is the corresponding ground-truth supervisor label.

Objective: Find an optimal predictive hypothesis function within a parameterized hypothesis class that minimizes an empirical risk objective function over the training distribution:

where is a task-specific loss function, is a regularization penalty, and is a shrinkage hyperparameter.

Depending on the topological nature of the target space , supervised learning splits into two fundamental branches:

  1. Classification ( is Discrete / Categorical):
    • The label belongs to a finite set of unordered nominal classes .
    • When , it is Binary Classification (e.g., for spam vs. non-spam).
    • When , it is Multi-Class Classification (e.g., classifying handwriting digits ).
    • Loss Function: Cross-Entropy Loss (Log-Loss).
    • Algorithms: Logistic Regression, Support Vector Machines (SVM), Decision Trees, Random Forests, -Nearest Neighbors (-NN), Naive Bayes.
  2. Regression ( is Continuous / Numerical):
    • The label takes real numerical values along the continuous real axis, or .
    • Predicts quantities such as asset prices, ambient temperatures, or engine failure hours.
    • Loss Function: Mean Squared Error (MSE), Mean Absolute Error (MAE), Huber Loss.
    • Algorithms: Ordinary Least Squares (OLS), Ridge/Lasso Regularized Regression, Support Vector Regression (SVR).

2. Unsupervised Learning

In Unsupervised Learning, the algorithm receives no supervisory target labels and no feedback signals. The training dataset consists solely of raw input feature vectors.

Definition

Unsupervised Learning Formulation

Given an unlabeled dataset:

Objective: Uncover the underlying probability density function , identify intrinsic geometrical manifolds, extract recurring structural relationships, or partition the data points into naturally cohesive clusters without external guidance.

The two primary tasks in unsupervised learning are:

  1. Clustering:
    • Partitioning heterogeneous data points into homogeneous clusters such that intra-cluster point similarities are maximized while inter-cluster separations are maximized.
    • Algorithms: -Means Clustering, Agglomerative Hierarchical Clustering, DBSCAN (Density-Based Spatial Clustering of Applications with Noise), Gaussian Mixture Models (GMM).
    • Applications: Market customer segmentation, genomic disease sub-typing, spatial crime pattern grouping.
  2. Dimensionality Reduction:
    • Projecting high-dimensional data into a lower-dimensional latent subspace (where ) while preserving maximal statistical variance or intrinsic topological distances.
    • Algorithms: Principal Component Analysis (PCA), Singular Value Decomposition (SVD), t-Distributed Stochastic Neighbor Embedding (t-SNE).
    • Applications: Data compression, multi-sensor noise filtering, 2D/3D exploratory visualization.
  3. Association Rule Mining:
    • Discovering probabilistic co-occurrence dependencies between discrete items in transactional records ().
    • Algorithms: Apriori algorithm, FP-Growth.

3. Reinforcement Learning (RL)

Unlike supervised learning (which relies on prescriptive teacher answers) or unsupervised learning (which operates without external evaluation), Reinforcement Learning is governed by sequential evaluative feedback. An autonomous agent navigates a dynamic environment, executing actions that transition the system state and yield scalar reward or penalty signals.

Definition

Reinforcement Learning: The MDP Framework

A Reinforcement Learning task is formally modeled as a Markov Decision Process (MDP) defined by the 5-tuple:

  1. Agent: The learner, decision-maker, and controller that interacts with the surrounding environment.
  2. Environment: The external physical or computational system with which the agent interacts.
  3. State Space (): The set of all possible environmental configurations. The state at discrete time step is denoted .
  4. Action Space (): The set of valid decisions or control maneuvers available to the agent: .
  5. Transition Probability Function (): The environmental dynamics governing the probability of transitioning to state given current state and action :
  1. Reward Function (): A scalar numerical feedback signal received from the environment immediately following an action:
  1. Discount Factor (): A mathematical discount applied to future rewards, balancing immediate payoff against long-term strategic returns.

Objective: Learn an optimal behavioral policy that maximizes the expected cumulative discounted future return :

                      +-------------------+
                      |    Environment    |
                      +-------------------+
                         ^             |
         Action $a_t$    |             |  State $s_t$
                         |             v  Reward $R_t$
                      +-------------------+
                      |       Agent       |
                      |  (Learns Policy)  |
                      +-------------------+

The Exploration vs. Exploitation Dilemma

A foundational challenge unique to reinforcement learning is balancing:

Comparative Synthesis of the Three Learning Paradigms

Comprehensive Comparison of Machine Learning Paradigms
DimensionSupervised LearningUnsupervised LearningReinforcement Learning
Input Data StructureLabeled pairs: Unlabeled vectors: States experienced sequentially
Supervisory SignalDirect, explicit ground-truth teacher targetsNone (autonomous structure discovery)Evaluative scalar rewards ()
Feedback TimingImmediate (per training instance)NoneDelayed across multi-step episodes
Primary GoalGeneralize functional mapping Model underlying density or cluster geometryMaximize expected cumulative return
Core ChallengesOverfitting, expensive human data annotationValidating cluster validity without labelsCredit assignment, exploration vs. exploitation
Canonical AlgorithmsLinear/Logistic Regression, SVM, Random Forest-Means, PCA, Hierarchical ClusteringQ-Learning, SARSA, Policy Gradients, PPO
Engineering BenchmarkLoan credit approval, tumor classificationCustomer segmentation, sensory noise filteringRobotics gait control, game agent navigation

Labeled vs. Unlabeled Data

Data is the fundamental fuel driving machine learning systems, but its operational utility depends heavily on whether it contains supervisory labels.

Definition

Labeled vs. Unlabeled Data

AspectLabeled DataUnlabeled Data
AvailabilityScarce; typically represents of an enterprise's data.Abundant; generated continuously by logs, IoT, and user events.
Acquisition CostHigh: requires domain expert annotators (e.g., radiologists labeling CT scans).Negligible: raw automated ingest into data lakes.
Susceptibility to NoiseProne to human annotator bias, subjectivity, and labeling drift.Free from subjective labeling bias, but contains raw sensor noise.
Applicable ParadigmsSupervised learning, semi-supervised verification.Unsupervised clustering, PCA, self-supervised representation learning.

Worked Examples

Example

Classifying Real-World Engineering Scenarios into ML Paradigms

Analyzing Scenario A (Bearing Failure Warning):

  • Input Data: Historical multivariate sensor readings paired with binary status logs (, ).
  • Target Output: Discrete binary label ().
  • Paradigm: Supervised Learning Binary Classification.
  • Justification: The training set contains explicit ground-truth failure records, and the decision space consists of discrete categorical outcomes.

Analyzing Scenario B (Customer Load Profiling):

  • Input Data: 24-dimensional daily load curves for 50,000 customers without prior demographic or tariff grouping labels.
  • Target Output: Cohesive groups or clusters of customer consumption patterns.
  • Paradigm: Unsupervised Learning Clustering.
  • Justification: There are no predefined "correct answers" or supervisory class labels; the objective is to uncover natural groupings based on geometrical proximity in load space.

Analyzing Scenario C (Data Center HVAC Control):

  • Input Data: Continuous stream of environmental state variables (rack temperatures, ambient humidity, IT server workload).
  • Control Actions: Setting damper angles, chiller motor speeds, and variable-frequency fan rates.
  • Feedback: Scalar rewards proportional to negative power consumption () with severe penalty spikes if temperatures exceed thermal danger thresholds.
  • Paradigm: Reinforcement Learning (RL).
  • Justification: The HVAC controller is an autonomous agent operating sequentially in an environment where control choices impact future thermal states, requiring exploration and long-term reward optimization.

Analyzing Scenario D (Turbofan Remaining Useful Life):

  • Input Data: Telemetry vectors with run-to-failure historical records.
  • Target Output: A continuous, positive real-valued numerical duration: hours.
  • Paradigm: Supervised Learning Regression.
  • Justification: The supervisor provides exact historical flight hours to failure, and the prediction space is continuous rather than categorical.

Analyzing Scenario E (Satellite Telemetry Compression):

  • Input Data: 120 continuous telemetry features without external labels.
  • Target Output: A 6-dimensional projection capturing maximal variance without loss of essential information.
  • Paradigm: Unsupervised Learning Dimensionality Reduction.
  • Justification: Unlabeled data is transformed into a compact latent representation by identifying and eliminating inter-channel correlations (e.g., via Principal Component Analysis).
Answer:
  • Scenario A: Supervised Learning (Binary Classification)
  • Scenario B: Unsupervised Learning (Clustering)
  • Scenario C: Reinforcement Learning (Sequential Decision Optimization via MDP)
  • Scenario D: Supervised Learning (Continuous Regression)
  • Scenario E: Unsupervised Learning (Dimensionality Reduction)
Example

Formalizing an Autonomous Robotic Vacuum as an MDP in Reinforcement Learning

Identifying the Agent and Environment:

  • Agent: The onboard embedded micro-controller and decision-making navigation algorithm of the robotic vacuum.
  • Environment: The physical indoor office room, including the spatial floor grid, furniture obstacles, dust/dirt distribution, wall boundaries, and recharging dock location.

Formulating the State Space (): At any discrete time step , the state vector encapsulates all sensory observations required for navigation:

where:

  • represents the robot's coordinates on the floor plane.
  • denotes the directional heading angle.
  • represents remaining battery charge.
  • indicates whether the current grid tile contains dirt.
  • denotes ultrasonic distance measurements to obstacles across 8 radial directions.

Formulating the Action Space (): The set of discrete actuator commands executable by the differential drive motors:

Formulating the Reward Function (): The scalar reward signal provides immediate numerical incentives aligned with the operational goals:

Policy Optimization Objective: The learning algorithm optimizes policy to maximize the expected cumulative discounted reward:

The discount factor encourages the robot to clean dirt promptly while maintaining enough foresight to avoid draining its battery before returning to the dock.

Answer:

The robotic vacuum is modeled as an MDP:

  • Agent: Onboard navigation processor.
  • Environment: Physical room, walls, obstacles, dirt patches, and dock.
  • State Space (): Coordinate position, heading, battery percentage, dirt presence, and 8-channel proximity sensors.
  • Action Space (): Forward movement, angular turns, suction activation, and docking.
  • Reward Function (): for cleaning dirt, for docking, for collisions, for battery exhaustion, and per step to encourage path efficiency.

Exercises

Exercises — Machine Learning: Supervised, Unsupervised & Reinforcement

Question 1

Define machine learning and explain how it differs from traditional programming. Illustrate both paradigms with workflow diagrams and concrete examples. [Assignment-6, Notebook p203, p205, p206 Q1]

Definition of Machine Learning: Machine Learning (ML) is an advanced branch of Artificial Intelligence concerned with the design and development of algorithms that learn empirical patterns and functional relationships directly from historical data. Instead of requiring human engineers to program every operating rule, an ML algorithm uses statistical estimation to improve its performance autonomously as it processes more experience.

Workflow of Traditional Programming: In traditional programming, humans write explicit logical rules, mathematical formulas, and algorithmic flowcharts:

The execution is strictly deterministic: the computer executes predefined instructions line-by-line. It excels when the underlying problem rules are fully known, structured, and static.

  • Example: A municipal payroll tax calculator that applies fixed statutory tax brackets to employee salary data.

Workflow of Machine Learning: In machine learning, the system is supplied with observed data and the desired output targets:

The machine learning algorithm extracts latent statistical associations to produce a generalized predictive model .

  • Example: An email spam filter. Hand-crafting rules for every permutation of phishing text is impossible. An ML classifier analyzes millions of historical emails tagged as spam or legitimate, learning weight vectors for indicative vocabulary and structural patterns.
Answer:

Machine learning enables computers to learn patterns from data and make predictions without being explicitly programmed. Traditional programming feeds Data + Rules into a computer to yield Output, whereas machine learning feeds Data + Output into an algorithm to induce the Model / Rules. Traditional code excels on deterministic calculations (tax tables), while ML excels on complex, perceptual problems (spam detection, computer vision).

Question 2

List and describe real-world applications of machine learning across healthcare, finance, recommendation systems, and autonomous transportation. [Assignment-6, Notebook p206, p207 Q2]

1. Healthcare & Medicine:

  • Disease Screening: Automated classification of medical imagery (detecting pulmonary nodules on chest X-rays or melanoma on dermatological scans).
  • Predictive Patient Risk: Forecasting sepsis onset or ICU mortality using time-series physiological monitors.
  • Drug Discovery: Generative modeling to predict molecular binding affinities against viral protein targets.

2. Finance and Banking:

  • Fraud Detection: Evaluating credit card transactions in real time to flag anomalous spending patterns.
  • Credit Scoring: Estimating the probability of loan default using customer credit histories, income stability, and macroeconomic regressors.
  • Algorithmic Trading: Quantitative models analyzing market order-book dynamics to execute optimal trades.

3. Recommendation Systems:

  • E-Commerce & Retail: Recommending complementary products on platforms such as Amazon based on purchase histories and collaborative filtering.
  • Media Streaming: Recommending movies or music tracks on Netflix and Spotify using matrix factorization and user listening embeddings.

4. Autonomous Transportation & Computer Vision:

  • Self-Driving Vehicles: Semantic segmentation of roadway scenes, pedestrian detection, and obstacle distance estimation from camera and LiDAR streams.
  • Fleet Route Optimization: Dynamic rerouting of logistics delivery trucks based on predictive traffic congestion models.
Answer:

Key real-world applications of machine learning include:

  1. Healthcare: Early disease screening from radiological imagery and patient risk forecasting.
  2. Finance: Millisecond transaction fraud detection and credit risk default scoring.
  3. Recommendation Engines: Personalized product and streaming media suggestions via collaborative filtering.
  4. Autonomous Transportation: Real-time pedestrian detection, scene segmentation, and predictive route optimization.
Question 3

Explain the conceptual distinction and hierarchical relationship between Artificial Intelligence (AI) and Machine Learning (ML). Provide concrete examples distinguishing rule-based AI from learning-based ML. [Assignment-6, Notebook p208, p210 Q3]

The Hierarchical Relationship: Artificial Intelligence is the broad umbrella discipline dedicated to building systems capable of simulating human intelligence, reasoning, decision-making, and problem-solving. Machine Learning is a specific subfield within AI:

All machine learning systems are AI systems, but not all AI systems use machine learning.

Rule-Based / Symbolic AI vs. Learning-Based ML:

  • Symbolic / Rule-Based AI: Operates on hardcoded ontologies, expert logic rules, and tree-search algorithms. Knowledge is manually authored by human knowledge engineers.
    • Example: The classic chess engine Deep Blue (1997), which combined hand-tuned board evaluation heuristics with massive minimax game-tree search; or MYCIN, an expert system containing hundreds of medical rules.
  • Learning-Based ML: Derives its rules by adjusting continuous numerical parameters against empirical data distributions through optimization algorithms (e.g., gradient descent).
    • Example: Modern evaluation systems that learn position valuations directly by training on millions of master chess games.

Comparative Breakdown:

  • Failure Modes: Symbolic AI is brittle; if an input falls outside programmed rules, it fails. ML handles noisy, novel inputs gracefully through probabilistic interpolation.
  • Knowledge Acquisition: Symbolic AI requires arduous human domain-expert interviews. ML relies on automated feature extraction and optimization over datasets.
Answer:

AI is the overarching science of creating machines capable of intelligent behavior (). Symbolic AI relies on hand-crafted rules, logical deduction, and explicit search heuristics (e.g., expert systems with rules), whereas Machine Learning automatically learns its decision functions from empirical data via statistical optimization (e.g., gradient-boosted decision trees).

Question 4

Compare supervised learning, unsupervised learning, and reinforcement learning in tabular format. State the primary mathematical objective, input data requirements, representative algorithms, and real-world engineering use cases for each. [Assignment-6, Notebook p209, p211-p215 Q5]

Constructing the Comparative Framework: Evaluating the three paradigms across five core engineering dimensions:

  1. Supervisory feedback mechanism.
  2. Input dataset formulation.
  3. Optimization objective.
  4. Representative algorithms.
  5. Real-world engineering benchmark.

Tabular Comparison:

DimensionSupervised LearningUnsupervised LearningReinforcement Learning
Input DataLabeled pairs: Unlabeled vectors: States observed sequentially
Feedback SignalExplicit target values provided for every sampleNo feedback or ground-truth targetsEvaluative scalar rewards ()
Mathematical ObjectiveMinimize empirical risk: Model underlying data density or minimize intra-cluster varianceMaximize expected cumulative return:
Feedback TimingImmediate (batch or online)AbsentDelayed across multi-action episodes
AlgorithmsLinear Regression, Logistic Regression, SVM, Random Forest-Means, Hierarchical Clustering, PCA, DBSCANQ-Learning, Deep Q-Networks (DQN), PPO, Actor-Critic
Engineering Use CaseMedical diagnosis, loan default risk predictionCustomer market segmentation, sensory noise filteringRobot manipulator control, autonomous driving
Answer:

Supervised learning trains on labeled pairs to learn predictive mappings. Unsupervised learning analyzes unlabeled data to uncover latent clusters and low-dimensional manifolds. Reinforcement learning trains an agent interacting with an environment to maximize cumulative scalar rewards via trial and error.

Question 5

Describe how labeled and unlabeled data are used in machine learning. Why is data labeling often the primary bottleneck in production supervised pipelines, and how does unsupervised learning bypass this constraint? [Assignment-6, Notebook p213, p215 Q6]

Role of Labeled Data in Supervised Learning: Labeled data provides the explicit ground truth necessary to calculate loss gradients:

These gradients guide the optimization algorithm to update model parameters toward correct predictions. Without reliable labels, supervised models cannot establish decision boundaries.

The Data Labeling Bottleneck:

  1. High Economic and Human Cost: Manual annotation requires human domain expertise (e.g., board-certified radiologists outlining lesions on thousands of MRI scans).
  2. Subjectivity and Inconsistency: Annotators often disagree on ambiguous edge cases, introducing label noise.
  3. Volume Disparity: In production systems, raw unlabeled data is generated continuously at high volume, while labeled data requires manual effort, creating a severe bottleneck.

How Unsupervised Learning Bypasses the Bottleneck: Unsupervised learning operates directly on raw, unlabeled data without needing human annotations. It extracts intrinsic geometric and statistical structures:

  • It discovers latent groupings through clustering algorithms (e.g., -means).
  • It eliminates redundant sensor channels through dimensionality reduction (e.g., PCA).
  • It enables self-supervised representation learning, where the model learns spatial or contextual representations from raw data before fine-tuning on a small labeled subset.
Answer:

Labeled data pairs inputs with verified ground-truth targets, enabling supervised algorithms to compute explicit loss gradients and adjust model parameters. Data labeling is the primary bottleneck in production pipelines because manual annotation is expensive, slow, and prone to human inconsistency. Unsupervised learning bypasses this bottleneck by operating directly on raw, uncurated data to discover natural groupings, density patterns, and lower-dimensional representations.

Question 6

Explain the main components of a reinforcement learning system: agent, environment, state, action, and reward. Illustrate their interaction cycle with a clear feedback loop and explain the exploration vs. exploitation trade-off. [Assignment-6, Notebook p213, p215-p217 Q7]

The Five Core Components:

  1. Agent: The active learning entity that perceives its surroundings, makes decisions, and executes actions.
  2. Environment: The external world or simulated system with which the agent interacts.
  3. State (): A complete description of the current environmental conditions perceived by the agent at step .
  4. Action (): A decision or physical control command executed by the agent that transitions the environment into a new state .
  5. Reward (): A scalar feedback value returned by the environment to quantify the immediate quality of the executed action.

The Sequential Interaction Cycle: At each discrete time step :

  1. The agent observes current state from the environment.
  2. The agent selects and executes an action based on its policy.
  3. The environment responds by transitioning to a new state .
  4. The environment returns an immediate numerical scalar reward .
  5. The agent updates its value function or policy parameters to maximize future discounted returns:

The Exploration vs. Exploitation Trade-Off:

  • Exploitation: The agent selects the action it currently believes yields the highest reward, maximizing short-term performance.
  • Exploration: The agent tests novel or unfamiliar actions to gather more information about the environment, potentially discovering superior long-term strategies.
  • An agent that only exploits risks getting trapped in suboptimal local policies, while an agent that only explores wastes actions and fails to maximize its cumulative return.
Answer:

A reinforcement learning system comprises:

  • Agent: The autonomous learner/decision-maker.
  • Environment: The external world providing observations and feedback.
  • State (): The current situational configuration.
  • Action (): The decision executed by the agent.
  • Reward (): The scalar feedback signal evaluating performance.

The interaction proceeds in a closed loop: Agent observes executes Environment transitions to and yields . The system must balance exploitation (capitalizing on current knowledge) with exploration (searching for better actions).

Question 7

Differentiate between Classification and Regression in Supervised Learning, and between Clustering and Dimensionality Reduction in Unsupervised Learning. Give mathematical formulations for the output space in each. [Added, Micro-Syllabus 6.1, 6.2]

Classification vs. Regression (Supervised Learning): Both learn a mapping from labeled data :

  • Classification: The target space is discrete and qualitative:

The model partitions feature space with decision boundaries to assign observations into distinct categories (e.g., identifying whether an incoming email is Spam or Ham).

  • Regression: The target space is continuous and quantitative:

The model fits a continuous surface to predict real numerical quantities (e.g., forecasting tomorrow's peak electrical demand in megawatts).

Clustering vs. Dimensionality Reduction (Unsupervised Learning): Both operate on unlabeled data :

  • Clustering: Partitions observations into disjoint subsets such that:

Objective: Minimize intra-cluster dispersion while maximizing inter-cluster separation.

  • Dimensionality Reduction: Maps each -dimensional feature vector to a lower-dimensional latent vector (where ):

Objective: Preserve maximal statistical variance or topological distances while eliminating multicollinearity and noise.

Answer:

In supervised learning, Classification predicts discrete categorical classes (), whereas Regression predicts continuous real values (). In unsupervised learning, Clustering groups points into distinct subsets, whereas Dimensionality Reduction projects high-dimensional vectors () into a compact latent subspace ().

Past exam questions for Foundation of Data Science

← Logistic RegressionModeling Process, Training/Validation & Cross-Validation →