PadhaikoAll subjects

Input: Concepts, Instances, and Attributes

Data Mining (ENCT 435)

Open in the Padhaiko app

2.1 Defining concepts, instances, and attributes

Every data mining task begins by deciding what we want to learn and how the raw world is represented in a tabular form that a learning algorithm can process.

Definition

Concepts, Instances, and Attributes

Four Styles of Learning Concepts

Depending on the nature of the concept to be learned, data mining recognizes four primary learning styles:

Four Styles of Concept Learning
Learning StyleSupervisionOutput TargetTypical Representation
Classification LearningSupervisedDiscrete class label (e.g., yes / no)Decision tree, classification rules
Association LearningUnsupervisedAny strong co-occurrence among attributesAssociation rules ()
ClusteringUnsupervisedGroup/cluster assignment for similar instancesCluster centroids, dendrograms
Numeric PredictionSupervisedContinuous real number (e.g., PRP)Linear equation, regression tree

Independence of Instances

Standard propositional learners assume that instances are independent and identically distributed (i.i.d.)—meaning one row does not directly reference or depend on another row's ordering. When data inherently involves relationships between different entities (such as customers and their multiple orders), those relationships must first be flattened into attributes on each instance.


2.2 Relational data and the process of denormalization

Operational databases are rarely stored as a single flat spreadsheet. Instead, relational database management systems (RDBMS) organize data into multiple normalized tables (typically in Third Normal Form, 3NF) to eliminate redundancy and prevent insertion, update, and deletion anomalies.

Why Normalization Conflicts with Data Mining Input

Most classical machine learning algorithms expect a single flat relation (one row per instance, fixed columns). If information about an entity is split across Customers, Orders, and OrderItems, a decision tree or linear classifier cannot directly traverse foreign-key joins during induction.

Definition

Denormalization

Denormalization is the process of combining multiple normalized relational tables back into a single flat table by joining along primary-key / foreign-key links (and aggregating or pivoting one-to-many child records) so that each row represents one complete mining instance.

Handling Join Cardinalities

  1. Many-to-One / One-to-One Joins (Direct Join): If each Order belongs to exactly one Customer, joining Orders with Customer on customer_id preserves one row per order and simply copies customer attributes onto each order row. @code sql "Direct Many-to-One Denormalization Join" SELECT o.order_id, o.order_amount, o.payment_type, c.customer_id, c.city, c.age_group, c.membership_tier FROM Orders o JOIN Customers c ON o.customer_id = c.customer_id; @endcode
  2. One-to-Many Joins (Aggregation / Feature Engineering): Suppose the target instance is a Customer (predicting customer churn), and each customer has or many rows in Orders. A raw SQL JOIN would duplicate the customer row once per order, distorting instance counts and violating row independence! Instead, child records must be summarized into fixed-width attributes per customer:
    • Total order count (num_orders),
    • Total and average spend (total_spend, avg_order_value),
    • Recency or maximum value (days_since_last_order, max_spend),
    • Category counts or boolean flags (used_coupon_flag).
Customer-Level Denormalization Using AggregationSQL
SELECT c.customer_id, c.city, c.membership_tier,
       COUNT(o.order_id) AS num_orders,
       COALESCE(SUM(o.order_amount), 0) AS total_spend,
       COALESCE(AVG(o.order_amount), 0) AS avg_spend
FROM Customers c
LEFT JOIN Orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_id, c.city, c.membership_tier;

2.3 Preparing input: ARFF format, attribute types (nominal, ordinal, interval, ratio), and missing/inaccurate values

Once a flat table is assembled, it must be encoded in a clean, unambiguous input format with properly typed attributes and clean values.

1. ARFF (Attribute-Relation File Format)

ARFF is the standard ASCII text file format popularized by WEKA for representing flat datasets. An ARFF file has two distinct sections: a Header (@relation and @attribute declarations) and a Data section (@data followed by comma-separated instances).

Weather Dataset in ARFF FormatText
% Title: Weather Problem (Nominal Version)
% Lines starting with '%' are comments.

@relation weather.symbolic

@attribute outlook {sunny, overcast, rainy}
@attribute temperature numeric
@attribute humidity numeric
@attribute windy {TRUE, FALSE}
@attribute play {yes, no}

@data
sunny, 85, 85, FALSE, no
sunny, 80, 90, TRUE, no
overcast, 83, 86, FALSE, yes
rainy, 70, 96, FALSE, yes
rainy, 68, ?, FALSE, yes
rainy, 65, 70, TRUE, no

Key ARFF rules:

2. Attribute Types (Measurement Scales)

Understanding an attribute's scale of measurement determines which mathematical operations (equality, sorting, subtraction, division) are valid:

Four Measurement Scales of Attributes
Attribute TypeKey PropertyPermissible OperationsRepresentative Examples
NominalUnordered distinct labels / categoriesEquality (, ), mode, frequency countsoutlook (sunny, overcast, rainy), blood_group, postal_code
OrdinalCategories with a meaningful rank order, unequal/unmeasured spacingEquality, ordering (, ), median, percentilesgrade (A, B, C), size (small, medium, large), Likert ratings
IntervalOrdered numeric scale with equal units, no true zero pointAddition, subtraction (), mean, standard deviationTemperature in or , calendar year
RatioOrdered numeric scale with equal units and an absolute zeroAll arithmetic including ratios (), geometric meanWeight (), age, income, memory size (), temperature in

3. Missing and Inaccurate Values

Real-world datasets frequently contain missing values (?) and inaccurate (noisy) values.

Missing Values

Inaccurate Values (Noise and Outliers)


Worked Examples

Example 1

Denormalizing Normalized Tables for Customer Churn Prediction (6 marks)

Suppose an e-commerce database has two normalized tables:

Table Customers:

cidtierchurned
C1GoldNo
C2SilverYes
C3BronzeNo

Table Orders:

oidcidamount
O101C1
O102C1
O103C1
O104C2
  1. Explain why a plain inner JOIN without aggregation is unsuitable for training a customer churn classifier.
  2. Construct the properly denormalized customer-level mining table using num_orders, total_spend, and avg_spend.

Why a plain inner JOIN fails:

  • Joining Customers and Orders on cid produces three rows for C1, one row for C2, and zero rows for C3 (who has no orders yet).
  • Customer C1 would count three times as heavily as C2, violating row independence, while C3 would vanish completely from the training data.

Aggregating Orders per customer (LEFT JOIN + GROUP BY):

  • For C1: orders (O101, O102, O103),
  • For C2: order (O104),
  • For C3: orders , , (using LEFT JOIN + COALESCE).

Resulting denormalized flat table (one row per customer):

cidtiernum_orderstotal_spendavg_spendchurned
C1GoldNo
C2SilverYes
C3BronzeNo
Answer:

A plain inner join duplicates multi-order customers (C1 appears times) and drops zero-order customers (C3). Using a LEFT JOIN with aggregation yields a clean -row customer table: C1 (Gold, 3, 300, 100.0, No), C2 (Silver, 1, 50, 50.0, Yes), and C3 (Bronze, 0, 0, 0.0, No).

Example 2

Classifying Attribute Scales and Writing an ARFF File (5 marks)

A smart-agriculture dataset records four attributes per greenhouse plot to predict crop status:

  1. Classify each of the five attributes as nominal, ordinal, interval, or ratio.
  2. Write a valid ARFF header and sample @data section containing instances, including one missing yield_kg value.

Classifying the measurement scale of each attribute:

  • soil_type: Nominal — distinct categories with no inherent order.
  • moisture_level: Ordinal — ordered categories (dry medium wet) without fixed numerical spacing.
  • temp_celsius: Interval — equal degree intervals, but is an arbitrary freezing point, not absence of thermal energy.
  • yield_kg: Ratio — equal units () with a true absolute zero ( means zero crop harvested).
  • status: Nominal — two discrete categorical class labels.

Constructing the ARFF representation: Nominal and ordinal attributes are declared in WEKA ARFF as enumerated sets {...}, while interval and ratio attributes are declared as numeric. Missing values use ?.

Greenhouse Crop Status ARFF FileText
@relation greenhouse_crops

@attribute soil_type {clay, loam, sandy}
@attribute moisture_level {dry, medium, wet}
@attribute temp_celsius numeric
@attribute yield_kg numeric
@attribute status {healthy, stressed}

@data
loam, medium, 24.5, 18.2, healthy
sandy, dry, 31.0, ?, stressed
clay, wet, 21.8, 15.6, healthy
Answer:

Scales: soil_type (nominal), moisture_level (ordinal), temp_celsius (interval), yield_kg (ratio), status (nominal). The ARFF file declares nominal/ordinal attributes via {...} lists, numeric attributes via numeric, and encodes the missing yield using ?.

Example 3

Detecting an Inaccurate Sentinel Value and Imputing Missing Data (5 marks)

A student survey table records age for respondents:

Suppose domain knowledge confirms ages of college respondents lie in , and 999 was an old database sentinel code for "not answered."

  1. Compute the distorted sample mean if 999 is mistakenly treated as a valid numeric age (over the non-missing numbers).
  2. Convert 999 to missing (?), then compute the mean and sample standard deviation of the valid ages and impute the missing slots.

Distorted mean before cleaning sentinel 999: Summing the non-? values :

A single sentinel value corrupts the mean to !

Cleaning 999 -> ? and computing valid statistics: The valid values are :

Sample variance ():

Imputing the two missing entries: Replacing both ? entries (original 999 and ?) with the clean mean produces the cleaned vector:

Answer:

Treating 999 as numeric yields a distorted mean of . Converting 999 to ? leaves valid ages with clean mean and sample standard deviation . Mean imputation produces .


Exercises

Practice Exercises — Input Representation

Question 1

Define concept, instance, and attribute in the context of data mining, and distinguish among the four basic styles of learning: classification, association, clustering, and numeric prediction.

Concepts, Instances, and Attributes:

  • A concept is the target pattern or mapping to be learned from data (whose learned output is a concept description).
  • An instance is a single independent example (row) of that concept.
  • An attribute is an individual measurable property (column) describing each instance.

Four Styles of Learning:

  • Classification: Supervised prediction of a discrete categorical class label.
  • Numeric Prediction: Supervised prediction of a continuous numeric target value.
  • Association Learning: Unsupervised discovery of strong co-occurrence rules among any attributes without a single designated target.
  • Clustering: Unsupervised grouping of instances into clusters so items within a cluster are similar and items across clusters differ.
Answer:

A concept is the target pattern to learn, an instance is one example row, and an attribute is one feature column. Classification predicts discrete classes, numeric prediction predicts continuous numbers, association learning discovers co-occurring attribute rules, and clustering partitions unlabeled instances into similarity groups.

Question 2

Explain why relational databases are normally kept in normalized form, why data mining requires denormalization, and how one-to-many relationships must be handled during denormalization.

Normalized Databases vs. Flat Mining Tables: Operational databases normalize data into linked tables (via primary/foreign keys) to avoid duplicate storage and update anomalies. However, most ML algorithms assume a single flat table of independent, fixed-length rows.

Denormalization and One-to-Many Relationships: Denormalization joins related tables into one flat table. For many-to-one links, parent attributes are copied directly onto each child row. For one-to-many links (e.g., one customer with multiple orders), a plain SQL join would duplicate the customer row multiple times and drop zero-child rows. Instead, child rows must be aggregated (COUNT, SUM, AVG, MIN, MAX, or indicator flags) using a LEFT JOIN and GROUP BY at the target instance's grain.

Answer:

Databases are normalized to avoid redundancy and anomalies, whereas ML models need a single flat table of independent instances. Denormalization joins normalized tables, handling one-to-many links through LEFT JOIN + GROUP BY aggregations (COUNT, SUM, AVG, etc.) so row counts and independence at the target grain are preserved.

Question 3

Compare the four attribute measurement scales (nominal, ordinal, interval, and ratio). Give one original example of each and state why treating a nominal or interval attribute carelessly can mislead a learning algorithm.

Four Measurement Scales:

  1. Nominal: Unordered categories; only equality () is valid (e.g., department = {CS, Civil, Electrical}).
  2. Ordinal: Ordered categories without fixed intervals; comparisons () are valid (e.g., severity = {low, moderate, critical}).
  3. Interval: Equal numeric units without a true zero; differences () are meaningful, ratios () are not (e.g., temperature in ).
  4. Ratio: Equal numeric units with a true absolute zero; ratios () are meaningful (e.g., packet_size in bytes).

Why Careless Encoding Misleads Learners: If nominal categories like department are encoded as integers , a linear or distance-based algorithm falsely infers that and that is the "average" of and . Nominal attributes should instead use one-hot (dummy) encoding or categorical splits. Similarly, ratio claims on interval scales (like treating as double ) fail because zero is arbitrary.

Answer:

Nominal (unordered labels, equality only), Ordinal (ranked labels, ), Interval (equal units, arbitrary zero, differences valid), Ratio (equal units, true zero, ratios valid). Encoding nominal values as arbitrary integers introduces false ordering and distance relationships.

Question 4

Describe the structure of an ARFF file and explain how missing values and sparse data are represented in ARFF.

Structure of an ARFF File: An ARFF file consists of:

  1. Optional comment lines beginning with %.
  2. A Header section with @relation <name> followed by ordered @attribute <name> <type> declarations (numeric, {v1, v2, ...}, string, date).
  3. A Data section starting with @data, followed by one comma-separated instance per line.

Missing Values and Sparse ARFF:

  • Missing values are written as a literal question mark ? in the corresponding attribute position.
  • Sparse ARFF omits zero/default entries by enclosing non-zero entries in curly braces with zero-based column indices, e.g., {1 4.5, 12 yes}.
Answer:

An ARFF file has % comments, a header (@relation and @attribute definitions), and a @data section of comma-separated rows. Missing values are written as ?, and sparse instances use {index value, ...} with zero-based attribute indices.

Question 5

Consider the numeric attribute humidity with recorded readings:

where -99 is an invalid sensor-error code outside the physical range .

  1. Identify the invalid entry and convert it to missing (?).
  2. Compute the mean and sample variance of the remaining valid readings.
  3. If the missing positions are imputed with the valid mean, verify that the mean of the -element imputed array remains unchanged, and explain one situation where keeping a missing_flag attribute is still useful.

Filtering the physically impossible reading: Humidity percentages must satisfy , so -99 is an inaccurate sentinel code. Converting -99 to ? leaves valid readings: .

Computing the valid mean and sample variance:

Imputation and informative missingness: Replacing both ? slots with gives , whose sum is and mean is (unchanged). Keeping a separate binary humidity_missing flag is valuable whenever missingness is informative—for example, if sensors fail specifically during heavy rainstorms, the flag itself helps predict the outcome even after imputation.

Answer:
  1. -99 is invalid and replaced with ?.
  2. Valid mean ; sample variance .
  3. Imputing with keeps the overall mean at ; a missing_flag preserves signal when sensor failure correlates with the target event.
← IntroductionOutput: Knowledge Representation →