Input: Concepts, Instances, and Attributes
Open in the Padhaiko app2.1 Defining concepts, instances, and attributes
Every data mining task begins by deciding what we want to learn and how the raw world is represented in a tabular form that a learning algorithm can process.
Concepts, Instances, and Attributes
- Concept: The phenomenon, pattern, or target mapping to be learned—such as deciding whether to play a game given weather conditions, classifying a flower species, or grouping customers by buying habits. The output produced by a learning algorithm is called a concept description.
- Instance (Example / Record / Tuple): A single, independent, fixed-length realization of the concept—corresponding to one row in a flat data table. Each instance is described by a fixed vector of attribute values.
- Attribute (Feature / Variable / Column): A specific measurable property, characteristic, or quantity describing one aspect of an instance (e.g.,
outlook,humidity,petal_length).
Four Styles of Learning Concepts
Depending on the nature of the concept to be learned, data mining recognizes four primary learning styles:
| Learning Style | Supervision | Output Target | Typical Representation |
|---|---|---|---|
| Classification Learning | Supervised | Discrete class label (e.g., yes / no) | Decision tree, classification rules |
| Association Learning | Unsupervised | Any strong co-occurrence among attributes | Association rules () |
| Clustering | Unsupervised | Group/cluster assignment for similar instances | Cluster centroids, dendrograms |
| Numeric Prediction | Supervised | Continuous real number (e.g., PRP) | Linear equation, regression tree |
Independence of Instances
Standard propositional learners assume that instances are independent and identically distributed (i.i.d.)—meaning one row does not directly reference or depend on another row's ordering. When data inherently involves relationships between different entities (such as customers and their multiple orders), those relationships must first be flattened into attributes on each instance.
2.2 Relational data and the process of denormalization
Operational databases are rarely stored as a single flat spreadsheet. Instead, relational database management systems (RDBMS) organize data into multiple normalized tables (typically in Third Normal Form, 3NF) to eliminate redundancy and prevent insertion, update, and deletion anomalies.
Why Normalization Conflicts with Data Mining Input
Most classical machine learning algorithms expect a single flat relation (one row per instance, fixed columns). If information about an entity is split across Customers, Orders, and OrderItems, a decision tree or linear classifier cannot directly traverse foreign-key joins during induction.
Denormalization
Denormalization is the process of combining multiple normalized relational tables back into a single flat table by joining along primary-key / foreign-key links (and aggregating or pivoting one-to-many child records) so that each row represents one complete mining instance.
Handling Join Cardinalities
- Many-to-One / One-to-One Joins (Direct Join): If each
Orderbelongs to exactly oneCustomer, joiningOrderswithCustomeroncustomer_idpreserves one row per order and simply copies customer attributes onto each order row. @code sql "Direct Many-to-One Denormalization Join" SELECT o.order_id, o.order_amount, o.payment_type, c.customer_id, c.city, c.age_group, c.membership_tier FROM Orders o JOIN Customers c ON o.customer_id = c.customer_id; @endcode - One-to-Many Joins (Aggregation / Feature Engineering): Suppose the target instance is a Customer (predicting customer churn), and each customer has or many rows in
Orders. A raw SQLJOINwould duplicate the customer row once per order, distorting instance counts and violating row independence! Instead, child records must be summarized into fixed-width attributes per customer:- Total order count (
num_orders), - Total and average spend (
total_spend,avg_order_value), - Recency or maximum value (
days_since_last_order,max_spend), - Category counts or boolean flags (
used_coupon_flag).
- Total order count (
SELECT c.customer_id, c.city, c.membership_tier,
COUNT(o.order_id) AS num_orders,
COALESCE(SUM(o.order_amount), 0) AS total_spend,
COALESCE(AVG(o.order_amount), 0) AS avg_spend
FROM Customers c
LEFT JOIN Orders o ON c.customer_id = o.customer_id
GROUP BY c.customer_id, c.city, c.membership_tier;2.3 Preparing input: ARFF format, attribute types (nominal, ordinal, interval, ratio), and missing/inaccurate values
Once a flat table is assembled, it must be encoded in a clean, unambiguous input format with properly typed attributes and clean values.
1. ARFF (Attribute-Relation File Format)
ARFF is the standard ASCII text file format popularized by WEKA for representing flat datasets. An ARFF file has two distinct sections: a Header (@relation and @attribute declarations) and a Data section (@data followed by comma-separated instances).
% Title: Weather Problem (Nominal Version)
% Lines starting with '%' are comments.
@relation weather.symbolic
@attribute outlook {sunny, overcast, rainy}
@attribute temperature numeric
@attribute humidity numeric
@attribute windy {TRUE, FALSE}
@attribute play {yes, no}
@data
sunny, 85, 85, FALSE, no
sunny, 80, 90, TRUE, no
overcast, 83, 86, FALSE, yes
rainy, 70, 96, FALSE, yes
rainy, 68, ?, FALSE, yes
rainy, 65, 70, TRUE, noKey ARFF rules:
@relation <relation-name>: names the dataset (quote strings containing spaces).@attribute <attr-name> <datatype>: defines each column in exact left-to-right order. Supported datatypes includenumeric(orreal/integer), nominal{val1, val2, ...},string, anddate.@data: marks the start of rows. Missing values are denoted by?.- Sparse ARFF: When most attributes are zero (e.g., text bag-of-words vectors), rows can be written as
{index value, ...}with zero-based attribute indices: e.g.,{0 sunny, 3 TRUE, 4 yes}.
2. Attribute Types (Measurement Scales)
Understanding an attribute's scale of measurement determines which mathematical operations (equality, sorting, subtraction, division) are valid:
| Attribute Type | Key Property | Permissible Operations | Representative Examples |
|---|---|---|---|
| Nominal | Unordered distinct labels / categories | Equality (, ), mode, frequency counts | outlook (sunny, overcast, rainy), blood_group, postal_code |
| Ordinal | Categories with a meaningful rank order, unequal/unmeasured spacing | Equality, ordering (, ), median, percentiles | grade (A, B, C), size (small, medium, large), Likert ratings |
| Interval | Ordered numeric scale with equal units, no true zero point | Addition, subtraction (), mean, standard deviation | Temperature in or , calendar year |
| Ratio | Ordered numeric scale with equal units and an absolute zero | All arithmetic including ratios (), geometric mean | Weight (), age, income, memory size (), temperature in |
3. Missing and Inaccurate Values
Real-world datasets frequently contain missing values (?) and inaccurate (noisy) values.
Missing Values
- Causes: Sensor failure, unrecorded survey fields, field not applicable (e.g., spouse's income for an unmarried applicant), or data corruption during export.
- Informative Missingness: Sometimes the fact that a value is missing carries predictive signal (e.g., an applicant omitting income or a patient skipping a specific lab test). Treating
?as a separate category or adding a binaryis_missingindicator preserves this signal. - Standard Handling Strategies:
- Ignore / Drop (Listwise Deletion): Discard records with missing values (safe only when very few rows are affected and missingness is completely at random).
- Global Constant / Separate Category: Replace missing nominal values with
'Unknown'or'Missing'. - Mean / Median / Mode Imputation: Replace missing numeric values with the attribute's mean (or median if skewed) and nominal values with the mode (most frequent label)—ideally computed only from the training set (or within the same class during training, though never using test labels!).
- Model-Based / -NN Imputation: Predict the missing attribute from other observed attributes.
- Algorithm-Native Handling: Algorithms like Naive Bayes ignore missing attributes in likelihood counts, and C4.5 splits fractional weights across branches.
Inaccurate Values (Noise and Outliers)
- Sources: Typographical mistakes (
199years old instead of19), sentinel codes masquerading as real numbers (-999or9999used for "unknown"), duplicate records with conflicting labels, or stale measurements. - Detection and Cleansing:
- Domain constraint checks (e.g.,
0 <= humidity <= 100,age >= 0). - Summary statistics and visualization (histograms, boxplots, IQR rule: values below or above flagged for inspection).
- Deduplication and consistency checks across related attributes (e.g.,
birth_datevs.age).
- Domain constraint checks (e.g.,
Worked Examples
Denormalizing Normalized Tables for Customer Churn Prediction
Suppose an e-commerce database has two normalized tables:
Table Customers:
cid | tier | churned |
|---|---|---|
| C1 | Gold | No |
| C2 | Silver | Yes |
| C3 | Bronze | No |
Table Orders:
oid | cid | amount |
|---|---|---|
| O101 | C1 | |
| O102 | C1 | |
| O103 | C1 | |
| O104 | C2 |
- Explain why a plain inner
JOINwithout aggregation is unsuitable for training a customer churn classifier. - Construct the properly denormalized customer-level mining table using
num_orders,total_spend, andavg_spend.
Why a plain inner JOIN fails:
- Joining
CustomersandOrdersoncidproduces three rows forC1, one row forC2, and zero rows forC3(who has no orders yet). - Customer
C1would count three times as heavily asC2, violating row independence, whileC3would vanish completely from the training data.
Aggregating Orders per customer (LEFT JOIN + GROUP BY):
- For C1: orders (
O101,O102,O103),
- For C2: order (
O104),
- For C3: orders , , (using
LEFT JOIN+COALESCE).
Resulting denormalized flat table (one row per customer):
cid | tier | num_orders | total_spend | avg_spend | churned |
|---|---|---|---|---|---|
| C1 | Gold | No | |||
| C2 | Silver | Yes | |||
| C3 | Bronze | No |
A plain inner join duplicates multi-order customers (C1 appears times) and drops zero-order customers (C3). Using a LEFT JOIN with aggregation yields a clean -row customer table: C1 (Gold, 3, 300, 100.0, No), C2 (Silver, 1, 50, 50.0, Yes), and C3 (Bronze, 0, 0, 0.0, No).
Classifying Attribute Scales and Writing an ARFF File
A smart-agriculture dataset records four attributes per greenhouse plot to predict crop status:
soil_type:{clay, loam, sandy}moisture_level:{dry, medium, wet}temp_celsius: measured in (e.g., )yield_kg: harvested weight in (e.g., , with one missing observation)status:{healthy, stressed}
- Classify each of the five attributes as nominal, ordinal, interval, or ratio.
- Write a valid ARFF header and sample
@datasection containing instances, including one missingyield_kgvalue.
Classifying the measurement scale of each attribute:
soil_type: Nominal — distinct categories with no inherent order.moisture_level: Ordinal — ordered categories (drymediumwet) without fixed numerical spacing.temp_celsius: Interval — equal degree intervals, but is an arbitrary freezing point, not absence of thermal energy.yield_kg: Ratio — equal units () with a true absolute zero ( means zero crop harvested).status: Nominal — two discrete categorical class labels.
Constructing the ARFF representation: Nominal and ordinal attributes are declared in WEKA ARFF as enumerated sets {...}, while interval and ratio attributes are declared as numeric. Missing values use ?.
@relation greenhouse_crops
@attribute soil_type {clay, loam, sandy}
@attribute moisture_level {dry, medium, wet}
@attribute temp_celsius numeric
@attribute yield_kg numeric
@attribute status {healthy, stressed}
@data
loam, medium, 24.5, 18.2, healthy
sandy, dry, 31.0, ?, stressed
clay, wet, 21.8, 15.6, healthyScales: soil_type (nominal), moisture_level (ordinal), temp_celsius (interval), yield_kg (ratio), status (nominal). The ARFF file declares nominal/ordinal attributes via {...} lists, numeric attributes via numeric, and encodes the missing yield using ?.
Detecting an Inaccurate Sentinel Value and Imputing Missing Data
A student survey table records age for respondents:
Suppose domain knowledge confirms ages of college respondents lie in , and 999 was an old database sentinel code for "not answered."
- Compute the distorted sample mean if
999is mistakenly treated as a valid numeric age (over the non-missing numbers). - Convert
999to missing (?), then compute the mean and sample standard deviation of the valid ages and impute the missing slots.
Distorted mean before cleaning sentinel 999: Summing the non-? values :
A single sentinel value corrupts the mean to !
Cleaning 999 -> ? and computing valid statistics: The valid values are :
Sample variance ():
Imputing the two missing entries: Replacing both ? entries (original 999 and ?) with the clean mean produces the cleaned vector:
Treating 999 as numeric yields a distorted mean of . Converting 999 to ? leaves valid ages with clean mean and sample standard deviation . Mean imputation produces .
Exercises
Practice Exercises — Input Representation
Define concept, instance, and attribute in the context of data mining, and distinguish among the four basic styles of learning: classification, association, clustering, and numeric prediction.
Concepts, Instances, and Attributes:
- A concept is the target pattern or mapping to be learned from data (whose learned output is a concept description).
- An instance is a single independent example (row) of that concept.
- An attribute is an individual measurable property (column) describing each instance.
Four Styles of Learning:
- Classification: Supervised prediction of a discrete categorical class label.
- Numeric Prediction: Supervised prediction of a continuous numeric target value.
- Association Learning: Unsupervised discovery of strong co-occurrence rules among any attributes without a single designated target.
- Clustering: Unsupervised grouping of instances into clusters so items within a cluster are similar and items across clusters differ.
A concept is the target pattern to learn, an instance is one example row, and an attribute is one feature column. Classification predicts discrete classes, numeric prediction predicts continuous numbers, association learning discovers co-occurring attribute rules, and clustering partitions unlabeled instances into similarity groups.
Explain why relational databases are normally kept in normalized form, why data mining requires denormalization, and how one-to-many relationships must be handled during denormalization.
Normalized Databases vs. Flat Mining Tables: Operational databases normalize data into linked tables (via primary/foreign keys) to avoid duplicate storage and update anomalies. However, most ML algorithms assume a single flat table of independent, fixed-length rows.
Denormalization and One-to-Many Relationships: Denormalization joins related tables into one flat table. For many-to-one links, parent attributes are copied directly onto each child row. For one-to-many links (e.g., one customer with multiple orders), a plain SQL join would duplicate the customer row multiple times and drop zero-child rows. Instead, child rows must be aggregated (COUNT, SUM, AVG, MIN, MAX, or indicator flags) using a LEFT JOIN and GROUP BY at the target instance's grain.
Databases are normalized to avoid redundancy and anomalies, whereas ML models need a single flat table of independent instances. Denormalization joins normalized tables, handling one-to-many links through LEFT JOIN + GROUP BY aggregations (COUNT, SUM, AVG, etc.) so row counts and independence at the target grain are preserved.
Compare the four attribute measurement scales (nominal, ordinal, interval, and ratio). Give one original example of each and state why treating a nominal or interval attribute carelessly can mislead a learning algorithm.
Four Measurement Scales:
- Nominal: Unordered categories; only equality () is valid (e.g.,
department = {CS, Civil, Electrical}). - Ordinal: Ordered categories without fixed intervals; comparisons () are valid (e.g.,
severity = {low, moderate, critical}). - Interval: Equal numeric units without a true zero; differences () are meaningful, ratios () are not (e.g., temperature in ).
- Ratio: Equal numeric units with a true absolute zero; ratios () are meaningful (e.g.,
packet_sizein bytes).
Why Careless Encoding Misleads Learners: If nominal categories like department are encoded as integers , a linear or distance-based algorithm falsely infers that and that is the "average" of and . Nominal attributes should instead use one-hot (dummy) encoding or categorical splits. Similarly, ratio claims on interval scales (like treating as double ) fail because zero is arbitrary.
Nominal (unordered labels, equality only), Ordinal (ranked labels, ), Interval (equal units, arbitrary zero, differences valid), Ratio (equal units, true zero, ratios valid). Encoding nominal values as arbitrary integers introduces false ordering and distance relationships.
Describe the structure of an ARFF file and explain how missing values and sparse data are represented in ARFF.
Structure of an ARFF File: An ARFF file consists of:
- Optional comment lines beginning with
%. - A Header section with
@relation <name>followed by ordered@attribute <name> <type>declarations (numeric,{v1, v2, ...},string,date). - A Data section starting with
@data, followed by one comma-separated instance per line.
Missing Values and Sparse ARFF:
- Missing values are written as a literal question mark
?in the corresponding attribute position. - Sparse ARFF omits zero/default entries by enclosing non-zero entries in curly braces with zero-based column indices, e.g.,
{1 4.5, 12 yes}.
An ARFF file has % comments, a header (@relation and @attribute definitions), and a @data section of comma-separated rows. Missing values are written as ?, and sparse instances use {index value, ...} with zero-based attribute indices.
Consider the numeric attribute humidity with recorded readings:
where -99 is an invalid sensor-error code outside the physical range .
- Identify the invalid entry and convert it to missing (
?). - Compute the mean and sample variance of the remaining valid readings.
- If the missing positions are imputed with the valid mean, verify that the mean of the -element imputed array remains unchanged, and explain one situation where keeping a
missing_flagattribute is still useful.
Filtering the physically impossible reading: Humidity percentages must satisfy , so -99 is an inaccurate sentinel code. Converting -99 to ? leaves valid readings: .
Computing the valid mean and sample variance:
Imputation and informative missingness: Replacing both ? slots with gives , whose sum is and mean is (unchanged). Keeping a separate binary humidity_missing flag is valuable whenever missingness is informative—for example, if sensors fail specifically during heavy rainstorms, the flag itself helps predict the outcome even after imputation.
-99is invalid and replaced with?.- Valid mean ; sample variance .
- Imputing with keeps the overall mean at ; a
missing_flagpreserves signal when sensor failure correlates with the target event.