Unsupervised Learning & Clustering: Foundations

Source: Unit 4 §1

Supervised learning always had a column to aim at. Unsupervised learning removes it: there is no target to predict, only the inputs X and the question of what structure is hiding in them. The main answer to that question is clustering.

Supervised vs unsupervised learning

	Supervised Learning	Unsupervised Learning
Target variable	Yes - trained under the supervision of a target/label	No target variable
Data	Independent variables + target	Only independent variables
Goal	Predict the target	Find hidden structure / groups
Example	Classification, Regression	Clustering

≡FactsThe distinction, spelled out

• Supervised: we train using the independent variables under the supervision of the target, which is where the name comes from.
• Unsupervised: sometimes there is no target to predict - we have only the inputs X and want to discover structure.
• Why supervised at all? it lets you learn from previous experience, optimize a performance criterion, and solve real-world computation problems.

What is clustering?

Clustering divides the entire dataset X into groups (clusters) such that objects within a group are similar to each other and different from objects in other groups.

◎Exam cueThe clustering objective, in one line

Minimize intra-cluster distance, maximize inter-cluster distance. Every clustering algorithm on the following pages is a different way of chasing those two numbers at once.

One clustering objective, two directions: shrink the arrow inside a group while stretching the arrow between groups.

Example applications: Marketing, Insurance, City-planning, Earthquake studies.

Four clustering approaches

Four families, one question each: is the output a tree, a flat partition, a set of dense regions, or a grid?

Type	Idea
Hierarchical	Build a tree (dendrogram) of nested clusters
Partitional	Divide data into non-overlapping clusters (e.g. K-Means)
Density-based	Clusters = dense regions (e.g. DBSCAN)
Grid-based	Quantize space into a grid

⇄ComparePartitional vs hierarchical

PartitionalA **single division** of the data into non-overlapping subsets: one cluster per point, and that is the whole answer.

HierarchicalA set of **nested clusters** organized as a tree, so every level of granularity is available at once.