Categorical Encoding

Categorical encoding turns non-numeric labels into numbers a model can use, with one-hot, ordinal, and target schemes.

From categories to numbers

Most models require numeric input, but real data is full of categories: colors, cities, device types. Categorical encoding converts these labels into numbers without inventing false relationships. The right scheme depends on whether the categories have an order and how many distinct values there are.

One-hot encoding

One-hot encoding creates a binary column per category, with a 1 in the column for the present value and 0 elsewhere. It adds no false ordering, so it is the safe default for nominal (unordered) categories. Its cost is width: a feature with many categories explodes into many columns, worsening the curse of dimensionality and slowing linear and distance-based models.

import pandas as pd
X = pd.get_dummies(df, columns=['color', 'city'], drop_first=True)
# drop_first avoids one redundant collinear column

Other schemes

• Ordinal encoding: map ordered categories to integers (low
• Target encoding: replace a category with the mean target for that category; compact but leakage-prone.
• Frequency encoding: replace a category with how often it appears.
• Hashing: map categories into a fixed number of buckets for very high cardinality.

Avoiding leakage and pitfalls

Fit the encoder on training data only, and handle categories unseen at training time gracefully. Target encoding is especially dangerous: computing category means on the full dataset leaks the target, so it must be done out-of-fold, exactly like inside cross-validation. Tree models tolerate high-cardinality encodings better than linear or distance-based models, so match the scheme to the algorithm as part of feature engineering.