Data Catalogs
A data catalog is an organized inventory of an organization's datasets with their descriptions, owners, schemas, and quality, making data findable and understandable.
The findability problem
In a large organization, the hardest data question is often "does the data I need already exist, and where?" Datasets multiply across systems with cryptic names and no documentation, so people rebuild what already exists or, worse, use the wrong table. A data catalog is the inventory that solves this: a searchable index of datasets enriched with the context needed to trust and use them.
What a catalog entry holds
- Schema: fields, types, and descriptions
- Ownership: the accountable owner and steward
- Provenance: where the data comes from and how it is produced
- Quality: freshness, completeness, and known issues
- Usage: who queries it and how often, as a popularity signal
Technical and business metadata
A catalog blends two kinds of metadata. Technical metadata is extracted automatically: schemas, partitions, sizes, last-updated times. Business metadata is curated by people: what a column means, which definition of a metric this table implements, and any caveats. Automated extraction keeps the catalog current; human curation makes it trustworthy. A catalog that has only one of the two is either stale or empty.
Harvesting versus curation
Modern catalogs harvest metadata by connecting to sources and scanning them, so the technical layer stays fresh without manual entry. This scales to thousands of datasets, but harvested metadata alone cannot say whether a dataset is fit for a purpose. The catalog's value comes from layering curation, ownership, quality signals, and lineage, on top of the harvested skeleton.
The catalog as a governance surface
Because it inventories every dataset, the catalog is the natural place to attach governance: sensitivity classifications, access policies, and retention rules. It becomes the map that both a curious analyst and a compliance auditor consult. Closely related is discovery, the experience of searching and browsing the catalog effectively. See data discovery, data lineage, and data governance.