Data Systems

Embedding Storage and Indexing

Storing embeddings well means choosing how to persist dense vectors, keep them synchronized with their source records, and rebuild indexes as data changes.

Embeddings as first-class data

An embedding is a fixed-length vector produced by a model from some input: a document, an image, a sensor trace. Because it is derived, it must be treated as a cache of a model plus an input, not as ground truth. If either the model version or the source record changes, the stored embedding becomes stale. Good embedding storage records which model version produced each vector so the whole set can be regenerated deterministically.

Layout choices

Kronos motion — data assimilation

Alongside source rows in a relational or document store, for simple joins
In a dedicated vector store with the source id as metadata
In flat binary files (one contiguous float array) for bulk index builds
Compressed via quantization when memory is the binding constraint

Dimensionality and footprint

A vector of d single-precision floats occupies 4d bytes. Ten million 768-dimensional vectors is roughly 30 gigabytes uncompressed. This footprint drives most engineering decisions: whether the index fits in RAM, how much quantization is needed, and how sharding is done. Reducing d through a smaller embedding model or dimensionality reduction is often the cheapest lever, but it can cost recall.

Keeping vectors fresh

When a source record is updated, its embedding must be recomputed and the index entry replaced. Deletes are harder for some index types: graph indexes such as HNSW cannot cheaply remove a node, so deletes are often tombstoned (marked invalid and skipped at query time) and cleaned up during periodic index rebuilds. A change-data-capture stream from the source table is a clean way to trigger re-embedding.

Versioning the whole set

Because embeddings depend on a model, upgrading the model invalidates every vector at once. Teams handle this with a blue-green swap: build a second index with the new model, verify recall on a held-out query set, then switch traffic. The old index stays available for rollback. This makes the embedding index a versioned artifact much like a trained model.