Real-Time Analytics
Real-time analytics answers queries over data that is seconds old, combining fast ingestion, incremental computation, and low-latency serving.
Freshness as a requirement
Traditional analytics runs over data that landed hours or a day ago. Real-time analytics narrows that gap to seconds, so a dashboard, alert, or automated decision reflects what is happening now. Achieving this reshapes the whole path: ingestion must be continuous, computation incremental, and serving fast, all at once.
The three pressures
- Ingestion latency: how quickly new events become queryable
- Query latency: how fast a question returns an answer
- Freshness under load: keeping both low as event volume grows
Ingestion and incremental compute
Data arrives as a stream rather than a batch, so it is ingested continuously from an event log. Rather than recompute aggregates from scratch, real-time systems maintain them incrementally: each new event updates a running count or sum. Pre-aggregation into materialized summaries trades storage and write work for near-instant reads, which is the right trade when reads vastly outnumber the questions asked.
Serving-layer stores
Real-time analytics uses specialized stores that ingest streams and answer aggregation queries in milliseconds. They typically keep data columnar for scan speed, index by time and a few dimensions, and hold recent data in memory. The design point differs from a warehouse: a warehouse optimizes throughput over huge history, a real-time store optimizes latency over recent data.
Accuracy versus latency
Real-time results sometimes trade exactness for speed. Approximate algorithms estimate distinct counts or percentiles far faster than exact ones, with bounded error. Late-arriving events may be incorporated as corrections rather than blocking the answer. The system's contract is not "perfectly correct eventually" but "good enough, right now, with known error." For monitoring a physical process, an answer that is a few seconds old and slightly approximate is far more useful than a perfect answer that arrives too late to act on. See stream processing, lambda and kappa, and time-series databases.