Data & System Architecture, from the ground up Lesson 21 / 80

Time-series databases: Influx, Timescale, Prometheus

When timestamp-plus-value is 99% of your data. The optimizations that let time-series stores beat general-purpose databases by 10x or more.

There is a class of workload where the same row repeats forever, only the timestamp moves, and the questions you ask about it are almost always shaped like “what was the average of this measurement over the last fifteen minutes.” Application metrics. Infrastructure metrics. IoT sensor readings. Financial tick data. If you have ever stared at a Grafana dashboard, you have looked at time-series data. This lesson is about why this shape deserves its own database family, what the family does to win, and which of the three real options you should reach for.

The headline number, before we get into the mechanics, is that a time-series database will outperform a general-purpose database on time-series workloads by a factor of ten to fifty, on the same hardware, for the same data. That is not a marketing claim. It is the consequence of three or four design decisions that the time-series stores all share, and that Postgres, MySQL, and friends do not. The decisions are interesting in their own right: they are a small case study in what happens when you specialise a storage layer to a single shape of data.

The shape of the data

A time-series point has a small, predictable structure. There is a timestamp (usually nanosecond or microsecond precision). There is a measurement name, which is the metric you are recording: cpu_usage, request_count, temperature. There is a set of tags, which are key-value pairs that identify which thing this measurement is about: host=web-01, region=eu-west, service=checkout. And there is one or more values, the actual numbers being recorded.

Conceptually a single point looks like:

2025-12-12T10:14:32.451Z  cpu_usage{host=web-01, region=eu-west}  47.3

You record millions of these per minute. The same (measurement, tag-set) combination, called a series, gets a new point every scrape interval, forever. Reads almost always take the form “for series matching this filter, give me the points between time A and time B, optionally aggregated into buckets of size N.” Writes are almost always inserts at the head of the series; updates and deletes of individual points are rare to non-existent.

This is a much narrower workload than a general-purpose database is designed for. The narrowness is the opportunity.

Why general-purpose databases struggle

Postgres can store time-series data. Plenty of people do exactly this for the first few months of a project. The trouble starts as the data grows.

The first problem is index size. A B-tree on (series_id, timestamp) works fine, but a B-tree of a billion entries takes up serious space, and the writes thrash the upper levels of the tree as new entries land at the maximum value of the key. The write amplification is real.

The second problem is storage cost. A row in Postgres has a fixed per-row overhead (around 24 bytes for the tuple header, plus per-column overhead, plus the actual data). A time-series point is maybe 24 bytes of actual content (timestamp + value + a series reference). The overhead doubles or triples your storage. At the scale where time-series matters, the storage bill is the bill.

The third problem is deletion. Time-series data has natural retention: you keep last week’s data at full resolution, last month’s at downsampled resolution, last year’s even coarser. In Postgres, deleting a billion old rows is an expensive operation, and the autovacuum work that follows is more expensive. Partition tables help, and dropping partitions is fast, but you have to set it up explicitly and keep up with it.

The fourth problem is query patterns that span huge ranges. “Average of cpu_usage over the last day, bucketed every minute” against a billion-row table is a sequential scan over a large slice. Without specialised indexes and pre-aggregation, it is just slow.

Time-series databases solve each of these explicitly.

What time-series databases do

The optimisations vary in detail across the products, but they all share roughly the same toolkit.

Append-mostly writes, sorted by time. Internally the storage is organised so that recent writes go into an in-memory buffer that flushes to immutable files, sorted by time. There is no random write to old data. Inserts cost almost nothing. The structure is similar in spirit to an LSM tree, specialised for time-ordered keys.

Columnar storage with compression. Instead of storing each point as a row, time-series stores keep each series as a column of values plus a column of timestamps. The same metric repeated row after row compresses extremely well: typical numbers are 10x to 50x compression depending on the metric (a CPU usage that hovers around 30% compresses much better than a random integer ID). Compression is the difference between a multi-terabyte storage bill and a multi-hundred-gigabyte one.

Aggregation at write time. As points come in, the database can maintain rolled-up summaries: per-minute, per-hour, per-day averages, sums, counts. When you ask for “average per hour over the last week,” the query reads the per-hour rollups, not the raw points. The rollup tables are tiny relative to the raw data, and the query cost drops by orders of magnitude.

TTL-based retention. “Delete points older than N days” is a built-in primitive. The implementation is usually to drop entire time partitions, which is a metadata operation rather than a row-by-row delete. Retention is cheap and reliable.

These are the four moves. Every time-series store does some version of all of them. The differences are in the surface area: the query language, the operational model, the integration with the rest of your stack.

The three real options

There is an entire ecosystem of time-series databases, but in 2026 the practical choices for a new project narrow to three.

Prometheus: the metrics standard

Prometheus is the dominant metrics system in the cloud-native world. It is pull-based: you run Prometheus as a server, and it scrapes metrics over HTTP from a list of targets every fifteen seconds (or whatever interval you set). The targets expose a /metrics endpoint with the current values; Prometheus pulls and stores. There is no client-side agent pushing data into the database. The model is unusual and, once you internalise it, very pleasant: services do not need to know where the metrics go, they just expose an endpoint.

Prometheus stores data locally, on a single server, in its own time-series format. Retention is typically fifteen to thirty days. The query language is PromQL, which is dense, powerful, and worth learning if you are doing any serious work in this space.

The crucial caveat: Prometheus is not a long-term store. It is a single-server system, and the recommended pattern for keeping years of data is to pair Prometheus with a long-term store. The standard names are Thanos, Cortex, and Grafana Mimir, all of which back Prometheus with object storage (S3, GCS) and provide horizontal scale and longer retention. Pick one based on the operational model you prefer; the data model is the same in all three.

InfluxDB: the classical time-series database

InfluxDB is the original “let’s build a time-series database from scratch” product, started in 2013 and still around. It is push-based by default, with a wire protocol that clients can write directly to. Retention policies and downsampling are built in. The query language has had a turbulent history: the original was InfluxQL (SQL-like), then they switched to Flux (a functional query language), then they walked Flux back and went back to InfluxQL plus optional SQL. If you are looking at InfluxDB today, make sure you understand which version you are looking at and which query language is current.

The product story has been bumpy enough that I would not pick InfluxDB today for a greenfield project unless there is a specific reason (an existing investment, a feature only Influx has). The two more boring options below cover most needs.

TimescaleDB: time-series in Postgres

TimescaleDB is a Postgres extension. You install it on your existing Postgres server, you create hypertables (Timescale’s wrapper around partitioned tables), and you get time-series performance with the full Postgres surface area: SQL, joins, indexes, foreign keys, the rest of the ecosystem.

For teams that are already on Postgres, this is often the right answer. You do not run a second database. The query language is the SQL you already know. You can join your time-series data against your relational data in a single query. The downsampling and retention features are first-class.

The trade-off is that Timescale’s per-node throughput is lower than a specialised store like InfluxDB or a columnar OLAP system like ClickHouse. For most application-metrics or business-metrics workloads, the throughput is sufficient. For very high-volume IoT or trading data, you may outgrow it.

ClickHouse: the columnar OLAP option

I will mention ClickHouse here even though it is not strictly a time-series database, because it does the time-series job extremely well. ClickHouse is a columnar OLAP database designed for analytical queries over large data volumes, and time-series is one of the workloads it excels at. The storage is columnar, the compression is excellent, the query language is SQL with extensions, and the throughput is very high.

We will cover ClickHouse properly in module 8 when we get to OLAP. For now, know that if your time-series workload also has analytics-style queries (group-bys across many dimensions, complex aggregations), ClickHouse is worth a look.

The pipeline

Most production metrics setups look the same, regardless of which store is in the middle. Services are instrumented (with Prometheus client libraries, OpenTelemetry, or vendor-specific agents). Prometheus scrapes them. Long-term storage holds the historical data. Grafana queries both for dashboards.

flowchart LR
    A[Service A: /metrics] --> P[Prometheus]
    B[Service B: /metrics] --> P
    C[Service C: /metrics] --> P
    P --> LT[Long-term store: Thanos/Mimir]
    P --> G[Grafana]
    LT --> G
    G --> U[User dashboards]

The split between Prometheus (recent, fast) and the long-term store (historical, cheaper) is the standard pattern. Queries for the last hour go to Prometheus; queries for last quarter go to Thanos. Grafana abstracts the difference.

The cardinality trap

There is one operational pitfall worth flagging because every team using a time-series database hits it eventually: cardinality explosion.

The number of distinct series in your database is the product of the number of distinct values for each tag. If you have one metric with three tags (host, region, service), and each tag has ten distinct values, you have a thousand series. Manageable. If one of those tags is user_id, with a million distinct values, you now have a hundred million series. The database will struggle. Memory usage explodes, query performance collapses, and the system may simply refuse to accept new writes.

The rule is: tags must be low-cardinality. Things like host, region, service, endpoint, status_code are fine. Things like user_id, request_id, session_id, ip_address are dangerous. If you find yourself wanting to put a high-cardinality identifier as a tag, you almost certainly want a different system: a logging store (Loki, Elasticsearch) or a tracing store (Jaeger, Tempo), not a metrics store.

Every team learns this once, the painful way. Now you have learned it the easy way.

Use cases that fit

The workloads where time-series databases are unambiguously the right answer:

  • Application metrics: request rates, latencies, error rates, saturation. The standard four “golden signals.” These power every production dashboard.
  • Infrastructure metrics: CPU, memory, disk, network. The substrate everything else runs on.
  • IoT sensor data: temperature readings, GPS positions, device telemetry. Often partitioned by device ID, which means you need to be careful about cardinality (a million devices is fine; a billion is not, without special design).
  • Financial tick data: per-instrument prices, sampled at high frequency. ClickHouse and KDB+ dominate here, but TimescaleDB shows up in this space too.

Putting it together

The recommendation, for most teams in 2026, is short. For metrics in a cloud-native stack, use Prometheus, paired with Thanos or Mimir for long-term storage, queried through Grafana. For time-series workloads embedded in an application that already uses Postgres, use TimescaleDB and avoid running a second database. For analytics-heavy time-series at very high volume, look at ClickHouse. If you find yourself reaching for InfluxDB, make sure you have a specific reason.

The shared property of all of these is that they are specialised. They beat Postgres on time-series workloads by a wide margin, and they lose to Postgres on anything that is not a time-series workload. Use the right tool. The cost of running a specialised store next to your main database is almost always less than the cost of trying to do everything in one.

Citations and further reading

  • Prometheus documentation, https://prometheus.io/docs/ (retrieved 2026-05-01). The reference for PromQL, the scrape model, and the long-term storage options.
  • TimescaleDB documentation, https://docs.timescale.com/ (retrieved 2026-05-01). Hypertables, continuous aggregates, retention policies.
  • InfluxDB documentation, https://docs.influxdata.com/ (retrieved 2026-05-01). The InfluxDB 3 line and the recent return to SQL.
  • Brian Brazil, “Prometheus: Up and Running” (O’Reilly, 2018). Still the best book for getting deep into PromQL and the operational model. A second edition is in progress.
  • Grafana Labs, “Grafana Mimir documentation”, https://grafana.com/docs/mimir/latest/ (retrieved 2026-05-01). The long-term-store option that has been gaining the most ground in recent years.
Search