Uber open-sourced their M3 metrics platform, and while skimming through how it works, I found an approach called LTTB downsampling. Here’s what it is all about…
LTTB stands for Largest Triangle Three Buckets, which is a downsampling algorithm used to query and visualize millions of time series data points efficiently. The approach keeps the visual shape of the plot while reducing the number of points required to render it.
This makes the frontend efficient with fewer data points to render and less data to be emitted over the network.
It works by dividing the time series into buckets and selecting points that preserve the largest visual area. So, instead of random sampling or simple averaging, LTTB picks points that maintain the peaks, valleys, and overall trend of the data.
The way it does this is by considering three points at a time (previous, current, next) and calculating the area of the triangle they form. The point that creates the largest triangle gets selected. This means important visual features survive the downsampling.
With this approach, the system can downsample 10,000 points to 500 and still get a chart that looks nearly identical to the original. Beyond M3, systems like Grafana, InfluxDB, and Prometheus have all adopted LTTB or similar downsampling techniques.
It is super interesting that something as seemingly simple as “which points should we keep?” turns into such an interesting problem statement.