1441 views • Backend System Design
DynamoDB does not support aggregation queries, but we need it for a use case; let’s build a real-time DDB aggregation today…
Deliveroo, a food delivery startup had a similar problem. On their app, people can mark a restaurant as “favorite” and now they wanted to render restaurants ordered by most favorite first.
We have a
favorites table in which we store users and their favorite restaurants. The table has
restaurant_id_user_id as their primary key and
user_id, as other attributes.
With the above data model, getting if a user marked a restaurant as a favorite is an O(1) lookup and so are the marking and unmarking activites.
With this data model, it becomes near impossible to get restaurants ordered by their favorite count, purely because DynamoDB does not support aggregations.
Maintain a separate table having aggregated favorite count as one of the attributes and use it to get tables ordered by favorite count.
Introducing a new table
aggregated_favourites having the following schema
rastaurant_idas the primary key
time_windowas the sort key
updated_atas other attributes.
We set up a DynamoDB stream that would contain all the events happening on the
favorites table. This stream will be consumed by an AWS lambda function.
The lambda function will transactional do
count++ upon every creation and
count-- on deletion.
This way, we maintain the aggregated favorite count for each restaurant in near-realtime without doing any fancy code changes.
If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter.
924 views • 54 likes • 2022-11-14
When a company scales, they adopt microservices and each service typically gets its own independent database. With data ...
2206 views • 98 likes • 2022-11-07
Authorization plays a critical role in ensuring that the platform is not abused. For example, Instagram ensures that if ...
2572 views • 152 likes • 2022-10-31
Do hyperlocal companies like Uber, Ola, Swiggy, Gojek, Zomato, etc share our phone numbers with the delivery people or t...
2193 views • 81 likes • 2022-10-24
Elasticsearch is a great search engine, but Yelp was not happy with its performance, so they built their own HTTP layer ...
A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.
Being a passionate engineer, I love to talk about a wide range of topics, but these are my personal favourites.
Arpit's Newsletter read by 21000+ engineers
Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.
Powered by this tech stack.