Backend System Design

27 videos 22 essays

Videos


Overview of Discord's data platform that daily processes petabytes of data and trillion points

924 views 54 likes 2022-11-14

When a company scales, they adopt microservices and each service typically gets its own independent database. With data ...

How Airbnb designed and scaled its central authorization system - Himeji

2206 views 98 likes 2022-11-07

Authorization plays a critical role in ensuring that the platform is not abused. For example, Instagram ensures that if ...

How Gojek masks and keeps users' phone numbers secure at scale?

2572 views 152 likes 2022-10-31

Do hyperlocal companies like Uber, Ola, Swiggy, Gojek, Zomato, etc share our phone numbers with the delivery people or t...

The architecture of Yelp's in-house Search Engine - nrtSearch

2193 views 81 likes 2022-10-24

Elasticsearch is a great search engine, but Yelp was not happy with its performance, so they built their own HTTP layer ...

How Giphy uses CDN to serve a billion GIFs every day

2942 views 120 likes 2022-10-17

Giphy is the world's most popular GIF website and it serves 10 billion media content every single day, and we can guess ...

How Dropbox efficiently serves and renders a large number of thumbnails

1586 views 99 likes 2022-10-07

A classic challenge that comes while building Instagram or Google Photos is about quickly and efficiently serving and re...

Aggregating DynamoDB data in realtime to list restaurants at Deliveroo

1441 views 68 likes 2022-10-03

DynamoDB is an extremely powerful, scalable, and fast KV store but it lacks Aggregations. In this video, we design a us...

How Razorpay scaled their notification system

4100 views 236 likes 2022-09-30

Notifications are extremely crucial for Fintech companies as it is a way to notify a user about an incoming transaction....

How Flipkart made their type ahead search hyper personalized

2031 views 159 likes 2022-09-28

Search is one of the most used features in any e-commerce and to give a better user experience it is almost customary to...

The Architecture of Pinterest's Time Series Database - Goku

1806 views 91 likes 2022-09-26

It is extremely critical to continuously monitor the health of the services and infrastructure. We use Time Series Datab...

The Architecture of Airbnb's Knowledge Graph

1770 views 87 likes 2022-09-23

When a product is catering to a large audience, providing contextual information becomes important and at the scale of A...

How Swiggy designed and scaled its chatbot

2293 views 128 likes 2022-09-21

We all love food but are lazy to go to a restaurant and pick it ourselves and this gave rise to Food Delivery Startups l...

How Instagram efficiently serves HashTags ordered by count

2693 views 213 likes 2022-09-19

Instagram has millions of HashTags and millions of people tag their photos every single day. To discover, Instagram allo...

An in-depth introduction to Rolling Deployments

1024 views 49 likes 2022-05-27

One of the simplest deployment strategies that make deployment a breeze is Rolling Deployment. It is the most widely ado...

Implementing Vertical Sharding

1481 views 92 likes 2022-05-25

Sharding is super-important when you want to handle the traffic that cannot be handled through one server. Sharding come...

An in-depth introduction to Blue Green Deployments

1387 views 63 likes 2022-05-18

Deployments are a pain if we are unsure about our release changes. But sometimes even if we know our changes well, somet...

An in-depth introduction to Canary Deployments

2081 views 124 likes 2022-05-16

Deployments are stressful; what if something goes wrong? What if you forgot to handle an edge case that was also missed ...

Introduction to Serverless Computing and Architecture

2254 views 135 likes 2022-05-02

Serverless Computing is one of the hottest topics of discussion today, but the term "serverless" is slightly misleading ...

Database Sharding and Partitioning

4782 views 301 likes 2022-04-25

Sharding and partitioning come in very handy when we want to scale our systems. These concepts operate on the database a...

Why, where, and when should we throttle or rate limit?

2414 views 135 likes 2022-04-18

It is a common belief that a rate limiter is always external and is designed to prevent our systems from being abused by...

How to approach System Design?

5010 views 168 likes 2022-04-13

System Design is tricky but it does not have to be difficult - be it a technical discussion at your workplace or your ne...

Implementing Idempotence in a Payments Microservice

4553 views 245 likes 2022-04-06

Idempotence is an extremely critical property that we must consider while implementing an API or designing a microservic...

Sending payload in an HTTP GET request

4474 views 157 likes 2022-04-04

Can we send data in an HTTP GET request? Most people think, No. The truth is, we can send the data in the request payloa...

Scaling Taxonomy Service and Database

1051 views 30 likes 2021-05-02

In this video, we do the high-level design of Udemy's Taxonomy Service and see how to scale databases in general. Outli...

Designing Taxonomy on SQL

2621 views 96 likes 2021-04-19

In this video, we design the Taxonomy of Udemy on top of a SQL-based Relational DB. We learn how to model any taxonomy o...

End-to-end Message Encryption

6047 views 244 likes 2021-04-05

In this video, we find how to implement a very very simple version of end-to-end WhatsApp-like message encryption. This ...

Designing Notifications Service for Instagram

27389 views 646 likes 2021-04-01

This video is a snippet from my System Design Masterclass and in it, we are discussing How Instagram Scales its notifica...

Essays


Indexing on Partitioned Data

380 reads 2022-02-07

In this essay, we will take a detailed look into how we could index the partitioned data, allowing us to query the data ...

Data Partitioning Strategies

302 reads 2022-01-31

In this essay, we take a detailed look into the two common approaches to horizontally partition the data - Hash Based an...

Data Partitioning

435 reads 2022-01-24

In this essay, we take a detailed look into Partitioning basics and understand how it can help us scale our Reads and Wr...

Leaderless Replication

444 reads 2022-01-16

In this essay, we take a look into a different way of replication, called Leaderless Replication, that comes in handy in...

Conflict Resolution

275 reads 2022-01-09

In this essay, go in-depth to understand ways to resolve and avoid conflicts in a multi-master setup....

Conflict Detection

709 reads 2021-11-28

In this essay, we talk about conflicts and understand what they are, how to detect them in a multi-master setup....

Multi-Master Replication

497 reads 2021-11-03

In this essay, we look at what Multi-Master Replication is, the core challenge it addresses, use-cases we can find this ...

Monotonic Reads

348 reads 2021-10-03

Asynchronous replication leads to a fascinating situation where it feels like we are going through a wormhole traveling ...

Read-your-write Consistency

1120 reads 2021-09-22

Read-Your-Writes consistency states that the system guarantees that, once an item has been updated, any attempt to read ...

Handling outages in a Master-Replica setup

375 reads 2021-09-07

This essay talks about the worse - nodes going down - impact, recovery, and real-world practices....

The New Replica

293 reads 2021-08-23

In this one, we take a look into how these Replicas are set up and understand some quirky nuances about Replication....

Replication Formats

334 reads 2021-08-15

When we are employing a Master-Replica pattern to improve availability, throughput, and fault-tolerance, the big questio...

Replication Strategies

821 reads 2021-08-10

In this essay, we take a quick yet verbose look into Synchronous, Asynchronous, and Semisynchronous replication strategi...

Master-Replica Replication

350 reads 2021-08-07

In this essay, we talk about everything we should know about Master-Replica replication pattern....

ACID - Durability

335 reads 2021-07-19

Durability seems to be a taken-for-granted requirement, but to be honest, it is the most important one. Let's deep dive ...

ACID - Isolation

301 reads 2021-07-05

Isolation is the ability of the database to concurrently process multiple transactions in a way that changes made in one...

ACID - Consistency

387 reads 2021-07-02

In the context of databases, Consistency is Correctness, which means that under no circumstance will the data lose its c...

ACID - Atomicity

748 reads 2021-06-28

A single database transaction often contains multiple statements to be executed on the database. In Relational Databases...

Architectures in Distributed Systems

563 reads 2021-06-22

While designing a Distributed System, it is essential to pick the right kind of architecture. Usually, architectures are...

Mistaken Beliefs of Distributed Systems

358 reads 2021-06-17

In this essay, we learn about a set of false assumptions that programmers new to distributed applications invariably mak...

The RUM Conjecture

670 reads 2020-05-31

While designing any storage system the three main aspects we optimize for are Reads, Updates, and auxiliary Memory. RUM ...

Sliding Window based Rate Limiter

1056 reads 2020-04-05

A rate limiter is used to control the rate of traffic sent or received on the network and in this article we dive deep a...

Be a better engineer

A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.


Paid Courses

System Design Masterclass

A masterclass that helps you become great at designing scalable, fault-tolerant, and highly available systems.

1000+ learners

Details →

Redis Internals

Learn internals of Redis by re-implementing some of the core features in Golang.

28+ learners

Details →

Free Courses

Designing Microservices

A free playlist to help you understand Microservices and their high-level patterns in depth.

17+ learners

Details →

GitHub Outage Dissections

A free playlist to help you learn core engineering from outages that happened at GitHub.

67+ learners

Details →

Hash Table Internals

A free playlist to help you understand the internal workings and construction of Hash Tables.

25+ learners

Details →

BitTorrent Internals

A free playlist to help you understand the algorithms and strategies that power P2P networks and BitTorrent.

42+ learners

Details →

Topics I talk about

Being a passionate engineer, I love to talk about a wide range of topics, but these are my personal favourites.





  • v13.8.5
  • © Arpit Bhayani, 2022

Powered by this tech stack.