Backend System Design

36 videos 22 essays

Videos


Thundering Herd Problem and How not to do API retries

6281 views 270 likes 2023-01-31

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Designing Idempotent API Endpoints for Payments at Stripe

6253 views 326 likes 2023-01-29

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Slack efficiently classifies emails at scale with an eventually consistent system

3085 views 103 likes 2023-01-13

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How @ShopifyEngineering avoids hot shards by moving data across databases without any downtime

4617 views 170 likes 2023-01-08

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

The architecture of Grab's data layer that serves millions of orders every day

8653 views 375 likes 2023-01-01

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How @twitter keeps its Search systems up and stable at scale

4565 views 177 likes 2022-12-25

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Zomato improved its search by identifying intent through NLP

4525 views 205 likes 2022-12-11

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Designing Uber's highly available Emergency SOS Service

4145 views 153 likes 2022-11-28

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Booking com designed and scaled their highly available and performant User Review System

7689 views 303 likes 2022-11-21

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Overview of Discord's data platform that daily processes petabytes of data and trillion points

2868 views 113 likes 2022-11-14

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Airbnb designed and scaled its central authorization system - Himeji

4579 views 158 likes 2022-11-07

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Gojek masks and keeps users' phone numbers secure at scale?

4031 views 218 likes 2022-10-31

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

The architecture of Yelp's in-house Search Engine - nrtSearch

3486 views 120 likes 2022-10-24

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Giphy uses CDN to serve a billion GIFs every day

4490 views 176 likes 2022-10-17

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Dropbox efficiently serves and renders a large number of thumbnails

2530 views 138 likes 2022-10-07

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Aggregating DynamoDB data in realtime to list restaurants at Deliveroo

2323 views 104 likes 2022-10-03

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Razorpay scaled their notification system

6430 views 331 likes 2022-09-30

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Flipkart made their type ahead search hyper personalized

4196 views 260 likes 2022-09-28

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

The Architecture of Pinterest's Time Series Database - Goku

2685 views 124 likes 2022-09-26

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

The Architecture of Airbnb's Knowledge Graph

2691 views 119 likes 2022-09-23

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Swiggy designed and scaled its chatbot

3944 views 189 likes 2022-09-21

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Instagram efficiently serves HashTags ordered by count

6614 views 420 likes 2022-09-19

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

An in-depth introduction to Rolling Deployments

1024 views 49 likes 2022-05-27

One of the simplest deployment strategies that make deployment a breeze is Rolling Deployment. It is the most widely ado...

Implementing Vertical Sharding

3247 views 169 likes 2022-05-25

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

An in-depth introduction to Blue Green Deployments

1387 views 63 likes 2022-05-18

Deployments are a pain if we are unsure about our release changes. But sometimes even if we know our changes well, somet...

An in-depth introduction to Canary Deployments

2081 views 124 likes 2022-05-16

Deployments are stressful; what if something goes wrong? What if you forgot to handle an edge case that was also missed ...

Introduction to Serverless Computing and Architecture

2254 views 135 likes 2022-05-02

Serverless Computing is one of the hottest topics of discussion today, but the term "serverless" is slightly misleading ...

Database Sharding and Partitioning

12419 views 603 likes 2022-04-25

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Why, where, and when should we throttle or rate limit?

4680 views 214 likes 2022-04-18

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How to approach System Design?

14050 views 386 likes 2022-04-13

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Implementing Idempotence in a Payments Microservice

8847 views 424 likes 2022-04-06

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Sending payload in an HTTP GET request

4474 views 157 likes 2022-04-04

Can we send data in an HTTP GET request? Most people think, No. The truth is, we can send the data in the request payloa...

Scaling Taxonomy Service and Database

1051 views 30 likes 2021-05-02

In this video, we do the high-level design of Udemy's Taxonomy Service and see how to scale databases in general. Outli...

Designing Taxonomy on SQL

4506 views 152 likes 2021-04-19

In this video, we design the Taxonomy of Udemy on top of a SQL-based Relational DB. We learn how to model any taxonomy o...

End-to-end Message Encryption

6047 views 244 likes 2021-04-05

In this video, we find how to implement a very very simple version of end-to-end WhatsApp-like message encryption. This ...

Designing Notifications Service for Instagram

36100 views 808 likes 2021-04-01

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Essays


Indexing on Partitioned Data

380 reads 2022-02-07

In this essay, we will take a detailed look into how we could index the partitioned data, allowing us to query the data ...

Data Partitioning Strategies

302 reads 2022-01-31

In this essay, we take a detailed look into the two common approaches to horizontally partition the data - Hash Based an...

Data Partitioning

435 reads 2022-01-24

In this essay, we take a detailed look into Partitioning basics and understand how it can help us scale our Reads and Wr...

Leaderless Replication

444 reads 2022-01-16

In this essay, we take a look into a different way of replication, called Leaderless Replication, that comes in handy in...

Conflict Resolution

275 reads 2022-01-09

In this essay, go in-depth to understand ways to resolve and avoid conflicts in a multi-master setup....

Conflict Detection

709 reads 2021-11-28

In this essay, we talk about conflicts and understand what they are, how to detect them in a multi-master setup....

Multi-Master Replication

497 reads 2021-11-03

In this essay, we look at what Multi-Master Replication is, the core challenge it addresses, use-cases we can find this ...

Monotonic Reads

348 reads 2021-10-03

Asynchronous replication leads to a fascinating situation where it feels like we are going through a wormhole traveling ...

Read-your-write Consistency

1120 reads 2021-09-22

Read-Your-Writes consistency states that the system guarantees that, once an item has been updated, any attempt to read ...

Handling outages in a Master-Replica setup

375 reads 2021-09-07

This essay talks about the worse - nodes going down - impact, recovery, and real-world practices....

The New Replica

293 reads 2021-08-23

In this one, we take a look into how these Replicas are set up and understand some quirky nuances about Replication....

Replication Formats

334 reads 2021-08-15

When we are employing a Master-Replica pattern to improve availability, throughput, and fault-tolerance, the big questio...

Replication Strategies

821 reads 2021-08-10

In this essay, we take a quick yet verbose look into Synchronous, Asynchronous, and Semisynchronous replication strategi...

Master-Replica Replication

350 reads 2021-08-07

In this essay, we talk about everything we should know about Master-Replica replication pattern....

ACID - Durability

335 reads 2021-07-19

Durability seems to be a taken-for-granted requirement, but to be honest, it is the most important one. Let's deep dive ...

ACID - Isolation

301 reads 2021-07-05

Isolation is the ability of the database to concurrently process multiple transactions in a way that changes made in one...

ACID - Consistency

387 reads 2021-07-02

In the context of databases, Consistency is Correctness, which means that under no circumstance will the data lose its c...

ACID - Atomicity

748 reads 2021-06-28

A single database transaction often contains multiple statements to be executed on the database. In Relational Databases...

Architectures in Distributed Systems

563 reads 2021-06-22

While designing a Distributed System, it is essential to pick the right kind of architecture. Usually, architectures are...

Mistaken Beliefs of Distributed Systems

358 reads 2021-06-17

In this essay, we learn about a set of false assumptions that programmers new to distributed applications invariably mak...

The RUM Conjecture

670 reads 2020-05-31

While designing any storage system the three main aspects we optimize for are Reads, Updates, and auxiliary Memory. RUM ...

Sliding Window based Rate Limiter

1056 reads 2020-04-05

A rate limiter is used to control the rate of traffic sent or received on the network and in this article we dive deep a...

Be a better engineer

A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.


Paid Courses

System Design for Beginners

A masterclass that helps early engineers and product managers become great at designing scalable systems.

180+ learners

Details →

System Design Masterclass

A masterclass that helps you become great at designing scalable, fault-tolerant, and highly available systems.

1000+ learners

Details →

Redis Internals

Learn internals of Redis by re-implementing some of the core features in Golang.

98+ learners

Details →

Free Courses

Designing Microservices

A free playlist to help you understand Microservices and their high-level patterns in depth.

823+ learners

Details →

GitHub Outage Dissections

A free playlist to help you learn core engineering from outages that happened at GitHub.

651+ learners

Details →

Hash Table Internals

A free playlist to help you understand the internal workings and construction of Hash Tables.

1027+ learners

Details →

BitTorrent Internals

A free playlist to help you understand the algorithms and strategies that power P2P networks and BitTorrent.

692+ learners

Details →