Handling timeouts in a microservice architecture

Watch the video explanation ➔

Microservices give us the flexibility to pick the best tech stack to solve the problem optimally. But one thing that ruins the real thrill is Timeouts.

Say we have a blogging website where a user can search for blogs. The request comes to the Search service, and it finds the most relevant blogs for the query.

In the response, a field called total_views should hold the total number of views the blog received in its lifetime. The search services should talk to the Analytics service synchronously to get the data. This synchronous dependency is the root of all evil.

The core problem: Delays can be arbitrarily large

Because the delay depending on service can be arbitrarily large, we know how long to wait for the response. We for sure cannot wait forever, and hence we introduce Timeout. Every time the Search service invokes the Analytics service, it starts a timer, and if it does not get a response in the stipulated time, it timeout and moves on.

There are 5 approaches to handling timeouts.

  • Approach 1: Ignore the timeout and move on
  • Approach 2: Use some default value if you timed out
  • Approach 3: Retry the request
  • Approach 4: Retry only when needed
  • Approach 5: Re-architect and make synchronous dependency an async one

Here's the video ⤵

Courses

Super practical courses, with a no-nonsense approach, are designed to spark engineering curiosity and help you ace your career.


System Design for Beginners

An in-depth, self-paced, and on-demand course that for early engineers to become great at designing scalable, available, and extensible systems at scale.

Details →

System Design Masterclass

A masterclass that helps experienced engineers become great at designing scalable, fault-tolerant, and highly available systems.

Details →

Redis Internals

A course that helps covers Redis internals by reimplementing its core features like - event loop, serialization protocol, pipelining, eviction, and transactions.

Details →



Writings and Videos

Videos

Essays and Blogs


Arpit's Newsletter read by 70000+ engineers

Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.