Handling timeouts in a microservice architecture

3816 views Designing μ-services

Microservices give us the flexibility to pick the best tech stack to solve the problem optimally. But one thing that ruins the real thrill is Timeouts.

Say we have a blogging website where a user can search for blogs. The request comes to the Search service, and it finds the most relevant blogs for the query.

In the response, a field called total_views should hold the total number of views the blog received in its lifetime. The search services should talk to the Analytics service synchronously to get the data. This synchronous dependency is the root of all evil.

The core problem: Delays can be arbitrarily large

Because the delay depending on service can be arbitrarily large, we know how long to wait for the response. We for sure cannot wait forever, and hence we introduce Timeout. Every time the Search service invokes the Analytics service, it starts a timer, and if it does not get a response in the stipulated time, it timeout and moves on.

There are 5 approaches to handling timeouts.

  • Approach 1: Ignore the timeout and move on
  • Approach 2: Use some default value if you timed out
  • Approach 3: Retry the request
  • Approach 4: Retry only when needed
  • Approach 5: Re-architect and make synchronous dependency an async one

Arpit Bhayani

Arpit's Newsletter

CS newsletter for the curious engineers

❤️ by 17000+ readers

If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter.

Other essays that you might like

BFF - Backend for Frontend - Pattern in Microservices

2635 views 131 likes 2022-07-04

As your application evolves, supporting multiple types of clients like Desktop, Mobile apps, etc becomes tricky. The bac...

Best practices that make microservices integration easy

938 views 52 likes 2022-06-27

Running microservices in isolation does not make any sense. To get something done, multiple microservices need to talk t...

Things to remember while building Microservices

938 views 41 likes 2022-06-20

An engineer working on Microservices should not only just focus on engineering; there are so many other aspects to look ...

Why should we have a standard way of building Microservices?

701 views 29 likes 2022-06-17

We all love creating microservices, but what if every team creates its own microservice uniquely and uses its own conven...

Be a better engineer

A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.

System Design Masterclass

A masterclass that helps you become great at designing scalable, fault-tolerant, and highly available systems.

800+ learners

Details →

Designing Microservices

A free playlist to help you understand Microservices and their high-level patterns in depth.

17+ learners

Details →

GitHub Outage Dissections

A free playlist to help you learn core engineering from outages that happened at GitHub.

67+ learners

Details →

Hash Table Internals

A free playlist to help you understand the internal workings and construction of Hash Tables.

25+ learners

Details →

BitTorrent Internals

A free playlist to help you understand the algorithms and strategies that power P2P networks and BitTorrent.

42+ learners

Details →

Topics I talk about

Being a passionate engineer, I love to talk about a wide range of topics, but these are my personal favourites.

Arpit's Newsletter read by 17000+ engineers

🔥 Thrice a week, in your inbox, an essay about system design, distributed systems, microservices, programming languages internals, or a deep dive on some super-clever algorithm, or just a few tips on building highly scalable distributed systems.

  • v12.7.8
  • © Arpit Bhayani, 2022

Powered by this tech stack.