Handling timeouts in a microservice architecture



2244 views Designing μ-services



Microservices give us the flexibility to pick the best tech stack to solve the problem optimally. But one thing that ruins the real thrill is Timeouts.

Say we have a blogging website where a user can search for blogs. The request comes to the Search service, and it finds the most relevant blogs for the query.

In the response, a field called total_views should hold the total number of views the blog received in its lifetime. The search services should talk to the Analytics service synchronously to get the data. This synchronous dependency is the root of all evil.

The core problem: Delays can be arbitrarily large

Because the delay depending on service can be arbitrarily large, we know how long to wait for the response. We for sure cannot wait forever, and hence we introduce Timeout. Every time the Search service invokes the Analytics service, it starts a timer, and if it does not get a response in the stipulated time, it timeout and moves on.

There are 5 approaches to handling timeouts.

  • Approach 1: Ignore the timeout and move on
  • Approach 2: Use some default value if you timed out
  • Approach 3: Retry the request
  • Approach 4: Retry only when needed
  • Approach 5: Re-architect and make synchronous dependency an async one

Arpit Bhayani

Arpit's Newsletter

CS newsletter for the curious engineers

❤️ by 14000+ readers

If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter.




Other videos that you might like


Introduction to RPC - Remote Procedure Calls

1531 views 97 likes 2022-05-13

One of the most interesting things that revived itself after a decade is Remote Procedure Calls, fondly called as RPCs; ...

Designing Workflows in Microservices - Orchestration vs Choreography

1659 views 111 likes 2022-05-11

In a microservices architecture there will always arise a need to design workflows; for example: when on an e-commerce w...

Synchronous and Asynchronous Communication between Microservices

1646 views 106 likes 2022-05-04

How should two microservices talk to each other? Picking the right communication pattern is super-important as a good de...

Should some microservices share a database?

1814 views 97 likes 2022-04-27

Microservices need to communicate with each other. Communication between them is always about getting or updating data t...


Arpit's Newsletter read by 14000+ engineers

🔥 Thrice a week, in your inbox, an essay about system design, distributed systems, microservices, programming languages internals, or a deep dive on some super-clever algorithm, or just a few tips on building highly scalable distributed systems.



  • v10.6.4
  • © Arpit Bhayani, 2022