Arpit's Newsletter read by 38000+ engineers
Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.
In the world of payments, reliability and consistency are crucial. Customers expect their transactions to be processed accurately and efficiently, without any errors or duplicates. However, in a distributed system, failures can occur, and the same request can be sent multiple times.
For example, consider a payment service API that transfers money from one user to another. No matter where the failure occurs, the system needs to ensure that one request is processed exactly once; leading us to build idempotent APIs.
An idempotent API is an API that can be called multiple times, but the result will always be the same. If a request is made twice or more, the API will only perform the operation once, ensuring that there are no duplicates or errors.
However, designing an idempotent API is not a trivial task. There are many things to consider, such as handling errors, passing idempotency keys, and maintaining consistency.
The core idea behind item potency keys is straightforward. Before making an API call, the client talks to the server to generate a random ID, which serves as the idempotency key.
The client then passes this key in all future requests to the server. The server stores the key and the purpose of the request in a backend database.
When the server receives a request, it extracts the idempotency key and checks if it has already processed the request. If it has, the server ignores the request. If it hasn’t, the server processes the request and deletes the idempotency key.
This gives us the much-needed exactly-once processing.
When designing an idempotent API, we need to consider what to do in case of failure. Depending on the application usecase and the transaction state, we can choose to
Retrying the request is often preferred because it provides a better user experience. However, we need to be careful when retrying requests, as we could end up processing the same payment multiple times.
There are several ways to pass the idempotency key to the server, such as using request headers or query parameters. Stripe, a popular payment processing company, requires clients to pass idempotency keys in request headers named Idempotency-Key
.
It is recommended to use a separate database or table to store idempotency keys to ensure that the server can quickly validate the processing status and load isolation.
In conclusion, designing an idempotent API for payments is crucial for ensuring reliability and consistency. By using idempotency keys, we can avoid processing the same payment multiple times.
This approach can significantly improve the reliability and user experience of your API, making it a must-have for any critical API that needs to be executed exactly once, no matter what.
Here's the video ⤵
Super practical courses, with a no-nonsense approach, are designed to spark engineering curiosity and help you ace your career.
An in-depth, self-paced, and on-demand course that for early engineers to become great at designing scalable, available, and extensible systems at scale.
A masterclass that helps experienced engineers become great at designing scalable, fault-tolerant, and highly available systems.
A course that helps covers Redis internals by reimplementing its core features like - event loop, serialization protocol, pipelining, eviction, and transactions.
Arpit's Newsletter read by 38000+ engineers
Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.