Designing Idempotent API Endpoints for Payments at Stripe



6253 views Backend System Design



In the world of payments, reliability and consistency are crucial. Customers expect their transactions to be processed accurately and efficiently, without any errors or duplicates. However, in a distributed system, failures can occur, and the same request can be sent multiple times.

For example, consider a payment service API that transfers money from one user to another. No matter where the failure occurs, the system needs to ensure that one request is processed exactly once; leading us to build idempotent APIs.

Idempotent APIs

An idempotent API is an API that can be called multiple times, but the result will always be the same. If a request is made twice or more, the API will only perform the operation once, ensuring that there are no duplicates or errors.

However, designing an idempotent API is not a trivial task. There are many things to consider, such as handling errors, passing idempotency keys, and maintaining consistency.

Core idea implementation

The core idea behind item potency keys is straightforward. Before making an API call, the client talks to the server to generate a random ID, which serves as the idempotency key.

The client then passes this key in all future requests to the server. The server stores the key and the purpose of the request in a backend database.

When the server receives a request, it extracts the idempotency key and checks if it has already processed the request. If it has, the server ignores the request. If it hasn’t, the server processes the request and deletes the idempotency key.

This gives us the much-needed exactly-once processing.

Handling Errors

When designing an idempotent API, we need to consider what to do in case of failure. Depending on the application usecase and the transaction state, we can choose to

  • ignore the failure,
  • pass the error to the user, or
  • retry the request.

Retrying the request is often preferred because it provides a better user experience. However, we need to be careful when retrying requests, as we could end up processing the same payment multiple times.

Passing the keys

There are several ways to pass the idempotency key to the server, such as using request headers or query parameters. Stripe, a popular payment processing company, requires clients to pass idempotency keys in request headers named Idempotency-Key.

Database decisions

It is recommended to use a separate database or table to store idempotency keys to ensure that the server can quickly validate the processing status and load isolation.

Conclusion

In conclusion, designing an idempotent API for payments is crucial for ensuring reliability and consistency. By using idempotency keys, we can avoid processing the same payment multiple times.

This approach can significantly improve the reliability and user experience of your API, making it a must-have for any critical API that needs to be executed exactly once, no matter what.


Arpit Bhayani

Arpit's Newsletter

CS newsletter for the curious engineers

❤️ by 38000+ readers

If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter.




Other essays that you might like


Thundering Herd Problem and How not to do API retries

6281 views 270 likes 2023-01-31

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

Designing Idempotent API Endpoints for Payments at Stripe

6253 views 326 likes 2023-01-29

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How Slack efficiently classifies emails at scale with an eventually consistent system

3085 views 103 likes 2023-01-13

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...

How @ShopifyEngineering avoids hot shards by moving data across databases without any downtime

4617 views 170 likes 2023-01-08

System Design for Experienced Engineers: https://arpitbhayani.me/masterclass Become a member for exclusive in-depth vide...


Be a better engineer

A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.


Paid Courses

System Design for Beginners

A masterclass that helps early engineers and product managers become great at designing scalable systems.

180+ learners

Details →

System Design Masterclass

A masterclass that helps you become great at designing scalable, fault-tolerant, and highly available systems.

1000+ learners

Details →

Redis Internals

Learn internals of Redis by re-implementing some of the core features in Golang.

98+ learners

Details →

Free Courses

Designing Microservices

A free playlist to help you understand Microservices and their high-level patterns in depth.

823+ learners

Details →

GitHub Outage Dissections

A free playlist to help you learn core engineering from outages that happened at GitHub.

651+ learners

Details →

Hash Table Internals

A free playlist to help you understand the internal workings and construction of Hash Tables.

1027+ learners

Details →

BitTorrent Internals

A free playlist to help you understand the algorithms and strategies that power P2P networks and BitTorrent.

692+ learners

Details →