Idempotency is a must-have for reliable AI agent operations

Arpit Bhayani

Arpit Bhayani

Jun 20, 2026 • 2 min read


AI agents will retry. They will always retry. We have to be ready for it. Given how long-running agentic loops are, network drops, timeouts, and rate limits kicking in are super common.

When that happens, the agent often does not know if the original tool call succeeded. So it calls again. Thus, if that tool charges a card, sends an email, or creates a record, a retry means it happens twice.

The fix is to make every mutating tool call idempotent. The second run of the tool should be a no-op that returns the original result. The standard way to do this is an idempotency key.

The caller generates a unique key per logical operation (a UUID tied to the user’s intent, not the HTTP request). We store the result keyed by that value. If a request with the same key arrives again, the server simply returns the stored response.

Idempotency key generation logic completely depends on what task you are doing, but a few patterns cover most cases.

In some cases, the client generates a stable operation ID. In others, a deterministic key can be derived from the business operation being performed.

For example, in a one-shot user action (place an order, send a payment), hash the meaningful inputs together: user ID, action type, and the request payload.

A common mistake is generating the key from something that changes on every attempt, like a timestamp or a fresh UUID with no link back to the original call. That defeats the purpose. The key has to be deterministic from the standpoint of “what is the agent trying to accomplish,” not “when did this packet leave the machine.”

The key has to live above the retry loop, not inside it. Generate it once, before the first attempt, and pass that same key into every retry of that operation. The network can fail ten times. The key does not change.

Idempotency is not a nice-to-have for agents; it is a must-have when it comes to building reliable agents.

Arpit Bhayani

Principal Engineer II at Razorpay - building Agent Studio, Ex-staff engg at GCP Memorystore & Dataproc, Creator of DiceDB, ex-Amazon Fast Data, ex-Director of Engg. SRE and Data Engineering at Unacademy. I spark engineering curiosity through my no-fluff engineering videos on YouTube and my courses