System design is not important anymore - if you believe this, just wait until your 2-hour agent run fails at step 95 of 100 :) I hope it does not happen, but it actually might.
Agentic apps have two classes: short-running and long-running. When you are building short agents, a simple retry loop is enough. Each LLM call is isolated, cheap, and stateless. Fail, retry, done.
But retrying a long-running agent spanning hours is expensive in API costs, tool calls, and wall clock time, and retries can also have side effects like resending notifications. Some failures that a long-running agent might hit in production are:
- LLM provider rate limiting
- Network timeouts for downstream calls
- Pod eviction due to OOM kills mid-run
- Pod rotation during deployment
- Context window overflow
- Human-in-the-loop pauses for days
Notice how most of these are not AI problems. They are classic software engineering problems that are showing up in a new system.
This is where we need checkpointing, idempotency, durable event logs, and resumable workflows. These are backend engineering fundamentals where we built robust, fault-tolerant systems by applying hundreds of patterns.
Good software engineering practices do not go out of fashion. Because someday, you will ship your “tutorial” to production. That is when system design suddenly matters again.
Fundamentals remain fundamentals, even when the abstraction layer changes.