One of the best mental models around for building reliable systems is to be a pessimist. Not in the “nothing will ever work” sense, but in the structured sense of asking hard questions before something goes wrong.
What could break here? What could break without us ever realizing it? What happens to the system if this dependency goes down? What is error-prone in this flow, and are we just getting lucky right now?
Most engineers think about the happy path and are done with the work. But it is important to think about edges and even the edges of the edges. Yes, the depth of your edge case handling is always a function of timeline and ROI on the effort.
By the way, we are not being pessimistic for its own sake. Here, pessimism is a forcing function. When you sit down and genuinely ask, “What could go wrong?”
- you write readable code
- you handle errors better
- you add the required alerting
- you design better fallbacks
- you don’t assume - DB will remain up
- you don’t assume - 3P API is available
- you don’t assume that your own code will always behave as expected under load
I would say: ship something reliable because, at the end of the day, you are accountable for the code you ship. If it breaks tomorrow, you will be asked questions. Plus, you should be proud of what you are shipping.
Remember, reliability is the output of asking uncomfortable questions early.