Interestingly, an out-of-memory error is not the worst-case failure, but often the best-case failure… let me explain…
When a process calls malloc, Linux does not necessarily hand over physical memory immediately. It gives the process virtual address space and promises to back those pages when they are actually touched.
This is classic virtual memory.
That promise is part of the kernel’s memory commitment model, and Linux can even overcommit memory, which means the total memory promised to processes can be larger than the physical memory available.
In most cases, processes do not use everything they asked for. But when they do, the kernel has to reclaim memory somehow. If it cannot satisfy an allocation, the OOM killer can step in and kill one or more processes, often based on its estimate of which process is the best candidate to sacrifice.
This is where PostgreSQL gets unlucky.
PostgreSQL uses multiple processes, with important shared memory regions used for things like shared buffers, WAL buffers, and lock-related state. If a backend process is killed unexpectedly, PostgreSQL cannot always establish that its shared state is still safe.
The postmaster takes the conservative approach. It terminates the remaining backends, drops existing connections, and restarts the database. PostgreSQL then performs crash recovery using WAL to bring the database back to a consistent state.
So a memory problem in one process can turn into a database-wide recovery event. One way to make this more predictable on Linux is:
vm.overcommit_memory = 2
This enables strict overcommit accounting. The kernel can reject allocations that would exceed the commit limit instead of allowing the system to reach a state where it has to invoke the OOM killer.
Now PostgreSQL gets an allocation failure instead.
The allocation fails, the statement errors, and the transaction can roll back without taking down unrelated sessions.
Databases are fascinating. Hope this helps.