For the last few weeks, I have been spending time implementing WAL for DiceDB and hence spent a ton of time exploring how other databases have implemented it. I stumbled upon selective logging, and here are some interesting details about it.
WAL is all about safeguarding data integrity and recoverability by ensuring changes are logged before they are applied. But should we log every single operation? probably not.
Although appending to a file is fast, it is still an overhead. So, most databases do not log all the operations, instead, they pick and choose critical operations and log them. This is called selective logging and it helps to tune and ensure that vital changes are recoverable while still
- maximizing the performance
- minimizing the logging overhead
PostgreSQL does not WAL following the following two things - unlogged tables and bulk operations.
PostgreSQL has a notion of an unlogged table where the data changes are not logged. Unlogged tables offer three key benefits
- massive improvements to write performance
- minimal impact on vacuum
- minimal load on WAL, leading to smaller backups
The only disadvantage of unlogged tables is - No durability! So, if the system crashes, the data in these tables vanishes since there’s no log to fall back on. You can create an unlogged table like this
CREATE UNLOGGED TABLE users (id int);By the way, PostgreSQL also does not log massive data imports where logging each row would be overkill. Digging a level deeper is always fun :)