Writes in CockroachDB are fascinating as it directly guarantees a Serializable isolation level. Let’s understand it in 6 easy steps ⚡
- Writes or Updates are initiated for a particular Row. A row is part of a range that is replicated across the cluster with some replication factor.
ex: Say, a row with ID 1 is part of range R1; and this range is replicated on nodes A, B, and C.
- For a particular range, one of the 3 nodes (where it is replicated) acts as the leader (leaseholder). We can think of it as the primary replica (leader) that will handle all the writes for that range.
The leader (leaseholder) registers the operation on its Raft Log (WAL) and proposes the write to the replicas.
ex: if A is the leader for the range R1, the write goes to A
- A appends it to its Raft log
- proposes the write to B and C
- The replicas write this operation in their Raft log and acknowledge the leader.
ex: B and C will register this operation on their own Raft Log (WAL) and respond to A acknowledging the success.
- Leader (leaseholder) now waits for the majority (quorum) before proceeding.
ex: A will wait for acknowledgment from B and C before proceeding
- Once the leader gets the ack from the majority, it writes the data to actual data storage (Pebble) and tells other replicas to commit to its storage.
ex: A will now write the update to its persistent data storage (Pebble database) and will tell B and C to commit.
- Any incoming reads always go to the leader (leaseholder) and because it is the singular authority for the range, it serves the freshly committed and consistent data.
⚡ How is it serializable isolation?
The leader (leaseholder) not only serves and writes but also reads for the row/range it is responsible for. Hence this singular node determines the total ordering of incoming updates and provides a consistent view across and thus serializable isolation.
Note: In cockroachDB, it is possible that the leader and leaseholder are two separate machines, but the algorithm tries to allocate them to the same node for one range replica.
⚡ This is how writes happen on CockroachDB. Hope you found it amusing.
This is the 6th post on CockroachDB internals. Recently, I started diving deep into CockroachDB internals and will share my learning in public. So, if you find it amusing, follow along.
Subscribe: youtube.com/c/ArpitBhayani