Yes, when the active node goes down, the passive (backup) node takes over - but how exactly? How are they configured? BTW, this is where keepalived comes in. Let’s dig deeper into the nuances.
Keepalived is used to create a virtual IP. This IP automatically follows your healthy instances, and clients use this virtual IP to connect to the healthy instance.
Keepalived implements the VRRP protocol, where multiple instances share a virtual IP address (their actual IP addresses would be different). The instance with the highest priority becomes the master and owns the VIP, while the others stay as backups (passive).
After configuring keepalived, the master node sends heartbeat advertisements every second to the multicast IP, and backup nodes listen for these packets.
When the master fails, the backup detects silence after ~3 seconds and promotes itself, moving the VIP instantly.
When a failover occurs, the new master announces that it now owns the virtual IP, updating all switch MAC tables immediately. Clients connecting to the VIP seamlessly reach whichever instance is currently healthy.
Thus, your load balancer or clients always connect to the same virtual IP regardless of which physical instance is serving - no DNS changes, connection resets, or manual intervention during failures.
So, three important keywords here - virtual IP, VRRP, and keepalived. Use your favorite LLM to dig deeper into the configuration and actually see seamless failover in action.
By the way, I cover this in depth in my cohort with a live demonstration of failover and auto-healing. Enrollments for the April cohort are open, in case you are interested in learning system design the right way. We go really in-depth into systems and form the right intuition to build, understand, and implement them.