DragonflyDB Architecture: Multi-threading, Shared-Nothing, and High Throughput for 6.43M Ops/Sec

DragonflyDB emerges as a compelling, high-performance drop-in replacement for Redis, boasting significantly higher throughput by optimizing for modern hardware capabilities. This document delves into its core architecture, design decisions, trade-offs, and implementation specifics that enable it to achieve staggering performance numbers.

Unprecedented Performance

DragonflyDB is engineered to extract maximum performance from the underlying hardware. When deployed on a C7 G9 16x large instance, equipped with 64 cores and 128 GB RAM, DragonflyDB can achieve an astounding 6.43 million operations per second. This level of throughput is a direct result of its architectural choices, which fundamentally differ from traditional in-memory data stores like Redis.

Redis’s Single-Threaded Bottleneck

To understand DragonflyDB’s innovation, it’s crucial to first grasp the inherent limitations of Redis:

The Single-Threaded Model

Redis operates as a single-threaded process. This design choice, while simplifying certain aspects of its internal logic, introduces a fundamental limitation:

• Underutilization of Hardware: Even on a powerful machine with 64 CPU cores, Redis can only utilize one core for its primary data processing. The remaining 63 cores remain idle, representing a significant waste of computational resources.
• Throughput Ceiling: Due to its single-threaded nature, Redis quickly reaches a maximum throughput point under load. Beyond this point, it cannot leverage additional hardware capacity to process more requests.

The Need for Horizontal Scaling

To overcome the single-threaded throughput limitation, Redis deployments often resort to horizontal scaling, creating a cluster of Redis instances. While this increases overall capacity, it introduces operational complexities and costs:

• Key Distribution: Managing a Redis cluster requires careful consideration of key distribution strategies to ensure even load balancing and data access.
• Cluster Management: Operating and maintaining a Redis cluster adds significant overhead in terms of infrastructure management and monitoring.
• Increased Infrastructure Cost: Scaling horizontally with multiple Redis instances inherently bloats infrastructure costs.

DragonflyDB’s Multi-Threaded Paradigm

DragonflyDB’s core differentiator is its multi-threaded architecture, designed from the ground up to fully utilize modern multi-core processors.

Fiber-Based Concurrency

Instead of traditional OS threads for every concurrent operation, DragonflyDB employs fibers, which are lightweight, user-managed threads, conceptually similar to Go routines.

Concurrency Model

The concurrency model in DragonflyDB can be visualized as a three-tier hierarchy:

1. Fibers (Lightweight Threads): These are numerous user-managed threads, representing the smallest unit of concurrency.
2. OS Threads: A smaller number of operating system threads. These are responsible for executing fibers.
3. CPU Cores: The underlying hardware resources where OS threads run.

The core idea is that a large number of fibers are multiplexed onto a fewer number of OS threads. These OS threads then compete for CPU time for their execution.

Advantages of Fibers

• Fine-Grained Concurrency: The ability to create a vast number of fibers allows for highly granular units of concurrency, enabling faster progress on many tasks simultaneously.
• Reduced Blocking: Because there are always numerous fibers ready for execution, OS threads are less likely to remain blocked for extended periods, ensuring continuous CPU utilization.

Implementation Details

DragonflyDB is written in C++ and leverages the boost.fibers library to manage user-level concurrency and fiber scheduling.

Shared-Nothing Architecture: Eliminating Contention

One of the most innovative aspects of DragonflyDB is its shared-nothing architecture, which directly addresses the contention issues common in multi-threaded systems.

The Contention Problem

In a typical multi-threaded in-memory key-value store, multiple threads might attempt to access and modify a single, global hash table. This necessitates mutual exclusive access mechanisms (like mutexes or locks) to prevent data corruption. However, frequent locking leads to significant contention, reducing parallelism and overall performance.

DragonflyDB’s Solution: Sharded Key Space

DragonflyDB avoids this contention by sharding its entire key space. Instead of a single hash table, it maintains multiple, mutually exclusive hash tables in memory:

• N Exclusive Key Spaces: The entire key-value data is sharded into N parts, where N is less than or equal to the number of logical CPU cores available on the machine (e.g., N <= 64 for a 64-core system).
• Dedicated Data Threads: Each of these N exclusive key spaces (hash tables) is owned and managed by a dedicated