Rockset Architecture: Distributed Query Execution and Horizontal Autoscaling

Rockset is a cloud-native search and analytics database engineered for low-latency, real-time queries across unstructured, semi-structured, and structured data. Delivering real-time analytical capabilities over high-velocity data requires decoupling compute from storage, intelligently distributing work across nodes, and pipelining query execution.

This architectural breakdown focuses on two critical aspects of Rockset’s internal engine:

1. Query Execution Mechanics: How an incoming SQL query is parsed, converted into a distributed Directed Acyclic Graph (DAG) of operators, and streamed across execution nodes.
2. Horizontal Elasticity: How stateless compute nodes and storage-backed leaf nodes scale out dynamically using RocksDB-Cloud on object storage (AWS S3).

---

1. High-Level System Architecture

Rockset’s query serving tier is split into two primary layers: Aggregator Nodes and Leaf Nodes.

graph TD
    Client([Client / Application]) -->|SQL Query| API[Rockset SQL API Layer]
    API -->|Validated SQL| Aggregator[Aggregator Nodes - Compute Tier]
    
    subgraph Query Execution & Coordination
        Aggregator -->|Operator Plan DAG| Leaf1[Leaf Node 1]
        Aggregator -->|Operator Plan DAG| Leaf2[Leaf Node 2]
        Aggregator -->|Operator Plan DAG| LeafN[Leaf Node N]
    end
    
    subgraph Storage Tier - RocksDB-Cloud
        Leaf1 -.->|Read SSTs| S3[(Shared Object Store - AWS S3)]
        Leaf2 -.->|Read SSTs| S3
        LeafN -.->|Read SSTs| S3
    end
    
    Leaf1 -->|Streamed Tuples| Aggregator
    Leaf2 -->|Streamed Tuples| Aggregator
    LeafN -->|Streamed Tuples| Aggregator
    Aggregator -->|Pipelined Results| API
    API -->|Pipelined Results| Client

• Rockset SQL API Layer: Handles authentication, authorization, query admission control, rate-limiting, and schema/metadata management.
• Aggregators (Compute Tier): Coordinate query planning, optimize execution plans, manage distributed joins, perform global aggregations, and stream final result sets back to the API.
• Leaf Nodes (Storage & Local Compute Tier): Own shards of data, manage inverted/columnar/document indices (Converged Index) via local RocksDB instances, execute local filters/projections/aggregations, and stream intermediate records.

---

2. Sharding Strategy and the Fanout Trade-Off

Document-Based Sharding

Rockset partitions data using document-based sharding. When a document is ingested, its primary identifier (_id) is passed through a consistent hash function modulo the number of active shards, deterministically placing the complete document onto a specific leaf shard:

$\text{Target Leaf} = \text{Hash}(\text{\_id}) \pmod N$

Because all indexed representations of that document (columnar, inverted, and row-oriented key-value structures) reside together on that leaf, writes and localized index updates do not require multi-phase distributed transactions.

The Fanout Query Pattern

Document-based sharding impacts read queries that do not filter strictly on _id:

SELECT year, COUNT(*)
FROM movies
GROUP BY year;

Because documents for any given year can be hashed across any leaf node in the cluster, the query cannot be routed to a single node. Instead, the query must fan out to all leaf nodes.

                 [ Aggregator Node ]
                 /        |        \
           (Fanout)    (Fanout)   (Fanout)
               /          |          \
         [ Leaf 1 ]   [ Leaf 2 ]   [ Leaf N ]

Why Fanout is an Advantage (MPP)

While a fanout pattern is traditionally viewed as expensive in operational transactional databases (OLTP), in an analytical database (OLAP) it enables Massive Parallel Processing (MPP):

• Rockset leverages the aggregated CPU cores, memory bandwidth, and disk I/O across every leaf node concurrently.
• Each leaf node computes local aggregations (COUNT(*) GROUP BY year) over its local subset of data.
• The network only transfers pre-aggregated intermediate results back to the aggregator, minimizing serialization overhead and network bottlenecks.

---

3. End-to-End Query Execution Lifecycle

sequenceDiagram
    autonumber
    participant C as Client
    participant API as SQL API Layer
    participant Agg as Aggregator Node
    participant Leaf as Leaf Nodes

    C->>API: Submit SQL Query
    API->>API: AuthN, AuthZ, Metadata Validation
    API->>Agg: Hand off SQL Query
    Agg->>Agg: Parse AST & Run Cost-Based Optimizer (CBO)
    Agg->>Agg: Construct Distributed Operator DAG
    Agg->>Leaf: Push Down Plan Snippets (Predecessor/Op/Successor)
    Leaf-->>Agg: Establish Persistent TCP Streaming Channels
    Leaf->>Leaf: Scan RocksDB SSTs (Range Lookups / Converged Index)
    Leaf-->>Agg: Continuously Stream Intermediate Tuples
    Agg->>Agg: Merge, Join, & Finalize Global Aggregations
    Agg-->>API: Stream Final Tuples
    API-->>C: Stream Results to Client Response Stream

Step 1: Parsing and Plan Optimization

1. Abstract Syntax Tree (AST): The aggregator parses incoming SQL into an AST.
2. Cost-Based Optimizer (CBO): The engine inspects index statistics, column cardinality, and data distribution to evaluate candidate join orders, index access paths, and predicate pushdown opportunities.
3. Operator DAG Generation: The plan is transformed into a Directed Acyclic Graph (DAG) of discrete operators (scans, filters, hash joins, local group-bys, and merges).

Step 2: Distributed Operator DAG Distribution

The aggregator slices the operator DAG into distinct per-node fragments. Each leaf node receives a self-contained execution recipe containing:

• Predecessor: Which storage index or upstream operator provides the input stream.
• Operation: The exact compute instruction (e.g., scan prefix c.year.*, filter, project, hash-aggregate).
• Successor: The downstream aggregator target address where output tuples must be routed.

Step 3: Pipelined Streaming with Persistent Connections

Traditional analytical engines often use stage-by-stage execution, materializing intermediate results in memory or on disk before passing them to the next phase. Rockset uses a fully pipelined streaming execution model:

• Persistent Connections: When execution kicks off, long-lived multiplexed connections are established across leaf and aggregator layers.
• Iterative Streaming: As a leaf node traverses index keys in RocksDB (e.g., executing a range lookup over an SST file), it yields tuples immediately into the network buffer rather than buffering the entire result set.
• Continuous Aggregation & Early Delivery: Aggregators process incoming tuple streams on the fly and immediately flush result batches to the SQL API and the client.
• Low Time-to-First-Token/Row: Clients can consume initial rows without waiting for full dataset aggregation, reducing perceived latency for interactive dashboards and real-time APIs.

---

4. Elastic Horizontal Autoscaling

Rockset separates compute scaling from storage scaling, allowing independent adjustments to query load and data volume.

[ Autoscaling Compute Tier ]
  Aggregator Pool (Stateless) <---> Scaled via Kubernetes HPA on CPU/Latency

[ Autoscaling Storage Tier ]
  Leaf Nodes (Zero-Copy)      <---> Pull Immutable SSTs from S3 on Demand
                                        |
                                 [ AWS S3 Bucket ]

Stateless Compute Scaling (Aggregator Tier)

Aggregators store no durable state. Their primary resource consumers are AST parsing, query optimization, join processing, and result aggregation.

• Mechanism: Aggregator instances run in containerized environments managed by Kubernetes.
• Autoscaler: Standard Horizontal Pod Autoscalers (HPA) monitor CPU utilization, memory thresholds, and query queue depths to spin up or tear down aggregator pods automatically without data movement.

Stateful Storage Scaling with Zero-Copy Read Replicas (Leaf Tier)

Scaling storage-bound leaf nodes dynamically is difficult in traditional shared-nothing architectures because rebalancing shards requires transferring gigabytes or terabytes of state over the network.

Rockset avoids this via RocksDB-Cloud (an open-source extension of RocksDB designed to decouple local execution from persistent cloud storage):

1. S3 as the Durability Anchor: While local SSDs on leaf nodes act as read caches, all RocksDB SST (Sorted String Table) files are flushed and durably written to cloud object storage (such as AWS S3).
2. Immutable SST Files: Because SST files are immutable, they can be read concurrently by multiple independent nodes without lock contention or write hazards.
3. Zero-Copy Scaling: When query workloads spike or specific shards experience read pressure, Rockset provisions new leaf nodes on demand. Instead of replicating state node-to-node:
   • The new leaf node mounts the metadata catalog.
   • It downloads only the relevant, immutable SST files directly from S3 (or warms its cache lazily).
   • It registers with the aggregators and immediately begins executing leaf scan fragments.
4. Scale-Down and Cost Optimization: When analytical traffic subsides, leaf nodes can be terminated without risking data loss, because the durable primary copy of the data lives in S3.

---

5. Architectural Summary

Architectural Component	Implementation Mechanism	Engineering Trade-off / Benefit
Data Partitioning	Document-based hashing on _id	Requires fanout for non-ID queries, but enables Massive Parallel Processing (MPP) across all cluster CPUs.
Query Planning	Cost-Based Optimizer (CBO) $\to$ Operator DAG	Customizes per-node execution plans with explicit predecessor/successor routing instructions.
Execution Model	Pipelined streaming via persistent network connections	Eliminates intermediate materialization; minimizes time-to-first-row for downstream consumers.
Aggregator Scaling	Kubernetes HPA on stateless compute pods	Fast, zero-overhead scaling based on concurrent query volume.
Storage Scaling	RocksDB-Cloud backed by AWS S3 SST files	Enables zero-copy read replicas; allows the storage tier to scale up for query bursts and down to optimize cost.