How to Pick a Garbage Collector: The Seven Core Trade-Offs

The Myth of the “Best” Garbage Collector

In systems engineering, there is no universally superior garbage collector (GC). A classic comparative study in the early 2000s demonstrated that virtually every garbage collection algorithm outperformed others by at least 15% in at least one specific workload or dimension.

A collector optimized for maximum batch processing throughput will degrade interactive user latency. Conversely, a collector optimized for sub-millisecond pause times sacrifices CPU throughput and introduces additional space overhead. Choosing or tuning a garbage collector requires plotting a multi-dimensional trade-off curve across your application’s requirements.

radar-chart
  title Garbage Collector Trade-Off Dimensions
  Safety : 5
  Throughput : 4
  Completeness : 4
  Pause Time : 2
  Space Overhead : 3
  Language Optimizations : 3
  Scalability : 4

To objectively evaluate a garbage collector—whether designing an engine runtime or tuning production runtimes like the JVM, Go runtime, or V8—engineers must analyze seven core characteristics.

---

The 7 Core Metrics of a Garbage Collector

1. Safety

Definition: A collector must never reclaim the memory of an object that is still live (reachable).

flowchart LR
    subgraph Mutator Context
        P1[Pointer 1]
        P2[Pointer 2]
    end
    
    subgraph Heap Memory
        LiveObj[Live Heap Object]
        DeadObj[Unreferenced Object]
    end
    
    P1 --> LiveObj
    P2 --> LiveObj
    
    DeadObj -.->|Reclaimed Safe| FreeList[Free Memory Pool]
    LiveObj -.->|Reclaiming Causes Dangling Pointers| Fault[Fatal Memory Corruption]

• If an object is referenced by active execution contexts (stack frames, global registers, or enclosing heap objects), reclaiming its memory produces dangling pointers.
• Dereferencing a dangling pointer leads to memory corruption, segmentation faults, or silent security vulnerabilities (use-after-free).
• While safety is treated as a non-negotiable invariant in managed runtimes, conservative collectors (often used in unmanaged runtimes like C/C++) can sometimes misidentify integer values as pointers or vice-versa, navigating delicate boundaries between conservative retention and premature freeing.

2. Throughput

Definition: The ratio of time the CPU spends executing business logic (mutator threads) versus executing garbage collection routines.

$\text{Throughput} = \frac{T_{\text{mutator}}}{T_{\text{mutator}} + T_{\text{GC}}}$

If an application allocates memory aggressively, the GC must run frequently to reclaim space. When GC cycles consume significant CPU cycles, the available computing budget for processing business transactions drops:

• Batch Systems: Care almost exclusively about total execution time and throughput. Running a heavy, multi-threaded GC sweep every few minutes is acceptable if total CPU efficiency remains near 95–99%.
• Burst Strategy: To balance throughput, many collectors interleave short, frequent collection phases (e.g., minor generational collections) with infrequent, expensive compaction sweeps. This provides adequate mutator compute cycles while bounding heap exhaustion.

3. Completeness

Definition: Every unreachable object must eventually be identified and reclaimed.

• A collector does not need to reclaim every single dead object in the immediate cycle during which it becomes unreachable.
• However, it must guarantee eventual reclamation.
• If an algorithm systematically fails to detect cyclic references (as pure reference counting does without a cycle detector) or prematurely promotes unreferenced objects into generational survivor regions indefinitely, the consequence is a persistent memory leak that eventually triggers an OutOfMemoryError.

4. Pause Time (Stop-the-World Latency)

Definition: The duration during which application mutator threads are suspended while the GC inspects or reorganizes heap memory.

sequenceDiagram
    autonumber
    participant Mutator as Mutator Threads (App)
    participant GC as GC Threads

    Mutator->>Mutator: Executing Business Logic
    Note over Mutator,GC: Allocation trigger or threshold reached
    GC->>Mutator: Signal Safepoint (Stop-The-World)
    Mutator-->>GC: Mutators Suspended
    GC->>GC: Scan Roots, Trace Graph, Compact Memory
    GC->>Mutator: Resume Mutators
    Mutator->>Mutator: Execution Resumes

Pause time is often the most critical operational metric for customer-facing systems (e.g., payment gateways, financial order routing, search engines):

Why Stop the World?

During memory reclamation, the heap can become fragmented. Allocating new objects requires contiguous free memory blocks. If the heap consists of scattered, isolated bytes, large allocations will fail even if total aggregate free memory is high.

To resolve this, the collector performs defragmentation (compaction):

1. Live objects are shifted to adjacent locations in memory.
2. Shifting an object changes its physical memory address.
3. If application threads modify pointers while objects are migrating, pointers will point to stale, invalid memory addresses.
4. Therefore, runtimes pause mutator threads, rewrite references to their new memory coordinates, and resume execution.

Minimizing pause times requires concurrent or incremental algorithms (such as read/write barriers and concurrent copying), which shift the trade-off toward lower throughput and higher CPU overhead.

5. Space Overhead

Definition: The auxiliary memory consumed by the garbage collector itself to manage metadata.

To track liveness, generation states, and pointers, collectors introduce metadata structures:

• Bitmaps and Mark Tables: Bit arrays representing whether an object address is marked, swept, or candidate for promotion.
• Card Tables and Remembered Sets: Data structures tracking inter-generational references (e.g., an object in an old generation pointing to an object in a young generation).
• Object Headers: Extra bytes prepended to every heap allocation to record GC age, mark bits, and lock states.

In environments with severe memory constraints (embedded hardware, edge microservices, memory-dense multi-tenant containers), a GC requiring 10–20% heap overhead for metadata reduces the memory budget available for business state.

6. Language-Specific Optimizations

Definition: The extent to which a collector exploits the semantic guarantees and memory layout patterns of a specific programming language.

A generic, language-agnostic collector misses critical optimization opportunities inherent to specific runtime models:

• Pure Functional Languages: Immutability guarantees that existing objects never point to newly created objects. Pointers only travel forward in time, simplifying generational assumptions.
• Persistent Data Structures: When data structures create structural copies upon write, short-lived nodes can be reclaimed using localized, deterministic collection strategies.
• Memory Layout Alignment: If a runtime guarantees that all objects conform to predictable memory arenas or strict stack/heap boundary rules, collection routines can traverse or free blocks in constant $O(1)$ time rather than traversing recursive object graphs.

7. Scalability

Definition: The ability of the garbage collection system to maintain performance as hardware scales across heap sizes and CPU core counts.

A garbage collection strategy designed in the 1990s for a machine with 64 MB of RAM and a single CPU core collapses when executed on modern machines with 512 GB of RAM and 128 cores:

• Parallelism: Can the GC distribute marking, tracing, and sweeping phases across multiple cores without lock contention?
• Heap Sizing: If a Stop-the-World collector must scan a 500 GB heap linearly, pause times escalate from milliseconds to tens of seconds or minutes, rendering services unresponsive.
• Concurrent / Low-Pause Collectors: Modern scalable runtimes utilize concurrent phases (marking and relocating while mutator threads continue running) to keep pauses bounded under 1ms, regardless of whether the heap is 8 GB or 8 TB.

---

Practical Runtime Tuning Knobs

Operating systems and runtime environments provide tunable configuration flags to balance these seven dimensions. In the Java Virtual Machine (JVM), common knobs illustrate these direct engineering trade-offs:

Configuration Flag	Primary Target	Trade-off Made
-XX:+UseSerialGC	Minimal footprint, simple execution	Single-threaded GC; stalls mutator threads entirely during collection.
-XX:+UseParallelGC	High mutator throughput	Multi-threaded collection optimizing throughput at the expense of longer individual STW pauses.
-XX:MaxGCPauseMillis=<N>	Low pause latency	Collector works in shorter, more frequent bursts; may reduce overall application throughput.
-XX:GCTimeRatio=<N>	Mutator CPU allocation balance	Sets target ratio of application time to GC time ($1 / (1 + N)$).
-XX:MinHeapFreeRatio=<N>	Heap expansion behavior	Influences memory reclamation aggression vs. operating system page releases.

---

Summary Decision Matrix

When evaluating or designing a garbage collector, map requirements across these operational profiles:

flowchart TD
    Start[Analyze Workload Profile] --> Q1{Is ultra-low latency required?}
    Q1 -- Yes --> A1[Prioritize Low Pause Time<br/>Trade-off: Lower overall throughput, higher space overhead]
    Q1 -- No --> Q2{Is hardware memory tightly constrained?}
    Q2 -- Yes --> A2[Prioritize Low Space Overhead<br/>Trade-off: Longer pause times, less sophisticated tracking]
    Q2 -- No --> Q3{Is peak batch compute efficiency key?}
    Q3 -- Yes --> A3[Prioritize Maximum Throughput<br/>Trade-off: Large, infrequent Stop-The-World compaction pauses]
    Q3 -- No --> A4[Prioritize Scalability and Hardware Utilization<br/>Multi-core concurrent collectors for large heaps]

Key Takeaways

1. No Absolute Winner: Every garbage collector optimizes for a subset of the seven metrics at the direct expense of others.
2. Pause Time vs. Throughput: Compaction requires pausing mutators to maintain pointer safety unless complex concurrent relocation barriers are introduced, which consume CPU cycles.
3. Hardware Context Matters: Algorithms must match the underlying hardware architecture; an algorithm optimized for small memory footprints will degrade severely when applied to massive multi-gigabyte modern heaps.