Understanding the Sidecar Pattern: Atlassian’s 70% Latency Reduction with TCS

The sidecar pattern is a powerful architectural approach that can significantly optimize distributed systems. While often discussed, its practical application and benefits are best understood through real-world examples. This article dissects Atlassian’s implementation of the sidecar pattern to reduce latency by 70% for their critical Tenant Contact Service (TCS), highlighting when and why to adopt this pattern.

The Criticality of Atlassian’s Tenant Contact Service (TCS)

Atlassian’s Tenant Contact Service (TCS) is a highly critical internal service. It provides essential tenant metadata, determining which tenant a request belongs to. This service is:

• High Throughput, Low Latency: Typically responds within 5-6 milliseconds or less.
• Highly Available: Boasts four to five nines of availability.
• Frequently Called: Invoked multiple times during every user request across Atlassian products like Confluence, Jira, and Bitbucket.
• System-Critical: If TCS is down, core Atlassian products are effectively down.

The Problem: Inconsistent Client Performance

Despite TCS’s robust performance (P99 latency in single-digit milliseconds, average even lower), one specific internal team consistently reported high latencies and failure rates when interacting with TCS. This discrepancy led to an investigation.

The root cause was identified not within the TCS service itself, but in the client implementation written by the consuming team. Their client code:

• Lacked best practices.
• Made numerous sequential calls that should have been parallelized.
• Was generally inefficient, leading to observed latencies significantly higher than the actual TCS service performance.

The Solution Dilemma: Solve for One or Solve for All?

The TCS team faced a choice:

1. Solve for One: Educate the specific team on best practices and ask them to refactor their client code.
2. Solve for All: Develop a horizontal solution that inherently enforces best practices for all consumers, preventing similar issues with other teams in the future.

The TCS team opted for the “solve for all” approach, recognizing that relying on every internal team to consistently implement best practices for client-side interactions was unsustainable and prone to future problems. This led to the adoption of the sidecar pattern.

Introducing the Sidecar Pattern

A sidecar is a separate process that runs alongside the main application process, typically within the same execution environment (e.g., the same EC2 instance, Kubernetes pod). Its primary role is to augment or extend the functionality of the main application without being tightly coupled to it.

Atlassian’s Sidecar Implementation for TCS

In Atlassian’s case, the sidecar works as follows:

1. Deployment: A dedicated TCS sidecar process is deployed alongside each microservice instance that needs to interact with the TCS service.
2. Communication:
   • The consuming microservice (e.g., Bitbucket, Jira) makes a simple, local HTTP call to its co-located TCS sidecar. This is not a network call across machines, but an inter-process communication on the same host.
   • The TCS sidecar then handles all interactions with the actual remote TCS service.
3. Responsibility: The TCS team owns and maintains the sidecar. This allows them to bake in all the necessary best practices for interacting with TCS, including:
   • Efficient parallelization of calls.
   • Robust retry mechanisms.
   • Error handling.
   • Connection pooling and other optimizations.

Sidecar vs. Library: Why Sidecar Was Chosen

An alternative to the sidecar pattern would have been for the TCS team to provide a client library. However, the sidecar offered distinct advantages:

Feature	Sidecar	Library
Language Agnostic	Communicates via local HTTP, so the client microservice can be written in any language.	Requires a separate library implementation for each programming language (Go, Java, Rust, etc.) used by clients.
Ownership & Updates	Owned and updated by the TCS team; updates are deployed centrally with the sidecar.	Client teams are responsible for integrating and updating the library in their own services.
Best Practices	Best practices are enforced and encapsulated within the sidecar, transparent to the client.	Client teams must explicitly use the library correctly and follow best practices.
Deployment	Deployed as a separate process alongside the application.	Integrated directly into the application’s codebase.

The language agnosticism and centralized management of best practices made the sidecar a superior choice for Atlassian’s diverse internal ecosystem.

Impact and Benefits of the Sidecar Pattern

Implementing the sidecar pattern yielded significant benefits for Atlassian:

1. Reduced Latency for Clients: The team initially struggling with high latencies saw a dramatic improvement, with the sidecar enforcing efficient communication patterns.
2. Overall System Optimization: Even teams that were already following best practices saw further improvements (e.g., a 0.3 millisecond reduction in P99 latency), demonstrating the sidecar’s ability to optimize beyond individual client efforts.
3. Reduced Load on TCS Service: By ensuring all client interactions were optimized, the total number of requests and the overall load on the central TCS service were significantly reduced. This improved the efficiency and stability of the core service itself.
4. Horizontal Solution: The sidecar provided a universal solution that benefited all internal consumers of TCS, rather than addressing issues on a per-team basis.

Other Use Cases for the Sidecar Pattern

The sidecar pattern is versatile and applicable in various scenarios beyond service-to-service communication optimization:

• Metrics Collection: A sidecar (e.g., Fluentd) can collect logs and metrics from the main application and forward them to a central observability system, offloading this responsibility from the application itself.
• Log Aggregation: Similar to metrics, a sidecar can handle log processing, filtering, and forwarding.
• Configuration Management: A sidecar can dynamically fetch and update configuration for the main application.
• Security Proxies: A sidecar can act as an authentication or authorization proxy, handling security concerns transparently.

Key Takeaways

1. Strategic Optimization: The sidecar pattern, when implemented correctly, can heavily optimize system performance and reliability by centralizing common concerns and best practices.
2. Horizontal Solutions: Prioritize building horizontal solutions that benefit an entire ecosystem rather than custom fixes for individual clients. This approach leads to more robust, maintainable, and efficient systems.