How Dropbox efficiently serves and renders a large number of thumbnails



1586 views Backend System Design



Dropbox has to render a lot of thumbnails when we browse a folder containing a bunch of images. Here’s how they serve them efficiently at scale.

Simple approach

Thumbnail is created for each image present in a Dropbox folder and uploaded to a blob store like s3 or hdfs. Each thumbnail thus has a path using which it can be fetched on the client and shown to the end user.

A simple approach to serve thumbnails is to send thumbnail URLs in an API response to the client and as the user scrolls through them, we keep fetching the images and render them.

This sounds good in theory but gives a very sluggish experience in practice; because of the browser limitation. Every browser caps the maximum number of concurrent connections it can make to a domain.

This limit is 6 for chrome and 8 for firefox, which means at max the browser will be able to fetch 6 images at a time (at best). When there are many photos to render, the experience becomes sluggish as the user will need to scroll and wait for the photos to load.

Dropbox’s Solution

Note: This is HTTP/1.1 based solution; you can do a lot more if you are using HTTP/2

Batching Requests

Expose an endpoint (GET) that accepts multiple thumbnail paths as query parameters.

GET https://photos.dropbox.com/tbatch?paths=1.png,2.png

The server upon receiving this request goes to the blob store and grabs the thumbnails. It then converts the thumbnails in Base64 and puts them in a text response.

The server then sends this response, containing base64 encoded image data of multiple images, back to the client. The client then identifies the image and its base64 data and renders it.

Chunked Transfer Encoding

The server need not wait to get image data of all the requested images before it can send the response; instead, it uses chunked transfer encoding to stream a partial response to the client.

The server gets the request and it initiates the fetching of thumbnails in parallel. As and when it receives image data, it creates a chunked response and sends it to the client.

A chunked response is a text file that looks like this

0: data:image/jpeg;base64, <image data>
2: data:image/jpeg;base64, <image data>
11: data:image/png;base64, <image data>

Once it has completed sending the response for all the images, it sends the terminating response informing the client that the entire response is sent, and closes the connection.


Arpit Bhayani

Arpit's Newsletter

CS newsletter for the curious engineers

❤️ by 21000+ readers

If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter.




Other essays that you might like


Overview of Discord's data platform that daily processes petabytes of data and trillion points

924 views 54 likes 2022-11-14

When a company scales, they adopt microservices and each service typically gets its own independent database. With data ...

How Airbnb designed and scaled its central authorization system - Himeji

2206 views 98 likes 2022-11-07

Authorization plays a critical role in ensuring that the platform is not abused. For example, Instagram ensures that if ...

How Gojek masks and keeps users' phone numbers secure at scale?

2572 views 152 likes 2022-10-31

Do hyperlocal companies like Uber, Ola, Swiggy, Gojek, Zomato, etc share our phone numbers with the delivery people or t...

The architecture of Yelp's in-house Search Engine - nrtSearch

2193 views 81 likes 2022-10-24

Elasticsearch is a great search engine, but Yelp was not happy with its performance, so they built their own HTTP layer ...


Be a better engineer

A set of courses designed to make you a better engineer and excel at your career; no-fluff, pure engineering.


Paid Courses

System Design Masterclass

A masterclass that helps you become great at designing scalable, fault-tolerant, and highly available systems.

1000+ learners

Details →

Redis Internals

Learn internals of Redis by re-implementing some of the core features in Golang.

28+ learners

Details →

Free Courses

Designing Microservices

A free playlist to help you understand Microservices and their high-level patterns in depth.

17+ learners

Details →

GitHub Outage Dissections

A free playlist to help you learn core engineering from outages that happened at GitHub.

67+ learners

Details →

Hash Table Internals

A free playlist to help you understand the internal workings and construction of Hash Tables.

25+ learners

Details →

BitTorrent Internals

A free playlist to help you understand the algorithms and strategies that power P2P networks and BitTorrent.

42+ learners

Details →

Topics I talk about

Being a passionate engineer, I love to talk about a wide range of topics, but these are my personal favourites.





  • v13.7.5
  • © Arpit Bhayani, 2022

Powered by this tech stack.