Why most TCP servers are multi threaded and how to build one from scratch

Watch the video explanation ➔

TCP is the most reliable method for communication between machines over a network. In this article, we will explore how a web server can handle and serve multiple TCP connections.

To gain a better understanding of this concept, we will build our own server from scratch using raw sockets. Throughout the process, we will delve into system calls, socket programming, and their limitations, and optimize our approach to handle multiple requests concurrently.

Understanding TCP Servers

A TCP server is a regular process that runs on a machine and listens to a specific port, enabling communication via TCP. Various web servers, such as Apache Tomcat, Spring Boot, or Flask development servers, operate by listening to a designated port.

Clients interested in communicating with the server must connect to this port.

Setting up a TCP Server

To begin building our TCP server, we need to create a process that listens to a specific port. Using the Go programming language for this demonstration, we can utilize the net package’s Listen function.

By invoking net.ListenTCP and specifying the desired port (e.g., 1729), we reserve that port for our server.

Accepting Client Connections

The next step involves the accept system call, which is a blocking call. When we call accept on the listener, the program halts until a client establishes a connection. We can use the listener.Accept() function to accept incoming connections, which returns a connection object and an error.

If an error occurs, we handle it accordingly.

Reading and Responding to Requests

Once a client establishes a connection, we need to read the incoming request, perform any necessary processing, generate a response, and close the connection.

We accomplish this by implementing a function, do, which accepts the connection object as a parameter. Within this function, we initiate a read operation to capture the request data and store it in a buffer. After processing the request, we generate a response and write it back to the client. Finally, we close the connection.

Implementing a Loop for Continuous Operation

To emulate real-world web servers, which continuously handle requests, we wrap the server logic within an infinite for loop. By doing so, our server remains active, accepting and processing client connections indefinitely.

This allows multiple clients to connect simultaneously and receive responses without affecting the server’s operation.


By building a simple TCP server from scratch using raw sockets, we gained insights into system calls, socket programming, and the process of handling multiple connections. We explored the crucial steps of listening for client connections, reading requests, processing data, generating responses, and maintaining continuous server operation.

Understanding these fundamentals is essential for developing robust and efficient web servers capable of handling concurrent requests.

Here's the video ⤵

Courses I teach

Alongside my daily work, I also teach some highly practical courses, with a no-fluff no-nonsense approach, that are designed to spark engineering curiosity and help you ace your career.

System Design Masterclass

A no-fluff masterclass that helps experienced engineers form the right intuition to design and implement highly scalable, fault-tolerant, extensible, and available systems.

Details →

System Design for Beginners

An in-depth and self-paced course for absolute beginners to become great at designing and implementing scalable, available, and extensible systems.

Details →

Redis Internals

A self-paced and hands-on course covering Redis internals - data structures, algorithms, and some core features by re-implementing them in Go.

Details →

Writings and Learnings

Knowledge Base



Arpit's Newsletter read by 90000+ engineers

Weekly essays on real-world system design, distributed systems, or a deep dive into some super-clever algorithm.