Published on 5th Jul 2017
3 min read
Share this article onMajority of the traffic over the internet is HTTP Traffic. There is a HTTP Client which wants some data from HTTP Server, so it creates a HTTP Request Message in the protocol understandable by the server and sends it. Server reads the message, understands it, acts accordingly and replies back with HTTP Response.
This complete process is abstracted by the tools like curl, requests libraries and utilities like Postman. Instead of using these tools and utilities, we shall go by the hard way and see HTTP messages in action.
For experimentation purpose let’s create a very basic webserver in Python Flask framework that exposes a trivial Hello World end point.
from flask import Flask
app = Flask(__name__)
@app.route('/hello')
def hello():
return "Hello, World!"
app.run(port=3000)
pip install flask
python hello.py
The server listens on port 3000 . If you hit from the browser http://localhost:3000/hello, you should see Hello, World! rendered.
A HTTP Client talks to HTTP Server via a common protocol that is understandable by the two parties. A sample HTTP request message looks something like
GET /hello.html HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.sample-server.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
To understand more about HTTP Request messages, see references at the end of this article.
The HTTP Communication happens over a TCP Connection. So we create a TCP connection with the server and try to get response from it. To get a TCP connection I will use netcat.
netcat is the utility that is used for just about anything under the sun involving TCP or UDP. It can open TCP connections, send UDP packets, listen on arbitrary TCP and UDP ports, do port scanning, and deal with both IPv4 and IPv6.
The webserver that was created above is listening on port 3000 . Lets create a TCP Connection and connect to it using netcat.
netcat localhost 3000
The command along with creating a TCP connection, will also open a STDIN. Anything passed in that input stream will reach the server via the connection. Lets see what happens when we provide This is a sample as input.
The input message given is not a valid HTTP message hence server responded with a status code of 400 which is for Bad Request. And if you closely observe the server logs on flask application, you will see an entry of our last request.
Since the server is a HTTP Server, so it understands HTTP request. Let’s create one to hit our exposed API endpoint /hello .
The HTTP request message for this request looks something like this
GET /hello HTTP/1.1
And you should see output like this
The HTTP Server understands the message sent from the client and it responded back as directed by the source code.
Following method exposes an endpoint which accepts a query parameter named name, and returns a response with name in it.
from flask import request
@app.route('/user')
def get_user():
name = request.args.get('name')
return "Requested for name = %s" % name
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
GET /user?name=arpit HTTP/1.1
Following method accepts form data via HTTP POST method and returns a dummy response with username and password in it.
from flask import request
@app.route('/login', methods=['POST'])
def login():
username = request.form.get('username')
password = request.form.get('password')
return "Login successful for %s:%s" % (username, password)
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
POST /login HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 32
username=arpit&password=welcome
Following method accepts JSON data that contains a field id with integer value via HTTP POST method and returns a dummy response with id in it.
from flask import request
@app.route('/save', methods=['POST'])
def save_user():
user_data = request.json
return 'Saving user with id = %d' % (user_data.get('id'))
Provide the HTTP request message below when STDIN opens up after you execute netcat command and connect with the server.
POST /save HTTP/1.1
Content-Type: application/json
Content-Length: 30
{"id": 1092, "name": "Arpit"}
The hard way to hit REST endpoints was not hard at all ;-) Stay curious and dive deep.
If you like what you read subscribe you can always subscribe to my newsletter and get the post delivered straight to your inbox. I write essays on various engineering topics and share it through my weekly newsletter 👇
Setting up Graphite and Grafana on an Ubuntu server
Part 2: Monitor your production systems and application analytics using Graphite. This article will ...
14th DecPublish python package on PyPI
If you have written something cool in Python and want to make it installable via pip and easy_instal...
10th NovMultiple MySQL server running on same Ubuntu server
Have multiple MySQL versions running on same server within 5 minutes....
13th MaySliding Window based Rate Limiter
A rate limiter is used to control the rate of traffic sent or received on the network and in this ar...
5th Apr