MrBool
You must be logged in to give feedback. Click here to login
[Close]

You must be logged to download.

Click here to login

[Close]

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

[Close]

MrBool is totally free and you can help us to help the Developers Community around the world

Yes, I'd like to help the MrBool and the Developers Community before download

No, I'd like to download without make the donation

Keep Alive Connections and Chunked Response in http 1.1 with Java

This post is about understanding HTTP protocol and the two important functionality (Keep Alive Connections and Chunked Response) added in HTTP protocol 1.1.

[close]

You didn't like the quality of this content?

Would you like to comment what you didn't like?

Introduction

The basic structure of any HTTP compliant request is like this:

METHOD_NAME URL HTTPVERSION
REQUEST_HEADER : REQUEST_HEADER_VALUE

REQUEST BODY

The first line consist of Http method name (most popular are GET , POST ,PUT and DELETE) , (SPACE) , Then the URL path (like /index.html) ,(SPACE), and then the HTTP Protocol with version (like HTTP1.1).

The second line has request header name value pair . You can have more than one headers name value pair which are separated by new line. The third line is blank and the last line will have a request body.

Each new line is defined by CR and LF characters.

In HTTP 1.1 , the Host request header is must . You have to pass the hostname (like this Host : www.javaroots.com) .

Keep Alive Connection : Keep Alive connection is introduces in Http 1.1 protocol to overcome the trouble of creating connection again and again. We all know Http is stateless , so every time a user request for a resource , server creates a new connection for it and after sending response to client , it closes the connection . To render a full HTML page , there will be more than one HTTP call , like for images , css , jss etc and also AJAX calls are there . It’s a burden to create connection every time . With Keep Alive connection , server keeps the connection open ,even after sending the response. When server encounter KeepAlive in request header , it will try to find the open connection in the pool , if it is present , it will write to the connection directly. If some error occurs (if client closed the connection meanwhile) , it will create a new connection and put it in the connection pool after writing to it .

This Keep Alive Connection is also called Persistent Connection and request pipe lining is related concept where we send requests one after another without using the same connection again and again.

On the other side , to support Keep Alive , client also creates a pool of open connections , and try to find the connection if it is already connected to server.

This is sample java class to show , how would the keep alive connection be implemented in a java client.

Listing 1: Simple Java Class

 
private Map<String, SocketChannel> connectionCache = new HashMap<String, SocketChannel>();
            private HttpProtocolWriter builder = new HttpProtocolWriter();
            private HttpProtocolParser parser = new HttpProtocolParser();
 
            public Response sendRequest(Request request) throws InvalidRequestException,IOException,InvalidResponseException,ConnectionException
            {
                        SocketChannel channel;
                        ResponseImpl response = new ResponseImpl();
                        String requestString = builder.buildReqestMessage(request);
                        String hostName = request.getHostName();
                        int port = request.getPort();
                        channel = getChannel(hostName, port, request.isKeepAlive());
                        ByteBuffer requestData = ByteBuffer.wrap(requestString.getBytes());
                        try
                        {
                                    channel.write(requestData);
                                    parser.parseResponse(channel, response);
                        }
                        catch(IOException e)
                        {
                                    //if IOException occured ..try again with new connection
                                    close(channel);
                                    channel = getChannel(hostName, port, request.isKeepAlive());
                                    //$REVIEW get connection based on keepalive
                                    requestData.flip();
                                    channel.write(requestData);
                                    parser.parseResponse(channel, response);
                        }
 
                        AbstractReader reader = (AbstractReader) response.getReader();
                        reader.setConnection(this);
                        reader.setKeppAlive(request.isKeepAlive());
 
                        return response;
            }
            public void finish(SocketChannel channel,boolean keepAlive)
            {
                        if (channel != null && channel.isConnected())
                        {
                                    Socket so = channel.socket();
                                    String hostName = so.getInetAddress().getHostName();
                                    int port = so.getPort();
                                    if (keepAlive)
                                                connectionCache.put(hostName + port, channel); //keep channel in cache
                                    else
                                                close(channel);
                        }
            }
 
            private SocketChannel getChannel(String hostName,int port,boolean keepAlive) throws IOException,ConnectionException
            {
                        SocketChannel channel = null;
                        if (!keepAlive)
                                    channel = createNewConnection(hostName, port);
                        else
                                    channel = getChannelFromConnectionCache(hostName, port);
                        return channel;
            }
 
            public SocketChannel getChannelFromConnectionCache(String hostName,int port) throws IOException,ConnectionException
            {
                        SocketChannel channel = connectionCache.remove(hostName + port);
                        if (channel == null || !channel.isConnected())
                        {
                                    channel = createNewConnection(hostName, port);
                        }
                        return channel;
            }
 
            public SocketChannel createNewConnection(String hostName,int port) throws IOException,ConnectionException
            {
 
                        try
                {
                  SocketAddress address = new InetSocketAddress(hostName, port);
                  SocketChannel channel = SocketChannel.open();
                  channel.connect(address);
                  return channel;
                }
                catch (IOException e)
                {
                  throw new ConnectionException("Could not connect to hostname : " + hostName + "       (" + hostName + "is down)");
                }
                catch (UnresolvedAddressException ex) {
                        throw new ConnectionException("Could not resolve the IP of hostname: " + hostName + "      (" + hostName + " is Invalid)");
                }
                       
            }
 
            public void close(SocketChannel channel)
            {
                        if (channel != null)
                                    try
                                    {
                                                channel.close();
                                    }
                                    catch(IOException e)
                                    {
                                                e.printStackTrace();
                                    }
            }
 
            public void removeChannelFromCache(String key)
            {
                        SocketChannel ch = connectionCache.remove(key);
                        close(ch);
            }
 
            public void clearConnectionCache() throws Exception
            {
 
                        Set<String> keySet = connectionCache.keySet();
 
                        for (String key : keySet)
                        {
                                    SocketChannel s = connectionCache.remove(key);
                                    close(s);
                        }
 
            }
 
 
 

Chunked Transfer Encoding: If a Http Request contains request body , the client provides a Content-Length header in request specifying the length of request body , so that server knows how much to read from request .Similarly ,Server has to send the Content-Length header in response , containing the length of response body. But many times , server does not know the content length in advance (i.e. reading from a stream ) that's where chunked transfer encoding is used. It is added in HTTP 1.1 so that server can send response in chunks , without specifying the Content Length header.

Server do not specify the Content-Length header , instead it specify a new Header called Transfer-Encoding : chunked

Now server splits the response in chunk.Size of every chunk is decided by server. Every chunk starts with chunk length followed by new line (CRLF) then the chunk content then new line(CRLF) . The last chunk will have length as 0 to denote end of response.

ChunkLength(in HexaDecimal) CRLF Content CRLF, example :

4
abhi
4
shek
6
somani
0

Now the response read by client will be : abhisheksomani

Here is the sample code as to how to read a chunked response in javaclient. Code uses bytebuffer and socket channels.

Listing 2: Bytebuffer and socket channels

private ByteBuffer buffer;
private int chunkLength = -1;
private int bufferPos = 0; //number of total bytes transferred to buffer

public ChunkReader(SocketChannel channel, ByteBuffer buffer)
{
super(channel);
this.buffer = buffer;
buffer.compact();
buffer.flip();
}

@Override
int _read(ByteBuffer fillBuffer) throws InvalidResponseException,IOException
{
if (!fillBuffer.hasRemaining())
throw new IllegalArgumentException("no space available in buffer");

if (chunkLength == 0)
{
//EOF reached return -1
finish(channel);
return -1;
}
int totalReadBytes = 0;
boolean readMore = true;
while (readMore)
{
if (!buffer.hasRemaining())//if buffer has no more bytes to read ..fill buffer from channel
{
buffer.clear();
channel.read(buffer);
buffer.flip();
}
if (chunkLength == -1)
{
//reads chunk length
chunkLength = readChunkLength();
}
//fills from buffer to input buffer
//will identify end of chunk and sets chunkLength=-1
int read = readChunk(fillBuffer);
totalReadBytes = totalReadBytes + read;
if (!fillBuffer.hasRemaining() || chunkLength == 0)
readMore = false;

}
fillBuffer.flip(); //prepare for reading

return totalReadBytes;

}

/**
 * transfers data from buffer to input buffer untill input buffer is fully filled 
 * or total bytes transferred equals to chunk length
 * or there are no more bytes to read from buffer
 * @param fillBuffer
 * @return
 * @throws InvalidResponseException
 */
private int readChunk(ByteBuffer fillBuffer) throws InvalidResponseException
{
int numberOfBytesRead = 0;
if (chunkLength == 0)
return 0;
if (bufferPos == chunkLength)
{
bufferPos = 0;
chunkLength = -1;
return 0;
}
int oldLimit = buffer.limit();

int remainingBytes = chunkLength - bufferPos;
if (buffer.remaining() > remainingBytes)
buffer.limit(buffer.position() + remainingBytes);
if (buffer.remaining() >= fillBuffer.remaining())
{
numberOfBytesRead += fillBuffer.remaining();
buffer.limit(buffer.position() + fillBuffer.remaining());
fillBuffer.put(buffer);
}
else
{
numberOfBytesRead += buffer.remaining();
fillBuffer.put(buffer);
}
buffer.limit(oldLimit);
bufferPos = bufferPos + numberOfBytesRead;
return numberOfBytesRead;
}

//read chunk length from buffer
private int readChunkLength() throws InvalidResponseException
{
int result = 0;
try
{
String s = readLine();
if (s.isEmpty())
s = readLine();
result = Integer.parseInt(s, 16);
}
catch(Exception e)
{
throw new InvalidResponseException(
"exception reading chunk Length " + e.getMessage(), e);
}
return result;

}
private String readLine() throws IOException
{
StringBuilder builder = new StringBuilder();
boolean newLine = false;

while (!newLine)
{
if (!buffer.hasRemaining()) //if buffer becomes empty , fill again
{
buffer.clear();
channel.read(buffer);
buffer.flip();
}
char a = (char) buffer.get();
if (a == '\r')
{
a = (char) buffer.get();
if (a == '\n')
newLine = true;

}
else
builder.append(a);
}
return builder.toString();

}

You can play with request and response header by this online tool www.onlinehttpclient.com). It has the keep alive functionality as well as support for chunked response. It also shows the response headers returned by server .

Hope you like the post , do comment and share !!



Abhishek is working as a senior java developer in a product start up .He has worked on various java related enterprise applications and frameworks. He loves to explore new technologies

What did you think of this post?

Did you like the post?

Help us to keep publishing good contents like this.

SUPPORT US

funded

remaining

[Close]
To have full access to this post (or download the associated files) you must have MrBool Credits.

  See the prices for this post in Mr.Bool Credits System below:

Individually � in this case the price for this post is US$ 0,00 (Buy it now)
in this case you will buy only this video by paying the full price with no discount.

Package of 10 credits - in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download few videos. In this plan you will receive a discount of 50% in each video. Subscribe for this package!

Package of 50 credits � in this case the price for this post is US$ 0,00
This subscription is ideal if you want to download several videos. In this plan you will receive a discount of 83% in each video. Subscribe for this package!


> More info about MrBool Credits
[Close]
You must be logged to download.

Click here to login