blogger templates blogger widgets
This is part of a list of blog posts.
To browse the contents go to

Async handling and Non-blocking IO - Servlet 3.0 and Servlet 3.1


The downside of the Servlet API 2.5 is that it only allows for handling messages in a synchronous way.
The HTTP request and response object have to be accessed within the scope of the request-handling thread.
This message-handling approach is sufficient for most classic use cases.

But what if the request handling thread needs to do a long-running task. An asynchronous servlet releases the request handling Thread back to the server pool and lets the long-running task to proceed in the background (another thread). This long running task takes with it a context object which can be used to intimate the server of the availability of the result and this result can then be relayed back to the client from the server.

This async support given by servlets enable efficient implementation of Comet/Server-push patterns.

Flow of events:

1. Container receives request

2. service() of the appropriate Servlet is invoked.

3. service() may delegate or dispatch

4. At some point in the execution, the code will call startAsync() on ServletRequest object. This returns a AsyncContext object.
AsyncContext is the environment which the asynchronous execution code will have access to. Through this object, the async code gets access
to request and response objects.

5. service() returns having left the asynchronous request to be executed in a parallel context (thread). The container will not commit
and close the response automatically (which happens for sync/normal requests) instead it is deferred for a later time.

Flow of events for the async thread:

1. It executes what it's made for!!!

2. It uses request and response from the AsyncContext object to read and write.

3. Once all is finished we call either of the 2 on the context object:

i) call complete() - is a signal to the container that the response could be committed and closed.
ii) call dispatch() - similar to RequestDispatcher.forward(). A new request/response cycle is started.
The dispatch() returns immediately but the resposne is not committed until the target of the dispatch completes its execution.

Differences between dispatch() and forward()

1. Dispatcher Type is set to ASYNC instead of FORWARD
2. Response buffers and headers are not re-setted, allowing content to be carried over.
3. It is legal to dispatch() even if the response has already been committed.

If the code doesn't call complete/dispatch within a configurable timeout, the container will abort the original request,
ensuring that the client gets some kind of response.

AsyncListener class could be used to listen on the 4 possible events:
- start,
- complete,
- timeout,
- error


The following examples are modifications of tomcat examples.

Example: Async0.java

- servlet does startAsync()
- background thread calls ctx.dispatch()



Now hit
http://127.0.0.1:8080/Async/async0?job=1

Console output:
[http-nio-8080-exec-1]status attr: null
[Job-Thread-1]work 0 completed
[Job-Thread-1]work 1 completed
[Job-Thread-1]work 2 completed
[Job-Thread-1]work 3 completed
[Job-Thread-1]work 4 completed
[Job-Thread-1]work 5 completed
[Job-Thread-1]work 6 completed
[Job-Thread-1]Job finished now dispatching...
[http-nio-8080-exec-2]status attr: true
[http-nio-8080-exec-2]Received dispatch, completing on the worker thread.

Note how the request handling thread [http-nio-8080-exec-1] is released and then later a different thread [http-nio-8080-exec-2] is used to continue the processing.

Now try hitting 2 browsers at the same time.
http://127.0.0.1:8080/Async/async0?job=1
http://127.0.0.1:8080/Async/async0?job=2

Console output:
[http-nio-8080-exec-6]status attr: null
[Job-Thread-1]work 0 completed
[Job-Thread-1]work 1 completed
[http-nio-8080-exec-7]status attr: null
[Job-Thread-1]work 2 completed
[Job-Thread-2]work 0 completed
[Job-Thread-1]work 3 completed
[Job-Thread-2]work 1 completed
[Job-Thread-1]work 4 completed
[Job-Thread-2]work 2 completed
[Job-Thread-1]work 5 completed
[Job-Thread-2]work 3 completed
[Job-Thread-1]work 6 completed
[Job-Thread-1]Job finished now dispatching...
[http-nio-8080-exec-8]status attr: true
[http-nio-8080-exec-8]Received dispatch, completing on the worker thread.
[Job-Thread-2]work 4 completed
[Job-Thread-2]work 5 completed
[Job-Thread-2]work 6 completed
[Job-Thread-2]Job finished now dispatching...
[http-nio-8080-exec-9]status attr: true
[http-nio-8080-exec-9]Received dispatch, completing on the worker thread.


We see this on the web page.
Before starting job
After starting job
Doing work (0)
Doing work (1)
Doing work (2)
Doing work (3)
Doing work (4)
Doing work (5)
Doing work (6)
Dispatch received. Worker thread completed


To compare this to a synchronous handling we write Sync0.java which is a plain non-async servlet.

Example: Sync0.java



Hit 2 browsers at the same time.

[http-nio-8080-exec-5]'status' attr: null
[Job-Thread-1]work 0 completed
[Job-Thread-1]work 1 completed
[Job-Thread-1]work 2 completed
[http-nio-8080-exec-6]'status' attr: null
[Job-Thread-1]work 3 completed
[Job-Thread-1]work 0 completed
[Job-Thread-1]work 4 completed
[Job-Thread-1]work 1 completed
[Job-Thread-1]work 5 completed
[Job-Thread-1]work 2 completed
[Job-Thread-1]work 6 completed
[Job-Thread-1]Job finished now dispatching...
[Job-Thread-1]work 3 completed
[Job-Thread-1]work 4 completed
[Job-Thread-1]work 5 completed
[Job-Thread-1]work 6 completed
[Job-Thread-1]Job finished now dispatching...

Nothing much different here.

But whats displayed on browser?

On one browser,
Doing work (1)
Doing work (2)
Doing work (3)
Before starting job
After starting job

On another browser,
Before starting job
After starting job

Notice that whatever has been written into the stream at that point of time comes through but the rest is lost. The output various based on lots of factors.


Let's consider another example

Example: Async1.java

- servlet does startAsync()
- background thread calls dispatch(/path/to/jsp)
- this is similar to request dispatcher



async1.jsp


<%@page session="false"%>
Output from async1.jsp Type is
<%=request.getDispatcherType()%>

<%
  System.out.println("["+Thread.currentThread().getName()+"]"+"Inside Async 1");
  if (request.isAsyncStarted()) {
    request.getAsyncContext().complete();
  }
%>

Completed async request at
<%=new java.sql.Date(System.currentTimeMillis())%>

Hitting the url
http://127.0.0.1:8080/Async/async1?job=1

And we see this output,

Before starting job
After starting job
Doing work (0)
Doing work (1)
Doing work (2)
Doing work (3)
Doing work (4)
Doing work (5)
Doing work (6)

Output from async1.jsp Type is
ASYNC

Completed async request at
2015-08-09



Example: Async2.java

- servlet does startAsync()
- background thread calls writes and calls complete()
- this is similar to the normal flow of a servlet but in normal servlets the method just returns and that means everything that needed to be written into the stream has been written but now as it happens asynchronously the container needs to be notified that the request-response cycle has ended. And for this we explicitly calls a method called complete()




Instead of manually creating threads we could give the thread creating task to the container itself.


final AsyncContext actx = request.startAsync();
actx.setTimeout(Long.MAX_VALUE);
actx.start(new HeavyTask(actx));


Non-blocking IO - Servlet 3.1

Asynchronous Processing provides a mechanism to execute application-specific blocking operations in a new thread, returning the thread associated with the request immediately to the container. Even if you use asynchronous processing for all the application-specific blocking operations inside your service methods, threads associated with client requests can be momentarily sitting idle because of input/output considerations.

For example, if a client is submitting a large HTTP POST request over a slow network connection, the server can read the request faster than the client can provide it. Using traditional I/O, the container thread associated with this request would be sometimes sitting idle waiting for the rest of the request.

Java EE provides nonblocking I/O support for servlets and filters when processing requests in asynchronous mode. The following steps summarize how to use nonblocking I/O to process requests and write responses inside service methods.

1. Put the request in asynchronous mode as described in Asynchronous Processing.

2. Obtain an input stream and/or an output stream from the request and response objects in the service method.

3. Assign a read listener to the input stream and/or a write listener to the output stream.

4. Process the request and the response inside the listener's callback methods.



Make a post request to the url using ajax or client's like postman.

http://localhost:8080/Async/asyncn
With payload data: This is a good day for people like us.

Response:
<html lang="en-US" xmlns="http://www.w3.org/1999/xhtml">
<body>Hello
This is a good day for people like us.</body>
</html>

And we see this output,

[http-nio-8080-exec-5]Before starting job
[http-nio-8080-exec-5]After starting job
[http-nio-8080-exec-5]AsyncReadListener: data available 
[http-nio-8080-exec-5]Reading data: This is a good day for people li
[http-nio-8080-exec-5]Reading data: ke us.
[http-nio-8080-exec-5]AsyncReadListener: All data read.. 
[http-nio-8080-exec-5]AsyncWriteListener: onWritePossible.. 
[http-nio-8080-exec-5]AsyncServletNIO onComplete() called



No comments:

Post a Comment