blogger templates blogger widgets
This is part of a list of blog posts.
To browse the contents go to

JDBC connection pooling - Container managed

Instead of relying on 3rd-party libraries/frameworks we could use the pooling mechanisms provided by the underlying container.

Let's take tomcat, the container we are using for these examples.

Tomcat comes with Tomcat-DBCP.

Here is excerpt from tomcat documentation:

The default database connection pool implementation in Apache Tomcat relies on the libraries from the Apache Commons project.
The following libraries are used:
• Commons DBCP
• Commons Pool
These libraries are located in a single JAR at $CATALINA_HOME/lib/tomcat-dbcp.jar.

The JDBC Connection Pool, org.apache.tomcat.jdbc.pool is a single replacement or an alternative to (org.apache.commons.pool + org.apache.commons.dbcp).

Here are few reasons why Tomcat-DBCP should be preferred if you are using tomcat. These are from Tomcat-Docs

  1. commons-dbcp is single threaded, in order to be thread safe commons-dbcp locks the entire pool, even during query validation. 
  2. commons-dbcp is slow - as the number of logical CPUs grow, the performance suffers, the above point shows that there is not support for high concurrency Even with the enormous optimizations of the synchronized statement in Java 6, commons-dbcp still suffers in speed and concurrency.
  3. commons-dbcp is complex, over 60 classes. tomcat-jdbc-pool, core is 8 classes, hence modifications for future requirement will require much less changes. This is all you need to run the connection pool itself, the rest is gravy.
  4. commons-dbcp uses static interfaces. This means you can't compile it with JDK 1.6, or if you run on JDK 1.6/1.7 you will get NoSuchMethodException for all the methods not implemented, even if the driver supports it.
  5. The commons-dbcp has become fairly stagnant. Sparse updates, releases, and new feature support.
  6. It's not worth rewriting over 60 classes, when something as a connection pool can be accomplished with as a much simpler implementation.
  7. Tomcat jdbc pool implements a fairness option not available in commons-dbcp and still performs faster than commons-dbcp
  8. Tomcat jdbc pool implements the ability retrieve a connection asynchronously, without adding additional threads to the library itself
  9. Tomcat jdbc pool is a Tomcat module, it depends on Tomcat JULI, a simplified logging framework used in Tomcat.
  10. Retrieve the underlying connection using the javax.sql.PooledConnection interface.
  11. Starvation proof. If a pool is empty, and threads are waiting for a connection, when a connection is returned, the pool will awake the correct thread waiting. Most pools will simply starve.

Features added over other connection pool implementations


  1. Support for highly concurrent environments and multi core/cpu systems.
  2. Dynamic implementation of interface, will support java.sql and javax.sql interfaces for your runtime environment (as long as your JDBC driver does the same), even when compiled with a lower version of the JDK.
  3. Validation intervals - we don't have to validate every single time we use the connection, we can do this when we borrow or return the connection, just not more frequent than an interval we can configure.
  4.  Run-Once query, a configurable query that will be run only once, when the connection to the database is established. Very useful to setup session settings, that you want to exist during the entire time the connection is established.
  5. Ability to configure custom interceptors. This allows you to write custom interceptors to enhance the functionality. You can use interceptors to gather query stats, cache session states, reconnect the connection upon failures, retry queries, cache query results, and so on. Your options are endless and the interceptors are dynamic, not tied to a JDK version of ajava.sql/javax.sql interface.
  6. High performance - we will show some differences in performance later on.
  7. Extremely simple, due to the very simplified implementation, the line count and source file count are very low, compare with c3p0 that has over 200 source files(last time we checked), Tomcat jdbc has a core of 8 files, the connection pool itself is about half that. As bugs may occur, they will be faster to track down, and easier to fix. Complexity reduction has been a focus from inception.
  8. Asynchronous connection retrieval - you can queue your request for a connection and receive a Future back.
  9. Better idle connection handling. Instead of closing connections directly, it can still pool connections and sizes the idle pool with a smarter algorithm.
  10. You can decide at what moment connections are considered abandoned, is it when the pool is full, or directly at a timeout by specifying a pool usage threshold.
  11. The abandon connection timer will reset upon a statement/query activity. Allowing a connections that is in use for a long time to not timeout. This is achieved using theResetAbandonedTimer.
  12. Close connections after they have been connected for a certain time. Age based close upon return to the pool.
  13. Get JMX notifications and log entries when connections are suspected for being abandoned. This is similar to the removeAbandonedTimeout but it doesn't take any action, only reports the information. This is achieved using the suspectTimeout attribute.
  14. Connections can be retrieved from a java.sql.Driver, javax.sql.DataSource or javax.sql.XADataSource This is achieved using the dataSource and dataSourceJNDIattributes.
  15. XA connection support

Now, lets try to run a standalone tomcat-dbcp program.

package com.prodapps;

import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;

import org.apache.tomcat.jdbc.pool.DataSource;
import org.apache.tomcat.jdbc.pool.PoolProperties;

//add tomcat-juli.jar to WEB-INF/lib folder for this main program to run 

public class PoolingDriverExample {

 public static void main(String[] args) throws SQLException {
  PoolProperties p = new PoolProperties();
        p.setUrl("jdbc:oracle:thin:@localhost:1521:XE");
        p.setDriverClassName("oracle.jdbc.OracleDriver");
        p.setUsername("SYSTEM");
        p.setPassword("password");
        p.setMaxActive(100);
        p.setInitialSize(10);

        DataSource datasource = new DataSource();
        datasource.setPoolProperties(p);

        Connection con = null;
        try {
          con = datasource.getConnection();
          Statement st = con.createStatement();
          ResultSet rs = st.executeQuery("select 777 from dual");
          
          while (rs.next()) {
             System.out.println(rs.getString(1));;
          }
          rs.close();
          st.close();
        } finally {
          if (con!=null) try {con.close();}catch (Exception ignore) {}
        }
 } 
}

Since it's a standalone application, which does not have access to tomcat runtime libraries, you need to add tomcat-juli.jar( found at {tomcat_installation_dir}/bin) to WEB-INF/lib folder else you get
java.lang.NoClassDefFoundError: org/apache/juli/logging/LogFactory

Download sample web application here. The above stand-alone class is available in the same application as PoolingDriverExample.java.

When you try out the web application example, chances are you will get runtime exception -
Unable to create initial connections of pool. java.lang.ClassNotFoundException: oracle.jdbc.OracleDriver

Solution:
Copy ojdbc6.jar from WEB-INF/lib folder to {tomcat-installation}/lib folder.

Why do we get ClassNotFoundException for oracle.jdbc.OracleDriver though we have it in WEB-INF/lib folder?

To understand why, let's look at how class loading happens in tomcat 7.

In a Java environment, class loaders are arranged in a parent-child tree. Normally, when a class loader is asked to load a particular class or resource, it delegates the request to a parent class loader first, and then looks in its own repositories only if the parent class loader(s) cannot find the requested class or resource.

When Tomcat is started, it creates a set of class loaders that are organized into the following parent-child relationships, where the parent class loader is above the child class loader:

Bootstrap
          |
       System
          |
       Common
       /     \
  Webapp1   Webapp2 ...


Note that child class-loader has access to classes loaded by the parent but not vice-versa simply because those classes aren't loaded yet.



  1. Bootstrap - loads all JVM classes.  This includes the singleton DriverManager class which is shared by all web apps.
  2. System - loads all tomcat related classes mainly, /bin/bootstrap.jar and /bin/tomcat-juli.jar.
  3. Common - This class loader contains additional classes that are made visible to both Tomcat internal classes and to all web applications. Application classes should NOT be placed here.
    The locations searched by this class loader are defined by the common.loader property in {tomcat installation}/conf/catalina.properties.
    The default setting will search the following locations in the order they are listed:
    ${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar
    By default, ${catalina.base} refers to tomcat installation directory and so the contents of {tomcat-installation}/lib/tomcat-dbcp.jar gets loaded here.
  4. WebappX — A class loader is created for each web application that is deployed in a single Tomcat instance. All unpacked classes and resources in the /WEB-INF/classes directory of your web application, plus classes and resources in JAR files under the /WEB-INF/lib directory of your web application are loaded.

Now, since Tomcat-DBCP code is loaded by Common class loader and Tomcat-DBCP internally tries to register the OracleDriver class which is only available when Webapp class loader runs.

This is evident from the logs:
Caused by: java.lang.ClassNotFoundException: oracle.jdbc.OracleDriver
at java.net.URLClassLoader$1.run(Unknown Source)
...
...
at java.lang.Class.forName(Unknown Source)
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:246)

Checking the PooledConnection class near line no:246 we see this

protected void connectUsingDriver() throws SQLException {

try {
if (driver==null) {
...                
driver = (java.sql.Driver) Class.forName(poolProperties.getDriverClassName(),
true, PooledConnection.class.getClassLoader()).newInstance();
}
} catch (java.lang.Exception cn) { ... }

To make the OracleDriver class available while Common class loader runs, we need to put ojdbc6.jar in {tomcat-installation}/lib folder.

Also, putting JDBC drivers into Tomcat's lib folder will help prevent memory leaks when you redeploy your web app without restarting Tomcat.

Output:
doGet
5
doGet
3
4
6

Let's check the tables.
SQL> select * from products;

ID TITLE
---------- -------------------------
1 ipad
2 galaxytab
3 htc
4 nexus

Continue reading:Why Datasource?

No comments:

Post a Comment