Configure Tomcat to support a maximum number of requests

It passed a while since we described why it is important to manage the number of connections your embedded Tomcat can handle and in that episode we promised that we’ll describe how to do that for a classical Tomcat container.

Actually, this is very easy to realize just by setting the value of maxThreads inside a connector. Default value is quite large – in Tomcat 8 by default there are 200 threads that should bring a traffic volume impossible to handle by a small host.
Let’s see in the next sections an automated approach for customizing the number of threads inside a Tomcat container managed by Beanstalk. This service offers an advanced way to configure the environment and the resources. We invite you to browse the official documentation. For sure after you get familiar with these concepts, things presented below will look very familiar. Thus, in the package, we’ll add a new file that will be run on each deployment, before the service start-up and if the maxThreads value is not already set, it will be added.

Under the root of the package, we’ll create a path, like it follows: /.elasticbeanstalk/.ebextensions/
In that folder, we create a config file that contains the script code to alter the Tomcat server.xml file with the value of maxThreads.

# ~/project/.ebextensions/tomcat.config
# Set maximum number of connections to value specified in AWS Beanstalk Console
# Logger messages can be viewed in /var/log/messages

files:
    "/tmp/update-tomcat-max-threads.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #! /bin/bash

            logger -t tomcat_conf "Attempting to set value for Tomcat maxThreads with value set in Beanstalk Console..."
            CONFIGURED=`grep -c '<Connector port="8080" maxThreads="'${TOMCAT_MAX_THREADS}'"' /etc/tomcat8/server.xml`

            if [ $CONFIGURED = 0 ]
            then
              sed -i 's/Connector port="8080"/Connector port="8080" maxThreads="'${TOMCAT_MAX_THREADS}'"/' /etc/tomcat8/server.xml
              logger -t tomcat_conf "/etc/tomcat8/server.xml updated successfully with maximum number of threads"
              exit 0
            else
              logger -t tomcat_conf "/etc/tomcat8/server.xml already updated"
              exit 0
            fi

container_commands:
    update-tomcat-max-threads:
        command: "sh /tmp/update-tomcat-max-threads.sh"

And there is one more thing left: to include that file in the war that is uploaded in Beanstalk. To do that, we are going to alter the pom.xml file and specify the resources to be copied:

<plugin>
 <artifactId>maven-war-plugin</artifactId>
 <configuration>
   <webResources>
     <resource>
       <directory>.elasticbeanstalk</directory>
       <targetPath>/</targetPath>
       <filtering>true</filtering>
     </resource>
   </webResources>
 </configuration>
</plugin>

You are now almost ready to go, with a small mention: the script created reads an environment variable named TOMCAT_MAX_THREADS that must be defined in Elastic Beanstalk console, in the Configuration -> Software Configuration section. In that way, if you have different environments for testing and production you have the freedom to set different values according to instance used in each of them.

Returning to our Elastic Beanstalk environment: the load balancer offers 2 very useful metrics that should be monitored for exceeding traffic: the surge queue length and spillover. These metrics should be used to create alarms and, consequently, autoscale the environment. We’ll end this post saying that there are many other Tomcat parameters to be tuned if you want to achieve the highest throughput and script we added above is just a starting point.

We would love to read your comments with other suggestions on this topic or with different approaches to achieve the same result.