How to customize the embedded Tomcat

We started several time ago a series of posts that show step by step how to build a microservice using Java platform and to run it in AWS Beanstalk. The microservice runs using an embedded version of the Tomcat web server. In this post, we’ll continue with a new topic about how to improve availability of your Tomcat service and, to be more specific, we’ll describe a safety action concerning on how to prevent an outage in case of a huge spike in traffic. Also, the same approach could be followed if you want to customize the embedded web server.

Let’s suppose you finished to write the business logic of the service we started together, you followed the general setup and monitoring best practices presented in a previous post and now your service is ready to go into production and handle traffic from clients. In almost all cases before this step a serious load test is performed in order to determine the hardware type to choose and the normal operating values – average response time, maximum number of requests a single host can handle, etc.

Normally, a service is scaled to handle an expected load. But what happens if it has to face a huge traffic spike? Technically speaking, each request is handled by one thread inside the Tomcat container. If this value is too high, it causes a huge CPU jump without seeing any concrete work result since most of the time processor is busy with context switching. What should we do in this case? The response is very simple: stop accepting new requests on that host and scale the service by adding more hardware. In other words, accept only requests you can handle without risking to affect all the calls. In Tomcat, this is very easy to do by setting the value of maxThreads inside a connector. Default value is quite large – in Tomcat 8 by default there are 200 threads that should bring a traffic volume impossible to handle by a small host. The same default value is set in the embedded version of Tomcat, but to change this, you have to follow a different set of actions that we’ll present step by step below. In a future post, we’ll show also how to automatize this process if you run a service inside a classic Tomcat Web container.

Let’s see what are the steps to change the number of threads:
1. Customize the embedded version of the Embedded Tomcat version

public class TomcatServletContainerCustomizer implements EmbeddedServletContainerCustomizer {

   private static final Logger LOG = LoggerFactory.getLogger(TomcatServletContainerCustomizer.class);

   @Override
   public void customize(ConfigurableEmbeddedServletContainer factory) {
       if(factory instanceof TomcatEmbeddedServletContainerFactory) {
           customizeTomcat((TomcatEmbeddedServletContainerFactory) factory);
       }
   }

   public void customizeTomcat(TomcatEmbeddedServletContainerFactory factory) {
       factory.addConnectorCustomizers(new TomcatConnectorCustomizer() {

           @Override
           public void customize(Connector connector) {

               String maxThreads = System.getProperty("tomcat.max.threads");
               if(maxThreads!=null) {

                   Object defaultMaxThreads = connector.getAttribute("maxThreads");
                   connector.setAttribute("maxThreads", maxThreads);
                   LOG.info("Changed Tomcat connector maxThreads from " + defaultMaxThreads + " to " + maxThreads);
               }
           }
       });
   }
}

2. Configure service to start using the customized version of the Tomcat

@Bean 
public EmbeddedServletContainerCustomizer embeddedServletCustomizer() {
    
    return new TomcatServletContainerCustomizer(); 
}

3. Add the configuration details to the JAVA_TOOLS_OPTIONS, in the Beanstalk console.

java -Dspring.profiles.active=dev -Dtomcat.max.threads=10 -jar target/demo-java-service-1.0-SAMPLE.jar

The code is available for download here.
Please add your thoughts below and share current post with anyone is interested!