Limit accepted requests using AWS ElastiCache

Let’s say you build an awesome service that exposes several APIs. We know: your service is scalable, robust and so on. But it cannot handle traffic spikes (5-10x) without being overscaled. And if it can, this is very expensive. A solution could be to reject calls that exceeds some limits, like calls from an IP or for a given client. Or maybe you can create some client pools and define some limits per pool.

If your service has only one host behind, this mechanism is simpler. But imagine we have deployed our service on tens of hosts. This means we need a storage where statistics are kept and all hosts have to sync their stats using that storage.

For a similar situation in a real-life service, we used a library named Spillway that uses Redis to persist records. Spillway is very easy to integrate in a Java microservice. We just created a Java filter that checks if another call is accepted and if not, 429 HTTP status code is returned.

And if that library needs Redis, we decided to use AWS ElastiCache for this. Even more this AWS service is part of free tier and this facility is enough for our needs.

To demonstrate this setup, we created a service deployed on 2 hosts:

Then we created a rule accepting only 2 requests per minute for a given name. Doing 3 calls, we received the following output:

curl -k http://cirrus-test.eu-west-1.elasticbeanstalk.com/hello?name=John

Hello, John. This is my first REST service!Simple text message from production stage


curl -k http://cirrus-test.eu-west-1.elasticbeanstalk.com/hello?name=John

Hello, John. This is my first REST service!Simple text message from production stage


curl -k http://cirrus-test.eu-west-1.elasticbeanstalk.com/hello?name=John

Too many calls!

The codebase for this example is available here. In our example we didn’t create a filter, because we considered in this way it is easier to understand how to integrate this library. You only need to open config-prod.properties file and put there the ElastiCache Redis endpoint. For the local/dev stage, you can use localstack, as we presented in an older post. In the near future, we’ll dedicate a whole post about how to connect to a Redis host from you local environment.

If you have any question, don’t hesitate to contact us!