Jobs scheduling with AWS ElastiCache

Even if there is the risk of repeating, this post is about how to use AWS ElastiCache (Redis, to be more specific) to schedule various jobs that have to be executed only once across an entire fleet of hosts.

Do you know that scenario when you want to trigger a report to be executed in an hour? Or next Monday at 8:00 in the morning?

Well, all the magic is based on a single concept: a distributed locking using a persistent data store. And in this case is Redis. A while ago, we discussed how to do that using Reddison, but today we are going to present another library that has a richer API for jobs scheduling.The library is named redis-scheduler (link) and comes with a very nice support for many Redis clients, from which we choose to use Spring support for Redis.

If you want to see how we integrated this library with Spring Boot, you can inspect our implementation here. After you clone the repository, make sure the Redis config values are correct. To start the service, follow the steps described in the README file.

For the sake of this demo, we created an API POST /request?id=xxx. It schedules a job with a delay of 5 seconds. If in this interval you’ll trigger another job with same id, the job is going to be ignored, but if ids are different, then you’ll see in the logs that all jobs are executed.

Schedule jobs with the same id

curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' 'http://localhost:8080/schedule?id=test'
curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' 'http://localhost:8080/schedule?id=test'

2019-01-23 18:27:23.326  INFO 82173 --- [nio-8080-exec-9] c.c.d.c.CirrusCloudDemoController        : /schedule request.
2019-01-23 18:27:55.156  INFO 82173 --- [nio-8080-exec-4] c.c.d.c.CirrusCloudDemoController        : /schedule request.
2019-01-23 18:28:07.139  INFO 82173 --- [heduler-polling] c.c.demo.task.CirrusTaskTriggerListener  : Task triggered: test

Schedule jobs with different ids

curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' 'http://localhost:8080/schedule?id=test3'
curl -X POST --header 'Content-Type: application/json' --header 'Accept: text/plain' 'http://localhost:8080/schedule?id=test3'

2019-01-23 18:37:22.356  INFO 82173 --- [nio-8080-exec-9] c.c.d.c.CirrusCloudDemoController        : /schedule request.
2019-01-23 18:37:23.156  INFO 82173 --- [nio-8080-exec-4] c.c.d.c.CirrusCloudDemoController        : /schedule request.
2019-01-23 18:37:28.139  INFO 82173 --- [heduler-polling] c.c.demo.task.CirrusTaskTriggerListener  : Task triggered: test2
2019-01-23 18:37:29.131  INFO 82173 --- [heduler-polling] c.c.demo.task.CirrusTaskTriggerListener  : Task triggered: test3


In the end, we want to remind you that AWS ElastiCache is part of free tier. If you want to access your Redis cluster from your machine, here is how to do it.

Feel free to add a comment with topics you want to discuss and subscribe to our newsletter to receive our newest posts!

Happy cloud computing!