CloudWatch can be cheaper

CloudWatch is the solution provided by Amazon to monitor your AWS resources, but also your service behaviour. Most probably, if you have ever used DynamoDB or EC2, you noticed some nice graphs with statistics about a table or about an EC2 instance. Most of the AWS services provide by default and for free some basic metrics. For others – like EC2 for example – you can choose to have more graphs and you’ll pay for that. All these charts are filled using data stored in AWS CloudWatch. Even more, you can push your own metrics there and after that, create your dashboard. Having as many metrics as possible could be considered sometime a caprice, but it is very useful in many situations:

  • Financial metrics (number of active clients, number of sales) are very useful for management in order to get the best business decisions
  • Operational metrics (latency, number of errors) to serve as input for alarms or for troubleshooting.

Now that we have seen why CloudWatch is so useful, let’s talk about pricing: it has 2 main components: a cost per metric stored and a cost per number of API calls. Investigating a little bit more the API for pushing metrics, you will discover that a single call can have up to 20 metrics and up to 40Kb. So the next question is: what should we do to reduce costs? And the response is very simple: buffer the CW metrics for a while and send them in batches.

In order to make this process very simple, we created a Java library that uses a blocking queue to buffer and send batches of metrics to CloudWatch. The source code of this library is available on Github, but the artifacts were uploaded into Maven Central.

So, to link this library in your maven project, just add these:

<repositories>
    <repository>
      <id>sonatype release</id>
      <url>https://oss.sonatype.org/content/repositories/releases/</url>
    </repository>
  </repositories>

<dependency>
  <groupId>cloud.cirrusup</groupId>
  <artifactId>cloudwatch-async-batch-metrics-publisher</artifactId>
  <version>1.0.0</version>
</dependency>

Then you have to instantiate the publisher:

MetricsPublisher publisher = new QueueBasedMetricsPublisher(cloudWatch, "my-service-namespace", 5000);

A small advice here: the third parameters is the maximum time interval in millis while publisher waits for the metrics. If you put this parameter very high (more than 24 hours), it could take a long time – up to 2 days – to see these metrics. So, our advice is to set this parameter high enough to accumulate some metrics, but not too big because it can delays your metrics.

And you are ready to use it:

boolean success = true;
Metric metric = metricsFactory.newMetric("GetUserDetails");
metric.open();
try {
   List<User> users = dataStore.getUsers();
   metric.addMeasure("NoUsers", users.size());
} catch (IOException e) {
   success = false;
   throw e;
} finally {
   metric.addMeasure("Success", success ? 1.0 : 0.0);
   metric.addMeasure("Failure", !success ? 1.0 : 0.0);
   metric.close();
}

Again, we invite you to share your thoughts about our articles in a comment and we’ll try to respond as soon as possible!