There are multiple ways you can interact with Amazon Web Services:
- AWS Management Console – a web application that offers an interactive user interface, allowing to perform many AWS task, such as launching an EC2 instance, creating a DynamoDB table, and so on.
- AWS Command Line Interface – a tool that offers a very powerful command line interface to manage AWS services. It is very easy to install and could be integrated in your scripts in order to build standard procedures for many tasks especially related to the maintenance of your application.
- AWS Software Development Kit (SDK) – definitely, it’s the most powerful and helpful way to interact with AWS, providing support for multiple programming languages.
Observation: In the upcoming sections, all examples will be offered using the Java SDK, but this blog post contains tips and tricks that are applicable to many other programming languages for which AWS offers support.
Let’s say that in our application we want to have a separate thread that each hour gets the logs file and stores them in AWS S3.
To accomplish this, first thing you have to do is to create an AWS S3 client object:
AWSCredentials awsCredentials = new BasicAWSCredentials("accessKey","secretKey"); AmazonS3 s3Client = new AmazonS3Client(awsCredentials);
It looks familiar, isn’t it? I imagine that at least credentials are kept in a properties file, but in most of the cases, the service client is created just as above: using only the credentials and ignoring the rest of the available possibilities. And there is one that must be considered in order to configure your service client according to the use case by setting appropriate values for timeouts, number of retries, proxy settings, etc.
Before seeing how to customize these settings according to your use case, let’s explain them:
- Connection timeout – the amount of time to wait initially to establish a connection before giving up and timing out.
- Socket timeout – the amount of time to wait for data to be transferred over an established, open connection before the connection times out and is closed.
- Maximum number of connections – maximum number of HTTP connections that could be simultaneously handled.
AWS SDK has global settings for socket and connection timeouts, with default values set very high for two reasons: to support the behaviour of all AWS services and also, to ensure that customers can connect even they have a very slow, high-latency network. For example, as of this writing, the Java SDK has default values for socket and connection timeout set to 50 seconds. The impact of these settings could be seen especially during operational issues, because SDK will hold connections open for unreasonably long time.
Ok, having cleared all these theoretical aspects, let’s see what should take to avoid all these issues:
- There is no reason to have a value for connection timeout bigger than several (1-3) seconds. In case a connection cannot be established, it is preferable to retry it than to wait 50 seconds – A dedicated post about SDK retry policy will appear soon 🙂
- Adjust socket timeout according to the service you are using: maybe it worth having a big socket timeout if you upload and download large object from S3, but, again, there is no reason to have such a large value if you want to send a SNS notification.
- Consider adjusting the maximum number of connection for AWS client to have at least one connection per each thread in your application. If you have a web service that reads objects from S3 and it can handle 100 simultaneous connections, you will see soon a bottleneck, because by default the AWS client can handle only 50 requests.
Consider creating distinct client configuration object for each AWS service client you create. Even this object could be shared across multiple AWS clients, it is recommended to have distinct instances, each of them having proper configuration values for the service and use case it was created.
Now, let’s see how to put into practice advice listed above:
AWSCredentials awsCredentials = new BasicAWSCredentials("accessKey","secretKey"); ClientConfiguration clientConfiguration = new ClientConfiguration() .withConnectionTimeout(1000) .withSocketTimeout(5000) .withMaxConnections(100); AmazonS3 s3Client = new AmazonS3Client(awsCredentials, clientConfiguration);
Consider setting distinct connection timeouts for each request using AmazonWebServiceRequest class methods if you need different configuration parameters for each request you made to AWS. This approach would be useful for example if you upload documents into S3 that have different sizes (some very small, some very large) and each of them should complete in a different time interval.
PutObjectRequest request = new PutObjectRequest("test-bucket","key", new File("/tmp/testfile.txt")) .withSdkClientExecutionTimeout(2000) .withSdkRequestTimeout(1000); amazonS3.putObject(request);
These are just few configuration options that could be set on an AWS client and, from what we noticed, there is not very much guidance on this topic, making this quite an advanced task. Also, of course there are many AWS users that are familiar with notions described in this post, but many of them began to consider these things only after they have been burnt by default timeouts. In the next days we’ll continue to detail this topic with a dedicated post about how to retry failed calls.
If you have questions, feel free to add them in the comments section. Also, if you know someone else that could be interested in this topic, please send her/him the link of this blog post.