Aws-sdk-ruby: Signed S3 URLs expire before the actual expiration time

[YorickPeterse]

At least when using V1 of the SDK it appears that signed S3 URLs expire _before_ their actual expiration time. For example, a certain URL that was generated 2 days ago has the Expires query string field set to 1431352538 (2015-05-11 13:55) but when requesting said URL one is greeted with the following:

<Error>
<Code>ExpiredToken</Code>
<Message>The provided token has expired.</Message>
<Token-0>...</Token-0>
<RequestId>C0ACB79ABACB6979</RequestId>
<HostId>YjQDfdSYumILl2e3zL44wfnlBX72l531IFhhO3jMYV9fY5XPH44GqcM/Da3BRX46phfRi53gBd0=</HostId>
</Error>

The code used for generating these URLs is along the lines of the following:

object = bucket.objects['foo']
object.url_for(:read, :expires => 604800).to_s

Reading the documentation I'm lead to believe this should work but reality shows otherwise. Am I perhaps overlooking something, or is this perhaps a bug in V1 of the AWS SDK?

[trevorrowe]

I haven't heard any reports of issues with pre-signed URLs expiring early. In the generated url, there should be an expires query parameter.

https://bucket.s3.amazonaws.com/key?AWSAccessKeyId=AKIA...&Expires=1432106071&Signature=Atb47Bskjc9DAfqHbbDeTVGdNys%3D

The number is a unix timestamp (epoch seconds). Is that date correct for the URL you have generated?

[YorickPeterse]

Yes, the URLs do include the Expires parameter. Here's an example URL pointing to a text file: https://s3-eu-west-1.amazonaws.com/storage.olery.com/aws-sdk-808-test.txt?AWSAccessKeyId=AKIAIGBLDAE2THGTJVQA&Expires=1432283350&Signature=442ZV51EMy5LojR36kWZq3R0dgI%3D

I just generated this URL, I'll check back in a few hours to see if it's still valid.

[trevorrowe]

Did you have a chance to verify if the URL work or had any issues?

[YorickPeterse]

Interesting enough said URL _is_ still valid, leading me to believe it might be the content type (our expired URLs have the content-type set to application/pdf). To test this I generated the following URL:

https://s3-eu-west-1.amazonaws.com/storage.olery.com/aws_808.pdf?AWSAccessKeyId=AKIAIGBLDAE2THGTJVQA&Expires=1432629378&Signature=7cwKkNguwRH9pW%2FZyL9U2kp0DTo%3D

Using the following code:

require 'aws-sdk-v1'

b = AWS::S3.new.buckets['storage.olery.com']
obj = b.objects['aws_808.pdf']
content = File.read('/home/yorickpeterse/Downloads/aws_808.pdf')
obj.write(content, :content_type => 'application/pdf')
obj.url_for(:read, :expires => 604800)

I'll check again if this URL expires prematurely.

[YorickPeterse]

So I just realized something, which might explain the problem we're seeing. We're using IAM instance credentials to generate pre-signed URLs from our applications. Instance credentials expire after a little while, probably resulting in the URLs also expiring. Is there a way around this, or is the only option to re-generate signed URLs in such a case? Sadly using static AWS credentials is not really an option due to it being less secure than IAM credentials.

[trevorrowe]

I strongly suspect the expiration of the credentials used to sign the URL are the cause of the failure with the pre-signed request.

I haven't dug into this, but I suspect you could give the instance profile role privileges to call the STS assume role operation with a longer expiration. The assumed role could be scoped to the appropriate S3 operations needed for the pre-signed URL.

[YorickPeterse]

This indeed seems to be caused by IAM credentials expiring. I'll close this issue as there's little the AWS SDK can do about this. Sorry for the noise.

[ShahriatHossain]

I am saving presignedUrl into our database and when I browse the url 10 days later or more the url is broken that means expired, can I make this enable for forever use? or has any alternate of presigned url so that I can direct show the file from amazon to our app like the presign work but without expire time or date?

[praveenv66]

@ShahriatHossain Did you find a solution to generate pre-signed url which doesn't expire or any possible maximum value? I'm using the below code to generate presigned url. client.generate_presigned_url('get_object',Params = {'Bucket': bucketName,'Key': sourceFilePath },ExpiresIn = 7200)

[lssachin]

this can happen when your local system is behind AWS S3's time

[jfcsantos]

@lssachin Did you confirm this?
I'm doing concurrent multipart upload of a large file and trying to renew credentials if they expire....getting this error at some point.
My test environment is running locally on probably +1h more than S3.

[lakshmankashyap]

I just want when i click on my dashboard logout button when it was expire