Aws-sdk-js: Limit the size of file upload with presigned urls

[basarat]

Support was removed in v2.1.6 https://aws.amazon.com/releasenotes/1473534964062833

The SDK will now throw an error if ContentLength is passed into an Amazon S3 presigned URL (AWS.S3.getSignedUrl()). Passing a ContentLength is not supported by the SDK, since it is not enforced on S3's side given the way the SDK is currently generating these URLs. See GitHub issue #457.

And now we get an error : ContentLength is not supported in pre-signed URLs.

I have seen issue #457 but there seems to be confusion around other SDKs still supporting limiting the size of the file that can be uploaded upfront when generating the presigned URL.

Question

• Are other SDKs e.g. go https://github.com/aws/aws-sdk-go/issues/625 mistaken (also works with Java) or is the JavaScript SDK mistaken?

Thanks :rose:

[basarat]

(also works with Java)

This is how to do it in Java. Would be great to get this supported in the JavaScript SDK :rose:

            SSEAlgorithm sseAlgorithm = SSEAlgorithm.KMS;
            GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(
                                                                            bucket,
                                                                            filePath,
                                                                            HttpMethod.PUT);
            generatePresignedUrlRequest.setSSEAlgorithm(sseAlgorithm);
            generatePresignedUrlRequest.setKmsCmkId(kmsCmkId);
            generatePresignedUrlRequest.setExpiration(expiration);
            generatePresignedUrlRequest.setContentType(contentType);
            generatePresignedUrlRequest.putCustomRequestHeader("Content-Length", contentLength.toString());

            URL url = s3client.generatePresignedUrl(generatePresignedUrlRequest);

[jeskew]

After removing the listener that throws the ContentLength is not supported in pre-signed URLs. and creating a pre-signed URL with a set ContentLength parameter, I observed that the URL could be used to put objects larger than the specified ContentLength without any error being thrown. I believe the listener should be kept, as content lengths specified in pre-signed URLs do not appear to be enforced.

The Go SDK's pre-signer works a bit differently from the JavaScript SDK's, which is how it is able to include content-length in the signature. Rather than converting headers to query string parameters and returning a pre-signed URL as a string, the Go SDK takes a pointer to an http.Request struct, modifies it in place with the appropriate signature, and returns a map of the headers with which the request was signed. Restricting ContentLength via presigning only appears to be enforced when the length is sent to S3 as a header and not as part of the query string.

[basarat]

So new question : Go SDK supports it. Can the JavaScript SDK support it? Would love to have it :heart:

@jeskew forgive me if I've read your comment wrong :rose:

[jeskew]

It would be possible to add a new operation presigner, but it would need to be a separate method with a different return type. Out of curiosity, would you want to use this pre-signer in node or a browser environment?

[basarat]

Out of curiosity, would you want to use this pre-signer in node or a browser environment?

node

[rhclayto]

So is there any way to enforce content-length/file size upload restrictions using s3.getSignedUrl? Content-Length header signing is not allowed & doesn't work anyway it seems, & ''content-length-range'' is not supported either the way it is when manually signing & doing a form POST. I would like to close any loophole allowing someone to generate a pre-signed URL then use it to PUT huge files to S3, but I haven't been able to find a way to do it using s3.getSignedUrl. Any clues? Node context.

[jeskew]

@rhclayto The best way to accomplish what you want is to use a signed policy to do a form-based POST upload. That will allow you to specify an acceptable range rather than a specific number of bytes.

My understanding of the Go SDK's presigner is that it will give you a presigned request (i.e., an object with a host, path, scheme, and headers) instead of a presigned URL (which cannot include headers). The JS SDK can add a feature that creates a presigned request with the appropriate headers, but you still would not be able to use a presigned URL for this purpose.

[rhclayto]

Oh well. The suggested presigned request creation feature would be handy in any event.

Thanks for your response.

[jainarchit1211]

(also works with Java)

This is how to do it in Java. Would be great to get this supported in the JavaScript SDK 🌹

            SSEAlgorithm sseAlgorithm = SSEAlgorithm.KMS;
            GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(
                                                                            bucket,
                                                                            filePath,
                                                                            HttpMethod.PUT);
            generatePresignedUrlRequest.setSSEAlgorithm(sseAlgorithm);
            generatePresignedUrlRequest.setKmsCmkId(kmsCmkId);
            generatePresignedUrlRequest.setExpiration(expiration);
            generatePresignedUrlRequest.setContentType(contentType);
            generatePresignedUrlRequest.putCustomRequestHeader("Content-Length", contentLength.toString());

            URL url = s3client.generatePresignedUrl(generatePresignedUrlRequest);

Hi Still not able to limit the size. getting the SignatureDoesNotMatch error while uploading

GeneratePresignedUrlRequest generatePresignedUrlRequest = new GeneratePresignedUrlRequest(
        bucketName,
        key,
        HttpMethod.PUT);
generatePresignedUrlRequest.setExpiration(expiration);
generatePresignedUrlRequest.putCustomRequestHeader("Content-Length", "87500");

[kasvith]

It would be good to have signature of go library for this, I think node still needs this