Hi i am uploading 10MB to 500 MB large video files to Amazon S3 bucket but during video file upload after reaching 20% to 50% progress my upload hangs in the middle and then stop their and my files does not upload on to the S3 bucket.
In my console i receive this error
XMLHttpRequest cannot load https://bucket_name.s3.amazonaws.com/folder/up鈥0UFHH8mn1UjgeUitR6yVAgkJpu1aqDeNpVlFiob283pEc8xu19.FmW3B7EmRxkwHYms.RoU4-. Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'https://myweb.com' is therefore not allowed access. The response had HTTP status code 403.
using JavaScript browser amazon sdk
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.77.0.min.js"></script>
here is my JavaScript code
var IdentityPoolId = 'us-east-1:xxxxx';
AWS.config.update({
credentials: new AWS.CognitoIdentityCredentials({
IdentityPoolId: IdentityPoolId
})
});
AWS.config.region = 'us-east-1';
var S3 = new AWS.S3({params: {Bucket: 'bucket-name'}});
var fileChooser = document.getElementById('file');
var folders = foldername;
var file = fileChooser.files[0];
var file_name = 'file_name';
if (file) {
var params = {Key: file_name, ContentType: file.type, Body: file};
S3.upload(params).on('httpUploadProgress', function(evt) {
var percentComplete = Math.round(evt.loaded * 100 / evt.total);
// document.getElementById('progressNumber').innerHTML = percentComplete.toString() + '%';
$('.progress-bar-success').css('width', percentComplete.toString() + '%').html(percentComplete.toString()+'%');
// console.log("Uploaded :: " + parseInt((evt.loaded * 100) / evt.total)+'%');
}).send(function(err, data) {
if(!err) {
alert('upload complete');
}
});
}
i have given permission to my IdentityPool to upload in my amazon s3 bucket.the above functions work for 2 to 3 mb files but not working for larger files.
here is response of request from Network tab
<Error><Code>AccessForbidden</Code>
<Message>CORSResponse: This CORS request is not allowed. This is usually because the evalution of Origin, request method / Access-Control-Request-Method or Access-Control-Request-Headers are not whitelisted by the resource's CORS spec.</Message>
<Method>DELETE</Method>
<ResourceType>UPLOAD</ResourceType>
<RequestId>BD66C8DEC2037046</RequestId>
<HostId>mQ+Xpdvx0MkKmw+B2cfT18y/rIXLwvfsKqtB5+wQ1/Nodcpzn7+cR0PGTOdkTS7/hfMEsG+lQzw=</HostId>
</Error>
kindly help me on this.
Thanks in advance.
Hi i have also tried multi-part upload
```
var options = {partSize: 5 * 1024 * 1024, queueSize: 1};
S3.upload(params,options).on('httpUploadProgress', function(evt) {
```
but the upload progress still hangs in the middle and gives me the same above exact error.
Can you log the error message of S3.upload() function? Also, can you share your CORS configuration?
@AllanFly120 i have received this error on my console
Error: No access to ETag property on response. Check CORS configuration to expose ETag header.
here is my configuration
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
It seems that 2 configurations are missing in your CORS configuration. First, you need to expose your ETag header so that the upload function can track which part it's going to upload. See this. Basically, in S3.upload() method, if the file size exceeds 5mb, it will be chopped into several 5mb parts. The SDK needs ETag in response headers to keep track on which part is uploaded successfully. But your CORS configuration doesn't expose ETag now.
Then, you need to allow DELETE method in your configuration. Since the upload method didn't go through, the SDK will need DELETE method to clean up broken parts. This is why you get <Error><Code>AccessForbidden</Code> error.
Here is an example of what the CORS configuration may look like:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<ExposeHeader>ETag</ExposeHeader>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Let me know if you have further error messages.
Thanks @AllanFly120 it worked but facing one major issue now my files are uploading very slowly right now on s3 can you please also give me some tip improve my file upload speed to S3.
Thanks.
and during the file uploads the file progress suddenly drops to 10% for example if i am at 50% it will drop to 35% or if am at 20% then it drop to 10%.
@AllanFly120 i have one new issue with the slow file upload in the console the new error is shown timeout error and after the error the file upload has been stop their.
@usamamashkoor Your upload progress drops due to file parts upload failed (possibly because of the time out). One way to solve this problem is to set a longer timeout in config so that the upload requests have more time to get through. Another way is to adjust your queue size setting, which is the number of parts concurrently uploaded. So that the upload process may be faster but browser usually has a restriction on concurrent requests at the same time.
Thanks i have set the queue size to 4 and part of each to be 5MB and i have set time out 0 is this is fine for faster uploads or should i need to change some other settings to.
Thanks again @AllanFly120
and @AllanFly120 i am uploading some files like around 70 MB and now faced this error in my console.
Error: Network Failure
Sorry for keep bothering you again.
Thanks in advance.
@usamamashkoor Many reasons can lead to network failure. Do you mind show me your network tab, the header information of your failed request as well as the respond?
It looks like the initial issue has been addressed, and we need a bit more information on the network congestion/errors in order to proceed. If you have not already, I would recommend cross-posting your questions about optimizing uploads to the S3 developer forum and to StackOverflow.
Closing due to inactivity and because the initial issue appears to be addressed.
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.
Most helpful comment
It seems that 2 configurations are missing in your CORS configuration. First, you need to expose your ETag header so that the upload function can track which part it's going to upload. See this. Basically, in
S3.upload()method, if the file size exceeds 5mb, it will be chopped into several 5mb parts. The SDK needs ETag in response headers to keep track on which part is uploaded successfully. But your CORS configuration doesn't expose ETag now.Then, you need to allow DELETE method in your configuration. Since the upload method didn't go through, the SDK will need DELETE method to clean up broken parts. This is why you get
<Error><Code>AccessForbidden</Code>error.Here is an example of what the CORS configuration may look like:
Let me know if you have further error messages.