Aws-sdk-net: Unity SDK - S3 PostObject Amazon.Runtime.Internal.HttpErrorResponseException when using S3CannedACL.PublicRead

[Jabbaquack]

AWS SDK version: aws-sdk-unity_3.1.57.0
Platform: Unity v5.3.4f1 (Mac OS Unity Editor and iOS 8.0 onwards)

When posting a file to a S3 bucket using the S3CannedACL.PublicRead, an Amazon.Runtime.Internal.HttpErrorResponseException is returned.
Using the same code but with S3CannedACL.Private, the file is successfully uploaded.

The request is formed as follows:

        var request = new PostObjectRequest()
        {
            Bucket = S3BucketName,
            Key = fileName,
            InputStream = stream,
            CannedACL = S3CannedACL.PublicRead
        };

Is S3CannedACL.PublicRead not a valid option? Or does the bucket need to be configured differently?

[johnnyhoffman]

Hi,
S3CannedACL.PublicRead is a valid option and I am able to use it with the same version of the SDK that you are using.
Our error reporting for PostObject is different from most other operations and is consequently less informative. We are working on fixing that, but in the meantime you can get more information about the error by enabling logging and looking at the HTTP request and response in the logs. You can enable logging by including a file named awsconfig.xml in a directory named Resources within your Assets directory, with the following content:

<?xml version="1.0" encoding="utf-8"?>
<aws>
    <logging
        logTo="UnityLogger"
        logResponses="Always"
        logMetrics="true"
        logMetricsFormat="JSON"
        logResponsesSizeLimit ="4096"/>
</aws>

If you share the info in the logs with us, we should have a better idea of what's going on here. Thanks.

[Jabbaquack]

Hi,
Unfortunately, adding the awsconfig.xml to Assets/Resources and playing from the Unity editor threw up the following exception:

NullReferenceException: Object reference not set to an instance of an object Amazon.AWSConfigs.GetSection[AWSSection] (System.String sectionName) Amazon.Util.Internal.RootConfig..ctor () Amazon.AWSConfigs..cctor () Rethrow as TypeInitializationException: An exception was thrown by the type initializer for Amazon.AWSConfigs Amazon.Runtime.Internal.Util.Logger.ConfigureLoggers () Amazon.Runtime.Internal.Util.Logger..ctor (System.Type type) Amazon.Runtime.Internal.Util.Logger.GetLogger (System.Type type) Amazon.Util.Internal.AmazonHookedPlatformInfo..cctor () Rethrow as TypeInitializationException: An exception was thrown by the type initializer for Amazon.Util.Internal.AmazonHookedPlatformInfo Amazon.UnityInitializer.Awake () UnityEngine.GameObject:AddComponent() Amazon.UnityInitializer:AttachToGameObject(GameObject)

for the code:
UnityInitializer.AttachToGameObject( this.gameObject );

this.gameObject is not the issue.
Without the awsconfig.xml, the code runs fine in the editor. Is there so other configuration required to get the logging working?

[oschvr]

I have the same issue ! :/

[ogoguel]

To fix the issue with the AWSConfig.XML, just add correctForClockSkew="true" in the aws tag
cf. https://github.com/aws/aws-sdk-unity/blob/master/Assets/AWSSDK/src/Core/Resources/awsconfig.xml

However, I also have the Amazon.Runtime.Internal.HttpErrorResponseException when trying to upload a file (whatever CannedACL I use), but cannot get any log to help me understand where it could come from. (The only difference with the sample is using BasicAWSCredentials instead of CognitoAWSCredentials)

Any pointer would be appreciated.

[Majter]

Hi Guys, I would suggest to change your post request to this ->

           var request = new PostObjectRequest()
           {
                Bucket = S3BucketName,
                Key = fileName,
                InputStream = stream,
                CannedACL = S3CannedACL.Private,
                Region = _S3Region
            };

In my case the default region used by the request was not the one that I need :)

[daves0]

Any word on this yet? The one-year anniversary is approaching. There are now three other Unity S3 upload bugs posted, and they all are difficult or impossible for us to debug because of the poor error handling here. Can we get some support for S3 uploads in Unity via .Net, or is it time to move to Azure? Sorry for the saltiness, but I've lost a LOT of time on this, chasing wild geese everywhere ("try private instead of publicread", "specify the region 'cuz it doesn't default correctly", "set AWSConfig in code before creating client") .. ugh. The lack of error reporting makes this whole process an even worse nightmare.

[ShayFB]

Guys some help will be appreciated.
I can't manage to upload a file using S3 unity sdk.
The error (even after I add the logging info in the XML) is always " Object reference not set to an instance of an object"

I manage to download file and also list the bucket. I allowed all permissions.

The example code doesn't work.

HELPPPP

[soapyigu]

Hi,

I guess the solution from @Majter is correct.

I had the same issue when running s3 sample app from the repo aws-sdk-unity-samples, the region parameter should be reset to make it work. AWS has already fixed the issue.

[maxCdev]

Hi,
Please help me. I'm trying to upload an object into my bucket, but I get only this exception.
Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
I get object list, bucket list and objects without any problems. It's sdk bug or i am crazy?
Code here:
```
public void UploadFile()
{
PostObject(@"D:\","2.png");
}
public void PostObject(string filepath, string fileName)
{
var BucketName = "mybuck";
ResultText.text = "Retrieving the file";

    var stream = new FileStream(filepath + fileName,
    FileMode.Open, FileAccess.Read, FileShare.Read);

    ResultText.text += "\nCreating request object";

    var request = new PostObjectRequest()
    {
        Bucket = BucketName,
        Key = fileName,
        InputStream = stream,
        CannedACL = S3CannedACL.Private,
        Region = RegionEndpoint.APSouth1,
        ContentType = "image/png"
    };

    ResultText.text += "\nMaking HTTP post call";

    client.PostObjectAsync(request, (responseObj) =>
    {
        if (responseObj.Exception == null)
        {
            ResultText.text += string.Format("\nobject {0} posted to bucket {1}",
            responseObj.Request.Key, responseObj.Request.Region);
        }
        else
        {
            ResultText.text += "\n" + responseObj.Exception.Message;
        }
    });
}

```

[daves0]

Hi Max,

I did finally get things to work, but unfortunately can't recall what specific steps I took to get that way. My code is very similar to yours, with the exception that I do not set the Region and ContentType in my request (my region is the same as that in my client, so it didn't matter despite what others have said, it appears). Also, and perhaps more applicable to your specific issue, I'm using a try/catch block around my PostObjectAsync call so that I can get more information from unhandled exceptions when things fail. You'll note the slight difference in the error string printed from the two error cases. Hope it helps! My code is below.

c# try { Client.PostObjectAsync(request, (responseObj) => { if (responseObj.Exception == null) { Debug.Log(string.Format("\nobject {0} posted to bucket {1}", responseObj.Request.Key, responseObj.Request.Bucket)); state = UploadState.success; } else { Debug.Log("Exception while posting the result object"); Debug.Log(string.Format("\n received error {0}", responseObj.Exception.ToString())); state = UploadState.fail; } }, new AsyncOptions { ExecuteCallbackOnMainThread = false }); } catch(Exception ex) { Debug.Log("Unhandled Exception while posting the result object"); Debug.Log(string.Format("\n received error {0}", ex.ToString())); state = UploadState.fail; }

[maxCdev]

Thanks for your reply Daves! I catch this error:

received error Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
  at Amazon.Runtime.Internal.UnityWwwRequest.EndGetResponse (IAsyncResult asyncResult) [0x00008] in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Core\Amazon.Runtime\Pipeline\HttpHandler\_unity\UnityWwwRequestFactory.cs:286 
  at Amazon.S3.AmazonS3Client.ProcessPostResponse (IAsyncResult result) [0x0001c] in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\_unity\AmazonS3Client.unity.cs:220 
UnityEngine.Debug:Log(Object)
Test:<Post2>m__F(AmazonServiceResult`2) (at Assets/Test.cs:169)
Amazon.S3.<>c__DisplayClass5_0:<PostObjectAsync>b__0(AmazonWebServiceRequest, AmazonWebServiceResponse, Exception, AsyncOptions) (at E:/JenkinsWorkspaces/v3-trebuchet-release/AWSDotNetPublic/sdk/src/Services/S3/Custom/_unity/AmazonS3Client.unity.cs:60)
Amazon.Runtime.Internal.RuntimeAsyncResult:InvokeCallback() (at E:/JenkinsWorkspaces/v3-trebuchet-release/AWSDotNetPublic/sdk/src/Core/Amazon.Runtime/Pipeline/_bcl+unity/RuntimeAsyncResult.cs:120)
Amazon.S3.AmazonS3Client:PostResponseHelper(IAsyncResult) (at E:/JenkinsWorkspaces/v3-trebuchet-release/AWSDotNetPublic/sdk/src/Services/S3/Custom/_unity/AmazonS3Client.unity.cs:263)
Amazon.S3.AmazonS3Client:ProcessPostResponse(IAsyncResult) (at E:/JenkinsWorkspaces/v3-trebuchet-release/AWSDotNetPublic/sdk/src/Services/S3/Custom/_unity/AmazonS3Client.unity.cs:234)
Amazon.Runtime.Internal.<>c__DisplayClass7_0:<InvokeRequest>b__0(Object) (at E:/JenkinsWorkspaces/v3-trebuchet-release/AWSDotNetPublic/sdk/src/Core/Amazon.Runtime/Pipeline/_unity/UnityMainThreadDispatcher.cs:229)

Maybe the problem is that I use access key and a secret key for authentification, and not cognito authentification?

AmazonS3Config config = new AmazonS3Config { ServiceURL = "https://s3.ap-south-1.amazonaws.com/" }; var client = new AmazonS3Client(accessKey, secretKey, config);

[Smorpheus]

Just a follow-up/update. I still cannot post a file with Public Read/Write operations, and I'm failing to get an error message out of the console even with the XML file properly placed in Resources. (It's just returning a null response object with no information.) Posting a Private file works fine.

[rzuf79]

Wow, it's been over a year, and this issue is still opened.
I'm having the same issue: setting CannedACL to anything but S3CannedACL.Private causes the HttpErrorResponseException.

[Otd2]

Same problem here!
Can't post any file without using S3CannedACL.Private.

I believe, all the AWS sdk's for Unity, need to some update . It is not easy to use and documentation for unity is lacking. It feels like aws-unity-sdks released when they were not ready. Even the error messaging is not working well.

Please, I need help desperately.

[arr0nax]

I am having the same basic problem too :(

[si294r]

I'm also using S3Example.cs from Unity AWSSDK and got HttpErrorResponseException, then after i setting AWSConfigs in Start()

        void Start()
        {
            UnityInitializer.AttachToGameObject(this.gameObject);
            GetBucketListButton.onClick.AddListener(() => { GetBucketList(); });
            PostBucketButton.onClick.AddListener(() => { PostObject(); });
            GetObjectsListButton.onClick.AddListener(() => { GetObjects(); });
            DeleteObjectButton.onClick.AddListener(() => { DeleteObject(); });
            GetObjectButton.onClick.AddListener(() => { GetObject(); });

            AWSConfigs.HttpClient = AWSConfigs.HttpClientOption.UnityWebRequest;

            AWSConfigs.LoggingConfig.LogTo = LoggingOptions.UnityLogger;
            AWSConfigs.LoggingConfig.LogResponses = ResponseLoggingOption.Always;
            AWSConfigs.LoggingConfig.LogMetrics = true;
            AWSConfigs.CorrectForClockSkew = true;
        }

now i can see the problem

Received error response: [<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message>
<RequestId>99EB18D75BE52010</RequestId>
<HostId>gl3VFqE2t4esxMl5vXsVROEPZlM5hoXlSTdl3QdH5lJJW+5UYvMDCIg4pqAOYMa+vlux0qrqonw=</HostId></Error>]

try with Full Access Policy and it's works

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "FullAccessS3",
            "Effect": "Allow",
            "Action": [
                "s3:*"
            ],
            "Resource": [
                "*"
            ]
        }
    ]
}

[bvschwartz]

The @Majter suggestion from Aug 28, 2016 worked for me. Try to explicitly specify the s3 region in the request:
Region = _S3Region

[JGuthrie-Bluebeck]

This is still not working for me either. I agree with the suggestions above that adding the Content-Length header is the most likely culprit and no idea how to fix it without a new DLL (or some messy reflection as posted above)

[swayam01]

Hello,
I am working on Unity SDK to send a csv file from my device to the S3 bucket using PutObject method with the example aws has share for uploading file to s3 bucket. The script works fine for the android but for ios i get error stating execption while uploading file, whereas downloading a file from S3 works fine on both.
Pls find my code attached to this post.

Suggestion Appreciated
S3Example.txt

Thanks

[kingsprayGraffiti]

I ran into the same issue using Unity 2017.3.

It seems they have defaulted the chunkedTransfer to true which is causing me the error

I've fixed up the SDK AWS.Core you can grab it here.. just replace this with the one inside Unity and you should be good to go.

https://drive.google.com/open?id=1TC_aTv3d6DWc6gtPdEND79jOhbEZ0v3A

[kossnick]

@kingsprayGraffiti big thanks for posting the dll! That fixed our identical problem. If you're ever in San Francisco, I owe you a beer :)

Any chance you'd be up for posting your fork of the AWS.Core code you changed and instructions to re-build the dll itself? Knowing Unity and AWS, this will not be the last workaround we'll need to do.

[sakai417]

@kingsprayGraffiti Thank you for your contribution!
My 3days error is fixed by your dll file.

[Adam-VisualVocal]

@kozzles: Here's my fork where I disabled chunkedTransfer on http put requests. Look in the VV_Fork branch: https://github.com/VISUAL-VOCAL/aws-sdk-net

To rebuild, just open the Unity solution in Visual Studio and update the reference paths for UnityEngine.dll to point to your local copy. Then rebuild the Core project in the solution.

[plavunov]

@kingsprayGraffiti God Bless You, friend! This issue has been driving me insane for the past 2 hours. You're a wonderful person. And the guys who defaulted the chunkedTransfer to true are the exact opposite.

[tkiener]

Not working for me. I replaced the dll with no success calling PutObjectAsync.
Tried setting both: ContentBody and Stream. Neither works.

Thans for any help.
Tim

HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
at Amazon.Runtime.Internal.UnityRequest.Dispose () [0x00000] in :0
at Amazon.Runtime.Internal.HttpHandler`1[TRequestContent].GetResponseCallbackHelper (System.Object state) [0x00000] in :0
Rethrow as AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method.

[plavunov]

@tkiener Are you using the latest AWS DLLS? Also check your link.xml, a lot of issues arise from that file. You need to restart Unity after changing link.xml

For me this worked: I placed this link.xml in Assets/Resources

link.xml.txt

[tkiener]

@plavunov thanks, my link.xml keeps reverting my changes. Makes no different if Unity runs or not.

[tkiener]

@plavunov ok, figured out that unity merges all existing link.xml files into link.xml in Assets/link.xml

[tkiener]

Hi all, quick update.
It works with Unity 2017.4.1f1 and the latest AWS SDK.

[Arc8ngel]

I've updated Unity to 2017.4.1f1, using the most recent AWS SDK (aws-sdk-unity_3.3.269.0), and swapped out the _AWSSDK.Core.dll_ for the one @kingsprayGraffiti posted. Still can't get the S3Example to POST to the bucket.

Regions are properly set. Permissions are set to public. HttpErrorResponseException throws and the responseObj comes back null.

Amazon.Runtime.Internal.HttpErrorResponseException: Exception of type 'Amazon.Runtime.Internal.HttpErrorResponseException' was thrown.
  at Amazon.Runtime.Internal.UnityRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00015] in <2d8240dc4b6b4c7d98e38b423fe6e222>:0 
  at Amazon.S3.AmazonS3Client.ProcessPostResponse (System.IAsyncResult result) [0x0001c] in E:\JenkinsWorkspaces\v3-trebuchet-release\AWSDotNetPublic\sdk\src\Services\S3\Custom\_unity\AmazonS3Client.unity.cs:266

[cecicosta]

I managed to upload the file using PutObjectRequest intead of post.

` Debug.Log("Creating request object");
var request = new PutObjectRequest() {
BucketName = S3BucketName,
Key = Path.GetFileName(path),
InputStream = stream,
CannedACL = S3CannedACL.Private
};

    Debug.Log("Making HTTP post call");
    lastRequestSucceed = false;
    try {

        Client.PutObjectAsync(request, (responseObj) => {

            if (responseObj.Exception == null) {
                Debug.Log(string.Format("object {0} posted to bucket {1}", responseObj.Request.Key, responseObj.Request.BucketName));
                lastRequestSucceed = true;
            } else {
                Debug.Log("\nException while posting the result object");
                Debug.Log(string.Format("\n receieved error {0}", responseObj.Response.HttpStatusCode.ToString()));
            }
            isLastRequestDone = true;
        });
    }
    catch (Exception ex) {
        Debug.Log("Unhandled Exception while posting the result object");
        Debug.Log(string.Format("\n received error {0}", ex.ToString()));

    }`

[jacobschellenberg]

@cecicosta how does this code even work?

Client.PutObjectAsync takes a CancellationToken, I can't even get this code to compile.

[cecicosta]

@jacobschellenberg In which platform are you working? The calls might change a little from each SDK. The code above is for unity3D.

[jacobschellenberg]

I'm also using Unity3D.

Hey do you have an issue where Visual Studio isn't seeing the AWSSDK.Core reference?

Most of my code has red squiggly underlines saying that "The type [type] is defined in an assembly that is not referenced." But when I right click on References in the solution, I only get "Add Connected Services". I see the AWSSDK.Core reference is in the references dropdown, but it has a yellow missing symbol.

The weird thing is, even though it shows it's missing, and shows an error in Visual Studio, it still compiles and I can still press play in Unity.

[cecicosta]

I had this problem with another project. To fix it I add the file to the project, so it could find the reference.
Right click-> add existing item -> locate the .dll.
Don't know why the Visual Studio don't show an option to add a reference. Maybe it is a lite version that unity install?

[cecicosta]

Also, check if you downloaded the SDK specific for unity, and not the .Net version. Didn't check but they may be similar

[jacobschellenberg]

I'm able to add the reference, but it doesn't fix the issue.

I think it's because the Unity specific SDK is targeted for .NET 3.5, but my project requires .NET 4.5.

So while it's still compiling for some reason, it's just not working in visual studio. It's mostly annoying because I don't have autocomplete, and anywhere I use the SDK stuff it just shows red underline errors.

But otherwise it works. Just annoying.

[cecicosta]

Oh, I see. Maybe you can recompile the SDK for .NET 4.5? If you think it is worth the trouble. Don't know how hard/easy is that...

[Arc8ngel]

I'm using Unity 2017.3.1f3 targeting .NET 4.6, and the current SDK didn't give me any dependency issues. Have you tried re-importing the AWS unity packages, @jacobschellenberg ?

Also, find Assets/AWSSDK/AWSSDK.Core.dll from within Unity and click on it to bring it up in the inspector. Make sure it's being included for your target platform (or check "Any Platform").

[ismaelash]

Very good !!

This resolve my problem: https://github.com/aws/aws-sdk-net/issues/332#issuecomment-365154153

[ismaelash]

You can now, use sdk of S3 modified and better: https://github.com/IsmaelNascimento/aws-sdk-s3-for-unity

[ashishdhingra]

Closing this since as per comments above, the issue appears to be resolved. Please open a new issue in case this is not the case.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.