Aws-sdk-net: DoesS3BucketExistAsync forces HTTP request

Created on 24 Oct 2018 · 7Comments · Source: aws/aws-sdk-net

Expected Behavior

DoesS3BucketExistAsync should use the protocol defined by service endpoint

Current Behavior

DoesS3BucketExistAsync forces HTTP

Possible Solution

Remove https://github.com/aws/aws-sdk-net/blob/efcd29fa8e00d59d1448dc87232d2b3a9b73c63a/sdk/src/Services/S3/Custom/Util/AmazonS3Util.cs#L573 , which should let GetPreSignedURL figure out the correct protocol.

Steps to Reproduce (for bugs)

Try to execute DoesS3BucketExistAsync against an HTTPS-only Minio instance.

Context

Your Environment

AWSSDK.Core version used: 3.3.28.1
Service assembly and version used: ?
Operating System and version: Linux
Visual Studio version: N/A
Targeted .NET platform: .NET Core 2.1

.NET Core Info

.NET Core version used for development:
.NET Core version installed in the environment where application runs:
Output of dotnet --info:

.NET Core SDK (reflecting any global.json):
 Version:   2.1.302
 Commit:    9048955601

Runtime Environment:
 OS Name:     debian
 OS Version:  9
 OS Platform: Linux
 RID:         debian.9-x64
 Base Path:   /usr/share/dotnet/sdk/2.1.302/

Host (useful for support):
  Version: 2.1.2
  Commit:  811c3ce6c0

.NET Core SDKs installed:
  2.1.302 [/usr/share/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.2 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.2 [/usr/share/dotnet/shared/Microsoft.NETCore.App]

Contents of project.json/project.csproj:

bug

Source

mausch

👍1

All 7 comments

Actually it is not enough to remove that line, because that means effectively setting the protocol to whatever the enum defaults to.
Instead, DoesS3BucketExistAsync should fetch the protocol from the service URL e.g. (yes this is hacky code):

            var serviceURL = ((AmazonServiceClient) s3Client).Config.DetermineServiceURL();

            var request = new GetPreSignedUrlRequest
            {
                BucketName = bucketName,
                Expires = s3Client.Config.CorrectedUtcNow.ToLocalTime().AddDays(1),
                Verb = HttpVerb.HEAD,
                Protocol = new Uri(serviceURL).Scheme.ToLowerInvariant() == "http" ? Protocol.HTTP : Protocol.HTTPS,
            };

Alternatively, the GetPreSignedUrlRequest.Protocol property could be made nullable (with null meaning don't override the protocol) but I don't know where else that is used.

mausch on 24 Oct 2018

Thank you for reporting this. The default protocol for the config is HTTPS, so this would seem to be a bug. The fix will presumably be to use the config's UseHttp property to determine the correct protocol.

klaytaybai on 27 Oct 2018

It appears that you are using Minio which is a third-party service provider. We are not currently prioritizing work to support third-party services.

boblodgett on 9 Nov 2018

@boblodgett this isn't just about Minio - the SDK is forcing an INSECURE request even when the endpoint is usually HTTPS. This is a pretty serious bug when you're using AWS S3 too. Please reopen.

mausch on 10 Nov 2018

we'll take another look at this. We are concerned that this maybe a breaking change for application in certain environments. At minimum, we should be able to make a v2 of the function as a way forward.