Google-cloud-dotnet: No way to specify CORS on resumable upload initiate session

[Lutando]

So I am sending consumers a session url to use as a point to do resumable uploads using the StorageClient.CreateObjectUploader() method

            var tempUploader = _client.CreateObjectUploader(myBucket, destination, properties.ContentType, new MemoryStream(), options);

            tempUploader.Body.Metadata = new Dictionary<string,string>()
            {
                { "foo", "bar" }
            };

            tempUploader.Body.ContentType = "some-content-type";

            tempUploader.Body.Size = 1337;

            var uploadUri = await tempUploader.InitiateSessionAsync();

I would like to add Origin : * or Origin : http://something.com headers to this InitiateSession request. how do I do that? using the package version 2.2.1, all help greatly appreciated :)

[jskeet]

I don't quite understand what you're trying to achieve - could you give a little more context? I'm not aware of any simple way to include extra headers in a resumable upload here... but it's not clear whether you'd need it to be in the initial request or in the "resume" calls later.

cc @chrisdunelm who may know more about this. (Not much about this is specific to Google.Cloud.Storage.V1; it's more about Google.Apis and ResumableUpload.)

[Lutando]

So we issue out the session url using a similar approach as outlined in my snippet above, we initiate the resumable upload without "starting" the upload however on subsequent calls to the resumable upload uri I get the following from the browser. maybe my real question is what is the proper way to issue out a resumable upload session uri that any origin can upload to.

Failed to load https://www.googleapis.com/upload/storage/v1/b/my-blob/o?uploadType=resumable&predefinedAcl=bucketOwnerFullControl&upload_id=AEnB2...0Ng: No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:8080' is therefore not allowed access. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

[jskeet]

When you say "we" - what is doing what here? Could you describe the components of your system? It's not clear to me what's issuing the error here - something at your client side, or the Storage server.

Note that the URL created by the upload here doesn't include any authentication information, so you'd need whatever was then uploading the file to have the same credentials available.

Perhaps you're looking for a signed upload URL? See the second example here, for instance: https://googleapis.github.io/google-cloud-dotnet/docs/Google.Cloud.Storage.V1/index.html#signed-urls

[Lutando]

I have a .NET backend that issues out the session URL for resumable uploading to a particular bucket for consumers in a very similar way to this , after initiating the session the backend would give consumers the uploadUri to do their resumable uploading

So basically the backend issues out this url so that front end browser based consumers can upload items to a particular location, uploading to this url actually works in most scenarios except for the scenarios where the requests come from an origin on the web.

Apologies if im not clear enough but the documentation on the resumable upload for the JSON API for this suggests that you should include the origin header for the origin that will issue subsequent requests to the session uri.

I can use the session uri to do resumable uploads from non-browser based uploaders (postman/server side) but when it comes to web we get the typical cors error that I snipped above.

[jskeet]

Okay, thanks, that documentation helps a lot.

I suspect we'll need more support in the underlying library (the one that contains ResumableUpload) for this. We'd then add the origin into UploadObjectOptions as well, so that's where you'd set it in your code.

While investigating this with @chrisdunelm I'll see if there's a workaround that will help you until we've got a more elegant approach.

[Lutando]

also fyi there was a similar question on the java sdk side of things that i found here.

[jskeet]

Aha - it looks like we should be able to add this option after all, using the ResumableUploadOptions.ModifySessionInitiationRequest property. Will look into that further today.

[jskeet]

@Lutando: I've now created a fix in #2756. Given my lack of experience in this particular area, I'd really like to check that it solves your issue before I merge it. Would it be possible for either:

• You to pull my branch (https://github.com/jskeet/google-cloud-dotnet/tree/cors) and build the library yourself
• You to use a .nupkg file that I create locally and put on Google Drive or somewhere similar, as part of a local NuGet repository on your development machine?

I'm happy to help you verify this in any way I can - please let me know the best way from your perspective.

[Lutando]

I'm on it. Thanks :)

[Lutando]

Preliminary Result - It works! but since we are doing chunked uploads we get the cors error on the last uploaded chunk request, but it still completes the resumable upload (despite the CORS error on resumable upload completion), I am under the impression that we might be doing something wrong on the client side.

[jskeet]

That's very odd - particularly that it did actually work. Presumably by that point the client library is not on the scene anyway?

[Lutando]

exactly, its out of band wrt the client library so its something on the our side but I will investigate further tomorrow

[jskeet]

Fabulous - thanks ever so much. I'll merge the change and close this issue then. Thanks so much for testing the change - I really, really appreciate it.

[shawnshaddock]

Preliminary Result - It works! but since we are doing chunked uploads we get the cors error on the last uploaded chunk request, but it still completes the resumable upload (despite the CORS error on resumable upload completion), I am under the impression that we might be doing something wrong on the client side.

I have the exact same scenario, my server is creating resumable urls and sending them back to the browser. When the browser attempts to POST the file to the URL I'm getting the CORS error. I've set CORS on my bucket and also included Orgin = "*" in my UploadObjectOptions when initiating the upload request.

The files do appear in my bucket, but they are only 2KB in size.

I also read that the API at storage.googleapis.com/your-bucket will not have CORS headers but your-bucket.storage.googleapis.com will. I tried manipulating the URI to be the latter but still with no success.

If a solution was found for this it would be greatly appreciated.

[jskeet]

@shawnshaddock: Everyone on the team is now on vacation until January, but if you could provide all the parts for me to reproduce this when we're back at work, I'll definitely have a look. Please open a new issue for this rather than using this one, just to keep things clearer.