Runtime: Configuring request options in Browser WebAssembly

[pranavkm]

Motivation

Before we look at our requirement in .NET, let's look at fetch. .NET WebAssembly implementation for HttpClient uses Browser's "fetch API" as it's transport.

In it's simplest format, all you need to perform a HTTP request using fetch is to specify a url:

const response = await fetch('products/get'); // Perform a GET request to products/get
const products = await response.json();       // Read the response body as json

Additional options to configure the request are passed in a secondary options type:

const response = await fetch('products/post', {
    method: 'POST',
    body: valueToSend,
    headers: {
        'Content-Type': 'application/json'
    }
});

fetch does not have a way to options that apply globally or to more than one request, all options must be configured per-request. Application developers may use helper methods for options that need to be configured frequently (for eg https://github.com/moll/js-fetch-defaults#using).

Let's see what this looks like when translated to using HttpRequestMessage:

```C#
var requestMessage = new HttpRequestMessage(HttpMethod.Post, "products/post");
requestMessage.Content = valueToSend;
requestMessage.Content.Headers.ContentType = "application/json";

In addition to the `method`, `body` and `headers` parameters, `fetch` has a few other options that are documented here: https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch#Syntax. In the example above, it was fairly easy to construct a `fetch` request given a `HttpRequestMessage` since all of these properties were available on the message instance. With HttpClient's programming model, some features are configurable per request, but many are not. In particular, platform-specific settings are always part of the message handler. We'll look at some common cases in webassembly where we really want per-request control of some of these platform-specific settings

1) Cache (https://developer.mozilla.org/en-US/docs/Web/API/Request/cache)

Browsers cache results and will return results from the caches. `fetch` has ways to force the browser to ignore locally cached resources and perform requests. Cache-busting is an important tool in a web developer's arsenal.

```js
// In this sample, we ensure the catalog is always updated from the server.
const response = await fetch('products/catalog.json', { cache: 'no-cache' });

If an equivalent option is offered to .NET WebAssembly developers, it is essential that it is available as a per-request setting. Changing cache settings globally is not desirable.

2) Integrity (https://developer.mozilla.org/en-US/docs/Web/API/Request/integrity)

Subresource integrity is a browser security feature that allows a response to only be read if the contents match the specified hash. It ensures that content downloaded to the browser haven't been tampered with in anyway. This is typically valuable if an application downloaded content from a 3rd party hosted site. For e.g.

// The application defers downloading a large payload until it's necessary. It uses integrity to ensure the contents are as expected.
if (shouldShowGrid()) {
    const data = await fetch('https://raw.githubusercontent.com/mydata/large.data.json', { integrity: 'precomuted-sha-goes-here' }).then(r => r.json());
    showGrid(data);
}

Like the cache option, integrity must be a per-request option.

3) Credentials (https://developer.mozilla.org/en-US/docs/Web/API/Request/credentials)

When performing a fetch request, browsers will default to not including the cookie header as part of the request. If your site relies on cookies for authentication, you need these to included. The credentials option allows configuring these:

const response = await fetch('products/get', { credentials: 'include' });

In .NET, credentials are configured on the handler. We could conceive of ways to solving this with what's already present in the HttpClient API, such as by using multiple HttpClient instances, or a handler that conditionally configures credentials. However, being able to configure a request setting on a per-request basis is very convenient.

4) Streamed versus buffered responses

The response returned from a fetch operation has methods to read it as bytes, strings, json etc. In addition, browsers allow reading the body as a raw (unbuffered) stream. Some applications such as gRPC Web's server streaming feature require streaming responses.
.NET WebAssembly has an implementation of StreamContent over the response body. When they attempted to make it the default, they received user feedback stating that performing sync reads on this content would result in application deadlocks (WASM is single-threaded). There was enough feedback where they feel the need to make returning an unbuffered stream content an option that users have to opt-into.

API Proposal

```C#
namespace System.Net.Http
{
public readonly struct HttpRequestOptionsKey
{
public HttpRequestOptionsKey(string key)
{
Key = key;
}

    public string Key { get; }
}

public sealed class HttpRequestOptions : IDictionary<string, object>
{
    // Explicit interface implementation
    public bool TryGetValue<TValue>(HttpRequestOptionsKey<TValue> key, out TValue value);
    public void Set(HttpRequestOptionsKey<TValue> key, TValue value);
}

public class HttpRequestMessage : IDisposable
{
    [Obsolete("Use Options instead.")]
    [EditorBrowseable(Never)]
    public IDictionary<string, object> Properties => Options;

    public HttpRequestOptions Options { get; }
}

}

<details>
<summary>Previous Proposal 2</summary>

```diff
+ // Inspired by https://github.com/dotnet/runtime/issues/1793
+ interface IHttpRequestOptions
+ {
+     bool TryGet<TValue>(HttpRequestOptionsKey<TValue> key, TValue value);
+ }
+ 
+ 
+ class HttpRequestOptions : IHttpRequestOptions
+ {
+     HttpRequestOptions Add(HttpRequestOptionsKey<TValue> key, TValue value);
+ }

// Limiting this list for the sake of brevity.
 public partial class HttpClient : System.Net.Http.HttpMessageInvoker
 {
     public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> DeleteAsync(System.Uri? requestUri, System.Threading.CancellationToken cancellationToken) { throw null; }
+    public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> DeleteAsync(System.Uri? requestUri, System.Net.Http.HttpRequestOptions requestOptions, System.Threading.CancellationToken cancellationToken) { throw null; }
     public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> GetAsync(System.Uri? requestUri, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) { throw null; }
+    public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> GetAsync(System.Uri? requestUri, System.Net.Http.HttpRequestOptions requestOptions, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) { throw null; }
     public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> SendAsync(System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Threading.CancellationToken cancellationToken) { throw null; }
+    public System.Threading.Tasks.Task<System.Net.Http.HttpResponseMessage> SendAsync(System.Net.Http.HttpRequestMessage request, System.Net.Http.HttpCompletionOption completionOption, System.Net.Http.HttpRequestOptions requestOptions, System.Threading.CancellationToken cancellationToken) { throw null; }
 }

+ static System.Threading.Tasks.Task<object?> GetFromJsonAsync(this System.Net.Http.HttpClient client, System.Uri? requestUri, System.Type type, System.Net.Http.HttpRequestOptions requestOptions, System.Threading.CancellationToken cancellationToken = default(System.Threading.CancellationToken)) { throw null; }

  partial class HttpRequestMessage : System.IDisposable
  {
+     public IHttpRequestOptions RequestOptions { get { throw null; } }
  }

Usage

1) Manually constructing a HttpRequestMessage

```C#
var request = new HttpRequestMessage(HttpMethod.Get, "products/catalog.json");

request.Options.Add(BrowserRequestOptions.CacheKey, BrowserRequestCache.NoCache);

// OR convenience extension methods if we deem necessary
request.Options.SetBrowserRequestCache(BrowserRequestCache.NoCache);

3) HTTP request performed by library or helper code.

```C#
var requestOptions = new HttpRequestOptions()
    .Add(BrowserRequestOptions.IntegrityKey, "precomuted-sha-goes-here");

var request = await httpClient.GetFromJsonAsync("https://raw.githubusercontent.com/mydata/large.data.json", requestOptions);

2) HttpRequestMessage constructed by external library

```C#
var requestOptions = new HttpRequestOptions();
requestOptions.Add(BrowserRequestOptions.Credentials, BrowserRequestCredentials.Include);

var request = myLibrary.CreateRequest(...);
httpClient.SendAsync(request, requestOptions, cancellationToken);

</details>

<details><summary>Previous Proposal 1</summary>

WebAssembly's `HttpClient` uses the [Fetch API](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API) as a transport for HTTP. Similar to how `WinHttpHandler` allow fine-grained platform specific config, there are options on `fetch` that a user may want to configure. This API introduces a `WebAssemblyHttpHandler` that supports this scenario.

## Proposed API

```c#
namespace System.Net.Http
{
    public partial class WebAssemblyHttpHandler : System.Net.Http.HttpMessageHandler
    {
        public WebAssemblyHttpHandler() { }

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/integrity
        public string? Integrity { get; set; }

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/cache
        public RequestCache? RequestCache { get; set; }

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/credentials
        public RequestCredentials? RequestCredentials { get; set; }

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/mode
        public RequestMode? RequestMode { get; set; }

        // Determines if the handler streams the response when supported by the platform. 
        // See https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts.
        public bool StreamingEnabled { get; set; }
    }

    enum RequestCache
    {
        Default = 0,
        NoStore = 1,
        Reload = 2,
        NoCache = 3,
        ForceCache = 4,
        OnlyIfCached = 5,
    }
    enum RequestCredentials
    {
        Omit = 0,
        SameOrigin = 1,
        Include = 2,
    }
    enum RequestMode
    {
        SameOrigin = 0,
        NoCors = 1,
        Cors = 2,
        Navigate = 3,
    }
}

Example

```c#
var client = new HttpClient(new WebAssemblyHttpHandler
{
RequestCredentials = RequestCredentials.Include,
RequestMode = RequestMode.Cors,
});

client.SendAsync(...)
```

[davidsh]

@scalablecory

[pranavkm]

The current implementation is located here: https://github.com/mono/mono/blob/master/sdks/wasm/framework/src/WebAssembly.Net.Http/WasmHttpMessageHandler.cs. Work's being done to rename the type and add these options as part of https://github.com/mono/mono/pull/19260.

/cc @marek-safar / @kjpou1 / @lewing / @terrajobst

[scalablecory]

namespace System.Net.Http

Given this type is very platform-specific, does it make sense have it in the System namespace or should it be somewhere else?

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/integrity
        public string? Integrity { get; set; }

It looks like this should be read-only based on the Mozilla docs. It appears to never return null there -- will we be normalizing "" into null for this API?

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/cache
        public RequestCache? RequestCache { get; set; }

It looks like this doesn't need to be nullable; RequestCache.Default is fine.

        // https://developer.mozilla.org/en-US/docs/Web/API/Request/credentials
        public RequestCredentials? RequestCredentials { get; set; }

Our other handlers have a ICredentials Credentials property. Would it make any sense to use the same here and have some sentinel values instead of an enum? We have precedent with sentinal implementations in CredentialCache.DefaultCredentials and CredentialCache.DefaultNetworkCredentials, and this would leave the door open for less browsery credential implementations in the future.

        // Determines if the handler streams the response when supported by the platform. 
        // See https://developer.mozilla.org/en-US/docs/Web/API/Streams_API/Concepts.
        public bool StreamingEnabled { get; set; }

Should this be static read-only? We have other APIs that use "Supports" instead of "Enabled"; may want to use that here.

[scalablecory]

Also, it looks like all of these APIs are at the request level in fetch, not the handler level (though fetch does not have a handler). We should think about how that shapes this here.

Maybe a derived HttpRequestMessage makes some sense, though I can see that causing hurdles for 3rd party libraries.

[pranavkm]

Given this type is very platform-specific, does it make sense have it in the System namespace or should it be somewhere else?

Based on an offline discussion, the plan was to mirror the platform-specific WinHttpHandler. I chose this namespaces to keep with the pattern there.

It looks like this should be read-only based on the Mozilla docs. It appears to never return null there -- will we be normalizing "" into null for this API?

The API the handler is attempting to mirror is window.fetch which allows configuring these values - https://developer.mozilla.org/en-US/docs/Web/API/WindowOrWorkerGlobalScope/fetch#Syntax. A usage for this would look like:

await fetch(myUri, { integrity: 'some-value' })

I wanted to leave these unspecified rather than specify a default, in the event the spec gets changed to have a new default at a future point.

We have precedent with sentinal implementations in CredentialCache.DefaultCredentials and CredentialCache.DefaultNetworkCredentials, and this would leave the door open for less browsery credential implementations in the future.

That's an interesting suggestion. The implementation could set the credentials option if it's one of the sentinel credentials, otherwise set the Authentication header. I'm not 100% certain if there's ever a need the header and the fetch option simultaneously, I'll ask around.

Should this be static read-only? We have other APIs that use "Supports" instead of "Enabled"; may want to use that here.

It's meant to be an option that influences whether HttpResponseMessage.Content is an unbuffered stream versus a buffered byte array. The unbuffered stream is required to enable application models that require streaming (such as gRPC). However, having it on by default or making it an implementation detail is problematic - performing a sync read on the returned stream in .NET deadlocks the application (.NET Wasm is single threaded). The property is meant to allow users to opt-in to using streams. In JavaScript, this is what the usage of the two APIs look like:

• https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#Examples
• https://developer.mozilla.org/en-US/docs/Web/API/Body/arrayBuffer#Examples

[terrajobst]

Video

• Shipping vehicle. Which assembly should this go in? Should this be shipped out-of-band?
• Namespace. We should consider either having a prefix for all types or introduce a new namespace.
• Scoping. It seems those are really per request, having them on the handler seems problematic unless we say they are just defaults. But we'd still need a way to pass them in per request.
• Mechanics. It seems we all like the concept of a loose property bag over message that a handler can inspect to perform request specific operations. However, we don't like the loose typing of HttpRequestMessage.Properties which is IDictionary<string, object>. It seems the easiest short term solution is an extension methods over HttpRequestMessage like SetRequestCache/GetRequestCache that will set/get the value in the property bag. Alternatively, we could expose a new type, such as HttpMessageOptions that people can pass around and that we can be stored in HttpRequestMessage.Properties. This way, libraries can accept an HttpMessageOptions and use that in case they are constructing HTTP messages themselves. @scalablecory also had this idea.

```C#

nullable enable

namespace System.Net.Http
{
public class WebAssemblyHttpHandler : HttpMessageHandler
{
public WebAssemblyHttpHandler();
public string? Integrity { get; set; }
public RequestCache? RequestCache { get; set; }
public RequestCredentials? RequestCredentials { get; set; }
public RequestMode? RequestMode { get; set; }
public bool StreamingEnabled { get; set; }
}
public enum RequestCache
{
Default = 0,
NoStore = 1,
Reload = 2,
NoCache = 3,
ForceCache = 4,
OnlyIfCached = 5,
}
public enum RequestCredentials
{
Omit = 0,
SameOrigin = 1,
Include = 2,
}
public enum RequestMode
{
SameOrigin = 0,
NoCors = 1,
Cors = 2,
Navigate = 3,
}
}
```

[pranavkm]

For the blazor-wasm release, we ended up most of the public API surface as part of Blazor's programming model. We'd like to replace it with a runtime feature for 5.0. I've updated the API proposal to include some background as to why we need this, as well as an API proposal based on some of the previous API review.

[marek-safar]

I'd suggest renaming WebAssemblyHttpHandler to BrowserHttpHandler or WebHttpHandler as the behaviour/implementation exposes web/js functionality and not WASM behaviour. If we had browser TFM I think this would be exactly API included there.

[terrajobst]

@scalablecory can you take another look? The new proposal is quite different. Please mark it api-ready-for-review once you're done.

[terrajobst]

Video

• It doesn't seem we need the IHttpRequestOptions
• We should combine the options with the Properties collection the message
• We don't believe we need the overloads on HttpClient that take options; it feels like an advanced scenario which people can achieve by constructing the message
• Users can store mismatched types via IDictionary<string, object>, but we should verify types on retrieval and throw an exception

```C#
namespace System.Net.Http
{
public class HttpRequestOptions : IDictionary
{
// Explicit interface implementation
public bool TryGetValue(HttpRequestOptionsKey key, out TValue value);
public void Set(HttpRequestOptionsKey key, TValue value);
}

public class HttpRequestMessage : IDisposable
{
    [Obsolete("Use Options instead.")]
    public IDictionary<string, object> Properties => Options;

    public HttpRequestOptions Options { get; set; }
}

}
```

[terrajobst]

@scalablecory if you disagree, feel free to mark it differently :-)

[karelz]

@pranavkm @marek-safar can you please run general Networking API changes first through @dotnet/ncl? We were not aware of the non-WebAssemblyHttpHandler part ...

@terrajobst I'd like @dotnet/ncl to look at the general API proposal to make sure it is aligned with our general direction - @stephentoub were you in the API review?

[stephentoub]

@stephentoub were you in the API review?

Yes, both today and on Thursday, and my very first comment in both was that it should be run through the networking team first before being reviewed more broadly.

[stephentoub]

(I'm also surprised to see it marked api-approved; I thought the whole discussion was contingent on being reviewed by @karelz and co.)

[pranavkm]

Bumped it back to ready-for-review. @karelz thinks @dotnet/ncl's review should be able to get to this by next week.

[terrajobst]

@stephentoub @karelz

(I'm also surprised to see it marked api-approved; I thought the whole discussion was contingent on being reviewed by @karelz and co.)

The goal was to unblock people, which is why I told @scalablecory to remove api-ready-for-review if he disagrees with the proposal.

[terrajobst]

Bumped it back to ready-for-review. @karelz thinks @dotnet/ncl's review should be able to get to this by next week.

@pranavkm please don't edit the API within a comment that represents approval if they are net-new suggestions. These comments are meant to express what was reviewed/approved at a given point in time. Otherwise it will confuse us in the next meeting :-)

Instead, update the issue description. I just did that and put the previous ones under a <details> tag.

[terrajobst]

Video

• Looks good as proposed

```C#
namespace System.Net.Http
{
public readonly struct HttpRequestOptionsKey
{
public HttpRequestOptionsKey(string key);
public string Key { get; }
}
public sealed class HttpRequestOptions : IDictionary
{
// Explicit interface implementation
public bool TryGetValue(HttpRequestOptionsKey key, out TValue value);
public void Set(HttpRequestOptionsKey key, TValue value);
}
public class HttpRequestMessage : IDisposable
{
[Obsolete("Use Options instead.")]
public IDictionary Properties => Options;

    public HttpRequestOptions Options { get; }
}

}
```

[marek-safar]

Implemented in master