Azure-functions-host: [.Net] HttpTrigger Model Binding - Collections not desearilized in v2.0.12050.0

[andyroschy]

I have an HTTP trigger function. The binding attribute is set to a custom POCO model. When sending a POST request with JSON content, array properties are not being deserialized in the model (they are left as null). This was working before the update.

Investigative information

• Function App version: 2.0.1-beta.36
• Function App name: Localy hosted function with Azure SDK. (Also happens on hosted functions)

Repro steps

1. Create a simple Azure function with HttpTrigger bound to a POCO model. The model should have at least one collection property (Array<>, ICollection<>, IEnuerable<>, etc)
2. Send a POST request to the function with correctly formed payload.

Expected behavior

The deserialized model should have collection properties correctly deserialized.

Actual behavior

All collection properties are null instead. Non collection properties deserialize as expected, including complex objects.

Related information

You can test the issue with the following:
Function code:

       [FunctionName("Test")]
        public static async Task<IActionResult> Test(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "test")] TestPayload payload,
            HttpRequest req,
            TraceWriter log,
            ExecutionContext context)
        {

            return new OkObjectResult(true);
        }

TestPayload class (you can change the IEnumerable for any collection type, Nested[], ICollection, IList, List, etc):

    public class TestPayload
    {
        public IEnumerable<Nested> Foo { get; set; }
    }

    public class Nested
    {
        public string Bar { get; set; }
    }

Example payloads (none of them work):

{
    "Foo": [{"Bar":'bar'}]
}

{
    "Foo": []
}



md5-04c28d1cf493154832a1b936a14dae73



{
    "Foo": [{"Bar":"bar"}, {"Bar":"baz"}]
}

I also tried with camelCasing and with the property names without double quotes.

[ChristianWeyer]

Same here for us - and is a major blocker, obviously 😅 @fabiocav
This still happens with Core Tools 2.0.3 for me.
Thanks!

[ChristianWeyer]

Maybe someone can confirm this? @tohling @fabiocav @paulbatum ?
Thank you!

[andyroschy]

I ended up creating an extension method to work around this:

        public static T DeserializeModel<T>(this HttpRequest request)
        {
            using (var reader = new StreamReader(request.Body))
            using(var textReader = new JsonTextReader(reader))
            {
                request.Body.Seek(0, SeekOrigin.Begin);
                var serializer = JsonSerializer.Create(new JsonSerializerSettings()
                {
                    ContractResolver = new CamelCasePropertyNamesContractResolver()
                });
                return serializer.Deserialize<T>(textReader);
            }
        }

[ChristianWeyer]

👌🏼 This is a working workaround (hence the name 😅)

Would be great to see it fixed in the product, though.
Maybe @fabiocav could look at it?

Here is a simple repro:
https://github.com/ChristianWeyer/azure-functions-serialization-repro

Thanks!

[damoodamoo]

I'm seeing the same, currently just having to accept a string and manually use JsonConvert.DeserializeObject<> to properly deserialise to my POCO.

[joshdcar]

Running into this issue as well. Manual workaround with JsonConvert is less then elegant but does work. I was about to create a new issue until I found this one so just in case you needed another repository to reproduce this issue you can find mine over here - https://github.com/joshdcar/azure-func-v2-httptrigger-model-binding-bug . It's probably not worth mentioning but this works as expected under Azure Function 1.x.

Hopefully this will get addressed soon :)

Thanks!

[joshdcar]

So I did some digging and I believe I found a potentially candidate for the guilty line of code within a binding utility class used by the HttpTriggerAttributeBindingProvider of the HttpTrigger.

https://github.com/Azure/azure-webjobs-sdk-extensions/blob/cdeb65faacf04c53845fb76c4b513c9998fc97de/src/WebJobs.Extensions.Http/Utility.cs#L29

It appears that the binding logic is explicitly excluding arrays for some reason (p.Value.Type != JTokenType.Array) while deserializing the json body contents.

JObject parsed = JObject.Parse(json);
 var additionalBindingData = parsed.Children<JProperty>()
         .Where(p => p.Value != null && (p.Value.Type != JTokenType.Array))
         .ToDictionary(p => p.Name, p => ConvertPropertyValue(p));

Please advice if I should create a new issue over on that Repo since it looks like the issue is actually out of that code base (azure-webjobs-sdk-extensions). Also feel free to tell me I'm going down the wrong rabbit hole :)

Thanks!

[ChristianWeyer]

Any idea when this will (finally) be merged @fabiocav ? Thanks!

[ChristianWeyer]

Hi guys - just wanted to check about the status of this.
It has been open since September 2018 - and it really is a very basic and essential feature of a Web API-like facade framework.

Some customers talked to me at conferences, and they really have a hard time in building faith and trust into Azure Functions when even such a basic thing is not working - or is not being fixed for such a long time.

@jeffhollan @fabiocav

Thanks!

[jeffhollan]

Thanks for the ping. Been working through and triaging some things. Not sure how this fell off but will move it to the top of the backlog and make sure we pick up in an upcoming sprint (or at least circle back if the fix is non-trivial for some reason)

[ChristianWeyer]

Thanks @jeffhollan !

[jeffhollan]

Has a workaroud so bumping to P1

[lukasvosyka]

This is still open! Just ran into the same issue. Using an ordinary custom model for HttpTrigger binding won't deserialize arrays, collections, etc. Instead need to go via HttpRequestMessage or similar and deserialize on it own.

[jason-system1]

Chop, chop!

[nmfisher]

Just ran into this one myself - any indication of when this will be merged? Workaround is fine but a touch messy for such a core feature.

[ChristianWeyer]

... I keep knocking for the upcoming years ... 😇 @fabiocav @jeffhollan

[fabiocav]

@ChristianWeyer and others, let me share a way to address this that requires a bit of code, but it's relatively clean...

You can use the DI support (add a reference to Microsoft.Azure.Functions.Extensions) to configure some of this behavior. The following should address your issue:

using System;
using System.Collections.Generic;
using System.Text;
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Formatters.Json.Internal;
using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection.Extensions;
using Microsoft.Extensions.Options;

[assembly:FunctionsStartup(typeof(MyFunctionApp.Startup))]

namespace MyFunctionApp
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
            builder.Services.AddTransient<IConfigureOptions<MvcOptions>, MvcJsonMvcOptionsSetup>();
        }
    }
}

We're looking into options to do this out of the box with an opt-in model (as some of the behavior differences introduced by this could break other customers). But this should give you what you need for now.

[ChristianWeyer]

Thanks for the idea @fabiocav - seems my Startup class is not being picked up when I run my function app.
Is there anything else to consider here?
Thanks!

[galvesribeiro]

@ChristianWeyer make sure the attribute and the base class are as on his example.

[ChristianWeyer]

They are, thanks for the idea @galvesribeiro

[macux]

@fabiocav - I've added the suggested code to my function's startup but I'm still seeing the issue described here: https://github.com/Azure/azure-webjobs-sdk-extensions/issues/511 - (array input binding broken in v2) should your code fix this as well? My startup is fine generally as I was already successfully using DI with the function in question.

[zpittmansf]

Is there an update on a fix for this or a pull request that is in progress at least? This workaround doesn't work with .net core 3.0. You get an error stating a provider is null on startup. Seems that something else needs to be defined in order for this to be injected as a workaround.

Could not load type 'Microsoft.AspNetCore.Mvc.Formatters.Json.Internal.MvcJsonMvcOptionsSetup' from assembly 'Microsoft.AspNetCore.Mvc.Formatters.Json, Version=3.0.0.0, Culture=neutral, PublicKeyToken=adb9793829ddae60'. Value cannot be null. (Parameter 'provider')

[CasperWSchmidt]

1½ years later and still not fixed? Reports say this worked in V1 so it is clearly a regression and should be fixed ASAP (no opt-in this should just work)

[galvesribeiro]

@CasperWSchmidt yeah... Look at the age of other issues/PRs. There is one for AI, and many others. Sometimes I wonder if Functions is something that will be long term supported since it takes ages to something get fixed, event the simple ones...

[fabiocav]

@CasperWSchmidt are you experiencing this in 3.0? The workaround for 2.0 was provided as there were potential issues with changes to the default serialization behavior there.

@galvesribeiro I’m sorry if that is the impression you’re getting. I can assure you the team is committed to the service and the product. Following our sprints is the best way to see what the team is actively working on. We try to make sure issues are correctly prioritized for assignment, and I’m sorry if some of the ones you’ve been following have sat for a while. We’re working to address that.

[galvesribeiro]

@fabiocav thanks for getting back. You are the only one from the team I see around. This impression comes exactly because of what @CasperWSchmidt said...

1½ years later and still not fixed?

Don't take me bad, this is not just me as you may see. There are multiple issues that are critical for many customers (like myself) like this one which makes things way harder...

[fabiocav]

Understood. We recognize that a set of issues haven’t been prioritized (mostly because of competing priorities) and we’re working to address that.

Are you still experiencing this in 3.0? For this particular issue, given that we have a viable workaround that made this behavior configurable, the priority hasn’t been as high as others where users were completely blocked, but I do agree it is an inconvenience.

[galvesribeiro]

@fabiocav I do have migrated all my functions to System.Text.Json to avoid those serialization issues:

[FunctionName(nameof(TestFunc))]
public async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "/test")] HttpRequest request)
{
    var obj = await JsonSerializer.DeserializeAsync<TestBody>(request.Body, Constants.SerializerOptions);
    ....
}

That way I don't need any hacks.

[galvesribeiro]

But it still, this issue should be addressed so people that still use JSON.Net can have it sorted.

I honestly think the best way to solve this is to let people register at the DI container their own serializer for the HttpRequest.Body on the Startup.cs.

An abstraction like this with the a default implementations (i.e. JSON.Net or System.Text.Json) would work perfectly:

public interface IBodySerializer
{
    string[] SupportedContentTypes { get; }
    ValueTask<TBody> Deserialize<TBody>(Stream body);
}

That would just work across the board. Multiple serializers could be registered for a set of content types and the appropriated one would be invoked based on the Content-Type.

[CasperWSchmidt]

@fabiocav We are not running V3 as that gave us some other interesting issues. As mentioned in https://github.com/Azure/azure-webjobs-sdk-extensions/issues/486#issuecomment-584621155 I can't get the work-around to run.

[jeffhollan]

@fabiocav / others can we confirm if this is fixed in v3? For others realize it's a bit more of a pain but since this was a breaking change the plan was to bring along with a major version. Until then the mitigation from fabio is the best option.

[vitalybibikov]

It is still reproducible in v3. (in 3.0.3), can't check in 3.0.4 as a Critical bug was introduced their and Startup is not being called.

[DenisBalan]

Still not fixed in v3

[proton888]

having errors with this in V_3.05 still

[CasperWSchmidt]

It's truly amazing that such a basic feature (which goes back to before 2013 in newtonsoft.json) is still an open issue here. Especially since it is a clear regression between v1 and v2/3.
@jeffhollan how can a regression and only partial object deserialization ever be a breaking change? Who on earth would rely on this "feature"?

[fredrikdaveus]

@fabiocav I do have migrated all my functions to System.Text.Json to avoid those serialization issues:

[FunctionName(nameof(TestFunc))]
public async Task<IActionResult> Run(
    [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "/test")] HttpRequest request)
{
    var obj = await JsonSerializer.DeserializeAsync<TestBody>(request.Body, Constants.SerializerOptions);
    ....
}

That way I don't need any hacks.

There's one problem though. It doesn't seem to be possible to use the batching features, as you cannot have HttpRequest[] requests as a parameter.

Or, is there a workaround here as well using custom deserialisation? (which we do anyways for performance reasons)

[fabiocav]

Work on this is currently in progress. Moving to sprint 75 to reflect the current status.

[alrod]

The issue is fixed in the V3.
Please reopen the issue if it's still reproducible in V3.

[CasperWSchmidt]

Where do I find the fix? And what about V2. This issue is raised against a 2.0.1 beta and according to https://github.com/Azure/azure-functions-host/issues/3370#issuecomment-436106870 it worked in V1.
I would expect this bug to be fixed in V2 as well unless it has publicly been deprecated?

[CasperWSchmidt]

@alrod I'm trying to get this to work in V3. Brand new project with Azure Functions V3, host version 3.0.13353.0 (as reported when I start debugging the project).

I have a simple POCO:

public class PostCurrenciesCommand : ICommand
    {
        public List<CurrencyModel> Currencies { get; set; }

        public PostCurrenciesCommand()
        {
            Currencies = new List<CurrencyModel>();
        }
    }

(setting a temp object in constructor since I'm also testing C# 8 nullable reference types)

Using the following JSON:

[
    {
        "code": "DKK",
        "name": "Danske kroner",
        "location": "Danmark"
    },
    {
        "code": "USD",
        "name": "American dollar",
        "location": "United States of America"
    },
    {
        "code": "EUR",
        "name": "Euro",
        "location": "European Union"
    }
]

I have also tried wrapping the array:

{
    "currencies": [
    {
        "code": "DKK",
        "name": "Danske kroner",
        "location": "Danmark"
    },
    {
        "code": "USD",
        "name": "American dollar",
        "location": "United States of America"
    },
    {
        "code": "EUR",
        "name": "Euro",
        "location": "European Union"
    }
]
}

I can't get my Function to work:

public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "v1/currency")] PostCurrenciesCommand command, CancellationToken cancellationToken)

I have also tried a simple List:

public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "v1/currency")] List<CurrencyModel> currencies, CancellationToken cancellationToken)

Whatever I do I get an empty list. Please let me know what I need to do to use the model binding with lists

[feminho]

I'm getting the same error with Azure Functions 3.1 when trying to deserealize a message from a queue using Newtonsoft.Json

    public class JsonMessageSerializer : ISerializer
    {
        public T Deserialize<T>(string message)
        {
            return JsonConvert.DeserializeObject<T>(message);
        }

        public string Serialize(object obj)
        {
            return JsonConvert.SerializeObject(obj);
        }
    }

I have log before and after call JsonMessageSerializer.Deserialize and log stream shows that code crashes at this point.

Error: One or more errors occurred. (Value cannot be null. (Parameter 'resource'))

The project where the serealization is running has .Net Core 3.0.
The Azure function has its SDK on 3.0.7 and target .Net Core 3.1