Runtime: Improvement: Do not merge arrays in configuration by default

[spragchris]

It seems like merging arrays from two files causes the arrays to merge by index. I don't think this is expected behaviour, and can even easily cause problems which aren't simple to diagnose.

I think the most obvious way to merge arrays, at least by default, is for one array to completely replace the other array, as if it were a single property, without merging.

In the following example, 'UnlimitedPower' is not assigned, and therefore some users would expect it to be false as a default (determined by the class not having set the value.) Instead, the value from base.json is applied to index [0] of the array, giving that user 'UnlimitedPower'.

using System;
using System.Collections.Generic;
using Microsoft.Extensions.Configuration;

namespace ConfigDemo1
{
    class Program
    {
        class User { public string Name { get; set; } public bool UnlimitedPower { get; set; }}
        static void Main(string[] args)
        {
            // base.json = { "Users": [ { "Name" : "Administrator", "UnlimitedPower" : "True" } ] }
            // override.json = { "Users": [ { "Name" : "Random Internet Guy" }, { "Name" : "Administrator", "UnlimitedPower" : "True" } ] }
            var users = new ConfigurationBuilder().SetBasePath(Environment.CurrentDirectory).AddJsonFile("base.json").AddJsonFile("override.json").Build()
                .GetSection("Users").Get<List<User>>();
            // users[0].Name = "Random Internet Guy", users[0].UnlimitedPower = True
        }
    }
}

[andrewslavin]

I have the same issue. Is this a bug or by design? If by design - why? This behaviour is quite counter-intuitive. You'd expect values in the overriding json file to replace values with the same names in the base file, not be merged with them.

@spragchris Can you please comment on the issue, thank you.

[spragchris]

@andrewslavin I'm not part of this project or anything, I just discovered this problem after scratching my head for a few days (actually it was a co-worker who discovered it.)

I think it's by design. I think this is a quality issue, the behaviour here was probably decided by the developer writing the functionality and not questioned. I'm a little shocked myself nobody from the team has read or commented on this case since I raised it...

[andrewslavin]

Sorry @spragchris, I saw you closing and re-opening the issue, so presumed you were on the project team. I did not notice that you were also the person who'd raised it.

This behaviour is quite dangerous, almost resulted in a bug for us. Noticed accidentally while testing. Ended up using bar-delimited strings instead of arrays.

Can someone from the project team please comment on the issue - is this a bug or by design? If by design - why? Thank you.

[spragchris]

@andrewslavin Hah no sorry, I accidentally closed the issue, then re-opened it.

[dpsenner]

Seconding that this behavior is strange at best. I would expect lists from different files to be union'ed. This would have the effect that the initial example yields a list with three users, the first and the last having both the name "Administrator".

[oliverjanik]

Any update here? The configuration binder has bunch these WTFs. Really violates principle of least surprise.

[n-shay]

Can someone at MS provide us with a bit of clarity whether there is any intention to improve this weird behavior?
It doesn't follow common sense, or patterns in other frameworks.

If someone have any good workaround, Please share. As of right now, my workaround bypasses the extension's PhysicalFileProvider (or AddJsonFile) which is not the desired outcome.

[CliffordSpainhour]

In my particular case I would want union of the arrays, but I can also see an argument for replacing the entire array.

[maryamariyan]

The discussion here is around understanding how configuration should behave (intentional or not), when there is a list of collections with the same name (should there be a union/merge/replace in this case).

The repro code is for json provider but it would also be worth putting together how the other configuration providers also behave in this case.

cc: @davidfowl @safern @HaoK

[HaoK]

RE The merging behavior, this is the only way you can compose multiple providers where they each add values to collections, which was the main focus for the binder. It doesn't seem unreasonable to add additional binder options to control this as long as they are off by default/opt-in.

[maryamariyan]

It doesn't seem unreasonable to add additional binder options to control this as long as they are off by default/opt-in.

Thanks @HaoK. Therefore, closing as dupe of https://github.com/dotnet/runtime/issues/36569 since that issue covers API suggestion to add a BinderOptions flag, to allow for this opt-in behavior.