Azure-functions-core-tools: better error message if the CLI sees nested values in local.settings.json

[alemag1986]

Hey guys,

I'm still having issues debugging locally with the local.settings.json and AzureWebJobsStorage - I'm not sure if that one is the real issue or the error is a different one but the exception is not right.

I'm running VS 2017 Enterprise Edition version 15.3.2
I have the last versions of Azure Functions CLI (and tried with the 2 latest: 1.0.11075.0, 1.0.11015.0)

This is how my local.setting.json looks like

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsDashboard": "<changed by alemag>",
    "AzureWebJobsStorage": "<changed by alemag>", 
    "commservicetopic": "<changed by alemag>",
    "EmailProvider": "<changed by alemag>",
    "SparkPostClientId": "<changed by alemag>",
    "SMSProvider": "<changed by alemag>",
    "ClickSendClientId": "<changed by alemag>",
    "ClickSendKey": "<changed by alemag>",
    "PushProvider": "<changed by alemag>",
    "MSNotificationHub": {
      "HubName": "<changed by alemag>",
      "ConnectionString": "<changed by alemag>"
    }
  }
}

*values are not encrypted

And this is how function.json looks like:

{
  "generatedBy": "Microsoft.NET.Sdk.Functions-1.0.0.0",
  "configurationSource": "attributes",
  "bindings": [
    {
      "type": "serviceBusTrigger",
      "connection": "commservicetopic",
      "topicName": "commservicetopic",
      "subscriptionName": "PushSub",
      "accessRights": "listen",
      "name": "message"
    }
  ],
  "disabled": false,
  "scriptFile": "..\\bin\\CommService.dll",
  "entryPoint": "CommService.MobilePush.Run"
}

Things that I tried:

• remove the Azure.Functions folder from C:\User\\AppData\Local.....
• run it from the command line as well as VS... no luck.
• put the local.setting.json in every possible debug/release/bin folder nothing.
• adding an applicationsettings.json and the Azure-Cli detected the 2 files and warned that application settings file will be ignored and local.setting.json was going to be used.

The function is triggered by a Service Bus (commservicetopic from local.settings.json key-values pairs).

Function is working perfectly fine in Azure Cloud with same endpoints for AzureWebJobsStorage and ServiceBus.

I was able to debug the function back in VS 2017 Preview 15.3.

Thanks!

[ahmelsayed]

@alemag1986 sorry for the delay.

This is because of this value

"MSNotificationHub": {
  "HubName": "<changed by alemag>",
  "ConnectionString": "<changed by alemag>"
}

Values collection is expected to be a Dictionary<string, string> and it's failing to parse it and returning an empty dictionary. I think parsing it as a JObject and displaying an error for any non-string values would be more correct here.

I know it's a bit confusing with the .NET Core appsettings.json, but this file local.settings.json is meant to just be a local equivalent to the Azure App Settings which are just key-value pairs.

You can test that locally by running func settings list in that directory. It should print empty values and connection strings. removing that MSNotificationHub property should get func settings list to work.

[alemag1986]

@ahmelsayed thank you very much for your answer - that finally worked!!! 👍

However, running the function from VS I had to copy the local.settings.json in every single bin/debug folder...I'll find out where really it is missing and let you know guys.

[alemag1986]

Update: I started playing with the output files and actually after removing obj and bin folders - running from VS started working properly. The configuration file local.settings.json is in just one place under: \bin\Debug\net461

Thx!

[riussi]

I am still having the same issue that running an Azure Function app from Visual Studio 2017 Community complains about: "Missing value for AzureWebJobsStorage in local.settings.json. This is required for all triggers other than HTTP. You can run 'func azure functionapp fetch-app-settings ' or specify a connection string in local.settings.json.:

Even though I have fetched and have the value for it.

[riussi]

The solution was to right-click on local.settings.json, go to properties, change "Copy to Output directory" from "Do not copy" to "Copy always". Now the CLI picks up the settings when running from within Visual Studio 2017.

[andrecarlucci]

I'm having the same problem but even with the "Copy always" setting I get the same error. Any thoughts?

[andrecarlucci]

Got it! If you use settings with more than one level in local.settings.json, it ignores the whole "Values" section and gives you this error.
Ex:
This is OK:

 "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "AzureWebJobsDashboard": "UseDevelopmentStorage=true",

    "Foo:Url": "http://localhost:3030",
    "Foo:Token": "token"
  }

But this generates an error:

 "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "AzureWebJobsDashboard": "UseDevelopmentStorage=true",

    "Foo":  {
        "Url" : "http://localhost:3030",
        "Token": "token"
      }
  }

[ferantivero]

for those who are not behind VS, the CopyToPublishDirectory is what did the trick in my environment:

Azure Functions Core Tools (2.0.1-beta.31)
Function Runtime Version: 2.0.11888.0

<None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
 </None>

with the following local.settings.json

{
    "IsEncrypted": false,
    "Values": {
        "AzureWebJobsStorage": "DefaultEndpointsProtocol=...",
        "AzureWebJobsDashboard": "",
        "FUNCTIONS_WORKER_RUNTIME": "dotnet"
    }
}

[MisinformedDNA]

This needs to fail with a better error message. Why would anyone expect valid, legal JSON to cause problems? If you aren't going to support nested JSON, then return an warning saying as much, highlighting which properties are in violation. Key-values that are the expected string/string should still just work fine.

[sinapis]

Since settings.json does not support nested JSON, can you please describe the recommended alternative to storing nested JSON in configuration?

[robinmanuelthiel]

Just found out, that

{
  "Something__ConnectionString": "Lorem Ipsum"
}

also does not work due to the double underscore (__).

[benbelow]

@alemag1986 sorry for the delay.

This is because of this value

"MSNotificationHub": {
  "HubName": "<changed by alemag>",
  "ConnectionString": "<changed by alemag>"
}

Values collection is expected to be a Dictionary<string, string> and it's failing to parse it and returning an empty dictionary. I think parsing it as a JObject and displaying an error for any non-string values would be more correct here.

I know it's a bit confusing with the .NET Core appsettings.json, but this file local.settings.json is meant to just be a local equivalent to the Azure App Settings which are just key-value pairs.

You can test that locally by running func settings list in that directory. It should print empty values and connection strings. removing that MSNotificationHub property should get func settings list to work.

@ahmelsayed I think this really needs some better warning on failure - I've just spent a lot of time trying to work out why my application settings weren't being read, and I don't think it's unreasonable to assume that valid JSON, in the format recommended elsewhere by Microsoft for .NET Core app settings, silently continues without loading any settings.

[ggirard07]

@ahmelsayed Any plan to support the nested object model in local.settings.json such as configurations in appsettings.json are in .Net Core?

We are sharing a common model for IOptions between API/Microservice/MVC and it is very disturbing to have azure functions behaving differently from any other .Net Core projects.... I understand I can write environment variables in local.settings.json as full string like MyObject:MySubObject:MyProp, but this becomes quickly complicated to maintain when you do refactoring.

I would probably prefer to create my own appsettings.json file, but as the IConfiguration service is already injected by the SDK, I have no entry point to customize it. I do not want to replace it as I might easily break injection required for the function bindings.

[ahmelsayed]

appsettings.json is problematic for any language that's not .NET. Also today the cli converts the settings into Azure App Settings, which an appsettings.json would further complicate since the runtime doesn't support loading one, and the cli is not in the picture when running on Azure.

I don't know what the right solution here so that it would work end to end. I think the runtime should handle appsettings.json if it's present.

[ggirard07]

Why don't you simply provide a way to customize the Configuration object on the .Net platform? Is this because you use DryIoc instead of Microsof.Extensions.DependencyInjection?

That way we can just continue to use appsettings.json for .Net projects. Other language can use their own appsettings.json equivalent. If they don't have any, or don't want to use it, then fallback to local.settings.json with the way it works today.

[brettsam]

@ggirard07 -- here's one reason why it's not quite that simple: https://github.com/Azure/azure-functions-host/issues/4464#issuecomment-513017446.

We think we may just add default support for appsettings.json (and the environment-specific files), which we think will cover 90+% of the requests to manipulate IConfiguration. We'll obviously need to make sure this works nicely with both the CLI and when deployed.

I've created this issue to track that: https://github.com/Azure/azure-functions-host/issues/4761.

@ahmelsayed -- sounds like the remaining issue here may be a better error message if the CLI sees nested values in local.settings.json? I'm going to move this over to the CLI repo in that case.

[Patrickkk]

Since settings.json does not support nested JSON, can you please describe the recommended alternative to storing nested JSON in configuration?

what i do now is use an additional local configuration file and loading the configuration myself (combined with DI)

private IConfigurationRoot GetConfiguration()
{
            var binDirectory = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
            var rootDirectory = Path.GetFullPath(Path.Combine(binDirectory, ".."));

            return new ConfigurationBuilder()
             .SetBasePath(rootDirectory)
             .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
             .AddJsonFile("local.settings.json", optional: true, reloadOnChange: true)
             .AddEnvironmentVariables()
             .Build();
}

[jnhaffey]

Also found out that arrays are not supported

...
    "MonthlyIntervals": [3, 6, 12, 18, 24]
...

[aliarias]

Any updates on this?

[ahmedelnably]

Closing this as obselete