Azure-cli: az monitor diagnostic-settings create - Troubleshooting JSON issues

[cj-anthony]

az monitor diagnostic-settings create - Troubleshooting JSON issues

Issue:

When trying to create a diagnostic setting on a Cosmos DB there does not seem to be any valid JSON for the --logs flag. Even json that has been exported from a working az monitor diagnostic-settings show does not work. Is there a way to get valid JSON for this command? I also want to get the --metrics flag working but have removed it from my CLI just so I can get this one thing working before proceeding.

Environment summary

Install Method: pip
CLI version:
azure-cli (2.0.27)

acr (2.0.21)
acs (2.0.26)
advisor (0.1.2)
appservice (0.1.26)
backup (1.0.6)
batch (3.1.10)
batchai (0.1.5)
billing (0.1.7)
cdn (0.0.13)
cloud (2.0.12)
cognitiveservices (0.1.10)
command-modules-nspkg (2.0.1)
configure (2.0.14)
consumption (0.2.1)
container (0.1.18)
core (2.0.27)
cosmosdb (0.1.19)
dla (0.0.18)
dls (0.0.19)
eventgrid (0.1.10)
extension (0.0.9)
feedback (2.1.0)
find (0.2.8)
interactive (0.3.16)
iot (0.1.17)
keyvault (2.0.18)
lab (0.0.17)
monitor (0.1.2)
network (2.0.23)
nspkg (3.0.1)
profile (2.0.19)
rdbms (0.0.12)
redis (0.2.11)
reservations (0.1.1)
resource (2.0.23)
role (2.0.19)
servicefabric (0.0.10)
sql (2.0.21)
storage (2.0.25)
vm (2.0.26)

Python location '/usr/bin/python'
Extensions directory '/home/tmac/.azure/cliextensions'

Python (Linux) 2.7.12 (default, Nov 20 2017, 18:23:56)
[GCC 5.4.0 20160609]
Shell Type: Bash on Windows

Problem Details

I've replaced some personally identifiable information in this section.

Command:
az monitor diagnostic-settings create
--name DBDiagnostics
--resource /subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/tmacdeveastus/providers/Microsoft.DocumentDB/databaseAccounts/mytestdb
[email protected]
--workspace mytestworkspace
--resource-group mytestdb

Logs JSON:

"logs": [
{
"category": "DataPlaneRequests",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
},
{
"category": "MongoRequests",
"enabled": true,
"retentionPolicy": {
"days": 0,
"enabled": false
}
}
],

Errors:

Extra data: line 1 column 9 - line 18 column 5 (char 8 - 319)

[tjprescott]

@troydai

[troydai]

If the data you show is the content of the entire file. it is not a valid JSON. A JSON can be a string, number, array, or dictionary/map. The example you gave is not any of the types. I assume you want to pass a dictionary, in this case it should be:

{
    "logs": [
    ...
    ]
}

[cj-anthony]

Thanks Troy, I originally had the json in a valid structure as you suggested but I got the following error when attempting the same command.

Unable to build a model: Cannot deserialize as [LogSettings] an object of type , DeserializationError: Cannot deserialize as [LogSettings] an object of type
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/knack/cli.py", line 194, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/local/lib/python2.7/dist-packages/azure/cli/core/commands/__init__.py", line 331, in execute
six.reraise(sys.exc_info())
File "/usr/local/lib/python2.7/dist-packages/azure/cli/core/commands/__init__.py", line 304, in execute
result = cmd(params)
File "/usr/local/lib/python2.7/dist-packages/azure/cli/core/commands/__init__.py", line 168, in __call__
return super(AzCliCommand, self).__call__(args, kwargs)
File "/usr/local/lib/python2.7/dist-packages/knack/commands.py", line 109, in __call__
return self.handler(args, *kwargs)
File "/usr/local/lib/python2.7/dist-packages/azure/cli/core/__init__.py", line 348, in default_command_handler
result = op(command_args)
File "/usr/local/lib/python2.7/dist-packages/azure/cli/command_modules/monitor/operations/diagnostics_settings.py", line 24, in create_diagnostics_settings
return client.create_or_update(resource_uri=resource_uri, parameters=parameters, name=name)
File "/usr/local/lib/python2.7/dist-packages/azure/mgmt/monitor/operations/diagnostic_settings_operations.py", line 141, in create_or_update
body_content = self._serialize.body(parameters, 'DiagnosticSettingsResource')
File "/usr/local/lib/python2.7/dist-packages/msrest/serialization.py", line 470, in body
SerializationError, "Unable to build a model: "+str(err), err)
File "/usr/local/lib/python2.7/dist-packages/msrest/exceptions.py", line 48, in raise_with_traceback
raise error
SerializationError: Unable to build a model: Cannot deserialize as [LogSettings] an object of type , DeserializationError: Cannot deserialize as [LogSettings] an object of type

The JSON I provided is as follows:

{
  "logs": [
    {
      "category": "DataPlaneRequests",
      "enabled": true,
      "retentionPolicy": {
        "days": 0,
        "enabled": false
      }
    },
    {
      "category": "MongoRequests",
      "enabled": true,
      "retentionPolicy": {
        "days": 0,
        "enabled": false
      }
    }
  ]
}

[troydai]

Hi @cj-anthony , underneath the CLI it is the Azure Python SDK. The model of the log settings parameter here is the LogSettings (Link). It is a dictionary, instead of an array.

So it would look like:

{
  "category": "...",
  "enable": true,
  "retention_policy" : {
    ...
  }
}

The monitor API had changed recently, so each diagnostic setting needs a name and has one log setting. To apply multiple log settings you need to create multiple diagnostic settings.

Here's more detail: Azure Monitor: Enhanced capabilities for routing logs and metrics.

I would close this issue for now since it is not a bug. Please feel free to leave further comments or reopen if there are other unsolved issues. Thanks.

[jkalwar]

@troydai
Having the below JSON structures for the --logs flag also do not work :

{ 
  "category":"NetworkSecurityGroupRuleCounter",
   "enabled":true
}

OR

[
{ 
  "category":"NetworkSecurityGroupRuleCounter",
   "enabled":true
}
]

Error Message :

Unable to build a model: Cannot deserialize as [LogSettings] an object of type <
class 'dict'>, DeserializationError: Cannot deserialize as [LogSettings] an obje
ct of type

[troydai]

@tjprescott ?

@jkalwar I'm not on the project more, but @tjprescott can help you.

[AvinashReddyVaka]

hi @troydai , issue doesn't seemed to be fixed. i am trying to enable the boot diagnostic logs on ADF through terraform local exec and below is the az cli command that i am using. I get a log error message class 'str'>, DeserializationError: Cannot deserialize as [LogSettings] an objec
t of type something like ""

command that i am using:
az monitor diagnostic-settings create --name adflogs --resource /subscriptions/0000-0000-0000-0000000/resourceGroups/resource-group-eg-lhbl9sxx/providers/Microsoft.DataFactory/factories/datafactorysample --storage-account xxxxxxxxxxx --resource-group resource-group-eg-lhbl9sxx --logs '[{"category":"PipelineRuns","enabled":true}]'

any help/suggestion on this?? Thanks in advance

[AvinashReddyVaka]

@tjprescott, can you please check this issue?

[tjprescott]

@AvinashReddyVaka I can repro this issue on CMD.exe on Windows and it is a quoting issue. This works:

--logs "[{\"category\":\"PipelineRuns\",\"enabled\":true}]"

[AvinashReddyVaka]

Thank you so much @tjprescott . This is very helpful for our motto of "Everything as code"