Powershell: Test-Json unable to parse MS ARM Schemas

[wembleyford]

I'm attempting to use test-json to validate some ARM template code I'm working up to push resources into Azure. The simplest test's I'm doing are to validate the paramter json files which provide parameter name/value pairs. The schema for which is provided by MS.

Unfortuntely, test-json appears to not be able to even parse the schema json, let alone validate my (valid) json parameter files.

What is particularly curious is that test-json $schema returns $true, but test-json -json "some-valid-json" -schema $schema errors, This might suggest that test-json is actually able to parse the json of the schema, but is not able to interpret the content correctly. This is rather disappointing given the schema is from Microsoft themselves. Naturally, I'm open to suggestions that this is a bug in the schema, rather than the cmdlet, but it also occurs with other ARM/Azure related schemas.

Steps to reproduce

Downloaded schema from https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json
And saved to schema.json - checked file and the json is formatted as expected, compressed onto a single line.

Attempted to test validity of basic parameter json:

$json = '{"$schema":"https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#","contentVersion":"1.0.0.0","parameters":{"foo":{"value":"bar"}}}'
$schema = get-content deploymentParameters.json
test-json -json $json -schema $schema

Expected behavior

True

Actual behavior

The error

Test-Json: Cannot parse the JSON schema.

Which seems odd given that

Test-json -json $schema

returns True as you might expect.

Environment data

PSVersion 7.0.3 on linux. Also same behaviour on 7.0.0

I've also noted that using the validation tool at https://www.liquid-technologies.com/online-json-schema-validator is able to validate the json+schema as expected but the validation tool at https://www.jsonschemavalidator.net/ is not.

[rjmholt]

Having tried this myself, the issue comes in two places:

• The ARM schemas are missing fields, so they have some invalid parts
• NJsonSchema, which is the 3rd party library Test-JsonSchema uses, does not tolerate partially valid schemas and throws at the first instance of a schema issue

You can test this with:

[NJsonSchema.JsonSchema]::FromUrlAsync('https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#').GetAwaiter().GetResult()

(In this particular case, notice there's no #/definitions/parameterValueTypes field in the schema at that URI)

[iSazonov]

I think it is better to ask in NJsonSchema repository.

[wembleyford]

Surely the cmdlet should be returning an 'False' if the schema is missing segments rather than an error message saying the schema cannot be parsed. If this is a bug in the underlying libraries as ISazonov suggests, I'm left wondering why Powershell is exposing the functionality if it's broken.

[iSazonov]

the cmdlet should be returning an 'False' if the schema is missing segments

I do not think we should ignore exceptions.

I'm left wondering why Powershell is exposing the functionality if it's broken.

We use the external library and we don't know whether there are bugs in the library.

If you have an interest please open new issue in NJsonSchema repository. We get the fix automatically for next version if it will be created.

[rjmholt]

Surely the cmdlet should be returning an 'False' if the schema is missing segments rather than an error message saying the schema cannot be parsed

The documented behaviour is for Test-Json to write an error and return false when the JSON being tested is invalid or does not comply with the schema. An invalid schema should at least do that, but it's not clear whether it should return anything (how can it give an answer when the schema it's comparing against is invalid) or whether the error it throws should be terminating or non-terminating.

If this is a bug in the underlying libraries as ISazonov suggests, I'm left wondering why Powershell is exposing the functionality if it's broken.

I don't think it's a bug in NJsonSchema, just how NJsonSchema works; it throws when given an invalid schema like the ARM schema you've pointed it at.

Test-Json should appropriately catch and wrap the error and either return false or throw a terminating error.

[iSazonov]

Test-Json should appropriately catch and wrap the error and either return false or throw a terminating error.

We catch all exceptions from NJsonSchema and wrap them as common PowerShell pattern.

[rjmholt]

Interesting, here's the relevant code:

https://github.com/PowerShell/PowerShell/blob/c22ccbebd2955ff67c03574ec26d252955e8ea6d/src/Microsoft.PowerShell.Commands.Utility/commands/utility/TestJsonCommand.cs#L163-L171

[rjmholt]

Ohhh, I've reread the original issue...

Test-Json -Json $schema returns $true. That's by design, since the schema is valid JSON.

test-json -json $json -schema $schema writes an error. Also by design, since the schema is broken; it's valid JSON but an invalid JSON schema.

[msftbot[bot]]

This issue has been marked as answered and has not had any activity for 1 day. It has been closed for housekeeping purposes.