Wot-thing-description: JSON Schema validation issue wr.t. "format": "iri-reference"

[danielpeintner]

I tried to use the JSON Schema file we have in the TD and used it with https://jsonschemalint.com.

At first the JSON schema seems fine but once I actually try to check a TD instance I get the following message

unknown format "iri-reference" is used in schema at path "#/definitions/anyUri"

Could it be the that it has been renamed to "uri-reference" instead?
see https://github.com/json-schema-org/json-schema-spec/issues/355

Once I change it to "uri-reference" everything works as expected.

[egekorkan]

The correct modeling is indeed iri-reference since but even the most popular json schema validation libraries do not support it as noted in the TD spec:

Please note that some JSON Schema validation tools do not support the iri string format.

Playground uses the same validator as in jsonschemalint.com and there is the following note: https://github.com/thingweb/thingweb-playground/blob/master/playground-core/td-schema.json#L8

https://www.jsonschemavalidator.net/ does support iri-reference though :) Also, here is an example validation: https://www.jsonschemavalidator.net/s/pk2OEcK9

[danielpeintner]

https://www.jsonschemavalidator.net/ does support iri-reference though :) Also, here is an example validation: https://www.jsonschemavalidator.net/s/pk2OEcK9

I am not sure whether https://www.jsonschemavalidator.net/ actually supports (or ignores it) given that you can use "format": "xxx-uri-reference" also without any error.

According to http://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.7.3.5 both is supported....
@handrews may I ask you what is the term to be used?

My goal is to have something that just works without any tweaks like this

[egekorkan]

I am not sure whether https://www.jsonschemavalidator.net/ actually supports (or ignores it) given that you can use "format": "xxx-uri-reference" also without any error.

It actually supports it and JSON Schema spec says that if the format is unknown, the validation should pass. See here: https://json-schema.org/draft/2019-09/json-schema-validation.html#rfc.section.7

To see that it works, you can have an é in one of the href values. If you change iri-reference to uri-reference, the validation will fail

[handrews]

TL;DR: format has never worked reliably and the real solution from 2019-09 onwards is extension vocabularies, which can be required through declaration in the meta-schema. This allows schema authors to ensure consistent handling of schemas, or if that is not possible, to ensure that a clear error is provided rather than silently failing.

format is the biggest mess in JSON Schema, sadly. Prior to 2019-09, support for it was optional, on by default if present, but could be turned off by whoever is running the validator. There was no way for a schema author to control the behavior at all.

However, the keyword has _never_ been consistently implemented, and when it is "supported" the "support" is usually best-effort, and varies depending on the format value. e.g. date-time is usually pretty solid, but email is notoriously variable in implementation. The most popular Python implementation validates format: email by checking for an @ sign, and doesn't do any further checking. The part of the spec that said that if you supported any of the format values in the spec you needed to fully support all of them has pretty much always been ignored.

And, as noted, unknown format values are supposed to pass, so you end up with things like iri-reference that should fail a given value, but instead pass because the implementation doesn't know what it is, even if it is supposed to.

In my view, format is not salvageable. Shoving all sorts of random junk, some of which is challenging to implement (it's challenging to even get people to agree on what constitutes a "valid email address"), spanning a wide range of unrelated use cases (date and time, network identifiers, regular expressions, ...) _and_ declaring it to be extensible with no mechanism of control over behavior available to schema authors is never going to work.

In 2019-09 we declared that by default format does not behave as a validation keyword anymore. There are some complicated things left in the spec to make it possible to retain the former (unreliable) behavior. And an option to require full and strict support, which almost no one has ever provided. You can collect format as an annotation and implement your own validation based on it, at which point you will at least get consistent results.

This state of affairs will be preserved while we encourage vocabulary development. If readily available vocabularies taking over all of format's functionality become popular, I hope to remove the keyword. Although that is my personal opinion and not an official decision.

[danielpeintner]

It feels a bit strange to me that we have to tweak our own JSON schema so that it works in our tools (e.g., playground).

My proposal would be to modify the JSON schema like @egekorkan did in the playground

        "anyUri": {
            "type": "string",
            "$comment": "Until iri-reference is supported by ajv, it will stay as uri-reference in playground",
            "format": "uri-reference"
        }

OR remove the format entirely (since @handrews mentions "format has never worked reliably") and mention our intent as comment.

        "anyUri": {
            "type": "string",
            "$comment": "should be format iri-reference however iri-reference is NOT supported properly by many tools and causes problems"
        }

Any thoughts?

[egekorkan]

We can modify the schema easily but should we leave anyURI as the type of href in the TD? So our schema would be less accurate according to the spec. I think it is fine but other opinions are appreciated

[danielpeintner]

We can modify the schema easily but should we leave anyURI as the type of href in the TD? So our schema would be less accurate according to the spec.

Yes, that would be my understanding/proposal also.

[relu91]

We can modify the schema easily but should we leave anyURI as the type of href in the TD? So our schema would be less accurate according to the spec. I think it is fine but other opinions are appreciated

Another option is to use pattern. Is gonna be fun to define a regex validation for IRIs (if it is possible at all). 😄

Edit
For reference, this is the IRI-reference grammar

[egekorkan]

FYI, Newtonsoft JSON Schema validator has the following test cases for IRI: https://github.com/JamesNK/Newtonsoft.Json.Schema/blob/4c98626aa1cd00844e5475861fc00700c0a122ff/Src/Newtonsoft.Json.Schema.Tests/Specs/tests/Draft7/optional/format/iri-reference.json

[relu91]

FYI, Newtonsoft JSON Schema validator has the following test cases for IRI: https://github.com/JamesNK/Newtonsoft.Json.Schema/blob/4c98626aa1cd00844e5475861fc00700c0a122ff/Src/Newtonsoft.Json.Schema.Tests/Specs/tests/Draft7/optional/format/iri-reference.json

Are you saying that we could just create a regex that satisfies that test case? it might work.

[egekorkan]

Well in theory yes. I actually wanted to find how they do the validation but coulnd't find it but found the test cases instead ^^.

[relu91]

Ah, thanks! I found the regex (pretty long :D):

https://github.com/JamesNK/Newtonsoft.Json.Schema/blob/4b598484926626e81b7adffa30d221a95e823fef/Src/Newtonsoft.Json.Schema/Infrastructure/FormatHelpers.cs#L14

[relu91]

I got it kinda work: https://www.jsonschemavalidator.net/s/UccJlKEn . We still have two problems:

1. It seems not to work with other not Latin characters (i.e. this http://ƒøø.ßår/?∂éœ=πîx#πîüx is not accepted). Newtonsoft used RegexOptions.CultureInvariant but we don't have it in normal regex
2. The error is super cryptic: https://www.jsonschemavalidator.net/s/oW6mG5NA. It seems that some JSON schema processors support custom error messages but it didn't work with the one that I used.

[danielpeintner]

_my personal view_: It is not worth the effort to come up with a super complex regex to check for iri entries.
I would rather be more permissive than to restrictive ...

[sebastiankb]

from today's TD call:

• for the TD 1.1 version we should not remove the term 'format'
• however, we should include a note that 'format' is not widely implemented and may be replaced by another mechanism that will be introduced by JSON Schema

[sebastiankb]

done

[egekorkan]

I think a side issue of this issue is fixed but should we continue using iri-reference in our JSON Schema?

[danielpeintner]

My opinion (as mentioned in a previous comment) is that we should remove "format": "uri-reference" all together and being more permissive.

@sebastiankb states

for the TD 1.1 version we should not remove the term 'format'

I think removing format is not an issue since we are just more permissive which should not cause any problem..

[egekorkan]

Small update on this: https://github.com/luzlab/ajv-formats-draft2019 has support for iri-reference but for some reason, it iterates through registered URI schemes at IANA so if one uses a URI scheme that is not registered, like MQTT, it fails validation.

[FadySalama]

Small update on this: https://github.com/luzlab/ajv-formats-draft2019 has support for iri-reference but for some reason, it iterates through registered URI schemes at IANA so if one uses a URI scheme that is not registered, like MQTT, it fails validation.

It also has a minor bug with regards to iri-references that include mailto scheme that would cause the validator to crash (see issue luzlab/ajv-formats-draft2019#9). I made a PR luzlab/ajv-formats-draft2019#10 to fix it, but development seems dead.

[FadySalama]

Small update on this: https://github.com/luzlab/ajv-formats-draft2019 has support for iri-reference but for some reason, it iterates through registered URI schemes at IANA so if one uses a URI scheme that is not registered, like MQTT, it fails validation.

It also has a minor bug with regards to iri-references that include mailto scheme that would cause the validator to crash (see issue luzlab/ajv-formats-draft2019#9). I made a PR luzlab/ajv-formats-draft2019#10 to fix it, but development seems dead.

The issue is now fixed and @egekorkan is working on getting mqtt recognized by the validator.

[danielpeintner]

Thanks!
Note: This fixes the problem for upcoming AJV versions. The issue persists for older AJV versions (that may or cannot be updated)

[egekorkan]

The repository linked above is actually not dead at all and my PR got merged in another repository that triggered a version update here: https://github.com/luzlab/ajv-formats-draft2019/pull/10