Ajv: Plugin for formats "iri", "iri-reference", "idn-hostname", "idn-email"

[epoberezkin]

[epoberezkin]

mentioned in docs, won't be part of Ajv, can be in a plugin

[fboutin-pmc]

Why is that?

[epoberezkin]

I think it’s better to implement as a plugin.

[fboutin-pmc]

On Ajv README file,

Ajv version 6.0.0 that supports draft-07 is released.

But IRIs are part of the spec, http://json-schema.org/latest/json-schema-validation.html#rfc.section.7.3.5

I mean, if it makes sense to implement IRIs as a plugin like you said, let's go. Until then, you cannot claim Ajv fully implement draft-07 because it is NOT.

[epoberezkin]

Formats are optional part of the spec, their support is not required

[fboutin-pmc]

Implementations MAY support the "format" keyword as a validation assertion. Should they choose to do so:
-they SHOULD implement validation for attributes defined below;

http://json-schema.org/latest/json-schema-validation.html#rfc.section.7.2

So because Ajv implements validation for attributes and that 7.3.5 defines an IRI, then you SHOULD implement it.

[epoberezkin]

1. My interpretation that partial support is better than no support
2. SHOULD according to https://tools.ietf.org/html/rfc2119 is the same as RECOMMENDED
3. I actually considered, quite a few times to move all formats to a plugin

Why don’t you make this plugin rather than trying to prove me wrong here?

[fboutin-pmc]

@epoberezkin I'm not trying to prove you wrong. I'm only trying to understand what's your thinking on the issue.

Off course, having something that is not feature-complete is better than nothing, we all agree on that. TBH you deserve to know that I think Ajv is a GREAT component, it works really well, is really fast and easy to use. IRI lack of support is literally the first limitation I encountered and sure I should contribute back to fix it.

I don't want you to take my questions and reasoning personally.

[fboutin-pmc]

Is there any way we could keep this issue (or a new one) open as a feature request - be it a plugin or a native support?

[epoberezkin]

@fboutin-pmc There is no need for it. I believe I've explained that I don't see it as the problem that this package itself should or would address. If you create a plugin that implements these formats, please submit a PR with the documentation change that mentions it.

[sebdeckers]

What I came up with for idn-hostname in an hour of research. HTH. YMMV.

Format plugin:

const regex = require('idn-allowed-code-points-regex')

module.exports.idnHostnameFormat =
function idnHostnameFormat (input) {
  if (typeof input !== 'string') return false
  if (input.length > 253) return false
  for (const part of input.split('.')) {
    if (part.length > 63) return false
    if (!regex.test(part)) return false
  }
  return true
}

Unit tests:

const test = require('blue-tape')
const { domainToASCII: toASCII } = require('url')
const { idnHostnameFormat } = require('../formats/idnHostnameFormat')

test('valid simple hostname', async (t) => {
  t.true(idnHostnameFormat('example.net'))
})

// https://www.verisign.com/en_US/channel-resources/domain-registry-products/idn/idn-policy/registration-rules/index.xhtml
// https://unicode.org/reports/tr46/
test('valid IDN hostname', async (t) => {
  for (const given of [
    'faß.de',
    'βόλος.com',
    'ශ්‍රී.com',
    'نامه‌ای.com',
    '스타벅스코리아.com'
  ]) {
    t.true(idnHostnameFormat(given))
  }
})

// https://tools.ietf.org/html/rfc5892
test('invalid IDN hostname', async (t) => {
  const given = '💩.example.net'
  t.false(idnHostnameFormat(given))
})

test('valid punycode hostname', async (t) => {
  const given = `${toASCII('💩')}.example.net`
  t.true(idnHostnameFormat(given))
})

test('hostname must be a string', async (t) => {
  for (const given of [123, null, true, {}, [], undefined]) {
    t.false(idnHostnameFormat(given))
  }
})

test('hostname label length exceeds 63', async (t) => {
  const given = 'w'.repeat(64) + '.example.net'
  t.false(idnHostnameFormat(given))
})

test('hostname length exceeds 253', async (t) => {
  const given = '.example.net'.repeat(100).substring(1)
  t.false(idnHostnameFormat(given))
})

[avanes]

Simple workaround for idn-email:

const ajv = new Ajv().addFormat('idn-email', /@/)

[joshuaavalon]

This package is using uri-js. uri-js supports IRI. Why don't we delegate validation to it?

Or this regex

[epoberezkin]

uri-js supports parsing IRIs - does it validate at all? You don’t need to validate in order to parse.

[carlo-quinonez]

@epoberezkin I see your point about validation vs parsing. Parsing is syntax and validation is semantics.Just because a string is syntactically a IRI/URI, doesn't mean it's valid.

I made a plugin ajv-formats as a first pass at support IRIs, idn-emails, and durations. I ended up validating IRIs by parsing with uri-js and then checking the scheme against a list of known schemes and that a path is present. If you check the tests in ajv-formats you can see some of the test cases.

I can go ahead and make a PR to update the readme if you'd like. And any suggestions for the plugin are welcome.

[epoberezkin]

I suggest to rename to something less generic if possible, as there are different possible "format" extensions and formats may be dropped out of core ajv at some point (and having core formats in ajv-formats makes more sense than extensions to core formats).

It seems like three formats should be in the plugin: idn-hostname and iri-reference should be added (they are related and it's actually in draft-07 that ajv implements) and duration seems out of scope (as it's neither related to idn nor in draft-07).

The reasonable name seems ajv-idn-formats.

[luzlab]

"duration" was added in the 2019-09 JSON schema release. Originally I picked ajv-draft2019-formats because I want to expand the library to support the complete draft2019 spec. But since the "idn" formats are a part of the 07 spec, I just dropped the "draft2019" since it applies to more than 1 draft.

I added 'iri-reference' and 'idn-hostname' but that's were things start to get tricky as far as how much to validate. I pushed a new version.

[luzlab]

I just published the new version that supports idn-hostname and iri-reference. I added an entry point for the international formats.

Take a look at the readme - comments and feedback are appreciated.

[Oloompa]

it would be usefull to add the plugin of @luzlab to plugin list (readme)

[luzlab]

I made a PR #1155 adding the plugin to the README. Also, I updated the idn-hostname validator to match the behavior of the stock hostname validator.

[luzlab]

I just published the ajv-formats-draft2019 npm module. The module under the old name ajv-formats is marked as deprecated.

[epoberezkin]

Thank you - I really appreciate it! So I guess I can split ajv formats in v7 beta and publish it there? I will reach out to npm support to ask what we should.

[luzlab]

@epoberezkin I just added you as a maintainer on the ajv-formats module on NPM. You can just leave me as a maintainer if you'd like. Otherwise I think we have to check with NPM support about transferring it... 🤦🏻‍♂️😩🙁

Fortunately I stuck to the interface in the README, so it's just a simple version bump to the new plugin.

Cheers

[epoberezkin]

@luzlab thank you. I will be splitting formats it in the next major version.