Ajv: Foolproof behaviour for addSchema

[alexkuz]

What version of Ajv you are you using?

[email protected]

What problem do you want to solve?

In my project I'm using one ajv instance, adding schemas to it when it's needed. The problem is that api doesn't provide a way to know if the schema with given id is already added, and trying to add without checking leads to "schema already exists" error.

Previously I used getSchema to check if schema is already there, but now it fails in case it's referencing non-existent schema.

For example:

const SCHEMA1 = { $id: 'test1.json', allOf: [{ $ref: 'test2.json' }] };
const SCHEMA2 = { $id: 'test2.json' };

// Add only the first schema
ajv.addSchema([SCHEMA1]);

// Now if we try to add both schemas, it fails since the first one is already there
ajv.addSchema([SCHEMA1, SCHEMA2]);

// but if we try to check that SCHEMA1 is there with avj.getSchema, it also fails since the reference leads to SCHEMA2, that is not added yet
[SCHEMA1, SCHEMA2].forEach(schema => {
    if (!ajv.getSchema(schema.$id)) {
        ajv.addSchema(schema);
    }
})

What do you think is the correct solution to problem?

Maybe an option for addSchema to override existing schema or ignore it?

Will you be able to implement it?

Probably :)

[epoberezkin]

What is the use case? I mean, why can’t you just add all the schemas you have?

[alexkuz]

Basically, I have a component that renders something based on schemas

<Component schemaRef={schemaRef} schemas={schemas} />

It uses global ajv instance by default (using multiple instances would be too expensive), so if schema set changes (they are fetched asyncronously for instance), I have to add absent schemas to ajv.

Another good-enough solution for me would be adding ajv.hasSchema(id) (or something) method that just checks if there's a schema with that id. Just like return (id in this._schemas)

[epoberezkin]

Why can't you ship all your schemas in advance and add them all together? Unless they are really large, it can be more efficient (=performant) bundling them all with code rather than downloading on demand.

If you still want to do lazy loading it can be better using compileAsync method with loadSchema option - it would probably achieve what you do now in a more reliable way.

[epoberezkin]

https://github.com/epoberezkin/ajv#asynchronous-schema-compilation

[alexkuz]

Yes, I could do that (just wait until all relevant schemas are loaded before rendering), but that's not really the point, it's more of a state consistency. In terms of the component, if current state is valid, it shouldn't matter that the previous one wasn't. But I can't get from invalid ajv instance state to valid one since I can't even know what schemas are in there already.

I mean, if there's a method (addSchema) that can throw an exception, there should be a way to avoid that exception (like checking it's state with hasSchema), isn't it?

[limzykenneth]

For my use case (on the backend) I won't be able to know all the schemas that are available at the point of ajv instantiation. More precisely, I am pulling schema definition from a database and additional schemas can be added to the database at runtime. As such I am not able to use loadSchema because 1. the schema isn't available from a URI and 2. I won't know what to pull at instantiation time.

What I resolve to do is when I need a particular schema for validation purpose, I will pull it from the database then add it to the ajv instance with addSchema, so that I can refer back to it later in the instance and not storing the compiled function, but there is no way for me to know if it has already been added previously.

[epoberezkin]

loadSchema is the function that you should provide, it can load schemas from database, file system, whatever. All you need is to retrieve a schema for a given ID.

[limzykenneth]

I don't really understand, when is loadSchema called and how is the parameter passed to it provided?

loadSchema: asynchronous function that will be used to load remote schemas when compileAsync method is used and some reference is missing (option missingRefs should NOT be 'fail' or 'ignore'). This function should accept remote schema uri as a parameter and return a Promise that resolves to a schema. See example in Asynchronous compilation.

I'm assuming that loadSchema is called when compileAsync is called but compileAsync can only be called with a parameter that is already a schema object. So how could I retrieve a schema from database, compile it, then use it to validate data, while providing with a given ID and no existing schema object?

Edit: Ok nevermind, I just figured it out to just give an object in the form of

{
  $ref: schemaID
}

[epoberezkin]

Probably cleaner to have getSchemaAsync that accepts schema ID, but passing {$ref:id} works.

[epoberezkin]

Re adding hasSchema, my concern is that it will be confusing for users as the implication could be that if the schema is present, you can validate against it, which is not the case - there may be missing dependencies. getSchema provides a stronger guarantee that the schema is present AND you can validate against it.

For your use case, you could simply add hasSchema to Ajv prototype that returns false if getSchema returns undefined and true if it returns a function or if it throws missingRef exception. I still think compileAsync for lazy loading is a better approach.

I am still happy to have getSchemaAsync in addition to compileAsync - it will be consistent with having getSchema and compile.

[alexkuz]

Yes, for now I guess I'll stick with checking for ajv._schemas (I don't want to alter Ajv prototype in case the user provides his own ajv instance to component).
I'm not sure how async would fit there - may be it will work better, I'll think about that. Adding getSchemaAsync wouldn't hurt :)

[fgarcia]

I wrote a related issue some time ago. @alexkuz if you want an example of how I did modify ajv._schemas check my gist https://gist.github.com/fgarcia/081a77518f33747fa61612a124c82d0b from https://github.com/epoberezkin/ajv/issues/947

I no longer use that approach. I am still thinking about these two issues:

1. If the app has some type of progressive loading, the architecture should default to have async validations (since bundling all schemas is no longer an option)

2. The app needs some type of SchemaService object to manage the verbosity of managing schemas. In my case it was enforcing a proper long/url id, adding defaults such as additionalProperties:true, setting up my ci/cd to generate+publish the schemas, combine with https://quicktype.io/, etc.

In the past I wanted a better addSchema, but the more I rely on schemas, the more nuances there are in the architecture and deployment of my app. I think the problem is that there are too many concerns when using Ajv directly, and instead, it should be hidden inside a bigger custom service.