Sails: Nested JSON inputs validation

[Sytten]

Hi there!
I was wondering if it is possible to have nested parameters in inputs. Currently my code uses the type json, but that basically means no validation is done on parameters that are not in the first JSON level. I am pretty sure I am not alone wondering how to do proper validation of parameters with a nested structure.
Thanks!

[sailsbot]

@Sytten Thanks for posting! We'll take a look as soon as possible.

In the mean time, there are a few ways you can help speed things along:

• look for a workaround. _(Even if it's just temporary, sharing your solution can save someone else a lot of time and effort.)_
• tell us why this issue is important to you and your team. What are you trying to accomplish? _(Submissions with a little bit of human context tend to be easier to understand and faster to resolve.)_
• make sure you've provided clear instructions on how to reproduce the bug from a clean install.
• double-check that you've provided all of the requested version and dependency information. _(Some of this info might seem irrelevant at first, like which database adapter you're using, but we ask that you include it anyway. Oftentimes an issue is caused by a confluence of unexpected factors, and it can save everybody a ton of time to know all the details up front.)_
• read the code of conduct.
• if appropriate, ask your business to sponsor your issue. _(Open source is our passion, and our core maintainers volunteer many of their nights and weekends working on Sails. But you only get so many nights and weekends in life, and stuff gets done a lot faster when you can work on it during normal daylight hours.)_
• let us know if you are using a 3rd party plugin; whether that's a database adapter, a non-standard view engine, or any other dependency maintained by someone other than our core team. _(Besides the name of the 3rd party package, it helps to include the exact version you're using. If you're unsure, check out this list of all the core packages we maintain.)_
  

  ---

Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.

For help with questions about Sails, click here.

[georgeben]

Hey @Sytten you can write a custom validation function for nested inputs. For example:

inputs: {
    myCustomInput: {
      type: 'json',
      description: 'A user\'s email notification settings',
      required: true,
      custom: function (value) {
        return (
          _.isObject(value) &&
          _.isBoolean(value.name) &&
          _.isBoolean(value.age)
        );
      }
    }
  }

[Sytten]

Someone on gitter hinted that sails uses rttc and they support nested verification, is that something that would be possible here. Honestly this solution of a custom validation is bad in all ways. People usually quite complex JSON payloads in production with multiple levels and it needs to be clear what the validation does.

[georgeben]

True. Custom validation isn't the best option especially in cases where the JSON payload has a complex schema. However, I don't know of any other better way to do it.

[johnabrams7]

Thanks @georgeben, the custom validation is definitely worth a try for nested inputs! 👍
@Sytten Have you tried this out?

[Sytten]

I mean, it works and its not the first time I see the solution. Its just not a good one IMO for mainly three reasons:

1. You don't get any detail about which field was wrong
2. Its harder the more nesting you have
3. It reduces the code legibility and it becomes hard to maintain long term
   It all boils down to: it this production ready or not?

[Sytten]

I kept digging a bit in the sails code (which is very confusing due to the nesting of small dependencies). I found in the code of machine (on which action2 is based) that nesting is not supported natively:
https://github.com/node-machine/machine/blob/850e87de1200b3985c04dadbd3554eea3c6f8613/lib/private/normalize-argins.js#L150-L154

HOWEVER, you can see at this line https://github.com/node-machine/machine/blob/850e87de1200b3985c04dadbd3554eea3c6f8613/lib/private/normalize-argins.js#L237 that the machine passes the .type property to rttc. Turns out we can exploit that functionality and give it an object. This will tell rttc that it is a faceted dictionary and it will verify the nesting! the syntax is a bit different but its easy to read:

inputs: {
    user: {
      type: {
        id: 'number',
      },
    },
}

[Sytten]

@mikermcneil Do you have plans to actually support this usecase as a first class citizen like it is written in the comment? Just want to know before I start migrating my 300 routes :)

[Noitidart]

@Sytten oh wow thanks for sharing that @Sytten !! THats awesome!

[Sytten]

The only problem I see is that is does not support optional nested fields, but I guess this is better than nothing...

[nahanil]

The only problem I see is that is does not support optional nested fields, but I guess this is better than nothing...

This is why custom validation methods or individual parameters are still _possibly_ a better method (for now at least), even if they're more verbose and/or delicate/mysterious.
I had a tinker with the "faceted dictionary" style like above after the gitter conversation. Like you I quickly discovered you lose "optional" fields, but also the finer-grained control other 'fancy' validations give you with individual parameters (think isInteger or maxLength, etc). Ragequit at that stage, but commenting here to see if there's any more developments :wink:

[navicstein]

The only problem I see is that is does not support optional nested fields, but I guess this is better than nothing...

True, but assuming sails used something like babel-node we would have had an option of nullish operator and optional chaining, and that would be a great feat for us

[wojo1086]

@Sytten Couldn't you use a JSON schema and validate the passed in JSON against that schema?You could easily do that using the custom function property.

[Sytten]

Sure we could do that, then I would ask what is the point of using sails if anyway you have to use other frameworks to get basic stuff done? This is such a basic feature for any production ready system i just don't understand what the fuss is all about.

[wojo1086]

Another framework? You would be using a library to do the JSON validation. Also, using helpers, you could easily create a reusable function to test the JSON. In fact, I don't know of any framework that does JSON schema validation out of the box. I couldn't find it in AdonisJs, NestJs, Express, or even any of the frontend frameworks like Angular or Vue.

Seeing as how SailsJS is open-source, you could add the functionality yourself. 😄

[mikermcneil]

This is why custom validation methods or individual parameters are still possibly a better method (for now at least), even if they're more verbose and/or delicate/mysterious.

@nahanil @georgeben @wojo1086 👍

The custom validation rule is what I would recommend.

[Noitidart]

I kept digging a bit in the sails code (which is very confusing due to the nesting of small dependencies). I found in the code of machine (on which action2 is based) that nesting is not supported natively:
https://github.com/node-machine/machine/blob/850e87de1200b3985c04dadbd3554eea3c6f8613/lib/private/normalize-argins.js#L150-L154

HOWEVER, you can see at this line https://github.com/node-machine/machine/blob/850e87de1200b3985c04dadbd3554eea3c6f8613/lib/private/normalize-argins.js#L237 that the machine passes the .type property to rttc. Turns out we can exploit that functionality and give it an object. This will tell rttc that it is a faceted dictionary and it will verify the nesting! the syntax is a bit different but its easy to read:

inputs: {
    user: {
      type: {
        id: 'number',
      },
    },
}

Hey @Sytten do you have examples of inputs you made using this. I would love to play with it.

[Sytten]

@Noitidart Sorry I dont have access to this code anymore and I am not working on the contract :S
The company decided to remove sails from their codebase.

[Noitidart]

Aw darn. Hope they come back soon to sails!

This shape is working great for me, i just don't know how to do optional keys. like i wanted "order" to be optional.

    values: {
      required: true,
      type: [
        {
          id: 'number',
          order: 'number',
        },
      ],
    },

[Sytten]

Like I said earlier its not possible: https://github.com/balderdashy/sails/issues/6958#issuecomment-598512874