Rocket: Default impl of FromForm could automatically map camel case to snake case

[lholden]

We are currently building out a rocket app that twilio communicates with to handle automated phone tree stuff. The main problem we have is that the query params twilio uses are all in camel case. The convention for field names in rust is snake case... and so, the only way we can take a FromForm for this is to manually implement the FromForm method and map each field directly.

It would be great if rocket supported either automatic mapping of camel case query params to snake case, or if it provided annotations on the fields to specify the name of the query param it is associated with (Such as how serde supports for serialization/deserialization).

So ex:

#[derive(FromForm)]
struct MyParams {
  one_field: String,
  two_field: String,
}

would work with query params: OneField and TwoField.

Or using custom field attributes:

#[derive(FromForm)]
struct MyParams {
  #[rocket(rename = "OneField")] 
  one_field: String,
  #[rocket(rename = "TwoField")] 
  two_field: String,
}

I think supporting a feature like one of the above will be important for Rocket as other people start to integrate with external services like twilio.

Thanks!

[SergioBenitez]

This is something I'd like like to have as well.

There are couple of things you can do now to get this functionality. First, as I'm sure you know, you could simple name the fields with the TitleCase/camelCase version and then use #[allow(non_snake_case)] on the struct.

More interestingly, you can create a custom FormForm implementation for the twilio form. This would look something like:

#[derive(Default)]
struct TwilioForm {
    one_field: String,
    two_field: String,
}

impl<'f> FromForm<'f> for TwilioForm {
    type Error = TwilioError<'f>;

    fn from_form_items(form_items: &mut FormItems<'f>) -> Result<Self, Self::Error> {
        let mut twilio_form = TwilioForm::default();
        for (key, value) in form_items {
            match key {
                "OneField" => twilio_form.one_field = String::from_form_value(value),
                "TwoField" => twilio_form.two_field = String::from_form_value(value),
                _ => return Err(TwilioError::UnknownField(key))
            }
        }

        Ok(twilio_form)
    }
}

Which you'd then use as:

#[post("/twilio/submit", data = "<form>")]
fn handler(form: Form<TwilioForm>) {
    ...
}

As far as the specific request goes, what do you think about this syntax?

struct MyParams {
    #[field("OneField")] 
    one_field: String,

    #[field("TwoField")] 
    two_field: String,
}

We could also allow more than one source field name:

struct MyParams {
    #[field("OneField", "oneField")] 
    one_field: String,

    #[field("TwoField", "twoField", "twofield")] 
    two_field: String,
}

[mehcode]

@SergioBenitez Maybe I'm missing something obvious but, moving forward, shouldn't we deprecate/remove FromForm and instead allow direct use of serde::Deserialize?

#[derive(Default, Deserialize)]
struct TwilioForm {
    #[serde(rename = "oneField")]
    one_field: String,
}

---

As to your specific suggestion, I'd rather rocket namespaced (as serde does) the non-obvious decorators (obvious ones being HTTP verbs).

struct MyParams {
    #[rocket(field("OneField", "oneField"))] 
    one_field: String,
}

[SergioBenitez]

I would love to use Serde here, but unfortunately, and I've talked to @dtolnay about this, it wouldn't be possible to use serde and allow custom FromFormValue implementations for custom validation.

[SergioBenitez]

Actually, I think simply being case and underscore agnostic might be the right thing here. So a field named a_b would match: a_b, A_B, A_b,a_B, ab, AB, Ab, and aB.

[lholden]

Sounds reasonable. What would happen on conflicts? Like... there is a field ab and a field a_b... and a param A_B, and a param AB?

[SergioBenitez]

@lholden Since I posted that, I've reconsidered. Instead, I think we should be camelCase, TitleCase, and snake_case agnostic. Thus, ab_cd_e, AbCdE, abCdE would match each other, but they wouldn't match abcd_e, for instance. Other matches are Ab_cdE, Ab_Cd_E, ab_CdE and variations thereof.

It will be a compile-time error if more than one struct field maps to a "normalized" version. In other words, you cannot have fields "ab_cd" _and_ "abCd". If a form contains multiple fields that match a single name, then they'll both map to the same struct field, just as if they had the same name. We don't do anything interesting in this case now, we just keep the last one. Ideally, we'd be able to group them into a Vec or so. #205 is tracking this.

[skinner]

Would being agnostic mean that it would accept multiple case/underscore versions of the field name from clients? Like, client A sends one_field and client B sends oneField and they both get accepted and mapped to the same struct field?

I'm all for allowing for the struct field name to differ from the JSON attribute, but I think it should accept only one case/underscore version in the JSON. In my mind, attribute names are a place to be strict, and I don't think I'm alone, e.g.
http://jsonapi.org/format/#document-member-names

edit: to clarify, I mean that IMO each struct field should only map to/from one attribute string. But different routes (or different structs, or different struct fields?) should be able to make different choices about camel/title/snake case.

In addition to (or instead of) the per-field attribute suggested above, maybe another option is to have a per-struct attribute that specifies a mapping to/from camel or title case?

[SergioBenitez]

I think the only reasonable, flexible thing to do here is to have a per-field attribute:

#[derive(FromForm)]
struct MyStruct {
    #[rocket(field = "myField")]
    my_field: u32
}

This says that the name of the field my_field in the form is myField; this is the _only_ name that will match. In particular, this means that the following is valid:

#[derive(FromForm)]
struct MyStruct {
    #[rocket(field = "myField")]
    my_field: u32,
    #[rocket(field = "my_field")]
    other_field: u32
}

And the following is invalid and will fail at compile-time:

#[derive(FromForm)]
struct MyStruct {
    #[rocket(field = "someField")]
    my_field: u32,
    #[rocket(field = "someField")]
    other_field: u32
}

[echochamber]

@SergioBenitez Would it be required for every field, or only fields that deviate from the default?

[SergioBenitez]

@echochamber Only fields that deviate from the default. Fields without the attribute will match exactly as written.

[mehcode]

@SergioBenitez 👍

Could we have parity with serde and use #[rocket(rename = "someField")] ?

[SergioBenitez]

@mehcode Frankly, I find "rename" really confusing. It makes it seem like it's renaming the structure's field when it's not; you still access the field with the same name. What it _really_ does is tell the library that the external name is something else. I'd like for the name to imply this more directly.

[mehcode]

Frankly, I find "rename" really confusing. It makes it seem like it's renaming the structure's field when it's not [...]

Look from the point-of-view of the external API. It is indeed _renaming_ the field received from the client.

---

I don't really care for #[rocket(field = "someField")] as to me it doesn't communicate enough.

Some other random ideas (this helps me at least get a feel for an API and whether a name might fit into the API space better):

#[rocket(name = "someField")] - Define the name of this field in the (de-)serialization process (alternative to rename/field).

#[rocket(path = "some.field.path") - Define the path of the field.

#[rocket(raw)] - Mark a field as "raw". Works only with the &str / String types. Possible alternative to RawStr.

#[rocket(with = "method_name")] - Possible alternative to https://api.rocket.rs/rocket/request/trait.FromFormValue.html (both ways would exist to be clear) for simple validation and easier impl for external types. Serde supports this type of thing.

#[rocket(default)] - Don't die if the value is not present and fill with Default::default

---

I think after that thought process I like #[rocket(name = "...")] the best.

[lholden]

Maybe: #[rocket(as = "someField")]

Though honestly, I don't mind 'field' or 'param'
#[rocket(param = "someField")]

or maybe from:
#[rocket(from = "someParam")]

[SergioBenitez]

This has been implemented in 7c19bf784d64936f90b1a61f86430d5d360072f0 via the #[form(field = "name")] attribute. See below for a complete example:

#[derive(FromForm)]
struct Form {
    single: usize,
    #[form(field = "camelCase")]
    camel_case: String,
    #[form(field = "TitleCase")]
    title_case: String,
    #[form(field = "type")]
    field_type: isize,
    #[form(field = "DOUBLE")]
    double: String,
}

This structure will parse for a form string that looks like:

single=100&camelCase=helloThere&TitleCase=HiHi&type=-2&DOUBLE=bing_bong