Actix-web: Format of sequence query parameters?

[mandreyel]

Can I pass a list of things as a query parameter with actix? Trying to deserialize into a Vec<i32> gives me the following error response:

Query deserialize error: invalid type: string "[1,2]", expected a sequence

Which suggests to me it is possible, but the format is incorrect.

I tried ids=[1,2], ids={1,2}, escaping special chars ([]{},), all to no avail. Insight would be much appreciated.

BTW : I'm really happy actix is back! Thanks again!

[tyranron]

@mandreyel ids=[1,2] and ids={1,2} is not a standard HTTP way of doing this, so actix-web doesn't deal with them out-of-the-box.

The appropriate one would be: ids[]=1&ids[]=2.

But still, if you need to deal exactly with ids=[1,2], you have the following options:

1. Just accept ids argument as string and deserialize it manually.
2. Create you own type to represent such arguments and implement FromRequest for it. Now you can use this type as a handler function argument.

[mandreyel]

@tyranron Thanks for the response! I'll try ids[]=1&ids[]=2.

[tyranron]

@mandreyel nice! Please, do not forget to close this issue once/if you will think it's resolved for you.

[onelson]

I could be remembering this incorrectly, but I think there was a limitation with the dependency actix-web uses in the Query extractor that actually squashes repeated keys in the query string.

We ended up using something like this to convert Query extracted fields from Option<String> to a vec of T.

/// A trait to help convert things into vecs of different values, such as shotgun ids.
///
/// The initial use case is the converting of `Option<String>`s coming in
/// from query params as CSV.
trait AsCSV {
    fn as_csv<T>(&self) -> Result<Option<Vec<T>>, ()>
    where
        T: std::str::FromStr;
}

impl<S> AsCSV for Option<S>
where
    S: AsRef<str>,
{
    fn as_csv<T>(&self) -> Result<Option<Vec<T>>, ()>
    where
        T: std::str::FromStr,
    {
        match self {
            Some(ref s) if !s.as_ref().trim().is_empty() => {
                let mut acc = vec![];
                for s in s.as_ref().split(',') {
                    let item = s.trim().parse::<T>().map_err(|_| ())?;
                    acc.push(item)
                }
                if acc.is_empty() {
                    Ok(None)
                } else {
                    Ok(Some(acc))
                }
            }
            _ => Ok(None),
        }
    }
}

https://gist.github.com/onelson/1b1d899ad4c616e171c7172809961671

[KVNU]

@tyranron Thanks for the response! I'll try ids[]=1&ids[]=2.

I'd be interested whether or not this works for you. In my experience param[] is recognized to be a different name from param, meaning I'd get a Query deserialize error: missing field 'param' when using this notation. A while ago, I've tried every sequence notation for querystrings I could dig up, but to no avail.

I've had this issue - trying to deserialize sequence query parameters - since I've started to implement some network services with actix-web a few months back. From discussing this in the actix gitter channel, I gathered that it's a limitation of the serde_urlencoded crate, which is used to parse querystrings in actix-web. There's an alternative, serde_qs, which may enable parsing querystrings with nested parameters. There's an actix interop available, with a pending pull request for v2 support.

I never got it to work for me, though.

Since my requirement was to merely parse sequence parameters in simple notation - meaning just one parameter with comma separated string value -, I bit the bullet and simply implemented trivial functionality to parse it myself. I had spend way to much time on trying to appease the deserialization mechanism supplied with actix-web - also driven by the error messages which seemingly suggested that I just had to get the notation right - to keep on bothering.

[zero-systems]

Actix-web uses serde for deserializing queries, I can suggest you try to impelement your own serde Deserializer trait for query struct. It should fix your problem

[mandreyel]

@Decisional Unfortunately it didn't work for me either. I'll experiment with other solutions suggested by zero-systems and onelson.

[sify21]

What's the status of this issue? I don't see any issues related in serde_urlencoded.
It's a common use-case. For example when you want to delete multiple entries, the http spec don't encourage you to put data in the delete body, so you have to pass ids in query string.

[jplatte]

I don't think there's been any movement on this. serde_urlencodeds author has closed any issues that asked for this feature, because it would complicate the code and at least for foo=bar&foo=baz (as opposed to foo[]=bar&foo[]=baz) requires additional state tracking and possibly allocations.

However, this doesn't need to be supported in serde_urlencoded for it to be possible to parse query strings. You can declare req: HttpRequest as an extractor on any route and use serde_qs::from_str(req.query_string()) to deserialize the query string using serde_qs. serde_qs only has an implementation of the notation with the square brackets, but I've also forked serde_urlencoded into ruma_serde::urlencoded to support the other syntax. I could put it in a separate crate if there is interest in that.

[mandreyel]

It's true that no changes are required in serde_urlencoded for this. We can just implement custom deserializers.

E.g. to parse the following query parameter: ids={uuid1},{uuid2},..., you could do:

#[derive(Deserialize)]
pub struct Query {
    #[serde(deserialize_with = "deserialize_stringified_uuid_list")]
    pub ids: Vec<Uuid>,
}


pub fn deserialize_stringified_uuid_list<'de, D>(deserializer: D) -> Result<Vec<Uuid>, D::Error>
where
    D: de::Deserializer<'de>,
{
    struct StringVecVisitor;

    impl<'de> de::Visitor<'de> for StringVecVisitor {
        type Value = Vec<Uuid>;

        fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
            formatter.write_str("a string containing a list of UUIDs")
        }

        fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
        where
            E: de::Error,
        {
            let mut ids = Vec::new();
            for id in v.split(",") {
                let id = Uuid::parse_str(id).map_err(E::custom)?;
                ids.push(id);
            }
            Ok(ids)
        }
    }

    deserializer.deserialize_any(StringVecVisitor)
}

This has the benefit of being able to use the query struct as is, without further conversions needed, as it's already type safe.

[xliiv]

We can just implement custom deserializers.

This is a nice walk around. However, frameworks should support multiple URL parameters out of the box, right?
Can we clarify it? Is it won't fix or there is an issue for it or new one should be created?

[robjtede]

Changing the way the current Query extractor works, even in a breaking release, would be pretty distruptive. There's a discussion to be had around how to support the standardized ways of passing array-ish values but not in this issue. This one has a satisfactory answer of custom deserializers.

[xliiv]

@robjtede good to know, thanks. Would be awesome to link the discussion here in future ;)

[robjtede]

@xliiv Feel free to get the discussion going. We'd be happy to look at proposals and experimental PRs if you'd like.