Actix-web: Actix-web JSON serializer disorders payload

[neilyoung]

My struct:

#[skip_serializing_none]
#[derive(Default, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Status {
    pub version: Option<String>,
    pub site_id: Option<String>,
    pub device_id: Option<String>,
    pub state: Option<isize>,
    pub is_started: Option<bool>,
    pub map_point_count: Option<isize>,
    pub marker_count: Option<isize>,
    pub loop_count: Option<isize>,
    pub fps: Option<f64>,
    pub fixed_at: Option<isize>,
    pub latitude: Option<f64>,
    pub longitude: Option<f64>,
    pub altitude: Option<f64>,
    pub level_id: Option<isize>,
    pub pitch: Option<f64>,
    pub yaw: Option<f64>,
    pub roll: Option<f64>,
    pub rotation_matrix: Option<[[f64;3];3]>,
    pub current_map_name: Option<String>,
    pub x: Option<f64>,
    pub y: Option<f64>,
    pub z: Option<f64>
}

My Config.toml:

serde_json = { version = "1.0.59", features = ["preserve_order"] }

The JSON is rendered by an actix-web based service:

async fn svc_utils_get_status() -> std::io::Result<web::Json<Status>> {
    Ok(web::Json(handle_get_status()))
}

The handle_get_status function returns an instance of Status above.

The result looks like so (prettyfied and None elements not rendered):

{
    "deviceId": "DF1IVJVV5R7I",
    "fps": 0.0,
    "isStarted": false,
    "loopCount": 7,
    "mapPointCount": 5,
    "markerCount": 6,
    "siteId": "1000",
    "state": 0,
    "version": "0.1.12/2.1.9"
}

So, it is alphabetically ordered, but not as defined preserved order. If I render the object to json using serde_json directly, the order is preserved.

Is there any JsonConfiguration item, which controls this?

[robjtede]

We do not use the preserve_order feature on serde_json. I recommend you keep serializing the payload yourself if this is required.

See the Responder impl for web::Json: https://docs.rs/actix-web/3.2.0/src/actix_web/types/json.rs.html#128. It would be reasonably easy for you to create a similar wrapper for using the serde_json settings you want.

[neilyoung]

@robjtede Yepp, was currently on it with a debugger. I'm not that sure if it really would be easy, since I'm a beginner, but I will try :)

Thanks

[neilyoung]

Maybe - if possible - you have an example by hand. Thanks

[robjtede]

Copy the responder impl I linked to so that it uses your serde_json.

[neilyoung]

That doesn't work for me. Thanks anyway

[neilyoung]

I'm wondering, how this is supposed to work. I'm missing a lot of types, e.g. Ready. There is an example here https://docs.rs/actix-web/3.2.0/actix_web/trait.FromRequest.html which suggest to just import

use futures_util::future::{ok, err, Ready};

But this ends up in use of undeclared crate or modulefutures_utilrustc(E0433)

[robjtede]

I'm not able to reproduce this. Using code

use std::io;

use actix_web::{web, App, Error, HttpRequest, HttpResponse, HttpServer, Responder};
use futures_util::future::{err, ok, Ready};
use serde::Serialize;

struct OrderedJson<T>(pub T);

impl<T: Serialize> Responder for OrderedJson<T> {
    type Error = Error;
    type Future = Ready<Result<HttpResponse, Error>>;

    fn respond_to(self, _: &HttpRequest) -> Self::Future {
        let body = match serde_json::to_string(&self.0) {
            Ok(body) => body,
            Err(e) => return err(e.into()),
        };

        ok(HttpResponse::Ok()
            .content_type("application/json")
            .body(body))
    }
}

#[serde_with::skip_serializing_none]
#[derive(Debug, Default, Serialize)]
#[serde(rename_all = "camelCase")]
struct NonAlphabetical {
    z_ned: u32,
    n: Option<u32>,
    a: u32,
    l: u32,
}

async fn handler() -> impl Responder {
    // toggle the commented line to see the difference

    println!("{}", serde_json::to_string(&NonAlphabetical::default()).unwrap());
    println!("{}", serde_json::to_string_pretty(&NonAlphabetical::default()).unwrap());

    web::Json(NonAlphabetical::default())
    // OrderedJson(NonAlphabetical::default())
}

#[actix_web::main]
async fn main() -> io::Result<()> {
    HttpServer::new(|| App::new().default_service(web::to(handler)))
        .bind(("127.0.0.1", 8080))?
        .run()
        .await
}

and cargo deps:

[dependencies]
actix-web = "3"
serde = { version = "1", features = ["derive"]}
serde_with = "1"
serde_json = "1"
futures-util = "0.3"

running curl localhost:8080:

{"zNed":0,"a":0,"l":0}

and in the server log:

{"zNed":0,"a":0,"l":0}
{
  "zNed": 0,
  "a": 0,
  "l": 0
}

It seems order is preserved for structs. The "preserve_order" feature only affects their Map type.

Perhaps your client is re-arranging them for you? (I know HTTPie does this)

[neilyoung]

Currently trying your code.

I realized I was missing futures-util = "0.3" in my Cargo.toml.

[neilyoung]

OK, I confirm, your code is OK. My client is curl...
I can also confirm, that serde_json output makes no diffference, regardless of whether the preserve_order feature is specified or not. It is always preserving order.

But I realized, what my problem was: I was prettyfying the output using python -mjson.tool pipe. This was the trick.

Give it a try

curl localhost:8080 | python -mjson.tool

I wasn't aware that this tool scrambles the order...

Sorry for the hassle and thanks for your efforts.

[neilyoung]

I'm still wrestling with the Serializer trait. I cannot really make it run. :(

[neilyoung]

Motivation is to make a pretty-serializer...

[neilyoung]

I think I didn't catch the idea yet :(

[robjtede]

I don't know what you mean. Might be better to ask Q on the gitter chat: https://gitter.im/actix/actix-web.

[neilyoung]

Yes, sorry. The problem is, that I can't make your suggestion "copy the code from the link above" run, even though I have satisfied all uses now...

[robjtede]

the code in this comment is _now_ a complete example of a re-impl

[neilyoung]

Yeah, thanks. You won't believe it, but a second before I saw your edited post above I could make it run by myself. My problem was, that I was obviously trying to kind of "overwrite" the default implementation. The key is to define a new type. That was new to me. Thanks for having taught me that this evening. Have a nice weekend. I think I'm enabled to run by myself now.

👍
Regards

[neilyoung]

That was extremely helpful. I never dealt with traits before in my one-week experience with Rust. But I begin to love it... Thanks again!!!

[robjtede]

You'll get there. We have all climbed the mountain 😅. Feel free to reach out in the Gitter or Discord if you got any other questions. There's more folks around who can offer quick assistance.

[neilyoung]

You'll get there. We have all climbed the mountain 😅. Feel free to reach out in the Gitter or Discord if you got any other questions. There's more folks around who can offer quick assistance.

I already used that and noticed it. There are a lot of great guys, who don't hesitate to share their knowledge. Good to know...