Rocket: Gzip Compression & Browser Caching

[mmstick]

Is there a way to enable gzip compression & browser caching with Rocket? Chrome/Firefox regularly warn me that my site could save about 70KB combined by enabling compression, and that my resources are missing a cache expiration.

[belst]

Since the recommended way to deploy rocket is behind a proxy such as ngnix anyway I guess it could manage caching and compression.

[mmstick]

@belst I have zero need for a proxy because I already implement page caching support in my web server with Rocket. No need for a middle man.

[SergioBenitez]

There is not built-in implementation of this, but you can easily implement it yourself via a custom Responder, likely using flate2. I'd like to have something like this in contrib eventually, but unfortunately there's no Rust-only gzip implementation, making integration difficult.

[mmstick]

There's the zopfli crate which has a 100% Rust gzip compression implementation.

[SergioBenitez]

Neat! I didn't know of it. Unfortunately, according to the zopfli documentation, it performs:

very good, but slow, deflate or zlib compression

Speed is _very_ important in an interactive setting such as the web.

[mmstick]

It's OK if it's not the fastest, given that I can utilize it with my caching state which would only need to perform compression for a specific page/media once every now and then.

[mmstick]

Okay, so I managed to quickly put together a solution with Zopfli and it's working perfectly. All of my JS, CSS, and HTML content is being compressed, and that compressed content is stored within my custom content cache which is using the new State mechanism.

For HTML pages which I am generating from templates, it looks something like this:

impl From<Template> for RequestedContent {
    fn from(template: Template) -> RequestedContent {
        let mut output = Vec::with_capacity(8*1024);
        zopfli::compress(&Options::default(), &Format::Gzip, template.to_string().as_bytes(), &mut output).unwrap();
        RequestedContent {
            data: output,
            content_type: ContentType::HTML,
            ttl: 600
        }
    }
}

Whereas my RequestedContent struct is as follows:

#[derive(Debug,Clone)]
pub struct RequestedContent {
    pub data: Vec<u8>,
    pub content_type: ContentType,
    pub ttl: usize,
}

impl<'a> Responder<'a> for RequestedContent {
    fn respond(self) -> response::Result<'a> {
        Response::build()
            .sized_body(Cursor::new(self.data))
            .raw_header("Cache-Control", format!("max-age={}, must-revalidate", self.ttl))
            .raw_header("Content-Encoding", "gzip")
            .header(self.content_type)
            .ok()
    }
}

[echochamber]

@mmstick Not sure if you intentionally omitted checking the Accept-Encoding request header in your rough implementation for simplicity of example. Just wanted point out you want to include a check for it in your final implementation :)

[SergioBenitez]

Looks like there is an answer to this question. I'm happy to revisit this once there's a dependency that's easy to depend on for fast compression. :)

[Razican]

This no longer works, as the Template.to_string() method no longer exists. Any other option? I would have used the show() method, but seems too slow as per the documentation.

[Razican]

Any news on this side, @SergioBenitez?

[SergioBenitez]

@Razican You can access the current Request from a Responder implementation, allowing you to call Template::respond_to().

[tafia]

I ended up doing this (with the deflate crate):

struct CompressedJson<T>(T);

/// Serializes the wrapped value into gzip compressed JSON. Returns a response with Content-Type
/// JSON and a fixed-size body with the serialized value. If serialization
/// fails, an `Err` of `Status::InternalServerError` is returned.
impl<T: Serialize> Responder<'static> for CompressedJson<T> {
    fn respond_to(self, req: &Request) -> response::Result<'static> {

        let json = to_string(&self.0)
            .map_err(|e| {
                error!("JSON failed to serialize: {:?}", e);
                Status::InternalServerError
            })?;

        let headers = req.headers();
        // check if requests accepts gzip encoding
        if headers.contains("Accept") && 
            headers.get("Accept-Encoding").any(|e| e.to_lowercase() == "gzip") {
            let data = ::deflate::deflate_bytes_gzip(json.as_bytes());
            Ok(Response::build()
               .status(Status::Ok)
               .header(ContentType::JSON)
               .raw_header("Content-Encoding", "gzip")
               .sized_body(::std::io::Cursor::new(data))
               .finalize())
        } else {
            Ok(Response::build()
               .status(Status::Ok)
               .header(ContentType::JSON)
               .sized_body(::std::io::Cursor::new(json))
               .finalize())
        }
    }
}

I'm happy with the result, thanks again for Rocket.

[adwhit]

I just came up against this. Here is my solution, using a Fairing and the flate2 crate.

pub struct Gzip;

impl fairing::Fairing for Gzip {
    fn info(&self) -> fairing::Info {
        fairing::Info {
            name: "Gzip compression",
            kind: fairing::Kind::Response,
        }
    }

    fn on_response(&self, request: &Request, response: &mut Response) {
        use flate2::{Compression, FlateReadExt};
        use std::io::{Cursor, Read};
        let headers = request.headers();
        if headers
            .get("Accept-Encoding")
            .any(|e| e.to_lowercase().contains("gzip"))
        {
            response.body_bytes().and_then(|body| {
                let mut enc = body.gz_encode(Compression::Default);
                let mut buf = Vec::with_capacity(body.len());
                enc.read_to_end(&mut buf)
                    .map(|_| {
                        response.set_sized_body(Cursor::new(buf));
                        response.set_raw_header("Content-Encoding", "gzip");
                    })
                    .map_err(|e| eprintln!("{}", e)).ok()
            });
        }
    }
}

Test with curl:

$ curl http://localhost:8000 --silent --write-out "%{size_download}\n" --output /dev/null
627
$ curl http://localhost:8000 --silent -H "Accept-Encoding: gzip, other things" --write-out "%{size_download}\n" --output /dev/null
299

[spawnia]

After looking at this issue while searching for a way to add compression, i found that the upcoming 0.5 release of rocket_contrib is going to support compression out of the box.

https://github.com/SergioBenitez/Rocket/blob/master/contrib/lib/src/compression/mod.rs