Docs.rs: Switch from html5ever to lol_html

Created on 2 Jul 2020 · 4Comments · Source: rust-lang/docs.rs

See https://github.com/kuchiki-rs/kuchiki/pull/74#issuecomment-653184255 for discussion. In a nutshell, kuchiki is not intended for low-memory usage: it uses lots of Rc and RefCell (one for each node in the tree!) LOL HTML is intended for exactly our use case:

Low Output Latency streaming HTML rewriter/parser with CSS-selector based API.

This would allow us to step the size of files rendered _way_ up, possibly removing the limit altogether (https://github.com/rust-lang/docs.rs/pull/834).

A-backend E-medium P-medium

Source

jyn514

👀1

Most helpful comment

I think having a limit even if it's absurdly high and will "never" be hit is something good, just as a safeguard

Edit: LOL Html has MemorySettings which allows us to specify the maximum and minimum memory used for parsing

Kixiron on 2 Jul 2020

👍2 ❤1

All 4 comments

LOL HTML is also developed by cloudflare so it has seen a lot of real-world usage.

jyn514 on 2 Jul 2020

I think having a limit even if it's absurdly high and will "never" be hit is something good, just as a safeguard

Edit: LOL Html has MemorySettings which allows us to specify the maximum and minimum memory used for parsing

Kixiron on 2 Jul 2020

👍2 ❤1

And according to Cloudflare's blog post LOL Html is vastly faster than html5ever and scales much better. I believe our performance will be somewhere in the ballpark of the tag scanner, but potentially even better than "normal" parsing since we grab two portions of the html and LOL doesn't parse innards if you specify a tag, while html5ever parses everything unconditionally