Rust: [tracking-ish issue] Doc pages are very slow to render

[GuillaumeGomez]

Message from discord:

the https://doc.rust-lang.org/std/iter/trait.Iterator.html page(and only that page?) seems to have a broken search bar that manifests as infinite? loading and high CPU usage in both Chrome and firefox, across stable, beta, nightly, and local versions.

Related:

• [x] [Minification makes JS parsing slower](https://github.com/rust-lang/rust/issues/57754)
• [ ] [Open
  search-index.js slows down documentation browsing](https://github.com/rust-lang/rust/issues/56545)
• [ ] Others?

[ollie27]

The HTML for that page is currently about 12.5MB thanks to #51885. We're now rendering a copy of every trait method for every impl. We could revert #51885 or only render methods that have documentation in the impl.

[JeanMertz]

Just to confirm this issue: I wanted to add that even loading that page is next to impossible on Safari. It takes a very long time, and after it loads, scrolling is so slow that the page is next to useless. I often have to switch to Firefox (my secondary browser) only to read this specific page in the docs.

[crumblingstatue]

Not only is it very slow to render initially, but due to the sheer amount of content on the page, it is also very sluggish to scroll through it on my system.

I also agree that we should show less information by default. A 12.5 MB doc page for a single item sounds ridiculous.

I'd very much prefer if implementors were just listed plainly like so: Implementors: std::ascii::EscapeDefault, EscapeDebug, std::char::EscapeDefault, EscapeUnicode, ToLowercase, ToUppercase, ..., with them being links so you can click on them to go to their doc page.

This amount of duplication is a lot of wasted data. It makes the documentation bloated and slow.

[Stargateur]

Also it would be good to fix this as fast as possible (I have not idea how still a newbie), Iterator are a lot used and be able to read the doc is primordial.

Freeze on Chrome Version 70.0.3538.110 (Build officiel) (64 bits)
Load quite fast on Firefox 63.0.3 (64 bits) ;)

[GuillaumeGomez]

The PR is open and waiting for reviews.

[nocduro]

Has this been deployed? I'm still having really sluggish performance on https://doc.rust-lang.org/std/iter/trait.Iterator.html

Related issue #57281

[steveklabnik]

I don't think it's made its way to stable, check the beta/nightly docs.

[nocduro]

Just tried beta and nightly in incognito and it works about the same. The page doesn't freeze or anything, but scrolling is a bit laggy. Chrome uses ~30-40% cpu when scrolling.

I think this patch is live though, because nightly shows some loading text while the sections are loaded instead of instantly trying to render?

[Stargateur]

yeah, that doesn't freeze but the page still load tons of information, is it possible to have the page that only load what user required, for example, a plus bottom that will only load content when opened ? That would allow to see method, without load everything.

Also, all theses informations are them really needed ? Why did we do this change ?

[GuillaumeGomez]

The rendering was slow in any case so whatever changes we want to perform afterwards, we'll still gain from them. We're debating about what information could/should be removed on the pages alongside making the HTML/JS lighter. It's moving forward.

[crumblingstatue]

I just checked the nightly doc page. On my computer, it's just as slow as it was before. It takes quite a lot of seconds before it loads, and even then the scrolling is sluggish.

[GuillaumeGomez]

I don't see such behavior (either on firefox or chrome) so the problem could come from your computer configuration?

[crumblingstatue]

I don't see such behavior (either on firefox or chrome) so the problem could come from your computer configuration?

I have a pretty old computer, but reading documentation shouldn't require a beefy computer. No other documentation I ever used had this problem. In fact, old versions of the Iterator page posed no problem either. But the new versions are way too large.

[JeanMertz]

I know this is being worked on, so my post is just "another datapoint". Ignore this if it doesn't add anything you don't already know.

Are we talking about this page? https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.html

If so, that one is (still) slow for me as well, on a 2017 MBP. It takes about 5 seconds to load, and once it does, scrolling down is jerky from time to time (but not as often as before), and shows white screens until rendering catches up.

I do believe it's quite a bit faster than before, but still not great.

It's also worth considering what information people want to get from that page. Anything above "Implementations on Foreign Types" is only a fraction of the contents of the page (understandably, as the rest is mostly about other implementors), while it contains all you need to know on _how_ to use Iterator (what that section lacks, is _what_ implements Iterator, which is the bulk of the page) so perhaps it makes sense to put a cut there, and load the rest of the page lazily on request.

[Stargateur]

there is no need to prove the page is slow:

curl "https://doc.rust-lang.org/nightly/std/iter/trait.Iterator.html" > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 13.1M  100 13.1M    0     0  3375k      0  0:00:04  0:00:04 --:--:-- 2761k

13.1M, for a html page ? there is a problem. Even, C11 standard only does 2M, https://port70.net/~nsz/c/c11/n1570.html, and it's already too big.

[GuillaumeGomez]

We agree on this and minification is being worked on like said previously: https://github.com/rust-lang/rust/pull/56869

[Stargateur]

I don't think micro optimization will fix the problem, we need a real solution that change how doc is working. 13 - 1 => 12 M. That will not change much. Also, that not a problem for me to download heavy thing, the problem is to render too many thing in the page.

We could argue that the problem is "only" in iterator, but iterator just show there is a problem, that could happen on other lib.

[DianaNites]

Wouldn't a better fix would be for pages to just not to repeat their own trait documentation in Implementors, just a list of what implements it and for what Item?(Or more generally whatever associated constants) Cut it out entirely.

Then the Iterator page wouldn't repeat it's entire documentation for everything. That seems to me to be where the bulk of the page is.

Plus the nightly page, while faster, is significantly less usable since theres no content until it finishes loading everything. On the current page you can scroll and read to what you need while it loads the huge implementors section. On nightly, you only get "loading content" until the entire page finishes.

[0xpr03]

The iter site works pretty fine on my side.
Yeah it takes some seconds but scrolling is fine on my laptop. I don't think we can do too much about this without removing relevant information. Sometimes it's just that: A pretty huge amount of stuff to display. Just as in high LoC files.

[GuillaumeGomez]

I don't think micro optimization will fix the problem, we need a real solution that change how doc is working. 13 - 1 => 12 M.

This is just one amongst others. There are PRs to reduce DOM size and other things. When we'll reach the end of those optimizations, if it's still not enough, we'll check for other solutions at this time.

[matthieu-m]

I would second the opinion that repeating trait methods for each and every implementor is overkill. The implementor's page will already provide those methods in full.

Beyond the performance issues, it also requires scrolling a WHOLE lot more, unless one uses the minimized mode, which I find even more concerning.

Sometimes less is more.

[steveklabnik]

There's always tension between different use-cases. Sometimes, people want docs for individual implementations, to talk about what they do, etc. That can only happen if things are repeated.

[DianaNites]

What use-case does the Implementors section on Traits solve? Right now all it does is bloat pages and repeat the methods already listed above. Note that it doesn't repeat their documentation. Just a straight list of documentation-less method signatures that are already listed above, for everything that uses Iterator.

The Trait Implementations section, which appears on structs, is fine as-is, and already does something different than the Implementors section, it has documentation.

[ebarnard]

When opening the Iterator page in the latest nightly docs (which include #56005) it takes about 45 seconds for "Loading content..." to disappear and the Iterator docs to become usable. This does not include network time. (Late 2013 MPB running Safari 12 and Firefox 64).

This renders the Iterator docs page unusable unless I am careful not to use the search feature, click on any links, or navigate away from the page.

I'm not sure whether others are experiencing similarly long load times but if they are it seems a convincing reason to revert #51885 until the performance issues can be fixed.

I'm also unsure of the intentions of the Implementors section. Is it meant to show docs written on methods in an impl block rather than on the methods in the trait declaration.

[GuillaumeGomez]

It takes 3 seconds for me, whatever the computer/OS.

[ebarnard]

Disabling extensions in Firefox brought it down to about 15s. Safari on Mac and iOS (iPhone 6S) is still about 45s.
Here's a Gecko profile for Firefox in case you're interested https://perfht.ml/2QBVmXi.

[GuillaumeGomez]

I'm on mac too (but never used safari though), strange. However, the profile tells that most of time is spent into rendering. So reducing DOM and discuss about removing trait implementations methods is still the good thing to do (and is being done already since a while).

[Stargateur]

10 minutes for my chrome. Also, try to avoid "work for me" argument, you seem to believe there is no problem here. I'm telling you that this will made crazy a lot of people who will use rust.

[GuillaumeGomez]

If I thought there were no problem, I wouldn't be working on it since I consider that a few seconds is still way too much. I was just saying that since you were using the same OS, having such rendering speed difference was strange.

[JeanMertz]

Thank you for the work you're doing on this, @GuillaumeGomez.

In case it helps, here's a recording of the request on my Mac that took about 40 seconds (without cache) until the screen was "painted" with some elements, and another 10 until text was shown.

https://www.icloud.com/iclouddrive/01XaTrKAGVgED_IAGNxerP6ig#doc.rust-lang.org.har

Also some attached screenshots of the inspector.

(for reference, this is on a 2017 Macbook Pro, with a wired 500Mbit connection)

[GuillaumeGomez]

Wo, this is insane! A lot of work remains to be done in here. Thanks for the screens!

[leonlag]

I can confirm this on fedora 29. Using Chromium (without addons and new profile for testing) it's close to unusable(~3FPS) just scrolling through the page. Firefox is acceptable (~50FPS).

[carols10cents]

A lot of work remains to be done in here.

I'm going to reopen this then, as a tracking issue of sorts? I think this could benefit from a checklist in the description of known issues and their status, perhaps with issues that need investigation as well?

Just a suggestion, because I'm pointing some folks to this issue, and it sounds like it isn't actually solved. Feel free to reclose and manage this in a different way!

[HarrisonMc555]

It is now March 1, 2019, and I can confirm that this is still an issue. Has there been more work on reducing the size of this page?

[GuillaumeGomez]

There was but no major improvement for the moment.

[kornelski]

The sheer size of it seems to be the problem. There's too much to load:

and too much for CSS to lay out:

I've never seen layout take so long. It's 2 seconds straight on Intel's i9 CPU.

[memoryruins]

Layout takes around 15-17 seconds on my laptop for both stable/nightly versions of the docs.

CPU: Intel i5-4300U (4) @ 2.900GHz
Browser: Firefox 66.0.2 (64-bit)

Safari on an iPhone 5s takes roughly 50 seconds before loading text at the top of the page.

[samcday]

Just weighing in with my 2c worth here.

Just because it's 2019 doesn't mean it's reasonable to be shipping 12mb HTML files to users. Browsers today are super smart and fast and all, but actually they're not really much different to how they were 10+ years ago in one important sense: virtually all painting / reflow events can only take place on the main UI thread.

As a result, forcing the poor browser to parse 12mb is somewhat absurd. Jump on the std::iter::Iterator page and run this in the Javascript console:

> $("#implementors-list").querySelectorAll("*").length
<- 172249

!!! 172k elements!

That said, I'm not claiming to know what's best for Rust or the Rust documentation. Maybe it is valuable to show every trait method implementation (blanket or otherwise) for every single implementor. For me personally, it's not. But what I will be opinionated about is if doing so requires DOM structures that large, a completely different path must be taken since you're never going to get that kind of beast to be performant.

[samcday]

I opened #59768 as a stopgap to alleviate this issue a bit.

It doesn't do anything to tackle the size of the Iterator doc page, but it does halve the time it takes to reach DOMContentLoaded on my machine. It does this by simply starting with all Implementors collapsed.

[kornelski]

edit: nevermind, that was a project issue

Did something happen to rustdoc in general?

Source code for actix-web is 1.3MB, but the generated documentation takes 347MB!

https://github.com/actix/actix-web/issues/770

[GuillaumeGomez]

Not as far as I know. Normally it should be reducing size, not the opposite... Can you provide the list of files with their associated size please?

[ollie27]

@kornelski checking out https://github.com/actix/actix-web/commit/3e986042973762e28fa6af25c8d161ce9f4de4d0 locally I'm only measuring 249MiB and I don't see anything too unusual. It includes documentation for all dependencies including things like libc which is 20MiB alone. The trait pages are very big like futures/stream/trait.Stream.html being 2.2MiB which is caused by the regressing caused by #51885 that this issue is tracking.

Can someone add a regression-from-stable-to-stable label to this issue?

[lzutao]

As you wish @ollie27
@rustbot modify labels: +regression-from-stable-to-stable
Edit: I'm sad.

[rustbot]

Error: Label regression-from-stable-to-stable can only be set by Rust team members

Please let @rust-lang/release know if you're having trouble with this bot.

[rustbot]

Error: Label regression-from-stable-to-stable can only be set by Rust team members

Please let @rust-lang/release know if you're having trouble with this bot.

[ollie27]

Maybe that will work for me.

@rustbot modify labels: +regression-from-stable-to-stable

[insanitybit]

https://rusoto.github.io/rusoto/rusoto_dynamodb/struct.PutItemInput.html

I've experienced serious lag/ hangs/ CPU spikes on rusoto docs.

Chrome 73.0.3683.103 on Ubuntu 18.04.

edit: I believe the major culprit is the extension DarkReader - pages like these really cause it to hang.

[wafflespeanut]

I have a suggestion related to trait pages (open to bikeshedding).

Most often, we're only interested in whether a type implements a trait (when we're in some trait's page) or finding a trait implementation in a type's page. Does it make sense to get rid of the detail on trait implementations (under "Implementations on foreign types" and "Implementors" sections)?

Basically, these:

For example, let's take the impl for Args shown in Iterator trait page. It's very useful to link to the source of that implementation, but it's probably not that useful to show the implementation detail with all those methods because we're already on the trait's page?

---

Instead, we could link to the trait impl in the type's page. Maybe an icon next to that impl? (again, this is just a suggestion, we could go for something else). The user can find the relevant impl information in the type's page.

This should improve the performance for trait pages (particularly Iterators) because we'll be cutting a huge chunk of nodes in the DOM.

(As a sidenote, if we land on some decision related to this, I'll be more than happy to work on it!)

[kornelski]

BTW, all content under these [+]'s is duplicate. You get all the same iterator methods repeated over and over again for every Itearator impl. It'd make sense to group implementations by trait, and include methods once.

[Manishearth]

I think that somewhat makes sense for common traits like Iterator, but not in general.

One useful side of the expanded doc view is being able to see what methods you have available. Often these are through traits.

[Stargateur]

Note that @shepmaster give me a trick, use a past version of the doc, https://doc.rust-lang.org/1.29.0/std/iter/trait.Iterator.html. Not perfect but that do the work until the problem is solved.

[matthieu-m]

I think that somewhat makes sense for common traits like Iterator, but not in general.

One useful side of the expanded doc view is being able to see what methods you have available. Often these are through traits.

There are two distinct use cases though:

1. Showing which traits are implemented by the current struct/trait A on the page of A, and the methods available.
2. Showing all implementations of a trait A on the page of A, and duplicate the trait methods for each implementation.

I agree with you that (1) is very useful, however (2) is a completely separate usecase and the cause of the issue for Iterator.

I find that @wafflespeanut adequately solves both (2) and the bloat issue: it shows all implementers, yet conveniently sidesteps the performance issue by referring to the implementer's page for further information.

[Manishearth]

Oh, I see. Hmm. That seems okay.

(Though I'll note the performance issues are due to various bugs and not the actual page content. Browsers are pretty good about not processing display:none things)

[wafflespeanut]

One useful side of the expanded doc view is being able to see what methods you have available. Often these are through traits.

This is true and this is the case for "type" pages. I never suggested to remove anything there. I was talking about removing the stuff in "trait" pages, because we already have all the trait methods up top (so why duplicate everything?). I'm also suggesting to provide a link (next to the trait impl in a trait's page) for the user to see that impl in the corresponding type's page.

Though I'll note the performance issues are due to various bugs and not the actual page content.

I agree that browsers are very good, but I still think the current state of rustdoc (specifically for Iterator trait) is an overkill, especially when we don't have a good reason for listing all trait methods for all trait impls.

[sinesc]

As soon as the last "Loading content..." appears the browser has processed the entire HTML. After that it is "just" layouting the blocks that were previously display: none.
One could flip those blocks individually from hidden to visible with a setTimeout(.. ,0) inbetween to allow the browser to layout the blocks individually. Total rendering time would go up slightly, but the stuff at the top would appear faster.

[Manishearth]

I agree that browsers are very good, but I still think the current state of rustdoc (specifically for Iterator trait) is an overkill, especially when we don't have a good reason for listing all trait methods for all trait impls.

Yeah I agree with that.

I'm more saying that this isn't the cause of the performance issues.

[GuillaumeGomez]

It could be part of it. That's something that has been suggested a while ago and that I wanted to give a try to. I'll try to find some time to check locally if that improves things.

[bendem]

Speaking of doc performance, you might want to fix the server's config to compress html documents. Those 13MB are 692K gzip'd, 74K brotli'd.

[GuillaumeGomez]

That's not up to rustdoc unfortunately. Please open an issue on docs.rs repository.

[ebarnard]

The Iterator docs are hosted on https://doc.rust-lang.org. Presumably that isn't hosted by docs.rs? I'm not sure where the right place to open an issue would be.

[GuillaumeGomez]

Like I said, on their repository: https://github.com/rust-lang/docs.rs

[SimonSapin]

I believe the hosting of rust-lang.org (including doc.rust-lang.org) is the responsibility of the @rust-lang/infra team. Maybe a new issue on https://github.com/rust-lang/infra-team/issues would be the appropriate place to discuss server configuration?

[GuillaumeGomez]

Arf sorry, I kept reading docs.rs instead of doc.rust-lang.org...

[pietroalbini]

Speaking of doc performance, you might want to fix the server's config to compress html documents. Those 13MB are 692K gzip'd, 74K brotli'd.

We already have automatic compression configured on the CloudFront side, but CloudFront won't compress files bigger than 10MB, so fixing this is not a simple configuration change.

[ebarnard]

Can I suggest a minimal change, as far as I can see without downsides, that should significantly reduce the size of the Iterator doc page:

In the Implementors and Implementations on Foreign Types sections, only show methods that appear in the impl block for that type. This has the benefit of

• Reducing the size of the Iterator page, and other large trait documentation pages.
• Retaining documentation on the impl blocks and functions in the impl blocks.
• Indicating which provided methods are overridden.
• Making the documentation match the structure of the code being documented.
• Being a small change that can be easily backed out if issues arise.

What this would look like for the Read trait:

This should be an easy change to make so I'll try and have a PR ready soon so we can see what impact it would have on page sizes and load times.

[ebarnard]

A set of Rust stdlib docs build with this change are available here.

The Iterator doc page is now 724kB.

[memoryruins]

Thank you for trying these changes and providing a demo!

Immediate impression:

• Current nightly Iterator docs: similar to my previous report https://github.com/rust-lang/rust/issues/55900#issuecomment-480550869,
  except causes my mobile browser to become unresponsive and crash.
• @ebarnard 's iterator docs: loads nearly immediately on laptop 🎉, and it scrolls smoothly without page flickering or laptop fans becoming loud. It loaded quickly on my phone too :)
  It appears to make docs pages clearer and easier to navigate in general.

Another example trait page that should benefit, which I'm unable to load on my laptop:
https://docs.rs/uom/0.23.1/uom/trait.Conversion.html

[GuillaumeGomez]

@ebarnard: Looks great! Can you open a PR with your changes please? It'll also allow to see if other people like this change or not.

[ebarnard]

@GuillaumeGomez It's open at #61505

[GuillaumeGomez]

Little update: I think I'll investigate wasm at some point and see if it improves the situation. But first, I want to check if there is a way to improve how we store the information in the search-index.js file.

[ollie27]

I think this issue should be closed. The original regression caused by #51885 has long since been fixed by #61505. I think we can track specific performance problems in dedicated issues like #56545.