Dvc.org: development server is slow to build

[elleobrien]

Working from a local clone to write new blogs, building a dev server (yarn develop) is getting pretty slow (>1 minute). Is there a way to make a smaller clone; i.e., maybe only blogposts after a certain date (%today-30) or not checkout the docs?

[rogermparent]

I'm on it! There's a few features I'm aware of that we can try to curb this issue.

1. Conditional Rebuilds - This has been in for a while, and is a basic way to only rebuild pages that have been edited. While it doesn't work on the Heroku builds, it has worked for me locally and may speed up writing.
2. Query-on-demand alpha - This is a new alpha feature that I was notified of in an email- it essentially makes Gatsby dev mode only run queries as you go to the individual page.

I'll try both in a branch.

[rogermparent]

The prior two features have showstopping issues that prevent them from even local use on dvc.org, however:

Is there a way to make a smaller clone; i.e., maybe only blogposts after a certain date (%today-30) or not checkout the docs?

We can adapt the LIMIT_BLOG_PAGES flag I have in already and bind it to an npm script to make invoking it easier. This way, we can have a separate limited mode for blog authoring that's quicker to build while maintaining the functionality of a full-site develop.

[elleobrien]

@rogermparent i love that idea!

[rogermparent]

I've made a new yarn author command that does just that at #1932, and it cuts development server spin-up time to 25s on a hot cache by building only the latest blog post and skipping Docs entirely.

I didn't time it on a cold cache yet, but the difference should be much bigger in that case as the images of skipped pages are also skipped.

[jorgeorpinel]

yarn author command ... building only the latest blog post and skipping Docs entirely

What about the opposite? A yarn docs command that only builds /docs and skips the whole blog? Would that be easy to include @rogermparent? (Assuming it's significantly faster than yarn develop)

Thanks

[jorgeorpinel]

Taking this to an extreme (prob not worth it) yarn develop could take arguments to specify which site components to work with (all by default):

• home (could be always included)
• land(ing) (Features and Support, could be part of home)
• docs
• blog or author
• com(munity)
• all

[rogermparent]

yarn author command ... building only the latest blog post and skipping Docs entirely

What about the opposite? A yarn docs command that only builds /docs and skips the whole blog?

Good idea! This would certainly be helpful for writing docs, as blog pages are easily the bulk of build time.

Taking this to an extreme (prob not worth it) yarn develop could take arguments to specify which site components to work with (all by default):

* `home` (could be always included)

* `land(ing)` (Features and Support, could be part of `home`)

* `docs`

* `blog` or `author`

* `com(munity)`

* `all`

This could be a good idea for the future, but author, docs, and develop should handle most of the cases this page limiting would be useful in. Since docs build very fast, a docs command could serve as a quick-build for all pages outside of the blog- maybe something like a pages command that cuts out both docs and blog could help when building out things like the Community page.

[jorgeorpinel]

Since docs build very fast, a docs command could serve as a quick-build for all pages outside of the blog

Yeah... That would definitely speed up things for first-time docs contributors, who have to wait for the whole blog to build before having a local dev env...

[jorgeorpinel]

BTW @rogermparent should we update https://dvc.org/doc/user-guide/contributing/docs#all-commands?