Lighthouse-ci: Only files in staticDistDir root are collected

[andypattenden]

Using autorun against a static site, only the files in the root of the directory are tested.

e.g. If I have a static site with the following structure:

.
|____news
| |____articles.html
|__index.html

then only index.html is collected.

[patrickhulce]

Thanks for filing @andypattenden! This was intentional to prevent ballooning the number of files tested and taking far too long with low marginal returns, but I guess it makes sense to support limiting this again with the url option or automatically limiting to the first 5 files found.

[andypattenden]

That does make sense.

I did start to work on a solution to this using readdirp in place of fs.readdirSync. Advantages of that package are that it allows limiting depth of recursion and additional filtering options.

I ran into issues running the unit tests locally. If I can get those resolved I'll have another go at this.

[einSelbst]

I'm also in favor of this. In our setup also only the main index.html and 404.html is found while all the interesting ones are in subfolders. Another idea would be to support sitemaps, but there also some kind of limiting might be needed.

[patrickhulce]

I'm curious do you also have a gatsby setup @einSelbst? I ran into that exact issue with a gatsby project recently :)

I think the recursive search with a low default limit on URLs makes sense.

[einSelbst]

I'm curious do you also have a gatsby setup @einSelbst? I ran into that exact issue with a gatsby project recently :)

Yes, this is with gatsby too.

[patrickhulce]

So we can't truly make this recursive by default without a breaking change since it might cause previously succeeding builds to start failing as new URLs are included, but manual support has been introduced in 93b674203a39520d40f6c985048d83d2ed46a2b8. We'll make sure this recursive support gets into 0.4.0 tracked by #119 👍

Once 93b674203a39520d40f6c985048d83d2ed46a2b8 is released you'd solve the issue above with

```
lhci autorun --collect.static-dist-dir=./
--collect.url=/news/articles.html
--collect.url=/index.html

[ClarkMitchell]

Thanks for filing @andypattenden! This was intentional to prevent ballooning the number of files tested and taking far too long with low marginal returns, but I guess it makes sense to support limiting this again with the url option or automatically limiting to the first 5 files found.

Is there a config option to not limit to 5?

[patrickhulce]

@ClarkMitchell nope but you're welcome to file a new issue to make that option configurable :)