Readthedocs.org: Allow for pull request builds to fail on warnings

[choldgraf]

Details

We have been building our documentation as a part of CI/CD by using this "strict" setting:

html-strict: 
    @$(SPHINXBUILD) -b html -nW --keep-going "$(SOURCEDIR)" "$(BUILDDIR)/html" $(SPHINXOPTS) $(O)

That way, the build will complete to the end, but it will _error_ if there are any warnings that are raised
during the build (e.g. if we've inserted an incorrect reference, or if a ref has become dead).

It would be helpful if readthedocs allowed for a "nitpicky" option when building forked pull-requests,
so that projects can use the docs preview as a form of quality control as well as being able to preview the docs.

Ideally this would also come with a warning message in the build preview link that would let authors check out
the error.

[stsewd]

You can enable nitpicky mode in the conf.py file https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-nitpick_ignore and enable fail on warnings on the readthedocs.yml file

https://docs.readthedocs.io/en/stable/config-file/v2.html#sphinx-fail-on-warning

[chrisjsewell]

the main thing I want highlight is the --keep-going: I want the build to report a failure on warnings, BUT I want it to try and keep going and still build the documentation.
For CI/PR reviews, the reason for this is so that you can
(a) get a report of all issues with the build in one run, so you can fix them all at once, rather than having to iterate on; fail, fix one issue, rerun, fail, fix the next issue, rerun, fail, ... (super tedious and a waste of resources!) and
(b) still preview the documentation

[stsewd]

I guess we can include the --keep-going option when using fail_on_warning

[chrisjsewell]

yeh that we be great ta, or have it as a separate option

[chrisjsewell]

Just why we are on the subject @stsewd, is there any specific build limits for PR submissions? I'm thinking of suggesting to switch this repository from CircleCI to RTD docs CI, which is fairly active and has a turn-over of up to a few PRs per day:
https://github.com/aiidateam/aiida-core/pulls?q=is%3Apr+is%3Aclosed, so just wanted to make sure if that's an issue.

[humitos]

is there any specific build limits for PR submissions?

Hi @chrisjsewell, the limitations are listed at https://docs.readthedocs.io/en/stable/builds.html

[Daltz333]

@humitos If the main repository build limits are increased, does this increase it for PRs?

[humitos]

@Daltz333 Yes.