Readthedocs.org: Build timed out for no apparent reason

[pllim]

Details

• Read the Docs project URL: https://readthedocs.org/projects/astropy/
• Build URL (if applicable): https://readthedocs.org/projects/astropy/builds/12279956/ (just one of many, as example)
• Read the Docs username (if applicable): pllim

Expected Result

Build to happen and succeed.

Actual Result

Timed out after about 1500s but we do not even see the sphinx-build command being run. The logs ends at:

# User's Sphinx configurations
language_user = globals().get('language', None)
latex_engine_user = globals().get('latex_engine', None)
latex_elements_user = globals().get('latex_elements', None)

# Remove this once xindy gets installed in Docker image and XINDYOPS
# env variable is supported
# https://github.com/rtfd/readthedocs-docker-images/pull/98
latex_use_xindy = False

chinese = any([
    language_user in ('zh_CN', 'zh_TW'),
    project_language in ('zh_CN', 'zh_TW'),
])

japanese = any([
    language_user == 'ja',
    project_language == 'ja',
])

if chinese:
    latex_engine = latex_engine_user or 'xelatex'

    latex_elements_rtd = {
        'preamble': '\\usepackage[UTF8]{ctex}\n',
    }
    latex_elements = latex_elements_user or latex_elements_rtd
elif japanese:
    latex_engine = latex_engine_user or 'platex'

# Make sure our build directory is always excluded
exclude_patterns = globals().get('exclude_patterns', [])
exclude_patterns.extend(['_build'])

We are hoping perhaps you can inspect the server log and give us more clue on how to fix this. Thank you very much!

xref astropy/astropy#10998

cc @saimn

[pllim]

@saimn noted that it started to happen about 4 days ago and might be related to using the new pip resolver.

[saimn]

Mentioned the pip resolver because I saw issues like #7639, but in our case it seems to work fine.
It's just that for some reason sphinx-build does not run.

[humitos]

Timed out after about 1500s but we do not even see the sphinx-build command being run.

It runs, but it's not logged because the process is killed before it completes and logs the output.

Are you doing something extra in the conf.py? I'm curious about why building the docs takes more than 1500s which is a lot :smile: -- I can probably increase the build time to 2000s, but still, I think it's a good idea to debug this a little to find out why it's taking too much and see if you can reduce that time a little.

[humitos]

I went ahead and increased the time limit for your project to 2000s and triggered a new build https://readthedocs.org/projects/astropy/builds/12295553/ --let's see if it passes. My suggestion to understand _why_ it takes too much time stands :sweat_smile:

[astrojuanlu]

Gracias @humitos :heart: Do you know if Sphinx offers any profiling or timing tool?

[humitos]

I went ahead and increased the time limit for your project to 2000s and triggered a new build readthedocs.org/projects/astropy/builds/12295553 --let's see if it passes

Oops. Super close!: "Build took 1928 seconds"

@astrojuanlu I know that Sphinx offers this extension, https://www.sphinx-doc.org/en/master/usage/extensions/duration.html that could be useful to understand what document is taking so much time. I never tried, tho.

[pllim]

Ah, thank you so much! We will investigate and get back to you. There had been a flurry of merges around release feature freeze time and it is possible that someone accidentally included something that caused performance loss in doc build. I will keep this issue open until we know why it is taken extra 500-ish seconds now.

[saimn]

Thanks @humitos. Astropy has a huge documentation, but this recent slowdown is a bit unexpected, since one month ago I did some timings with sphinx.ext.duration (https://github.com/astropy/astropy/issues/10807) and it was not so slow. And when building locally now I get about the same timings as one month ago, so we will to figure out what's slowing down the build on RTD.

[humitos]

Maybe it's related to https://github.com/sphinx-doc/sphinx/issues/8350 ? :thinking:

[pllim]

@humitos , good guess, though astropy does not use autodoc_mock_imports.

[pllim]

Traced it to Sphinx 3.3.0 release. I think we can close this one now over here. Thank you for your help!

[pllim]

xref sphinx-doc/sphinx#8403

[saimn]

Maybe just one more thing, @humitos : the output of sphinx.ext.duration does not show on RTD, not sure why, but it would be great to have it.

[humitos]

Traced it to Sphinx 3.3.0 release. I think we can close this one now over here. Thank you for your help!

@pllim Great! Thanks for report this back here! I'm happy that you were able to reduce the build time :)

Maybe just one more thing, @humitos : the output of sphinx.ext.duration does not show on RTD

@saimn want to open a new issue with examples and pointing to the builds where it was enabled, etc? I don't know how that extension works and _where_ it should show the output --but we can discuss all of this in a new issue.

[pllim]

New issue opened at #7659