Jupyter-book: How to utilize local Sphinx extensions in Jupyter Book?

[hashslingrz]

Hey all,

Sphinx has a great tutorial on developing extensions. Of note, it explains, by way of conf.py, how to utilize a local Sphinx extension (i.e. one not available on PyPI):

import os
import sys

sys.path.append(os.path.abspath("./_ext"))

extensions = ['helloworld']

Of course, one can do the latter part of appending an extension to the extension list in Jupyter Book. Your documentation explains that one should modify _config.yml like so:

sphinx:
  extra_extensions:
   - extension1
   - extension2

However, the important part — adding the local extension's path — is impossible via _config.yml alone.

How would I go about using local Sphinx extensions as part of the Jupyter Book build process?

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:

If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

Welcome to the EBP community! :tada:

[chrisjsewell]

Heya, you could always just install the extension locally; add a minimal setup.py and run pip install -e .

[hashslingrz]

Thank you @chrisjsewell I'll likely resort to that as it solves my problem in a pinch. 😄

I suppose that means Jupyter Book does not have "native" support for this functionality? I'd happily contribute code to handle this case, but am pretty swamped at the moment.

[choldgraf]

Jupyter Book does not have "native" support for this functionality

correct - right now it's not possible to point to a local extension directly through configuration. The pattern @chrisjsewell suggests is probably the best route.

we should document that as an "advanced" option. @hashslingrz if you get this working, any interest in writing a PR to document it?

longer-term, I feel like this is related to the issue of letting users give conf.py-style python code that will be run along with the Sphinx environment. If that were possible then all of the extra complexity of sphinx (like adding local extensions via sys.path.append) would be available

[hashslingrz]

I have added some documentation on using a local Python package as a Sphinx extension in PR #1058

It would be interesting to have the following syntax for _config.yml, but I am unsure as to how feasible it is:

sphinx:
  extra_extensions:
   - extension1
   - some_dir/local_extension1
   - extension2

That is, the entries within the extra_extensions category are parsed and those with a relative path are sys.path.appended to the Python path.

However, I agree with your comment @choldgraf about providing users the full power of conf.py-style Python code.

[choldgraf]

Yeah I think that could be a good way to support this, though I am not sure how to implement it as under the hood, we are not using conf.py...maybe there's another way to specify custom code in sphinx?

[mmcky]

so to support this we need the name and path of the local extension.

Perhaps it would be simple enough to add a local_extensions: option in the yaml:

sphinx:
  local_extensions:
    <name>: <path>

https://www.sphinx-doc.org/en/master/usage/extensions/index.html#where-to-put-your-own-extensions and add it to sys.path.append

[hashslingrz]

@mmcky, this has already been suggested in the existing discussion on this issue. @choldgraf mentioned that since Jupyter Book is not using conf.py internally, it isn't as easy as doing a sys.path.append.

[mmcky]

@choldgraf jupyter-book is written in python so I'm not sure I understand why we couldn't parse this list from _config.yml and append to sys.path.append in the jupyter-book cli? Does the path need to be appended in the sphinx.app?

[mmcky]

@choldgraf @hashslingrz I have added support for local_extensions in PR #1102.
Let me know what you think re: syntax and then I can finalise with a test case etc.

[hashslingrz]

Huh, well I'll be darned, nice job @mmcky! Your PR works just fine for my local extension. Bang up job 😄

I also like your syntax choice more than my previous suggestion. 👍

[mmcky]

thanks for testing this out @hashslingrz.