Jupyter-book: Using myst_parser's new "substitutions" feature

[firasm]

Describe the bug

I'm trying to get the new "substitution" directive from myst_parser v0.13 and running into some trouble translating the "conf.py" file to jupyterbook's "_config.yml".

Problem 1: dependency issues

Trying to upgrade myst_parser to v0.13 caused some issues with dependencies for myst-nb

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
myst-nb 0.10.1 requires myst-parser~=0.12.9, but you have myst-parser 0.13.0 which is incompatible.
jupytext 1.6.0 requires markdown-it-py~=0.5.2; python_version >= "3.6", but you have markdown-it-py 0.6.0 which is incompatible.

I think this was just a warning, and I ended up upgrading myst_parser anyway.

Problem 2: myst_enable_extensions

The Myst docs say:

To enable all the syntaxes explained below:

myst_enable_extensions = [
    "amsmath",
    "colon_fence",
    "deflist",
    "dollarmath",
    "html_image",
    "linkify",
    "replacements",
    "smartquotes",
    "substitution"
]

What is the way to enable these in jupyterbook? Here is my attempt in _config.yml:

# bunch of stuff that's not relevant here

sphinx:
  config:
    html_show_copyright: false
    html_extra_path: ['files']

parse:
  myst_extended_syntax: true

myst_enable_extensions: 
  - amsmath
  - colon_fence
  - deflist
  - dollarmath
  - html_image
  - linkify
  - replacements
  - smartquotes
  - substitution

myst_substitutions:
  course: 'THISISAMAZING1010'

Problem 3: Using substitutions

I imagine that with the _config.yml file above, this should work in any md file:

This is the official syllabus and course outline for COSC 123 {{ course }}.

... but {{ course }} doesn't replace the text as I expected.

I also tried to add the substitution definition at the top of the file as "front-matter" using three dashes, but that doesn't work either (no error, just leaves it as {{ varname }}.

E.g.,

---
substitutions:
  course2: "I'm a substitution"
---

To Reproduce

See above.

Expected behavior

{{ course }} should be replaced with "THISISAMAZING1010"

Environment

• Python Version [e.g. 3.7.1]: 3.8.3
• Package versions or output of jupyter-book --version:

Jupyter Book: 0.9.1
MyST-NB: 0.10.1
Sphinx Book Theme: 0.0.39
MyST-Parser: 0.13.0
Jupyter-Cache: 0.4.1
NbClient: 0.4.1

• Operating System: macOS 11.1

Additional context

RE: Problem 2, I see the same warnings as seen in issue #1152...

[chrisjsewell]

Heya, as I said there, jupyter book does not yet officially support this version of myst-parser.
But if you do want to try it out, both options have to be placed in the sphinx: config:
section

[chrisjsewell]

sphinx:
  config:
    html_show_copyright: false
    html_extra_path: ['files']

    myst_enable_extensions: 
    - amsmath
    - colon_fence
    - deflist
    - dollarmath
    - html_image
    - linkify
    - replacements
    - smartquotes
    - substitution

    myst_substitutions:
      course: 'THISISAMAZING1010'

[choldgraf]

I opened up https://github.com/executablebooks/jupyter-book/issues/1158 to track the improvement to the docs that we'll need to make

[firasm]

Aha! Works! 🎉

Thanks @chrisjsewell ! Yes I was aware it's not yet supported but I really needed that feature 😃.

Thank you!

[chrisjsewell]

great, let me know how it goes 😄

[firasm]

@chrisjsewell substitutions are working great!

Only found one minor issue, if you pass an integer instead of a string, all the substitutions break (show the code {{ <sub name> }} and give this warning:

WARNING: myst configuration invalid: 'substitutions' must be <class 'str'> (got 63995 that is a <class 'int'>).

Probably would be better/nicer if that was an error rather than a warning, or better yet, attempt to cast that int to a string.

Let me know if you'd like me to file this as a new issue in myst_parser. It could be that this is handled with a small note in the docs also.

[chrisjsewell]

if you pass an integer instead of a string

yeh I can extend the validation to ints and floats
jinja2 will actually allow the values to be anything, even functions! but I'm not sure if functions can be saved (pickled) in the stored sphinx environment

[choldgraf]

+1 to ints and floats, can we think of any other types that are:

• commonly used by people
• assumed to behave like strings when those people type them in

?

[firasm]

Well, I can think of passing in "directives" or chunks of code via substitution, but I feel that's asking for trouble and too fancy.

[chrisjsewell]

You can already pass in directives, the text is parsed before it is substituted in