Sphinx: ..include:: statement doesn't parse markdown files correctly

[kdebrab]

Inspired by this issue comment and this example, I try to combine .rst restructured text files (for automatically generated documentation) with .md markdown files (for manual documentation), using the CommonMark parser. This works well (after #2841) when including markdown files in a toctree, but does not seem to work for .. include:: statements.

E.g. with the statement .. include:: ../README.md inside the (restructured text) INDEX.rst master file, the contents of README.md file are not parsed as markdown text but as restructured text!

What is the recommended way to include in the Sphinx documentation markdown files that typically reside in the root of a package (such as README.md, CHANGELOG.md or CONTRIBUTING.md), while parsing them correctly as markdown files?

[tk0miya]

The include directive only reads a text from specified file and put it on the position.
So the included text is recognized as reStructured Text.

At this moment, there is no way to include markdown document into reST one.
Please use toctree directive instead.

[tk0miya]

Now I close this issue.
Please reopen if you still have a problem.

[swiesend]

As this answer wasn't very satisfying to me I investigated further and found the following solution.

It is possible to use the AutoStructify transformations from recommonmark for that purpose. The AutoStructify transformations enrich the markdown syntax with .rst capabilities:

https://recommonmark.readthedocs.io/en/latest/#autostructify

One may create a symbolic link to the README.md and may call it docs/index.md. The library itself shows how they do it:

https://github.com/rtfd/recommonmark

[eric-wieser]

I've made a patch to allow this via ..includedoc in https://github.com/sphinx-doc/sphinx/pull/7739

[DominiqueMakowski]

@eric-wieser If I understand correctly, this patch would allow to easily integrate markdown files by using ..includedoc instead of ..include (as for regular .rst), without the need of additional extensions to sphinx?

As I see that this PR is not yet merged, do you (or anyone else 😁) know by any chance how could I use it with readthedocs (i.e., how to tell to readthedocs to use a particular version of sphinx, namely, https://github.com/sphinx-doc/sphinx/pull/7739)?

Thanks a lot

[eric-wieser]

If I understand correctly, this patch would allow to easily integrate markdown files by using ..includedoc instead of ..include (as for regular .rst), without the need of additional extensions to sphinx?

If you have a sphinx extension that supports some_file.ext files, then that PR will let you use .. includedoc:: some_file.ext to include it. So you'll need recommonmark installed if ext == md.

As I see that this PR is not yet merged, do you (or anyone else 😁) know by any chance how could I use it with readthedocs

Yes - you can a line like this, swapping the commit sha for the one containing my patch:

https://github.com/pygae/galgebra/blob/master/doc/readthedocs-pip-requirements.txt#L7-L9

[DominiqueMakowski]

Thanks @eric-wieser, Here are the steps that I followed, and modified 4 files:

In docs/conf.py, added:

import recommonmark
from recommonmark.transform import AutoStructify

extensions = [..., 'recommonmark']

# Mention this on the reconmark doc
def setup(app):
    app.add_config_value('recommonmark_config', {
        # 'url_resolver': lambda url: github_doc_root + url,
        'auto_toc_tree_section': 'Contents',
        'enable_math': False,
        'enable_inline_math': False,
        'enable_eval_rst': True,
        'enable_auto_doc_ref': True,
    }, True)
    app.add_transform(AutoStructify)

In the docs/environment.yml:

dependencies:
  - python>=3.5
  #- sphinx>=1.4  # Get WIP version of sphinx with MD fix
  - pip
  - ...
  - pip:
    - git+https://[email protected]/sphinx-doc/sphinx.git@5460ad6925199b57b27b7d059825d3560872edbb#egg=sphinx 
    - recommonmark

And in the index.rst, added:

Contents:

.. toctree::
   :maxdepth: 1

   file

And finally, in file.rst, added:

.. includedoc:: ../file.md

But it doesn't seem to work, and the webpage remains empty.

And in the build log I have:

/home/docs/checkouts/readthedocs.org/user_builds/neurokit2/checkouts/246/docs/benchmarks/ecg_preprocessing.rst:1: WARNING: Unknown directive type "includedoc".

Any idea what is wrong here?

[eric-wieser]

5460ad6925199b57b27b7d059825d3560872edbb is the wrong commit, you need to edit that to be the one you want. Your other steps look fine.