Tox: ensure tox.ini examples in our documentation can be parsed

[gaborbernat]

Would be nice if we could ensure that our examples in our documentation parse correctly.

@asottile @nicoddemus how hard would it be to generate for pytest extra tests by parsing the rst files inside the documentation, looking for .. codeblock:: ini and wrapping it into a generated test (e.g. parse_config("ini")). I assume this requires adding an additional test discovery mechanism and an additional test runner for it. Does anyone have examples of similar? Docstrings are sort of similar to this.

[nicoddemus]

Hi @gaborbernat,

I have little experience with parsing RST, @asottile and @RonnyPfannschmidt seems to have more experience with that; but with that information on hands should be simple to generate new tests for it automatically.

[asottile]

Here's a quick sketch, based on the regex used for blacken-docs:

import os.path
import re
import textwrap

import pytest

from tox.config import parseconfig


RST_RE = re.compile(
    r'(?P<before>'
    r'^(?P<indent> *)\.\. (code-block|sourcecode):: ini\n'
    r'((?P=indent) +:.*\n)*'
    r'\n*'
    r')'
    r'(?P<code>(^((?P=indent) +.*)?\n)+)',
    re.MULTILINE,
)


rst_files = []
for root, _, filenames in os.walk('doc'):
    for f in filenames:
        if f.endswith('.rst'):
            rst_files.append(os.path.join(root, f))


@pytest.mark.parametrize('filename', rst_files)
def test_all_rst_ini_blocks_parse(filename):
    with open(filename) as f:
        contents = f.read()
    for match in RST_RE.finditer(contents):
        parseconfig(textwrap.dedent(match['code']))

[asottile]

oh I see, assigned to me -- I'll make a PR :D

[gaborbernat]

I assigned to you more for the question than the solution, but thanks for the PR :tada: