Sphinx: Request of implementation: global substitutions

[andreacassioli]

I have a fairly large project, say 1000 files, and a set of say 40 substitutions. I use rst_prolog to pre-pend the set of substitutions to each file. This seems to slow quite a lot the doc generation, and my guess is that I need to:

1. process 40x1000 lines of text
2. generate 40x1000 substitution nodes

That looks overkilling and useless to me. Why is there no global substitution table? I know that Docutils is where substitutions happen. But I guess there should be a way to add substitutions in a way pretty much we do with the default ones (date, version...).

If it were possible, one may just provide a python dictionary in conf.py. Or a specific markup

.. |my_subs| global_replace:: my text

If it is not possible to hack on docutils, what about a role like

:sub:`my_subs`

but that would be quite cumbersome.

[lehmannro]

There is some functionality like this in DefaultSubstitutions already, which replaces |version|, |release|, and |today|. You could modify which config variables are inlined from your conf.py by monkeypatching sphinx.transforms.default_substitutions.

I think the easiest way to do this would be another transform though. Use Sphinx.add_transform to augment the document's substitution_defs (see standard SubstitutionsTransform.)

I'd be happy to review a pull request to merge this upstream. (In which case you wouldn't need to add_transform it, of course, and just add it to the SphinxStandaloneReader.)

[andreacassioli]

Hi,
thank you for the hint. I will take a look to the options you gave me.

I guess this is not considered an issue unless a project starts to be fairly big. I am not sure I have enough understanding of Sphinx to work on a pull request, but I'll see if it is not too hard.

[andreacassioli]

Hi again,
I have patch my conf.py this way:

• create a dictionary global_substitutions

global_substitutions = {}

that I populate in conf.py.

• create an extension with Transform subclass that basically does the substitution

from docutils.transforms import Transform
from docutils import nodes

class GlobalSubstitutions(Transform):
    default_priority = 200

    def apply(self):
        config = self.document.settings.env.config
        global_substitutions = config['global_substitutions']
        to_handle = set( global_substitutions.keys() ) - set(self.document.substitution_defs)
        for ref in self.document.traverse(nodes.substitution_reference):
            refname = ref['refname']
            if refname in to_handle:
                try:
                    text = global_substitutions[ refname ]
                    ref.replace_self(nodes.Text(text, text))
                except :
                    pass

def setup(app):
    app.add_transform(GlobalSubstitutions)

It seems to work and the speed up is impressive (40%) ! Only one thing does not work: the standard markup layout like **word** for bold fonts seems not to be processed. Is it a matter of priority?

[andreacassioli]

Oh I got it! I create a simple text node....

But what should I do to have the text parsed? I have been looking to what @lehmannro suggested but I could not figured out how to do that.

In my understanding one should add suitable entries in self.document.substitution_defs but I could not understand which kind of node....any clue?

[lehmannro]

Text nodes are not post-processed for markup. You'd need to create a docutils.parsers.rst.RSTParser and an empty document (docutils.utils.new_document), parse your source into the document, and then extract the resulting nodes. See the Locale transform for an example.

(DefaultSubstitutions won't help, since these are used as Text nodes as well.)

[Peque]

Just for reference: https://github.com/missinglinkelectronics/sphinxcontrib-globalsubs

[ScriptAutomate]

Wanted to check-in here: @andreacassioli -- were you able to get further with this?

One issue I run into still is exactly what you mentioned here:

_Only one thing does not work: the standard markup layout like word for bold fonts seems not to be processed. Is it a matter of priority?_

This seems to be a problem that impacts rst_prolog, too.