Pandoc: Pandoc does not process RST (replacement) substitutions defined in an included file.
pandoc version: 2.0.3
Here is the sample a.txt with explanations:
I have a target to the following line.
.. _TARGETID:
I would like to jump to this TARGET line from this file
and other files.
I use replacement substitution: See last line.
The links are generated in a separate file:
.. include: links_html.txt
``links_html.txt`` has entries of this kind and it works with Sphinx::
.. |TARGETID| replace:: :ref:`TARGETID<file.html#TARGETID>`
``links_docx.txt`` has entries of this kind::
.. |TARGETID| replace:: `TARGETID <file.docx#TARGETID>`_
In this example I use the same link for both DOCX and HTML::
.. |TARGETID| replace:: `TARGETID <#TARGETID>`_
Command to convert to html::
pandoc -f rst -t html a.txt -o a.html
Command to convert to DOCX::
cat a.txt | sed -e's/links_html/links_docx/g' | pandoc -f rst -t docx -o a.docx
.. admonition:: BUG
Pandoc does NOT process the substitution, if it is defined in an include file,
neither for DOCX nor for HTML.
It is a problem of how the RST reader processes ``.. include::``.
Pandoc produces working links for HTML and DOCX, if I put the replacement substitution inside this file::
.. |TARGETID| replace:: `TARGETID <#TARGETID>`_
As a side note, ``rst2html.py`` does not accept this substitution in neither case.
This should jump to the |TARGETID|.
.. include: links_html.txt
rpuntaie
All 12 comments
For my case I have a workaround:
cat file.rst links_docx.txt | sed -e's/.. include:: links_html.txt//g' | pandoc -f rst -t docx -o file.docx
I'm not sure I understand all the details here. But we try to match the official docutils behavior, so if we're doing the same thing as rst2html.py, there's no bug. Sphinx may have its own special extensions?
jgm
on 15 Dec 2017
Sphinx has some extensions, but substitution is just that of docutils.
Here is a minimal example with two files a.txt and b.txt:
a.txt
.. include:: b.txt
b.txt
|substituteme|
.. |substituteme| replace:: I've been substituted.
rst2html a.txt > a.htmlworks as expected.pandoc -f rst -t html a.txt > a.htmlproduces
[WARNING] Reference not found for 'Key "substituteme"' at b.txt line 1 column 15
Thanks for the simpler case. I see the issue now. The RST
parser does a two-step parse, grabbing references first and
then parsing. Processing of include directives takes place
in the second step, so the references defined in other
files aren't available. This isn't a good design; we should
really change the reader to use the F monad, as the Markdown
reader does, and do a single pass. This would be a bit of
work.
jgm
on 17 Dec 2017
I think this might be a separate issue actually, but pandoc version 1.16.0.2 doesn't seem to process substitutions at all:
test.rst:
.. |date| date::
Hello world. It is currently |date|.
pandoc -o test.tex test.rst:
Hello world. It is currently \textbar{}date\textbar{}.
expected would be something like:
Hello world. It is currently 2019-03-15.
For reference: rst2html5.py test.rst | tail:
</style>
</head>
<body>
<div class="document">
<p>Hello world. It is currently 2019-03-15.</p>
</div>
</body>
</html>
Apologies if this is fixed in a version newer than what my distribution provides.
cheshirekow
on 15 Mar 2019
Pandoc 1.16 is ancient. Try a more recent version (we're on 2.7.1; linux binaries are available on our Releases page.)
jgm
on 19 Mar 2019
@cheshirekow This issue is about substitutions (replace:: directives) found in the included file.
Your issue is specifically about the date:: directive. That also does not work in a recent pandoc (my version is 2.9.2), so I've opened a new issue for that date:: directive here: https://github.com/jgm/pandoc/issues/6276
@jgm on this current issue, it seems like the replace:: in included files are also not working for my version 2.9.2 either.
[605] $ pandoc --version
pandoc 2.9.2
Compiled with pandoc-types 1.20, texmath 0.12.0.1, skylighting 0.8.3.2
Default user data directory: /home/brian/.local/share/pandoc or /home/brian/.pandoc
Copyright (C) 2006-2019 John MacFarlane
Web: https://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.
```
[605] $ pandoc -f rst -t html a.txt
[WARNING] Reference not found for 'Key "substituteme"' at b.txt line 1 column 15
[WARNING] Reference not found for 'Key "substituteme"' at b.txt line 2 column 18
.. replace:: I've been substituted.
The result is even stranger when you change it to define the replacement text before it's used, i.e. changing `b.txt` to the following ...
```$ cat b.txt
.. |substituteme| replace:: I've been substituted.
|substituteme|
fails to produce any output when started from a.txt and works when started with b.txt
[638] $ pandoc -f rst -t html a.txt
[WARNING] Reference not found for 'Key "substituteme"' at b.txt line 3 column 15
[639] $ pandoc -f rst -t html b.txt
<p>I've been substituted.</p>
brianonn
on 12 Apr 2020
@jgm on this current issue, it seems like the replace:: in included files are also not working for my version 2.9.2 either.
Correct, that's why this issue is still open!
jgm
on 12 Apr 2020
Ok, what about a half year after?
RarogCmex
on 6 Nov 2020
What about it?
If the comment is meant to be read as a passive-aggressive demand to implement the feature: please don't.
If you are asking whether there is ongoing work which you can join to bring a solution forward: I don't think anybody is directly working on this right now. Your help would be most welcome.
tarleb
on 6 Nov 2020
@tarleb Both, sure 馃憤 :)
This moment I learn Haskell, and because I can't learn it full time, its slows. So after another half of year I could try to fix it.
P.S. I use ReST to write notes about Haskell learning.
RarogCmex
on 6 Nov 2020
Feel free to ping us with questions if you feel ready to giving it a shot. The fpchat slack and Haskell Discourse are good resources for general Haskell questions, while the pandoc-discuss mailing list is full of people with deep pandoc knowledge. Ask away!
tarleb
on 6 Nov 2020
Related issues
nrnrnr
路
49Comments
jgm
路
48Comments
jgm
路
51Comments
kevinushey
路
79Comments
phyllisstein
路
84Comments
Most helpful comment
What about it?
If the comment is meant to be read as a passive-aggressive demand to implement the feature: please don't.
If you are asking whether there is ongoing work which you can join to bring a solution forward: I don't think anybody is directly working on this right now. Your help would be most welcome.