Pandoc: Table of contents not generated for DOCX

[retorquere]

What steps will reproduce the problem?

1. Create a Markdown file with headings
2. Use pandoc --toc to create DOCX file

_What is the expected output? What do you see instead?_
I expect a table of contents at the top; I see none. When I render to PDF, the TOC is present.

_What version of the product are you using? On what operating system?_
1.9.1.2

[jonasvp]

I can confirm this for pandoc 1.8.1.1 on Ubuntu, using ODT as an output. There is no table of contents, even though I asked for it using "--toc".

[rcrowe]

Also happening with pandoc 1.9.2 on OSX when outputting as .docx

[daroczig]

I am pretty sure TOC is not supported in docx and odt, but it's not mentioned in the documentation.

[stashluk]

Also happening with pandoc 1.11.1 from cabal-install on Ubuntu 12.04 LTS

[lcn918]

A tip to the guys that really need this functionality when it's still away: table of contents can be easily generated for a docx converted by pandoc in Word, because head lines maintain their styles after conversion and it's right these styles that MS Word uses to generate the TOC.

[pizaninja]

Nothing for 1.12
https://github.com/jgm/pandoc/blob/master/changelog

[jgm]

While it would be possible to have pandoc insert a table of contents into a docx, it doesn't seem desirable to me. It's trivial to insert a table of contents in the docx generated by pandoc, as pointed out by @lcn918 - just go Insert -> Index and Tables, then select the Table of Contents tab and choose the particular styling you want.

[tolot27]

Indeed, it is trivial to insert a TOC manually, but it is quite annoying doing it every time a new version of the document is generated. Even fixing a single typo requiries this step again.
It gets more complicated, if you want to have a TOC with level > 3 and keeping the header of the TOC (i. e. 'Content'). In this case you have to do some more steps and of course repeat is gain and again.

I already wrote a global macro to do this but it is still quite annoying.

It would be really great if pandoc would support --toc and --toc-depth for docx as well.

[jgm]

Inserting this into document.xml should work, I think:

      <w:r>
        <w:fldChar w:fldCharType="begin" />
      </w:r>
      <w:r>
        <w:instrText xml:space="preserve">                                       
 TOC \o "1-3"                                                                    
</w:instrText>
      </w:r>
      <w:r>
        <w:fldChar w:fldCharType="separate" />
      </w:r>

[jgm]

Partial docs here:
http://officeopenxml.com/WPtableOfContents.php

At the very least, we need to add to the above

      <w:r>
        <w:fldChar w:fldCharType="end" />
      </w:r>

but that alone is not sufficient.

[jmuheim]

Is there any news on this? Is there a way to add a TOC to a generated docx file without needing to start World manually or automatically (using a script or equivalent)?

[tolot27]

I could manage to create the TOC programmatically but was not able to update the page numbers. Hence, the easiest way is to insert the following code into document.xml:

        <w:sdt>
            <w:sdtPr>
                <w:docPartObj>
                    <w:docPartGallery w:val="Table of Contents" />
                    <w:docPartUnique />
                </w:docPartObj>
            </w:sdtPr>
            <w:sdtContent>
                <w:p>
                    <w:pPr>
                        <w:pStyle w:val="TOCHeading" />
                    </w:pPr>
                    <w:r>
                        <w:t>Table of Contents</w:t>
                    </w:r>
                </w:p>
                <w:p>
                    <w:r>
                        <w:fldChar w:fldCharType="begin" w:dirty="true" />
                        <w:instrText xml:space="preserve"> TOC \o "1-3" \h \z \u </w:instrText>
                        <w:fldChar w:fldCharType="separate" />
                        <w:fldChar w:fldCharType="end" />
                    </w:r>
                </w:p>
            </w:sdtContent>
        </w:sdt>

Instead of using the attribute w:dirty="true", you can also add the following to the end of settings.xml (before </w:settings>):

  <w:updateFields w:val="true" />

If you now open the document in Word, you will ask to update the fields. Click yes, and the TOC is created automatically.

The enhancements proposed in #1612 can be applied easily.

BTW: Calculating the page numbers within pandoc is not possible because it depends on the (customized) styles and page layout settings in reference.docx.

[tolot27]

If you are on Windows and have MS Word installed, then save the following script to toc.js:

if (WScript.Arguments.length == 0) {
    WScript.Echo("No file name provided!");
}

var WshShell = WScript.CreateObject ("WScript.Shell");
var wordApp = new ActiveXObject ('Word.Application');
try {
    var doc = wordApp.Documents.Open(WshShell.CurrentDirectory + "\\" + WScript.Arguments(0));
    doc.Save();
} finally {
    wordApp.Quit();
}

Calling wscript toc.js mydocument.docx will automatically update the fields of the given document in the current directory.

[tolot27]

To create a list of figures, just use

<w:instrText xml:space="preserve"> TOC \h \z \c "Figure" </w:instrText>

For a list of tables, provide the common table label (i. e. "Table") to the \c parameter.

[tolot27]

There is stil one problem regarding to the header of the TOC. As long as it cannot be provided via YAML (#1612) and as it is hardcoded as shown in the XML snipet above, it can be in the wrong language if the documents language is for instance german.
Unfortunately, Word does not insert a header automatically.

[tolot27]

An option would be to ignore the <w:sdt> node of a reference.docx, if the docx is generated. But that requires that an adapted reference.docx is always provided. It is much less elegant than the --toc switch.

[tolot27]

What is the best way to continue with this issue? Shall I open a thread in Pandoc Google Group?

I've checked out the source code already. Unfortunately, I'm not experienced in writing Haskell code.

[lierdakil]

It should be possible to implement described behavior with filters. I am not entirely convinced that --toc should behave like this though.

[tolot27]

Thanks for the feedback.

How should --toc behave?

IMHO, I suggest a similar solution like for the other output formats. Even for Latex/PDF the toc parameters are just 'injected' into the reference template file and rendered by the downstream engine (i. e. pdflatex).

Indeed, it is possible to 'pre-create' the whole TOC including links but without the user dialog. However, no page numbers are possible as explained above. The epub or html writer behave very similar.

I'll check if I can provide an implementation as a filter to have a starting point.

[lierdakil]

I'm more concerned that proposed solution will essentially create "dirty" docx files, which have to be opened with Word at least once (and will likely ask some strange question about external files), before sending them out for a review for example. This is not a behaviour I'd expect from this. Then again, not sure if we have any choice here.
As for filter, here is a place to start: https://gist.github.com/lierdakil/461cf6b0ef67f8cee47e

[tolot27]

I can understand your concern but the situation is identical if it is implemented as a filter.

I believe this proposol is a much better but not ideal solution to not having a TOC at all and creating it manually every time the source document gets converted.
In any way, you have to open Word to include/automatically update the TOC before sending them out except if you are on a Windows or Mac system. Then you can include the execution of the toc.js script as provided in my comment https://github.com/jgm/pandoc/issues/458#issuecomment-58720613 and you'll get a final Word document without any user intervention.
But there is no solution for *nix based systems.

[lierdakil]

Yes, it certainly will be the same for filter and pandoc option. My question is this: should pandoc really offer such half-baked solution out of the box? Users who often need automatic docx table of contents could use filter instead of --toc option. It's not that much different. Others could be painfully surprised with suggested implementation (and will likely complain).

[tolot27]

As long as there is no free renderer for docx, there will be no complete solution on all platforms. The users have to learn/already know the drawback of the (maximum possible) proposed solution - either as pandoc implementation or external filter.

I see your point implementing this not fully automated toc generation into the pandoc code. But otherwise users might wonder why they have to use a filter (separate to install) for docx and the command line for other output formats. That's an additional inconsistency.

I don't know the code policies of pandoc. Maybe imperfect solutions are left out until they become perfect? In this case that would never happen.

Personally, I can use filters. I've just adapted my make files. But non-experienced users?

[lierdakil]

I suggest discussing this with people in https://groups.google.com/forum/#!forum/pandoc-discuss . If general consensus will be that an incomplete solution is better than no solution at all, I'm willing to put some time into implementing suggested behavior properly..

[lierdakil]

Ok, I've dug around a little, and it seems this will need a new reference.docx for TOC heading. There is nothing close in reference.docx atm. This will likely have the same implications as international headings (i.e. #1607).
As a side note, I am unsure, where exactly TOC should be placed relative to other metadata. Writer currently supports title, subtitle, authors, date and abstract. At the moment, I chose to insert toc before abstract, but this is easily changed.

[lierdakil]

Okay, since #2014 and #1968 are merged, it's now possible to implement --toc with relatively little effort. #2037 does exactly that.

[tolot27]

Great, thanks!

It should be noted/documented that this implementation supports --toc-depth as well.

I would also suggest moving it after abstract rather than before. That is the typical order for thesis, dissertation and scientific/technical articles. See also http://en.wikipedia.org/wiki/Table_of_contents.
There are only few other cases where the TOC is before the abstract. In most cases it is after the abstract.

Issue #1612 suggests for the TOC placement.

[ousia]

Issue #1612 suggests for the TOC placement.

@tolot27, I suggest to comment in #1612, since that issue was marked as _More discussion needed_.

[lierdakil]

Right. I'll move it to after abstract then. I'm reluctant to implement suggestions from #1612 here, since
a) it behaves quite differently from other writers
b) there is an ongoing discussion on syntax

[tolot27]

My comment https://github.com/jgm/pandoc/issues/458#issuecomment-87384315 was related to this issue here, IMHO, because it was about the hardcoded placment (the defaults).
I've just referenced #1612 again and will continue to discuss configurable placement of TOC there.
@lierdakil: You are right not to implement #1612 here because it #1612 just depends on this issue.

[jgm]

Closed by #2037.

[brainchild0]

Although activity ended some time ago on this issue, I am unable to find documentation on the current expected behavior. Presently, I use a reference document featuring a table of contents. When I open the generated document, the table of contents is visible but has no content. It must be updated manually.

Is the expected behavior that the table of contents updates automatically when the document is opened?

Am I experiencing a limitation of my application, LibreWriter, compared to Microsoft Word, which would behave differently?

Is documentation available on this subject?

[jgm]

brainchild0 [e-mail] writes:

Although activity ended some time ago on this issue, I am unable to find documentation on the current expected behavior. Presently, I use a reference document featuring a table of contents. When I open the generated document, the table of contents is visible but has no content. It must be updated manually.

Yes, that's as described in the thread above. Note
this linked comment:
https://github.com/jgm/pandoc/issues/458#issuecomment-58720613

I'm sorry, that's the best we can do; there seems to
be no way to make it update the fields without manual
intervention, as far as I can see.

[brainchild0]

In the earlier comments, it was implied that as long as the document was opened once, the application would detect the _dirty_ flag and regenerate the index. Reading down the chain of comments, I am not finding when, if ever, it was determined otherwise. Indeed, it is sensible that the application would behave as such, since if a document were generated interactively, as most are, then the index would need to be regenerated whenever it became obsolete. I am confused about why I the manual step is required.

Would I be seeing different behavior if were using Microsoft Office, not Libre Office?

[jgm]

I just tried with my version of Word (a rather old
Word for Mac) and when I opened the document, it
popped up a dialogue box asking whether I wanted
to regenerate the fields. I clicked yes and it did.
So, yes, it may be a Libre Office limitation.

[brainchild0]

I confirmed that it is a known issue that LibreOffice never updates indices unless the update feature is explicitly invoked. I found the report for the issue in the tracker for LibreOffice, apparently unresolved after being reported 15 years ago.