Pandoc: segfault when rendering file from Markdown to HTML

[kevinushey]

I really wish I could distill this into a more minimal document, but the various edits I make all cause the segfault to vanish for me...

The associated file is here: https://dl.dropboxusercontent.com/s/juv01st40m5udbe/process_raw.md

If you want to reproduce, you can run this directly in the Windows command prompt (assume that the 64-bit build of Pandoc 2.3.1 is on the PATH)

curl https://dl.dropboxusercontent.com/s/juv01st40m5udbe/process_raw.md -O
pandoc process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained

When I run this, I see:

C:\Users\kevin\scratch\ouch>pandoc process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x74eb050

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x521daf0    0x74eb050
 * 0x521daf8    0x7ffb574f00f0 C:\Windows\System32\IMM32.DLL+0xf0
 * 0x521db00    0x521de60
 * 0x521db70    0x7ffb57e24581 C:\Windows\SYSTEM32\ntdll.dll+0x34581
 * 0x521db78    0x76de641
 * 0x521db80    0x7471d80
 * 0x521db88    0x7471d65
 * 0x521db90    0x7471ce0
 * 0x521db98    0x85
 * 0x521dba0    0x7686968
 * 0x521dba8    0x76de6a1
 * 0x521dbb0    0x8e834d8
 * 0x521dbb8    0x8e834f2
 * 0x521dbc0    0x7546928
 * 0x521dbc8    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dbd0    0x3e87e98 C:\Program Files\Pandoc\pandoc.exe+0x3a87e98
 * 0x521dbd8    0x74bb000
 * 0x521dbe0    0x3d6a75c C:\Program Files\Pandoc\pandoc.exe+0x396a75c
 * 0x521dbe8    0x3d6a7f1 C:\Program Files\Pandoc\pandoc.exe+0x396a7f1
 * 0x521dbf0    0x44d47ca C:\Program Files\Pandoc\pandoc.exe+0x40d47ca
 * 0x521dbf8    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc00    0x3d541e0 C:\Program Files\Pandoc\pandoc.exe+0x39541e0
 * 0x521dc08    0x74bb148
 * 0x521dc10    0x4643851 C:\Program Files\Pandoc\pandoc.exe+0x4243851
 * 0x521dc18    0x3d541d1 C:\Program Files\Pandoc\pandoc.exe+0x39541d1
 * 0x521dc20    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc28    0x3d541c1 C:\Program Files\Pandoc\pandoc.exe+0x39541c1
 * 0x521dc30    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc38    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc40    0x3e7a34a C:\Program Files\Pandoc\pandoc.exe+0x3a7a34a
 * 0x521dc48    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc50    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc58    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc60    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc68    0x3d6a73b C:\Program Files\Pandoc\pandoc.exe+0x396a73b

   ... (maximum recursion depth reached.)

This is with Pandoc as downloaded and installed with the installer at https://github.com/jgm/pandoc/releases/tag/2.3.1, using the 64-bit installer.

C:\Users\kevin\scratch\ouch>pandoc -v
pandoc 2.3.1
Compiled with pandoc-types 1.17.5.1, texmath 0.11.1.1, skylighting 0.7.3
Default user data directory: C:\Users\kevin\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

Any idea what might be going on? For what it's worth, I don't see this crash with the 32-bit build of Pandoc.

For posterity, this is with Windows 10 Pro (10.0.17134 Build 17134)

[jgm]

I don't have a Windows setup to test on, but maybe
someone else can try to reproduce this. What version
of Windows, what architecture, and are you using
pandoc in powersrhell or regular cmd prompt?

Kevin Ushey [e-mail] writes:

I really wish I could distill this into a more minimal document, but the various edits I make all cause the segfault to vanish for me...

The associated file is here: https://dl.dropboxusercontent.com/s/juv01st40m5udbe/process_raw.md

If you want to reproduce, you can run this directly in the Windows command prompt (assume that Pandoc 2.3.1 is on the PATH)

curl https://dl.dropboxusercontent.com/s/juv01st40m5udbe/process_raw.md -O
pandoc process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained

When I run this, I see:

C:\Users\kevin\scratch\ouch>pandoc process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x74eb050

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x521daf0    0x74eb050
 * 0x521daf8    0x7ffb574f00f0 C:\Windows\System32\IMM32.DLL+0xf0
 * 0x521db00    0x521de60
 * 0x521db70    0x7ffb57e24581 C:\Windows\SYSTEM32\ntdll.dll+0x34581
 * 0x521db78    0x76de641
 * 0x521db80    0x7471d80
 * 0x521db88    0x7471d65
 * 0x521db90    0x7471ce0
 * 0x521db98    0x85
 * 0x521dba0    0x7686968
 * 0x521dba8    0x76de6a1
 * 0x521dbb0    0x8e834d8
 * 0x521dbb8    0x8e834f2
 * 0x521dbc0    0x7546928
 * 0x521dbc8    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dbd0    0x3e87e98 C:\Program Files\Pandoc\pandoc.exe+0x3a87e98
 * 0x521dbd8    0x74bb000
 * 0x521dbe0    0x3d6a75c C:\Program Files\Pandoc\pandoc.exe+0x396a75c
 * 0x521dbe8    0x3d6a7f1 C:\Program Files\Pandoc\pandoc.exe+0x396a7f1
 * 0x521dbf0    0x44d47ca C:\Program Files\Pandoc\pandoc.exe+0x40d47ca
 * 0x521dbf8    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc00    0x3d541e0 C:\Program Files\Pandoc\pandoc.exe+0x39541e0
 * 0x521dc08    0x74bb148
 * 0x521dc10    0x4643851 C:\Program Files\Pandoc\pandoc.exe+0x4243851
 * 0x521dc18    0x3d541d1 C:\Program Files\Pandoc\pandoc.exe+0x39541d1
 * 0x521dc20    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc28    0x3d541c1 C:\Program Files\Pandoc\pandoc.exe+0x39541c1
 * 0x521dc30    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc38    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc40    0x3e7a34a C:\Program Files\Pandoc\pandoc.exe+0x3a7a34a
 * 0x521dc48    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc50    0x454f519 C:\Program Files\Pandoc\pandoc.exe+0x414f519
 * 0x521dc58    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc60    0x4643821 C:\Program Files\Pandoc\pandoc.exe+0x4243821
 * 0x521dc68    0x3d6a73b C:\Program Files\Pandoc\pandoc.exe+0x396a73b

   ... (maximum recursion depth reached.)

This is with Pandoc as downloaded and installed with the installer at https://github.com/jgm/pandoc/releases/tag/2.3.1, using the 64-bit installer.

C:\Users\kevin\scratch\ouch>pandoc -v
pandoc 2.3.1
Compiled with pandoc-types 1.17.5.1, texmath 0.11.1.1, skylighting 0.7.3
Default user data directory: C:\Users\kevin\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

Any idea what might be going on? For what it's worth, I don't see this crash with the 32-bit build of Pandoc.

--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/jgm/pandoc/issues/5037

[kevinushey]

I'm using a 64-bit build of Windows 10 Pro, build 17134. I'm using Pandoc in a regular command prompt.

[kevinushey]

I'm no longer able to reproduce this crash with Pandoc 2.5.

[kevinushey]

Unfortunately, some of our users are seeing this again. One user is reporting the following:

Access violation in generated code when reading 0x8cc72f0

 Attempting to reconstruct a stack trace...

   Frame    Code address
 * 0x561dac0    0x4018450 C:\PROGRA~1\RStudio\bin\pandoc\pandoc.exe+0x3c18450

pandoc.exe: internal error: evacuate: strange closure type 0
    (GHC version 8.6.1 for x86_64_unknown_mingw32)
    Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug

It looks like the latest release of GHC is 8.6.3. Are there plans to bump the GHC version in the next release of Pandoc, in case that resolves whatever might be going on here?

[jgm]

I just changed the appveyor setup to use ghc 8.6.2 (the latest on chocolatey).
We can see if that fixes things. If the build completes, you should be able to get a binary from artifacts.

[kevinushey]

Thanks! I'll report back if we learn anything more.

[znmeb]

Just hit this a few minutes ago:

https://github.com/rstudio/rstudio/issues/3661#issuecomment-458795874

It appears solidly reproducible - not intermittent - and I can post the source document if that's any help.

[znmeb]

Good news - the 32-bit Pandoc release https://github.com/jgm/pandoc/releases/download/2.5/pandoc-2.5-windows-i386.msi works!

[jgm]

@znmeb as noted above, this was caused by a bug in the ghc compiler. We fixed it by using the latest ghc (8.6) for our builds. You should be fine if you use our latest release binary, or compile from source using ghc 8.6.

[jgm]

See #4283

[jgm]

@kevinushey did the version compiled against 8.6 work for you?
I suspect this issue can be closed.

[znmeb]

@jgm @kevinushey I got an "error 11" today on 64-bit Windows 10 with a freshly-installed RStudio v1.2.1256-3. I won't be back at that machine till tomorrow but it was reproducible and using the released 32-bit Pandoc from this repo fixed it.

[dpprdan]

RStudio v1.2.1256-3 ships with pandoc 2.5 which has been compiled with GHC version 8.6.1, apparently. See the end of my comment here: https://github.com/rstudio/rstudio/issues/4072#issuecomment-458979070

@jgm Is there a pandoc version compiled with a GHC version > 8.6.1 available somewhere? I cannot build it myself.

@znmeb Can you post your source document to see if we can reproduce the error?

[agusmba]

@dpprdan you can always test the latest and greatest at appveyor. They are built with ghc 8.6.2 currently.

[mb21]

Or since a few days jgm switched pandoc from appveyor to https://circleci.com/gh/jgm/pandoc

[dpprdan]

@agusmba @mb21 That's great, but I don't see where I can get a pandoc.exe there to test it together with RStudio.

[mb21]

Can't find it in CircleCI either, but here's one: https://ci.appveyor.com/project/jgm/pandoc/build/job/a1xjd0spdsv706f9/artifacts

[znmeb]

Here's the R Markdown dialog:

==> rmarkdown::render('C:/Users/znmeb/Projects/sqlpetr/vignettes/building-the-dvdrental-docker-image.Rmd',  encoding = 'UTF-8', output_dir = 'C:/Users/znmeb/AppData/Local/Temp/RtmpuypBtl/preview-22d826a73966.dir');


processing file: building-the-dvdrental-docker-image.Rmd
  |......................                                           |  33%
   inline R code fragments

  |...........................................                      |  67%
label: setup (with options) 
List of 1
 $ include: logi FALSE

  |.................................................................| 100%
  ordinary text without R code


"C:/Program Files/RStudio/bin/pandoc/pandoc" +RTS -K512m -RTS building-the-dvdrental-docker-image.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output pandoc1c345a96125.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Users\znmeb\OneDrive\Documents\R\win-library\3.5\rmarkdown\rmd\h\default.html" --highlight-style pygments --css "C:\Users\znmeb\OneDrive\Documents\R\win-library\3.5\rmarkdown\rmarkdown\templates\html_vignette\resources\vignette.css" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" --filter pandoc-citeproc 
output file: building-the-dvdrental-docker-image.knit.md


Access violation in generated code when reading 0xffffffffffffffff

 Attempting to reconstruct a stack trace...

   Frame    Code address
 * 0x561dac0    0x3fc9711 C:\PROGRA~1\RStudio\bin\pandoc\pandoc.exe+0x3bc9711

Error: pandoc document conversion failed with error 11
Execution halted

The R Markdown document is a vignette - the source is https://github.com/smithjd/sqlpetr/blob/master/vignettes/building-the-dvdrental-docker-image.Rmd

I haven't been able to find the knitted ordinary Markdown file though.

[jgm]

I see that I switched to using ghc 8.6.2 for the
appveyor builds in January. Pandoc 2.5 was released
in November, so that explains why the released
binaries have the issue.

As noted you can get build artifacts from the dev
version of pandoc (future 2.6) from appveyor.
e.g.
https://ci.appveyor.com/project/jgm/pandoc/build/job/npgqhnf9v9xgy1vq/artifacts

We did switch to CircleCI, but only for linux CI;
for Windows we still have appveyor.

Please test with the recent build and report back; if
it fixes the issue, I will try to do a 2.6 releases
before long.

[znmeb]

Well ... it looks like the 2.6 still has problems. I've created a pandoc-testing branch in the repo - this is MIT licensed and I can leave the branch there until you can figure out what's happening.

https://github.com/smithjd/sqlpetr/tree/pandoc-testing

The action is all in the vignettes directory. Here's the log of several consecutive runs of the pandoc_tester.R script:

https://github.com/smithjd/sqlpetr/blob/pandoc-testing/vignettes/pandoc_tester.console

All of this seems to be working normally on Linux so my plan is to leave the branch there as long as you need it and go back to my development in my regular branch.

[jgm]

@znmeb In order to isolate this, we really need a small reproducible example that just uses pandoc, not RStudio.

If you're able to extract the intermediate file building-the-dvdrental-docker-image.utf8.md then that would be a start, since the log you give shows the exact pandoc command rstudio uses on this file.

[jgm]

Another thought: your command line uses pandoc-citeproc; I assume you're probably using a version of pandoc-citeproc compiled using older ghc, so it's possible (though not likely) that the problem traces to this. Anyway, it would really be great if you could get a repro that just uses pandoc.

[jgm]

@znmeb can you try just running pandoc on the sample linked in the first comment on this issue, using the same command line as @kevinushey used? It would be helpful to know whether you get the issue on that file with the pandoc 2.6 binary. We never heard back about this.

[znmeb]

OK - I've got my test case narrowed down a bit. The input file is

https://github.com/smithjd/sqlpetr/blob/pandoc-testing/vignettes/building-the-dvdrental-docker-image.utf8.md

There's a command file to run it with both versions of Pandoc at https://github.com/smithjd/sqlpetr/blob/pandoc-testing/vignettes/pandoc_tester.cmd.

Even though RStudio is out of the picture, there's still some inputs coming from the rmarkdown library, so I've saved a zip archive of that if you need it at https://github.com/smithjd/sqlpetr/blob/pandoc-testing/vignettes/rmarkdown.zip

Note that the symptom of the crash changes between runs for both versions of Pandoc. This and the fact that it all works on Linux leads me to believe that there's something broken in the Haskell Windows runtime rather than the Pandoc code.

I'll try Kevin's test case and post the results in a separate comment.

[znmeb]

@kevinushey @jgm I've got a new repo for Kevin's test case. I had to comment out the images to get Pandoc to run but other than that it's the same Markdown file. It crashes with 2.5 from RStudio and works with 2.6 from the 64-bit MSI installer.

https://github.com/znmeb/pandoc-tests

C:\Users\znmeb\Projects\pandoc-tests>pandoc_tester.cmd

C:\Users\znmeb\Projects\pandoc-tests>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\znmeb\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Users\znmeb\Projects\pandoc-tests>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when reading 0xffffffffffffffff

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x561dac0    0x3e297fa C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3a297fa


C:\Users\znmeb\Projects\pandoc-tests>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.5
Default user data directory: C:\Users\znmeb\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Users\znmeb\Projects\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

[jgm]

@znmeb thank you. So, 2.6 works with kevinushey's case but not with yours.
I tried spinning up a Windows box to test. Couldn't get an error but I didn't download all the associated files and try with your exact command line. (Would have had to install pandoc-citeproc as well, and Windows frustrates me.)

Could you try just paring down your command line to see what causes the problem? First, remove the --filter pandoc-citeproc. Then try taking out some of the other things, e.g. --self-contained.

[znmeb]

OK - I'll be on Windows again tomorrow (PST8PDT time zone).

[dpprdan]

I've also tried https://github.com/znmeb/pandoc-tests and experimented with the command line options. I have used v2.5 from the RStudio install and v2.6 from here: https://ci.appveyor.com/api/buildjobs/90ndpd77k3jumcab/artifacts/windows%2Fpandoc-windows-x86_64.msi (hoping that this direct link works).

So @znmeb's example failes with 2.5 💥 and succeeds with 2.6 ✅. This is the base example, so all changes mentioned below are relative to this.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when reading 0xffffffffffffffff

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x561daf0    0x3fc9711 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3bc9711
 * 0x561daf8    0x7ff8f4aa00f0 C:\WINDOWS\System32\IMM32.DLL+0xf0
 * 0x561db00    0x561de60
 * 0x561db70    0x7ff8f75abaa1 C:\WINDOWS\SYSTEM32\ntdll.dll+0x3baa1
 * 0x561db78    0x7a668d1
 * 0x561db80    0x88ca578
 * 0x561db88    0x88ca555
 * 0x561db90    0x88ca4d0
 * 0x561db98    0x85
 * 0x561dba0    0x8ff68a8
 * 0x561dba8    0x7a66931
 * 0x561dbb0    0x7b940d0
 * 0x561dbb8    0x7b940ea
 * 0x561dbc0    0x7b053b8
 * 0x561dbc8    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dbd0    0x40e5b88 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3ce5b88
 * 0x561dbd8    0x8807c68
 * 0x561dbe0    0x8807c31
 * 0x561dbe8    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dbf0    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dbf8    0x40e5b68 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3ce5b68
 * 0x561dc00    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dc08    0x8807db0
 * 0x561dc10    0x7ae94b3
 * 0x561dc18    0x7a102a0
 * 0x561dc20    0x7a102b8
 * 0x561dc28    0x7a102d0
 * 0x561dc30    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc38    0x40d800a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3cd800a
 * 0x561dc40    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc48    0x4904359 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x4504359
 * 0x561dc50    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc58    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc60    0x40536d3 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3c536d3
 * 0x561dc68    0x4126b69 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3d26b69

   ... (maximum recursion depth reached.)


C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.5
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Remove --self-contained 2.5 ✅ 2.6 ✅

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.5
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Remove --email-obfuscation none: v2.5 💥 , v.2.6 ✅

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x7a71080

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x561daf0    0x7a71080
 * 0x561daf8    0x7ff8f4aa00f0 C:\WINDOWS\System32\IMM32.DLL+0xf0
 * 0x561db00    0x561de60
 * 0x561db70    0x7ff8f75abaa1 C:\WINDOWS\SYSTEM32\ntdll.dll+0x3baa1
 * 0x561db78    0x87cf8d1
 * 0x561db80    0x8be5578
 * 0x561db88    0x8be5555
 * 0x561db90    0x8be54d0
 * 0x561db98    0x85
 * 0x561dba0    0x88ea7c8
 * 0x561dba8    0x87cf931
 * 0x561dbb0    0x8d6c470
 * 0x561dbb8    0x8d6c48a
 * 0x561dbc0    0x8707db0
 * 0x561dbc8    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dbd0    0x40e5b88 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3ce5b88
 * 0x561dbd8    0x7b3f000
 * 0x561dbe0    0x7a44fc9
 * 0x561dbe8    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dbf0    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dbf8    0x40e5b68 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3ce5b68
 * 0x561dc00    0x488920a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x448920a
 * 0x561dc08    0x7b3f148
 * 0x561dc10    0x7bd847b
 * 0x561dc18    0x790e2a0
 * 0x561dc20    0x790e2b8
 * 0x561dc28    0x790e2d0
 * 0x561dc30    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc38    0x40d800a C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3cd800a
 * 0x561dc40    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc48    0x4904359 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x4504359
 * 0x561dc50    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc58    0x49f8661 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x45f8661
 * 0x561dc60    0x40536d3 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3c536d3
 * 0x561dc68    0x4126b69 C:\Program Files\RStudio\bin\pandoc\pandoc.exe+0x3d26b69

   ... (maximum recursion depth reached.)


C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.5
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Add --standalone: v2.5 ✅ , v2.6 💥 ❗️

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.5
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x7c24080

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x57adaf0    0x7c24080
 * 0x57adaf8    0x1

Remove --self-contained, add --standalone 2.5 ✅ 2.6 ✅

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" --version
pandoc 2.5
Compiled with pandoc-types 1.17.5.4, texmath 0.11.1.2, skylighting 0.7.4
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2018 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/RStudio/bin/pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.5
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Temp\pandoc_test>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

[znmeb]

@dpprdan Thanks!! I'm curious - what's the Haskell equivalent of gdb and does it work on Windows?

[jgm]

So --self-contained is what triggers the issue on 2.6, if I'm understanding correctly?

gdb: see https://wiki.haskell.org/Debugging
But to get the -xc stack traces, you need to compile with special options.

[dpprdan]

@jgm No, it is "C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone that is failing, so the --self-contained --standalone combo compaired to the others.

From my tests over at https://github.com/rstudio/rstudio/issues/4072#issuecomment-458979070 (and my post further down) it seems that the bug is also quite dependent on the content of the document (and, like here, the comand line options used).

[mkoohafkan]

For what it's worth, I've hit this issue (in Windows 10 + pandoc release, haven't tried the latest build) both with self-contained and without. In most cases, adding or deleting do-nothing lines of code in a chunk and sometimes even a blank line will fix the problem. I've never been able to reproduce it on e.g. a Travis build. I'm not sure if the options are actually the issue, it seems like something much weirder.

[jgm]

so the --self-contained --standalone combo compaired to the others.

Well, --self-contained implies --standalone, so this combo is equivalent to --self-contained by itself.

I agree that this is probably a GHC runtime issue. It seems to be partly dependent on the exact content of the document and partly dependent on the GHC version used to compile pandoc. If we can isolate a bit better what causes it, we might be able to submit a bug report to GHC. @mkoohafkan it would be good to know if you experience any problems with the latest release.

[jgm]

See https://github.com/haskell-stdio/stdio/issues/14 for a related report. (If you have ghc 8.6.3 installed on your windows box, you might try reproducing the case there, which is very simple.) There is also a link to a possible workaround, which you might try.

[znmeb]

If removing --self-contained gives a reproducible success I can live with that for now. The self-contained HTML pages are kind of annoying; I like the images to be downloadable as-is rather than being regenerated by the browser.

[jgm]

Also https://stackoverflow.com/questions/48232637/haskell-stack-access-violation-error-during-build

Note: the windows binary is compiled using ghc 8.6.2, not 8.6.3.

[znmeb]

By the way, one can automate Windows testing using Microsoft's browser-test virtual machines in most cases. https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

[jgm]

"M. Edward (Ed) Borasky" [e-mail]
writes:

By the way, one can automate Windows testing using Microsoft's browser-test virtual machines in most cases. https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/

Is this a full VM on which I could install ghc and
pandoc? Or just something that runs the Edge browser
(that's how it seems to be advertised).

[znmeb]

It's a full VM - my recollection is that they work in either VMWare Player / Workstation or VirtualBox. They may also work in GNOME Boxes / Virtual Machine Manager. I remember trying them but don't remember if it was successful or whether I had to do qemu-img finagling to bring them up.

[mb21]

I'm not sure if the options are actually the issue, it seems like something much weirder.

It may not actually even be dependent on specific options / input, but simply be a GHC runtime issue that results in a segfault or not, depending on your current system memory state. So more likely to get triggered if you have little memory, or maybe on large documents. So can you guys try running the exact same command a couple of times and see whether it's actually deterministic?

Either way, if we're to report this to GHC, it should probably be reproducible with the newest GHC release...

[mkoohafkan]

@mb21 I think it is deterministic, as I would repeat the command and get a segfault. But I usually tried this in fairly quick succession, and didn't try anything like force-clearing memory before trying again. I'll do some more testing the next time I run into the issue.

[dpprdan]

@jgm

Well, --self-contained implies --standalone, so this combo is equivalent to --self-contained by itself.

That makes a lot of sense. However, the v2.6 I mention above fails for me with "C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone but succeeds if you leave --standalone out. For v2.5 it is the other way around. 🤷‍♂️

I tried this (https://github.com/jgm/pandoc/issues/5037#issuecomment-459322394) on another laptop (same Windows version, I think, and same pandoc versions), and I get the same results.

[jgm]

Daniel Possenriede [e-mail] writes:

@jgm

Well, --self-contained implies --standalone, so this combo is equivalent to --self-contained by itself.

That makes a lot of sense. However, the v2.6 I mention above fails for me with "C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone but succeeds if you leave --standalone out. For v2.5 it is the other way around. 🤷‍♂️

That's quite bizarre, because the code paths with
--self-contained --standalone and --self-contained
should be almost identical. There would only be a
small difference at the very beginning, in option parsing.

[znmeb]

@jgm What's the toolchain for building the Windows executable? Can it be cross-compiled on Linux?

[jgm]

Easiest method is to get the Haskell 'stack' tool and
do 'stack install' from the pandoc source directory.

https://docs.haskellstack.org/en/stable/README/

There's no cross-compilation at this point, as far as
I know.

[znmeb]

Thanks! I'll take a shot at it sometime next week - if I get snowed in it will be earlier. ;-)

[mkoohafkan]

OK, I have an R package vignette that fails when trying to build on my windows machine but succeeds on Travis. Upgrading to Pandoc 2.6 seems to have fixed the issue. The information below relates to v2.5 but may be helpful if it turns out the issue is not completely resolved by v2.6.

---

Offending vignette | Github Repo | Travis build

Summary: The crash can be avoided by changing the contents of the document, by changing certain options, or by rerunning the execution in quick succession.

I reproduce the error when trying to build the package with devtools::check() and/or by running rmarkdown::render directly on the file.

"C:/Users/mkoohafk/AppData/Local/Pandoc/pandoc" +RTS -K512m -RTS visit-processing.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output visit-processing.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Programs\R-3.5.2\library\rmarkdown\rmd\h\default.html" --highlight-style pygments --css "C:\Programs\R-3.5.2\library\rmarkdown\rmarkdown\templates\html_vignette\resources\vignette.css" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" 

Access violation in generated code when executing data at 0x98fe849

 Attempting to reconstruct a stack trace...

   Frame    Code address
 * 0x561daf0    0x98fe849
 * 0x561daf8    0x561dbb0

One interesting thing is that if I add the chunk option echo = FALSE to the second chunk, the document compiles just fine, even though the pandoc execution string is exactly the same:

output file: visit-processing.knit.md

"C:/Users/mkoohafk/AppData/Local/Pandoc/pandoc" +RTS -K512m -RTS visit-processing.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output visit-processing.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Programs\R-3.5.2\library\rmarkdown\rmd\h\default.html" --highlight-style pygments --css "C:\Programs\R-3.5.2\library\rmarkdown\rmarkdown\templates\html_vignette\resources\vignette.css" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" 

Output created: visit-processing.html

This implies an issue with the contents of the document.

If I remove the following header information, the document also processes without issue:

### remove this stuff
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Processing Visit Sheets}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
###

"C:/Users/mkoohafk/AppData/Local/Pandoc/pandoc" +RTS -K512m -RTS visit-processing.utf8.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output visit-processing.html --email-obfuscation none --self-contained --standalone --section-divs --template "C:\Programs\R-3.5.2\library\rmarkdown\rmd\h\default.html" --no-highlight --variable highlightjs=1 --variable "theme:bootstrap" --include-in-header "C:\Users\mkoohafk\AppData\Local\Temp\RtmpAPRo9n\rmarkdown-str46b46c2e6e4e.html" --mathjax --variable "mathjax-url:https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML" 

The main change here is that the --highlight-related options and a css file reference are changed.

You can get the intermediate files by running rmarkdown::render with the option clean = FALSE. This gives me files visit-processing.utf8.md and visit-processing.knit.md which appear to be identical. If I run pandoc on the original file (which causes the memory error but still generates the intermediate .md files) and then run the pandoc execution string directly in the command prompt, it will crash the same way---sometimes!

• If I keep rerunning it in quick succession, it will sometimes succeed.
• If I remove the --self-contained option, it will succeed every time.
• If I remove the --standalone, it will work---but when I first tried this, it would always fail...

If I remove a bunch of other options, it also compiles just fine, even with --stand-alone:

"C:/Users/mkoohafk/AppData/Local/Pandoc/pandoc" +RTS -K512m -RTS visit-processing.knit.md --to html4 --from markdown+autolink_bare_uris+ascii_identifiers+tex_math_single_backslash+smart --output visit-processing.html --self-contained --standalone 

[kevinushey]

Sorry, I've been out of office for a couple weeks so haven't been following along / testing to see if and where things are fixed.

That said, we're planning to update Pandoc to 2.6 and will be publishing this is as part of a new preview release in the near future: https://github.com/rstudio/rstudio/pull/4274

[znmeb]

@kevinushey I've created a testing repo for my own testing on Windows - https://github.com/znmeb/pandoc-tests

Your specific failure case works with 2.6 pandoc on Windows now. Mine doesn't, but removing self-contained makes it work, and I'm not a huge fan of self-contained anyhow.

[dpprdan]

@znmeb @kevinushey's case is not (completely) fixed with pandoc 2.6 (at least the version I tried), see my comments here and here.

[znmeb]

@dpprdan Ah - still open then? Should I invite you and @kevinushey to the repo?

[dpprdan]

@znmeb It is public already, is it not? I.e. I've used your repo ( https://github.com/znmeb/pandoc-tests ). 😉

[znmeb]

@dpprdan Yeah, MIT license. I just wondered if you wanted to put stuff in as well.

[dpprdan]

Update: I've put my tests from above into a .cmd and also added tests for --to html5. I've sent a PR to @znmeb if someone wants to try these as well.

I still get failures with v2.6.1 (built with GHC_VERSION=8.6.2)

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" --version
pandoc 2.6.1
Compiled with pandoc-types 1.17.5.4, texmath 0.11.2, skylighting 0.7.6
Default user data directory: C:\Users\daniel\AppData\Roaming\pandoc
Copyright (C) 2006-2019 John MacFarlane
Web:  http://pandoc.org
This is free software; see the source for copying conditions.
There is no warranty, not even for merchantability or fitness
for a particular purpose.

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x7dc5080

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x57ddaf0    0x7dc5080
 * 0x57ddaf8    0x1


C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html5 --output process_raw.html --email-obfuscation none --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x881ba01

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x57ddaf0    0x881ba01
 * 0x57ddaf8    0x1


C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html5 --output process_raw.html --email-obfuscation none
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html5 --output process_raw.html --self-contained
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html5 --output process_raw.html --email-obfuscation none --self-contained --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>"C:/Program Files/Pandoc/pandoc" process_raw.md --from markdown --to html5 --output process_raw.html --email-obfuscation none --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

C:\Users\daniel\Documents\R\github\pandoc-tests>pause

[dpprdan]

Could these help: https://ghc.haskell.org/trac/ghc/ticket/13112, and https://github.com/haskell-tools/haskell-tools/issues/277#issuecomment-332917356, via https://github.com/commercialhaskell/stack/issues/3765#issuecomment-460651087
(I know nothing about Haskell, so sorry for the noise if it isn't helpful).

[kevinushey]

Do you have any better luck with GHC 8.6.3? Might also be worth testing with older versions of the toolchain as well.

[aphalo]

Could this error be indirectly triggered by a file-access problem?

I have three different keyboard layouts installed: Finnish, UK English and Latin American Spanish. I normally use the Finnish keyboard layout.

This first test I did with the Finnish keyboard layout enabled.
The User Guide of 'ggpmisc' (user-guide.Rmd) consistently triggers the same Pandoc Error 11 and the access violation, but very occasionally I receive instead locked file error for user-guide.R
Furthermore, if I delete user-guide.R before rebuilding the documentation, knitting seems to consistently succeed and if I do not delete it, it seems to consistently fail.

I also see that only in the case of building just the documentation (Build > More.. > Documentation in RStudio) the error is consistently triggered only with the Finnish keyboard layout active but not with the UK English or Latin American Spanish layouts. If I switch back to the Finnish layout building the documentation succeeds until I restart RStudio. On the other hand Build > Check,, Build > More..> Source Package or building the vignette when building the documentation with pkgdown they all fail whichever keyboard layout is active.

Setting self_contained: FALSE solves the problem in every cases and with any of the three keyboard layouts active.

I am on Windows 10 Academic in a laptop with FI, UK and ESP keyboard layouts installed. RStudio 1.2.1326.

[kevinushey]

I tried testing with Pandoc 2.7.1 on Windows (x64) and am still seeing a segfault on Windows with @znmeb's pandoc tester.

C:\Users\kevin\scratch\pandoc\pandoc-tests>pandoc process_raw.md --from markdown --to html4 --output process_raw.html --email-obfuscation none --self-contained --standalone
[WARNING] Could not convert TeX math '\textrm{TWS} = \sqrt{u^2 + v^2}', rendering as TeX

Access violation in generated code when executing data at 0x7b3d080

 Attempting to reconstruct a stack trace...

   Frame        Code address
 * 0x57edac0    0x7b3d080

I'll see if I can learn anything more...

[jgm]

We're still building with ghc 8.6.2 (on appveyor, bc that's what's in choco).
If someone wants to try with 8.6.3/4, maybe there will be an improvement.

[kevinushey]

Giving that a shot with LTS-13.12.

[kevinushey]

Some good news. I'm not able to reproduce the crashes using Pandoc 2.7.1 built with stack + the LTS-13.12 resolver (with GHC-8.6.4).

[dpprdan]

Some good news. I'm not able to reproduce the crashes using Pandoc 2.7.1 built with stack + the LTS-13.12 resolver (with GHC-8.6.4).

Same here. @kevinushey's pandoc version is available at https://s3.amazonaws.com/rstudio-buildtools/pandoc-rstudio/2.7.1/pandoc-2.7.1-windows-x86_64.zip

I asked @Mistuke (chocolatey's ghc Package Maintainer) about an update of ghc on chocolatey.

[Mistuke]

I'm waiting for the GHC release manager to finish the tarballs for 32 bit windows to release 8.6.4 on chocolatey which is why I haven't put it up yet.

It would make for a very weird user experience otherwise having them point to different versions :) Hopefully he has it sorted soon. Sorry for the wait.

[Mistuke]

Oh and ghc 8.6.3 was there but I pulled it since it's a Dao release. It's not stable for anything on Windows.

[jgm]

Tamar Christina [e-mail] writes:

I'm waiting for the GHC release manager to finish the tarballs for 32 bit windows to release 8.6.4 on chocolatey which is why I haven't put it up yet.

It would make for a very weird user experience otherwise having them point to different versions :) Hopefully he has it sorted soon. Sorry for the wait.

Terrific! It looks like this will solve our problem when it's ready.

[jgm]

We could also consider switching back to stack in appveyor for our binary builds.
I can't recall now why we switched to cabal, but I think it was to get access to the newer ghc versions. Now that stack is caught up, we could switch back.

[Mistuke]

Switching to stack won't help you if you build 32 bit releases as well. 32 bit windows is a broken time bomb until a patch that is in 8.6.3, and so 8.6.4.

Stack also doesn't have a 32 bit 8.6.4 quite simply because one wasn't made.

[jgm]

Okay, right. @Mistuke any estimates on when the 8.6.4 release will be ready?

[Mistuke]

Just asked the release manager, he's busy testing the new CI setup now for
it. It was delayed because of the switch to a new way to generate the
bindists.

The chocolatey package is already written I just need the build so should
be quick after that.

On Mon, Mar 18, 2019, 18:57 John MacFarlane [e-mail]
wrote:

Okay, right. @Mistuke https://github.com/Mistuke any estimates on when
the 8.6.4 release will be ready?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jgm/pandoc/issues/5037#issuecomment-474032291, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABH3KRP1s9E6bLyWekyjSRxJkbUEoC8Vks5vX9OVgaJpZM4YIjCC
.

[Mistuke]

I've submitted the package for 8.6.4 https://chocolatey.org/packages/ghc/8.6.4 once it passes through automated validation it should be made available. Should take 30-60mins

[jgm]

Great, thank you @Mistuke. I have told appveyor to use 8.6.4.
If this build completes, it just remains for someone to test the binary (from artifacts):
https://ci.appveyor.com/project/jgm/pandoc/build/job/x0cmmyf5pfbva8kp

[kevinushey]

I'll test that build when it's ready.

[jgm]

Unfortunately we're hitting the 60 minute limit on appveyor, so the build is failing.
I've asked for more time, which I see has been granted for some other projects. Fingers crossed!

[kevinushey]

:( Looks like the post is still awaiting moderation. I hope they're receptive to the request.

[jgm]

Got tired of waiting, so I hacked appveyor by running a partial build and caching the results.
The build with ghc 8.6.4 has now completed, so @kevinushey you should be able to test.

[kevinushey]

Just gave it a spin; happy to report that both the x86 and x64 builds pass the tests!

[jgm]

Excellent! Then we can finally close this. Thanks, all, for your help.

[XiaoqiLu]

So... what's the solution?

[jgm]

Solution is to use pandoc 2.7.2 compiled with ghc 8.6.4 (as our release binaries are).