Nim: NIM compiler looses track of sub-processes on Linux running MinGW

Created on 10 May 2017 · 8Comments · Source: nim-lang/Nim

This is a follow up from the NIM forum trying to avoid polluting it with listings. This is my environment

jordan@black:~/Desktop/problem$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 9.0 (stretch)
Release:    9.0
Codename:   stretch

jordan@black:~/Desktop/problem$ i686-w64-mingw32-gcc --version
i686-w64-mingw32-gcc (GCC) 6.3.0 20170425
[..]

jordan@black:~/Desktop/problem$ nim --version
Nim Compiler Version 0.16.1 (2017-05-09) [Linux: amd64]
[..]
git hash: fa3436fb657141127038d88431b4aad113c27cf6
active boot switches: -d:release

jordan@black:~/Desktop/problem$ cat nim.cfg 
cpu = i386
i386.windows.gcc.path = "/usr/bin"
i386.windows.gcc.exe = "i686-w64-mingw32-gcc"
i386.windows.gcc.linkerexe = "i686-w64-mingw32-gcc"

The problem is reproducible on my system when I dispatch NIM with a clean environment as in

env -i PATH=/usr/local/bin:/usr/bin:/bin nim cc --os:windows --verbosity:3 --listCmd problem.nim

where the source looks like

# nim test file: problem.nim

# C source equivalent of:
#  #define WIN32_LEAN_AND_MEAN
#  #include <windows.h>
#  #include <wincrypt.h>
#  printf(">>>  PROV_RSA_FULL=%d\n", PROV_RSA_FULL);

{.passC: "-DWIN32_LEAN_AND_MEAN".}
var PROV_RSA_FULL {.importc,
                    header: "<windows.h>",
                    header: "<wincrypt.h>".}: int

echo ">>>  PROV_RSA_FULL=", PROV_RSA_FULL

The NIM compiler just hangs at

[..]
problem.nim(8, 6) Hint: 104005 [Processing]
/usr/bin/i686-w64-mingw32-gcc -c  -w  [..] /home/jordan/Desktop/problem/nimcache/problem.c
/usr/bin/i686-w64-mingw32-gcc -c  -w  [..] /home/jordan/Desktop/problem/nimcache/stdlib_system.c

which results in a a process table like (some line feeds added)

jordan@black:~/Desktop/problem$ ps xafe
[..]
11369 pts/12   S+     0:00  |   \_ /usr/local/share/nim/bin/nim cc --os:windows --verbosity:3 --listCmd problem.nim
                                       PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                       PWD=/home/jordan/Desktop/problem
11372 pts/12   S+     0:00  |       \_ /bin/sh -c /usr/bin/i686-w64-mingw32-gcc -c  -w -DWIN32_LEAN_AND_MEAN  -I/usr/local/share/nim/lib -o /home/jordan/Desktop/problem/nimcache/problem.o /home/jordan/Desktop/problem/nimcache/problem.c
                                           PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                           PWD=/home/jordan/Desktop/problem
11374 pts/12   S+     0:00  |       |   \_ /usr/bin/i686-w64-mingw32-gcc -c -w -DWIN32_LEAN_AND_MEAN -I/usr/local/share/nim/lib -o /home/jordan/Desktop/problem/nimcache/problem.o /home/jordan/Desktop/problem/nimcache/problem.c
                                               PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                               PWD=/home/jordan/Desktop/problem
11376 pts/12   S+     0:00  |       |       \_ /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1 -quiet -I /usr/local/share/nim/lib -U_REENTRANT -D WIN32_LEAN_AND_MEAN /home/jordan/Desktop/problem/nimcache/problem.c -quiet -dumpbase problem.c -mtune=generic -march=pentiumpro -auxbase-strip /home/jordan/Desktop/problem/nimcache/problem.o -w -o /tmp/ccMA30uS.s
                                                   PATH=/usr/local/share/nim/bin:/usr/local/bin:/usr/bin:/bin
                                                   PWD=/home/jordan/Desktop/problem COLLECT_GCC=/usr/bin/i686-w64-mingw32-gcc
                                                   COLLECT_GCC_OPTIONS='-c' '-w' '-D' 'WIN32_LEAN_AND_MEAN' '-I' '/usr/local/share/nim/lib' '-o' '/home/jordan/Desktop/problem/nimcache/problem.o' '-mtune=generic' '-march=pentiumpro'
11373 pts/12   Z+     0:00  |       \_ [sh] <defunct>
[..]

Just to make sure the context is all right I have a look at the cgroups

jordan@black:~/Desktop/problem$ systemd-cgls
Control group /:
-.slice
[..]
├─user.slice
│ └─user-1000.slice
│   ├─session-2.scope
│   │ ├─ 3044 lightdm --session-child 12 21
[..]
│   │ ├─11369 /usr/local/share/nim/bin/nim cc --os:windows --verbosity:3 --li...
│   │ ├─11372 /bin/sh -c /usr/bin/i686-w64-mingw32-gcc -c  -w -DWIN32_LEAN_AN...
│   │ ├─11374 /usr/bin/i686-w64-mingw32-gcc -c -w -DWIN32_LEAN_AND_MEAN -I/us...
│   │ ├─11376 /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1 -quiet -I /usr/loca...

Tracing the last MinGW command I get

jordan@black:~/Desktop/problem$ strace -p 11376
strace: Process 11376 attached
write(2, "/usr/share/mingw-w64/include/bcr"..., 367
^Cstrace: Process 11376 detached
 <detached ...>

and the open file streams look like

jordan@black:~/Desktop/problem$ lsof | grep 11376
lsof: WARNING: can't stat() aufs [..]
lsof: WARNING: can't stat() tmpfs [..]
lsof: WARNING: can't stat() nsfs [..]
cc1       11376              jordan  cwd       DIR               0,56     4096    3410577 /home/jordan/Desktop/problem
cc1       11376              jordan  rtd       DIR              253,0     4096          2 /
cc1       11376              jordan  txt       REG              253,7 22042016     940171 /usr/lib/gcc/i686-w64-mingw32/6.3-win32/cc1
cc1       11376              jordan  mem       REG              253,0  1685264      58073 /lib/x86_64-linux-gnu/libc-2.24.so
cc1       11376              jordan  mem       REG              253,0  1063328      58094 /lib/x86_64-linux-gnu/libm-2.24.so
cc1       11376              jordan  mem       REG              253,0   105088      57708 /lib/x86_64-linux-gnu/libz.so.1.2.8
cc1       11376              jordan  mem       REG              253,0    14640      58089 /lib/x86_64-linux-gnu/libdl-2.24.so
cc1       11376              jordan  mem       REG              253,7   537448     131530 /usr/lib/x86_64-linux-gnu/libgmp.so.10.3.2
cc1       11376              jordan  mem       REG              253,7   422200     131324 /usr/lib/x86_64-linux-gnu/libmpfr.so.4.1.5
cc1       11376              jordan  mem       REG              253,7    97736     136991 /usr/lib/x86_64-linux-gnu/libmpc.so.3.0.0
cc1       11376              jordan  mem       REG              253,7  1738240     138496 /usr/lib/x86_64-linux-gnu/libisl.so.15.3.0
cc1       11376              jordan  mem       REG              253,0   153288      57359 /lib/x86_64-linux-gnu/ld-2.24.so
cc1       11376              jordan    0r     FIFO               0,10      0t0    7995392 pipe
cc1       11376              jordan    1w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    2w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    3r     FIFO               0,10      0t0    7995392 pipe
cc1       11376              jordan    4w      REG               0,37        0    7996413 /tmp/ccMA30uS.s
cc1       11376              jordan    5r      REG              253,7   237724     719406 /usr/share/mingw-w64/include/wincrypt.h
cc1       11376              jordan    6w     FIFO               0,10      0t0    8003585 pipe
cc1       11376              jordan    7r      REG              253,7    23915     709325 /usr/share/mingw-w64/include/bcrypt.h
cc1       11376              jordan    8w     FIFO               0,10      0t0    8003586 pipe

All the other commands below the NIM compiler are explicitly waiting for the child (wait4(<pid>,..)) except the shell which waits for any child (wait4(-1,..)).

I have no clue what is happening here.

OArch specific Stdlib

Source

mjfh

Most helpful comment

@yatesco no worries. I wasn't directing at anyone in particular, just saying it for anyone that wishes to look further into this issue. The osproc module is the likely culprit.

Glad you're enjoying Nim :)

dom96 on 3 Jul 2018

❤1 👍1

All 8 comments

@mjfh in my project's nim.cfg I'm using this:

# Windows cross-compile via mingw
@if crosswin:
  gcc.linkerexe = "x86_64-w64-mingw32-gcc"
  gcc.exe = "x86_64-w64-mingw32-gcc"
  gcc.path = "/usr/bin"
  gcc.options.linker = ""
  os = "windows"
  define:windows
@end

Yardanico on 10 May 2017

@mjfh oh, yeah, your example hangs forever on compilation even on my PC with mingw 64bits...

Yardanico on 10 May 2017

👍2

lol - this was my surprise as well.

Problem is that the MinGW compiler aborts with errors (see the NIM forum thread).

My solution is basically here.

And NIM correctly reports errors on native NimGW but this has a different process mangement. I do not see this annoyance a high priority as it is easily circumvented. It is just a bit annoying. So people should just know about.

Jordan

mjfh on 10 May 2017

👍1

Same issue on Debian. Running command via --listCmd:

gcc -c  -w -mno-ms-bitfields -I/usr/x86_64-w64-mingw32/include/ \
    -static  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.o \
    /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c'

results in (tail end):

/home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c:1247:1: error: expected ‘{’ at end of input
 }
 ^
In file included from /usr/x86_64-w64-mingw32/include/minwindef.h:146:0,
                 from /usr/x86_64-w64-mingw32/include/windef.h:8,
                 from /usr/x86_64-w64-mingw32/include/windows.h:69,
                 from /home/jamieson/nim-0.17.2/nimcache/stdlib_md5.c:12:
/usr/x86_64-w64-mingw32/include/winnt.h:138:25: note: the ABI of passing struct with a flexible array member has changed in GCC 4.4
 #define DECLSPEC_IMPORT __declspec (dllimport)
                         ^
/usr/x86_64-w64-mingw32/include/winnt.h:251:18: note: in expansion of macro ‘DECLSPEC_IMPORT’
 #define NTSYSAPI DECLSPEC_IMPORT
                  ^
/usr/x86_64-w64-mingw32/include/winnt.h:6895:5: note: in expansion of macro ‘NTSYSAPI’
     NTSYSAPI WORD NTAPI RtlCaptureStackBackTrace (DWORD FramesToSkip, DWORD FramesToCapture, PVOID *BackTrace, PDWORD BackTraceHash);
     ^

Maybe it was just md5, so took out md5; ten it hung on osproc. Also took that out and now strace says it's hanging on 31002.

write(1, "gcc -c  -w -mno-ms-bitfields -st"..., 213gcc -c  -w -mno-ms-bitfields -static -I/usr/x86_64-w64-mingw32/include/  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/stdlib_unicode.o /home/jamieson/nim-0.17.2/nimcache/stdlib_unicode.c 
) = 213
close(85)                               = 0
close(86)                               = 0
close(3)                                = 0
close(84)                               = 0
wait4(31002,

jamieson 31002  0.0  0.0   4336   808 pts/5    S+   21:26   0:00                          \_ /bin/sh -c gcc -c  -w -mno-ms-bitfields -static -I/usr/x86_64-w64-mingw32/include/  -I/home/jamieson/nim-0.17.2/lib -o /home/jamieson/nim-0.17.2/nimcache/compiler_shim.o /home/jamieson/nim-0.17.2/nimcache/compiler_shim.c

~ strace -p 31002
Process 31002 attached
wait4(-1,

jamieson99 on 17 Oct 2017

Also hangs on macOS:

brew install nim mingw-64
mkdir problem && cd problem
cat problem.nim:

# nim test file: problem.nim

# C source equivalent of:
#  #define WIN32_LEAN_AND_MEAN
#  #include <windows.h>
#  #include <wincrypt.h>
#  printf(">>>  PROV_RSA_FULL=%d\n", PROV_RSA_FULL);

{.passC: "-DWIN32_LEAN_AND_MEAN".}
var PROV_RSA_FULL {.importc,
                    header: "<windows.h>",
                    header: "<wincrypt.h>".}: int

echo ">>>  PROV_RSA_FULL=", PROV_RSA_FULL

cat nim.cfg:

cpu = i386
i386.windows.gcc.path = "/usr/local/bin/"
i386.windows.gcc.exe = "i686-w64-mingw32-gcc"
i386.windows.gcc.linkerexe = "i686-w64-mingw32-gcc"

env -i PATH=/usr/local/bin:/usr/bin:/bin nim cc --os:windows --verbosity:3 --listCmd problem.nim hangs.

Unfortunately, I am not skilled enough in nim to even call myself a nim newbie! so I haven't the first clue of how to debug this.

yatesco on 2 Jul 2018

Possibly an osproc issue?

dom96 on 2 Jul 2018

@dom96 - if that was to me, the fact I don't even know what osproc means should tell you at what level I am at with nim :-). I have spent too many decades being an "enterprise" developer (Java, Spring, Clojure) etc., I am absolutely bowled over with joy as I read about nim, but it is going to take a while before I can contribute.

yatesco on 2 Jul 2018

👍1

@yatesco no worries. I wasn't directing at anyone in particular, just saying it for anyone that wishes to look further into this issue. The osproc module is the likely culprit.

Glad you're enjoying Nim :)

dom96 on 3 Jul 2018

❤1 👍1

Was this page helpful?

0 / 5 - 0 ratings

Related issues

ospaths.isAbsolute: uncovering out of bound bugs after updating to 0.18.1 from 0.18.0: empty string and nil string now checked for out of bound errors

timotheecour · 3Comments

Support default arguments in sugar.`=>` and sugar.`->`

hlaaftana · 3Comments

[style guide] [low priority] Naming conventions for modules?

awr1 · 3Comments

deprecation syntax inconsistent for objects, not possible for const/let/var

timotheecour · 3Comments

[RFC] Error for exported procs that take or return private types

hlaaftana · 3Comments