Nuitka segfaults in ld call on aarch64

[itdaniher]

Python3.6.3
Linux 4.4
GCC 7.2.0
Aarch64

trying to compile a simple 'hello world' with:

pipenv run nuitka --show-scons --verbose print.py

segfaults as per in a call to ld as per:

https://gist.github.com/itdaniher/6343be4097dd35e78df5df06b3f88c8b

note that my compiler / linker chain is tested-working, built python and the kernel from source.

[kayhayen]

Any segfault of "ld" is its own. For all I know, the Debian tool chain works fine with Nuitka.

[shubjain26]

Getting the same error, while compiling on Jetson Tx2 (aarch64)

@itdaniher did you find any workaround

gcc -o hello.dist/hello.exe -Wl,-R,'$ORIGIN' -Wl,--disable-new-dtags -Wl,-b -Wl,binary -Wl,hello.build/__constants.bin -Wl,-b -Wl,elf64-littleaarch64 -Wl,-defsym -Wl,constant_bin=_binary_hello_build___constants_bin_start hello.build/__frozen.o hello.build/module.__main__.o hello.build/__constants.o hello.build/__helpers.o hello.build/MainProgram.o hello.build/CompiledCellType.o hello.build/CompiledFunctionType.o hello.build/CompiledMethodType.o hello.build/CompiledGeneratorType.o hello.build/CompiledFrameType.o hello.build/CompiledCodeHelpers.o hello.build/InspectPatcher.o hello.build/MetaPathBasedLoader.o hello.build/fibers_gen.o -L/usr/lib -ldl -lpython2.7
collect2: fatal error: ld terminated with signal 11 [Segmentation fault], core dumped
compilation terminated.
scons: *** [hello.dist/hello.exe] Error 1
scons: building terminated because of errors.

[kayhayen]

How about adding more memory, or swap file? Since you are having this on 64 bits, I wouldn't think it's a 2GB limit hit.

[itdaniher]

@shubjain26 I did not. I just stopped using Nuitka as I could not get it to behave and @kayhayen just chalked it up to ld.

[benelgiac]

Just so you know, even with the latest binutils 2.32 the problem is still there. At no point RAM is exhausted and I have a 16 GB swap file. Very quickly reproducible if you have a TX2.

[kayhayen]

@benelgiac You do realize, that a segmentation fault of ld when used with Nuitka is not a bug of Nuitka, but of ld and your specific binary? There is no way, I will debug ld crashes. But if it's reproducible, turn it into a minimal test case of ld and take it to the binutils maintainers.

[benelgiac]

Of course I do, I am not blaming anyone or asking you to debug anything but please consider that it's already the third tx2 user reporting the same issue, so probably it's worth understanding what's going on. Even if it has nothing to do with Nuitka, I think it would be useful if we could find a solution on this thread.

Since I cannot, am not able and do not want to even start considering looking into ld, I have tried to overcome the problem by installing lld, which should be a drop-in replacement for ld. I have compiled it and installed on TX2 and got an error as well, slightly better than a crash at least:

ld: error: unknown -format value: elf64-littleaarch64 (supported formats: elf, default, binary)
collect2: error: ld returned 1 exit status

Maybe this tells you something more than it tells me (i.e. lld is not very much a "drop-in" replacement)

Facts:

• ld version 2.30 segments on simple hello world nuitka compilation
• ld version 2.32 segments on simple hello world nuitka compilation
• lld version 9.0.0 gives error on simple hello world nuitka compilation
• I can successfully link other complex projects (i.e. OpenCV) on TX2 using both ld and lld
• I have successfully used the same version of nuitka on tx2 on previous version of TX2 linux (based on ubuntu 16.04 and supposedly with a different version of ld)

If I will be able to spend more time on this, I will try to make it reproducible without nuitka.

[kayhayen]

@benelgiac You might want to try and change SingleExe.scons so it doesn't use resource_mode = "linker" but instead value code. I can imagine that it makes it easier for ld, or that your lld will then work.

Should it be that, coming up with a bug report to the binutils people would be possible.

Yours,
Kay

[benelgiac]

Hi @kayhayen I have not yet been able to test your suggestion, but in the meantime I found a workaround. Just using clang instead of gcc "fixes" the problem. It must be that the ld/lld bug is triggered only when attempting to link object files generated by gcc.

I am happy with this at the moment. Thanks for you help!

[kayhayen]

I wonder, if we should get into the business of warning about using clang on aarch64, as this may be happening a lot.

Yours,
Kay