Emscripten: Assertion `gotIndex == INVALID_INDEX' failed

[Beuc]

Hi,

I ran into an assertion error when linking zlib.

Steps to reproduce:

wget http://prdownloads.sourceforge.net/libpng/zlib-1.2.11.tar.gz
tar xf zlib-1.2.11.tar.gz 
cd zlib-1.2.11/
emconfigure env CFLAGS="-O3" ./configure --prefix $(pwd)/destdir
make

Output:

...
emcc -O3 -fPIC -D_LARGEFILE64_SOURCE=1 -DHAVE_HIDDEN -o libz.so.1.2.11 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo  -lc 
wasm-ld: /b/s/w/ir/cache/builder/emscripten-releases/llvm-project/lld/wasm/Symbols.cpp:115: void lld::wasm::Symbol::setGOTIndex(uint32_t): Assertion `gotIndex == INVALID_INDEX' failed.
Stack dump:
0.  Program arguments: /home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld -o libz.so.1.2.11 --allow-undefined --import-memory --import-table --lto-O0 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024 --relocatable 
 #0 0x00007f3d675d69f4 PrintStackTraceSignalHandler(void*) (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d79f4)
 #1 0x00007f3d675d473e llvm::sys::RunSignalHandlers() (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d573e)
 #2 0x00007f3d675d6ca8 SignalHandler(int) (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/../lib/libLLVM-10svn.so+0x6d7ca8)
 #3 0x00007f3d6a4a9730 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12730)
 #4 0x00007f3d66a547bb raise (/lib/x86_64-linux-gnu/libc.so.6+0x377bb)
 #5 0x00007f3d66a3f535 abort (/lib/x86_64-linux-gnu/libc.so.6+0x22535)
 #6 0x00007f3d66a3f40f (/lib/x86_64-linux-gnu/libc.so.6+0x2240f)
 #7 0x00007f3d66a4d102 (/lib/x86_64-linux-gnu/libc.so.6+0x30102)
 #8 0x00000000006ac4fb (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x6ac4fb)
 #9 0x00000000006c691b lld::wasm::GlobalSection::assignIndexes() (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x6c691b)
#10 0x00000000006b1466 (anonymous namespace)::Writer::run() (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x6b1466)
#11 0x00000000006adaaf lld::wasm::writeResult() (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x6adaaf)
#12 0x0000000000690ae7 (anonymous namespace)::LinkerDriver::link(llvm::ArrayRef<char const*>) (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x690ae7)
#13 0x000000000068b6d8 lld::wasm::link(llvm::ArrayRef<char const*>, bool, llvm::raw_ostream&) (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x68b6d8)
#14 0x000000000041eadb main (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x41eadb)
#15 0x00007f3d66a4109b __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2409b)
#16 0x000000000041e669 _start (/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld+0x41e669)
shared:ERROR: '/home/me/workdir/emtests/renpyweb/toolchain/emsdk/upstream/bin/wasm-ld -o libz.so.1.2.11 --allow-undefined --import-memory --import-table --lto-O0 adler32.lo crc32.lo deflate.lo infback.lo inffast.lo inflate.lo inftrees.lo trees.lo zutil.lo compress.lo uncompr.lo gzclose.lo gzlib.lo gzread.lo gzwrite.lo -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024 --relocatable' failed (-6)
make: *** [Makefile:282: libz.so.1.2.11] Error 1

$ emcc --version
emcc (Emscripten gcc/clang-like replacement) 1.38.42 (commit da4900b83f6bce7511189b9307569d6c9c344e36)

[supernow]

I got this error ,too

[sbc100]

Certainly looks like an lld bug. Will try to reproduce locally.

[Beuc]

I got it again when linking SDL2_mixer 2.0.1 and freebidi 0.19.2.
Let me know if you're interested in more traces.

[sbc100]

Fix is in progress in https://reviews.llvm.org/D66784. As a workaround (assuming are you are not actually trying to use MAIN_MODULE/SIDE_MODULE) you can make sure you are building everything without -fPIC.

[Beuc]

Are there instructions on how to manually test this patch? (it is still waiting for review at LLVM)
(I tried getting rid of all -fPIC but apparently I missed one and the error doesn't tell me where)
Building Emscripten from Source is quite vague for "upstream", and I don't know how the official binaries are built.

[pmp-p]

same question there :)

[kripken]

Oh, what's missing from the docs in that link? It links to the upstream docs and it mentions the extra flag -DLLVM_ENABLE_PROJECTS=lld;clang.

[Beuc]

The upstream doc has like 50 project-specific options that one may or may not need (the previous fastcomp doc set 5 precise options). Each build requires at least one hour, each mistake requires a complete rebuild. Consequently I don't want to spend the week-end running builds and finding a working configuration by trial-and-error, I'd rather use a tested set of options -- preferably the one you use for official builds.
With the added insurance that errors I'll report will come from the patch and not from my build :)

[kripken]

Oh, yeah - LLVM does have a lot of options. But you don't need any, in most cases just a standard cmake build should work, so

mkdir mybuild
cd mybuild
cmake .. [extra flags from emscripten docs]
make -j8

If you do need any special flags to work around a local issue (like if the cmake step fails on something), then the LLVM docs can be very helpful. I don't think there's anything we can add in the emscripten docs, except to add the code snippet I just wrote, would that help?

[Beuc]

Thanks for the info!

This is typically the kind of steps where, if one is already familiar with the target system, things appear natural, but if one is new, one has to assemble information from numerous pages. Also the LLVM documentation is adamant about reading the full documentation before trying anything, which is daunting. So providing explicit instructions, or pointing to the official build scripts (which I didn't find in e.g. .circleci) would help.

In either case, IMHO, the needed information include:

• where to checkout llvm from (repository "llvm-project" - https://github.com/llvm/llvm-project.git)
• how to identify to version to checkout (no tags for v10.x - always build "master"?)
• clarify whether clang needs to be checked-out separately or not (since we had to that with fastcomp, and that's optional now AFAIU; there's a warning about that in the LLVM doc)
• clarify whether "make install" is required or if the build is intended to be used in-place (installing would require carefully specifying a working install prefix, most probably non-default)

The cmake instructions seems to target ../llvm, rather than ...
There's a missing starting quote in the current documentation (-DLLVM_ENABLE_PROJECTS=lld;clang').
The phrase "using something like -DLLVM_..." was implying there was more configuration work/fiddling to perform, but since this fortunately doesn't seem to be the case, let's leave it out :)

I'm compiling as we speak, I'll add more remarks if I find some.

[Beuc]

Possibly, mention that this requires at least 70GB build space and 2h compilation. If there's a simple way to lower these requirements, it would be good to note them.

It seems my LLVM is 10x slower than the Emscriten build when compiling SDL2, I guess I missed an option, possibly CMAKE_BUILD_TYPE=Release. I'd have to spend 2 more hours to test this hypothesis.

-j8 will require a huge amount of memory at the linking phase and the build will probably fail (even with -j1 linking clang filled my 16GB RAM), so I would recommend a more conservative value.

--

binaryen/wasm-opt gives me Unknown option '--asyncify' - the binaryen port doesn't fetch the right version (86 vs. 89).
This may be a bug in the current port file.
If not, we also need:

• how to identify the required binaryen version
• the need to specify the location of binaryen in emconfig with BINARYEN_ROOT

--

At last I can build my project for the first time, but the browser greets me with:
uncaught exception: could not load memory initializer http://localhost:8000/libz.raw.js.mem
This file is nowhere to be found. Though given all the above, I don't have a high confidence in my build.
I hope this emphasizes that testing a simple patch could be made smoother! ;)

[Beuc]

2 long builds and testings later I eventually refined to what I wanted to read from emscripten.org:

cmake ../llvm -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS='lld;clang' -DLLVM_TARGETS_TO_BUILD="host;WebAssembly" -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF

--

This particular issue sounds fixed, but I get several other errors in my project.
No more time to investigate this week :/

[kripken]

Thanks @Beuc!

I think you're right, we should have an example of reasonable build flags for LLVM - I didn't realize the defaults are to debug, and that debug is so painful :( Thanks for the findings, I opened https://github.com/emscripten-core/emscripten/pull/9371 with them.

[sbc100]

Sorry, the upstream change is not finished so if you are trying to build llvm so that you can integrate https://reviews.llvm.org/D66784 then I would not advise that for now.

A much quicker solution would be to filter out -fPIC in the emcc.py driver. You can do this in parse_args.. you can either assert that -fPIC is not in the newargs in order to find out which of your libraries is being compiled wrong.. or you can just filter it out to have it ignored by the underlying clang.

[Beuc]

I tried preventing -fPIC in emcc.py, good idea. The project now compiles (with tot-upstream), and I didn't get the weird cython error that I got when trying out D66784.

Thanks for adding the LLVM compilation line. There's a mistake, and this doesn't treat the other doc points I raised, but I'll open a separate bug/pr.

[pmp-p]

A much quicker solution would be to filter out -fPIC in the emcc.py driver. You can do this in parse_args.. you can either assert that -fPIC is not in the newargs in order to find out which of your libraries is being compiled wrong.. or you can just filter it out to have it ignored by the underlying clang.

out of topic but can that method be used to easy fix removing "-s USE_*" and other things when emcc is required to act as a preprocessor ( clang -E ) ?

[sbc100]

A much quicker solution would be to filter out -fPIC in the emcc.py driver. You can do this in parse_args.. you can either assert that -fPIC is not in the newargs in order to find out which of your libraries is being compiled wrong.. or you can just filter it out to have it ignored by the underlying clang.

out of topic but can that method be used to easy fix removing "-s USE_*" and other things when emcc is required to act as a preprocessor ( clang -E ) ?

I'm not quite sure what you are asking. Are you saying that the '-s USE_*' flags are being passed down the the underlying clang when -E is used? If this is the case it is certainly a bug, please open a seperate issue. If not, perhaps I misunderstand you.

[pmp-p]

it is certainly a bug, please open a seperate issue

i wasn't sure emcc is meant to be used also as preprocessor with multiple files input ( but we need to in micropython javascript port) so i will open an issue thanks for clarifying.

edit/ my bad issue was not with input args but with number of files input : "I tried defining CPP = emcc -E but this doesn't like multiple input files"
sorry for the noise

[Beuc]

Hi!
Are there news on static linking with -fPIC? :)

[sbc100]

The upstream change is still not ready: https://reviews.llvm.org/D66784. Once it lands this issue can closed. I should be able to take another look before the end of the week.

[Beuc]

I did check the reviews.llvm.org link but the information there is apparently stalled since Aug 26th.
Hence why I asked.
Let me know if there is was a better way :)

[sbc100]

The code is simply not ready to review yet and no work has been done on it since that date. There is no better way. Asking about status fine.

[Shachlan]

@sbc100
As a novice to the complexities of the c++ build system, I'm not sure - how can a library be built without fPIC? Wouldn't that make it unusable in other code?

[sbc100]

No, compiling code without fPIC simply means that the resulting objects files cannot be linked into a shared library. Its perfectly possible for the shared library and that static library to contains different object files, one set built with fPIC and the other set built without fPIC. In fact this is technically correct and depending the platform will result in a faster static build since fPIC can have performance costs.

[sbc100]

The upstream change is now landed! It should make its way into the next emscripten release (1.38.46)

[pmp-p]

just tested the new tag with : ./emsdk install --embedded 1.38.46-upstream
embuilder.py build --pic sdl2 gives strange messages now:

embuilder.py build --pic sdl2
embuilder:INFO: building and verifying sdl2
cache:INFO: generating port: libSDL2.a... (this will be cached in "/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/libSDL2.a" for subsequent builds)
wasm-ld: error: -r and -pie may not be used together
wasm-ld: error: -r and -pie may not be used together
shared:ERROR: '/opt/sdk/emsdk-upstream/upstream/bin/wasm-ld -o /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/src/SDL_dataqueue.c.o --allow-undefined --lto-O0 /tmp/emscripten_temp_TjwJxW/SDL_dataqueue_0.o -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/local/lib -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/lib -L/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --no-gc-sections --export-dynamic --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -pie -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --relocatable' failed (1)
shared:ERROR: '/opt/sdk/emsdk-upstream/upstream/bin/wasm-ld -o /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/src/SDL_assert.c.o --allow-undefined --lto-O0 /tmp/emscripten_temp_tLAHBb/SDL_assert_0.o -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/local/lib -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/lib -L/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --no-gc-sections --export-dynamic --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -pie -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --relocatable' failed (1)
wasm-ld: error: -r and -pie may not be used together
shared:ERROR: '/opt/sdk/emsdk-upstream/upstream/bin/wasm-ld -o /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/src/SDL_error.c.o --allow-undefined --lto-O0 /tmp/emscripten_temp_UinHVE/SDL_error_0.o -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/local/lib -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/lib -L/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --no-gc-sections --export-dynamic --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -pie -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --relocatable' failed (1)
Traceback (most recent call last):
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py", line 3652, in <module>
    sys.exit(run(sys.argv))
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py", line 1907, in run
    compile_source_file(i, input_file)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py", line 1889, in compile_source_file
    cmd = get_clang_command([input_file]) + ['-c', '-o', output_file]
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py", line 1860, in get_clang_command
    args = system_libs.process_args(args, shared.Settings)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/system_libs.py", line 1620, in process_args
    args = port.process_args(Ports, args, settings, shared)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/ports/sdl2.py", line 70, in process_args
    get(ports, settings, shared)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/ports/sdl2.py", line 59, in get
    return [shared.Cache.get(libname, create, what='port')]
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/cache.py", line 133, in get
    temp = creator()
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/ports/sdl2.py", line 54, in create
    ports.run_commands(commands)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/system_libs.py", line 1393, in run_commands
    run_commands(commands)
  File "/opt/sdk/emsdk-upstream/upstream/emscripten/tools/system_libs.py", line 73, in run_commands
    pool.map_async(run_build_command, commands, chunksize=1).get(999999)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
tools.shared.Py2CalledProcessError: Command '['/usr/bin/python2', '/opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py', '-s', 'RELOCATABLE', '-Wno-int-in-bool-context', '/Users/user/.emscripten_ports/sdl2/SDL2-version_18/src/SDL_dataqueue.c', '-O2', '-o', '/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/src/SDL_dataqueue.c.o', '-I/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/include', '-O2', '-DUSING_GENERATED_CONFIG_H', '-w']' returned non-zero exit status 1
wasm-ld: error: -r and -pie may not be used together
shared:ERROR: '/opt/sdk/emsdk-upstream/upstream/bin/wasm-ld -o /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/ports-builds/sdl2/src/SDL.c.o --allow-undefined --lto-O0 /tmp/emscripten_temp_lSK_yp/SDL_0.o -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/local/lib -L/opt/sdk/emsdk-upstream/upstream/emscripten/system/lib -L/opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj --import-memory --import-table -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --no-gc-sections --export-dynamic --export __wasm_call_ctors --export __data_end --export main --export malloc --export free --export setThrew --export __errno_location -pie -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --relocatable' failed (1)
shared:ERROR: embuilder: emcc command failed with 1: '/usr/bin/python2 /opt/sdk/emsdk-upstream/upstream/emscripten/emcc.py /tmp/tmpGWdUoM/src.cpp -s USE_SDL=2 -s RELOCATABLE -o /tmp/tmpGWdUoM/out.js'
user@pp /opt/sdk/emsdk-upstream $ wasm-ld: error: -r and -pie may not be used together

[sbc100]

This looks unrelated to this issus. Would you mind opening another one since this one is already quite long?

(I can repro this BTW and am investigating).

[pmp-p]

done #9510

[pmp-p]

i triplechecked for SIDE MODULE not being present, added -fPIC everywhere
but wasmld failed on
wasm-ld: error: duplicate symbol: fprintf

defined in /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj/libc.a(fprintf.c.o)
defined in /opt/sdk/emsdk-upstream/.emscripten_cache/wasm-obj-pic/libc.a(fprintf.c.o)

MAIN_MODULE pulls wasm-obj-pic/libc.a

pipeline was :
a bunch of .o => lib.a with $(AR) rcs $(LIBMICROPYTHON) $(OBJ) => -o program.html main.c lib.a -ldl -lm -lc

i'm not sure if it's related more to #8995 or -fPIC for allowing use of ffi dlfcn from a static build.

sorry for the noise, static build without MAIN_MODULE is fully ok, so it's only really a MAIN_MODULE problem on target linking but with a different output than previously and unrelated to pic. FFI will stay broken for my project for now so no C++ lib integration in wasm compiled script languages yet.

[sbc100]

It looks like somehow two versions of libc are being linked in at the same time. Obviously not a good thing. Can you open a new issue for that?