Emscripten: gcc-CMake-compatible mode?

[saschanaz]

I'm trying building LLVM with Emscripten and am getting this error:

[  9%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/Types.cpp.o
[  9%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86DisassemblerTables.cpp.o
[  9%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86ModRMFilters.cpp.o
[  9%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/X86RecognizableInstr.cpp.o
[  9%] Building CXX object utils/TableGen/CMakeFiles/obj.llvm-tblgen.dir/CTagsEmitter.cpp.o
[  9%] Built target obj.llvm-tblgen
Scanning dependencies of target llvm-tblgen
[  9%] Linking CXX executable ../../bin/llvm-tblgen
[  9%] Built target llvm-tblgen
Scanning dependencies of target AttributeCompatFuncTableGen
[  9%] Building AttributesCompatFunc.inc...
/bin/sh: 1: ../../bin/llvm-tblgen: Permission denied
lib/IR/CMakeFiles/AttributeCompatFuncTableGen.dir/build.make:90: recipe for target 'lib/IR/AttributesCompatFunc.inc.tmp' failed
make[2]: *** [lib/IR/AttributesCompatFunc.inc.tmp] Error 126
CMakeFiles/Makefile2:789: recipe for target 'lib/IR/CMakeFiles/AttributeCompatFuncTableGen.dir/all' failed
make[1]: *** [lib/IR/CMakeFiles/AttributeCompatFuncTableGen.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

By default emcc generates a bitcode file instead of an executable when the target path does not have file extension, and that behavior is problematic in LLVM CMake script. Can we introduce gcc-compatible mode so that extension-less target will generate javascript with a leading shebang #!/usr/bin/env node, for example?

Edit: you can try this using emmake cmake ../emscripten-fastcomp/ -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release -DLLVM_TARGETS_TO_BUILD="X86;JSBackend" -DLLVM_INCLUDE_EXAMPLES=OFF -DLLVM_INCLUDE_TESTS=OFF -DCLANG_INCLUDE_TESTS=OFF -DLLVM_COMPILER_IS_GCC_COMPATIBLE=1 -DLLVM_ENABLE_TERMINFO=OFF

[tbfleming]

LLVM builds a couple *-tblgen executables then runs them. The problem is they're not native executables; emscripten produced them. You need to build LLVM twice:

• A native build
• An emscripten build. You need to point these two CMake flags at the executables the native build produced: -DLLVM_TABLEGEN=... and -DCLANG_TABLEGEN=...

Skip -DCLANG_TABLEGEN if you're not building clang.

[kripken]

@saschanaz "Can we introduce gcc-compatible mode so that extension-less target will generate javascript with a leading shebang #!/usr/bin/env node, for example?"

There is an attempt at something similar to what you suggest, look for EMCONFIGURE_JS in emcc.py. Actually that might be a lot more feasible today with your new NODERAWFS mode, since I think filesystem access was the main blocker there, hmm - maybe worth trying that.

[tbfleming]

Interesting. If that works then it will save me a build step.

[saschanaz]

I tried EMCONFIGURE_JS=2 EMMAKEN_JUST_CONFIGURE=1 (emmaken is deprecated but anyway this is required) and got a long list of strange linking warnings:

warning: unresolved symbol: _ZNKSt3__210error_code7messageEv
warning: unresolved symbol: _ZNKSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE4findEcj
warning: unresolved symbol: _ZNKSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7compareEjjPKcj
warning: unresolved symbol: _ZNKSt3__220__vector_base_commonILb1EE20__throw_length_errorEv
warning: unresolved symbol: _ZNKSt3__221__basic_string_commonILb1EE20__throw_length_errorEv
warning: unresolved symbol: _ZNSt3__211__call_onceERVmPvPFvS2_E
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE5eraseEjj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6appendEPKc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6appendEPKcj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6assignEPKc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertENS_11__wrap_iterIPKcEEc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertEjPKc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6insertEjPKcj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE6resizeEjc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7replaceEjjPKc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7replaceEjjPKcj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE7reserveEj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE9__grow_byEjjjjjj
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEE9push_backEc
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEC1ERKS5_
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEC1ERKS5_jjRKS4_
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED1Ev
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEED2Ev
warning: unresolved symbol: _ZNSt3__212basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEaSERKS5_
warning: unresolved symbol: _ZNSt3__215system_categoryEv
warning: unresolved symbol: _ZNSt3__216generic_categoryEv
warning: unresolved symbol: _ZNSt3__26__sortIRNS_6__lessIiiEEPiEEvT0_S5_T_
warning: unresolved symbol: _ZNSt3__26__sortIRNS_6__lessIjjEEPjEEvT0_S5_T_
warning: unresolved symbol: _ZNSt3__26chrono12steady_clock3nowEv
warning: unresolved symbol: _ZdaPv
warning: unresolved symbol: _ZdlPv
warning: unresolved symbol: _Znaj
warning: unresolved symbol: _Znwj
warning: unresolved symbol: _ZnwjRKSt9nothrow_t
warning: unresolved symbol: __cxa_guard_acquire
warning: unresolved symbol: __cxa_guard_release
warning: unresolved symbol: __errno_location
warning: unresolved symbol: access
warning: unresolved symbol: close
warning: unresolved symbol: dup2
warning: unresolved symbol: fstat
warning: unresolved symbol: getcwd
warning: unresolved symbol: getpid
warning: unresolved symbol: getrlimit
warning: unresolved symbol: isalnum
warning: unresolved symbol: isalpha
warning: unresolved symbol: isatty
warning: unresolved symbol: isdigit
warning: unresolved symbol: islower
warning: unresolved symbol: isprint
warning: unresolved symbol: ispunct
warning: unresolved symbol: isspace
warning: unresolved symbol: isupper
warning: unresolved symbol: isxdigit
warning: unresolved symbol: lseek
warning: unresolved symbol: lstat
warning: unresolved symbol: memchr
warning: unresolved symbol: memcmp
warning: unresolved symbol: mkdir
warning: unresolved symbol: mmap
warning: unresolved symbol: munmap
warning: unresolved symbol: open
warning: unresolved symbol: posix_spawn_file_actions_adddup2
warning: unresolved symbol: posix_spawn_file_actions_addopen
warning: unresolved symbol: posix_spawn_file_actions_destroy
warning: unresolved symbol: posix_spawn_file_actions_init
warning: unresolved symbol: pread
warning: unresolved symbol: pthread_sigmask
warning: unresolved symbol: qsort
warning: unresolved symbol: rand
warning: unresolved symbol: read
warning: unresolved symbol: readlink
warning: unresolved symbol: realpath
warning: unresolved symbol: remove
warning: unresolved symbol: setrlimit
warning: unresolved symbol: sigaltstack
warning: unresolved symbol: snprintf
warning: unresolved symbol: srand
warning: unresolved symbol: stat
warning: unresolved symbol: strchr
warning: unresolved symbol: strcmp
warning: unresolved symbol: strerror_r
warning: unresolved symbol: strlen
warning: unresolved symbol: strncmp
warning: unresolved symbol: strrchr
warning: unresolved symbol: strsignal
warning: unresolved symbol: strtoll
warning: unresolved symbol: strtoull
warning: unresolved symbol: tolower
warning: unresolved symbol: toupper
warning: unresolved symbol: uname
warning: unresolved symbol: unlink
warning: unresolved symbol: write
warning: unresolved symbol: _ZSt7nothrow

Any idea why this happens? The following is the full command generated by CMake:

/mnt/c/Users/sasch/Documents/GitHub/emsdk/emscripten/incoming/em++    -fPIC -fvisibility-inlines-hidden -ffunction-sections -fdata-sections   -Wl,-allow-shlib-undefined  -Wl,-rpath-link,/home/saschanaz/cmaketmp/./lib  -Wl,-O3 -Wl,--gc-sections CMakeFiles/obj.llvm-tblgen.dir/AsmMatcherEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/AsmWriterEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/AsmWriterInst.cpp.o CMakeFiles/obj.llvm-tblgen.dir/Attributes.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CallingConvEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeEmitterGen.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenDAGPatterns.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenInstruction.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenMapTable.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenRegisters.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenSchedule.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CodeGenTarget.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DAGISelEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherGen.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcherOpt.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DAGISelMatcher.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DFAPacketizerEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/DisassemblerEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/FastISelEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/FixedLenDecoderEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/GlobalISelEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/InstrInfoEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/IntrinsicEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/OptParserEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/PseudoLoweringEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/RegisterInfoEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/SearchableTableEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/SubtargetEmitter.cpp.o CMakeFiles/obj.llvm-tblgen.dir/SubtargetFeatureInfo.cpp.o CMakeFiles/obj.llvm-tblgen.dir/TableGen.cpp.o CMakeFiles/obj.llvm-tblgen.dir/Types.cpp.o CMakeFiles/obj.llvm-tblgen.dir/X86DisassemblerTables.cpp.o CMakeFiles/obj.llvm-tblgen.dir/X86ModRMFilters.cpp.o CMakeFiles/obj.llvm-tblgen.dir/X86RecognizableInstr.cpp.o CMakeFiles/obj.llvm-tblgen.dir/CTagsEmitter.cpp.o  -o ../../bin/llvm-tblgen ../../lib/libLLVMSupport.a ../../lib/libLLVMTableGen.a ../../lib/libLLVMSupport.a -lrt -ldl -lm ../../lib/libLLVMDemangle.a -Wl,-rpath,"\$ORIGIN/../lib"

Edit: export EMCC_FORCE_STDLIBS=1 largely reduces the list but still the list remains (It shouldn't be required, though?):

warning: emitted code will contain very large numbers of local variables, which is bad for performance (build to JS with -O2 or above to avoid this - make sure to do so both on source files, and during 'linking')
warning: unresolved symbol: posix_spawn_file_actions_adddup2
warning: unresolved symbol: posix_spawn_file_actions_addopen
warning: unresolved symbol: posix_spawn_file_actions_destroy
warning: unresolved symbol: posix_spawn_file_actions_init
warning: unresolved symbol: pthread_sigmask
warning: unresolved symbol: sigaltstack

[saschanaz]

It seems it needs -s USE_PTHREAD=1 but can emmake pass it through cmake?

Edit: Yes, EMMAKEN_CFLAGS

[kripken]

I suspect it's not linking in system libraries normally in that mode. EMCC_DEBUG=1 logging should show what it is deciding to link in.

[saschanaz]

I suspect it's not linking in system libraries normally in that mode.

Not sure I understood correctly, you mean that EMCONFIGURE_JS is expected to omit system libraries? Sorry, ignore this 😅

The linking problem still happens when without EMCONFIGURE_JS and doing mv bin/llvm-tblgen bin/llvm-tblgen.bc && emcc llvm-tblgen.bc -o llvm-tblgen.js.

[saschanaz]

Linker only wants to link dlmalloc.bc and nothing else.

[kripken]

Odd. With -s VERBOSE=1 it will also explain its linking choices, so that could help figure out why it isn't linking in libc. For example here is how it decides to link in libc for hello world:

DEBUG:root:undefs: set(['malloc', '__errno_location', 'free', 'printf', 'fflush', 'main'])
DEBUG:root:considering libc.bc: we need set(['malloc', '__errno_location', 'fflush', 'free', 'printf']) and have set([])
DEBUG:root:including libc.bc

[saschanaz]

I tried it on my other machine and it does include libcxx, no idea why the difference. Hmm.

Edit: My git clone was problematic 😅 Opened #6090 for remaining linker errors.

[dschuff]

My general opinion is, if you are using emscripten, you are doing a cross build. If you are doing a cross build, the thing you are building (i.e. its build scripts) will have to be compatible with cross builds. It is easy to mess this up by making certain assumptions in your build scripts.
We (emscripten, or any other cross-compilation environment) can try to fake or paper over this by trying to enable build artifacts like tablegen to be run on the host system, but it's even easier to mess that up, and hard to catch all the possible use cases and assumptions that build systems/scripts might make, that it will never truly be like a native build (to say nothing of the fact that even if we allow intermediate JS artifacts to be run in e.g. node, that's still different in interesting ways from the web). So I think everyone is better served by making build scripts support cross-building.

[saschanaz]

Generally I agree that build scripts should support cross building, but I still think that extension-less targets should be able to be executable. Anyway Emscripten cannot do anything with extension-less file input so IMO it's a better behavior.

[kripken]

I agree with @saschanaz that extension-less outputs should be executable when doing configure/cmake, it should be easy to make that work with the new NODERAWFS, has very little risk, and more closely matches how configure/cmake expect things to work. But I agree with @dschuff that anything more complicated on top of that is not worth doing, and those projects will have to add proper cross-compilation support. In particular any node/browser differences are not worth trying to paper over.

I'm not sure where tblgen falls between those two things. But I'd hope it would just work with NODERAWFS.

[juj]

I still think that extension-less targets should be able to be executable

By extension-less targets do you refer to CMake add_executable targets that don't specify an extension to them in the name of the target? E.g. https://github.com/kripken/emscripten/blob/incoming/tests/cmake/target_js/CMakeLists.txt#L87 ?

This should produce a .js file that can be executed. For example,

src.c:

int main(){}

CMakeLists.txt

project(test)
add_executable(app src.c)

creates an app.js file when built. This is governed by CMAKE_EXECUTABLE_SUFFIX parameter that is set here https://github.com/kripken/emscripten/blob/incoming/cmake/Modules/Platform/Emscripten.cmake#L196.

That can be adjusted to e.g. .html to cause an app.html to be built instead of an app.js:

project(test)
set(CMAKE_EXECUTABLE_SUFFIX ".html")
add_executable(app src.c)

Currently if one does set(CMAKE_EXECUTABLE_SUFFIX "") however, that will cause an emcc src.c -o app build to happen, which causes emcc to not produce a .js or a .html file.

Would it be preferred that in this scenario the file app was a JavaScript file? Or would it be preferred that even when emcc src.c -o app command line was used, a file app.js with JavaScript was still generated?

If the former, then perhaps we could extend support here to the build script by adding a --output-type html/js/bc/wasm/a type of extra directive that could be added, e.g. emcc src.c -o app --output-type js would create a JavaScript file app? If this was an extra directive, we would not have to make a defined claim as to what an extensionless output was, and emcc src.c -o app could still be kept ambiguous, perhaps even go as far to start producing an error?

[saschanaz]

If the former, then perhaps we could extend support here to the build script by adding a --output-type html/js/bc/wasm/a type of extra directive that could be added

Yeah, that's what #5575 does but it didn't show any further progress. No signal there, perhaps we should open a new PR?

[kripken]

For cmake it's nice to have the suffix customization option, yeah. I think though that the topic in this issue is more specific, it's for cmake and configure when doing their configuration work, not normal compilation. configure just wants to create a bunch of small programs and run them to see what their output is, so in configure mode (emconfigure was run) we have a bunch of things we do to get that to work. We build natively, but pass it flags to make it use our headers, for example.

It would better and safer to build to js instead, that's really what configure and cmake want when configuring. Filesystem access was the main blocker for that, so I am hoping with NODERAWFS we can resolve this once and for all.

[saschanaz]

I wonder whether we have a use case for defaulting to bitcode for extension-less output. Will something get broken if we default to JS?

[kripken]

Defaulting to JS would increase build time. It would also build in -O0 (as likely it won't pass a specific opt level in, it think it's just linking), and would miss the other flags that are useful for compiling to JS, so it's risky - instead, right now you must explicitly build to JS, which is safer. But, for configure/cmake configuration, I think it's ok to do that (build time doesn't matter, those are tiny files anyhow, and we just want them to run, -O0 is ok).

Another risk is that some libraries might build to an extension-less output that is a shared or static library. I'm not sure if that's common or not.

[saschanaz]

Defaulting to JS would increase build time.

Yeah, but only when someone is actually doing non-executable non-JS build with extension-less output. I doubt anyone will do it when emcc itself cannot receive extension-less inputs.

It would also build in -O0

The LLVM CMake script in fact only passes -Wl,-O3. Can we read it as -O3 when all input files are binaries?

[kripken]

Yeah, but only when someone is actually doing non-executable non-JS build with extension-less output.

True, but many projects build multiple executables by default when doing make (binaryen does this, for example, it builds wasm-as, wasm-dis, wasm-opt, etc.). It's not that the user would request it, but that doing configure, make would emit a bunch of executables. Normally that's fine, each is just a quick link, but our asm.js/wasm compilation does all the codegen at the link stage, so this would be a very noticeable change for those projects.

(But again, I think this would be a good change to make during cmake/configure.)

The LLVM CMake script in fact only passes -Wl,-O3

Interesting, I'm not sure if that's not something special about LLVM, though? I thought most projects don't do it, but I could be wrong.

[saschanaz]

Interesting, I'm not sure if that's not something special about LLVM, though?

Ah, I mean it only passes -Wl,-O3 only when all the command does is linking. (So it build in -O0, causes a large JS file as you said.) Sorry to confuse you 😱

[stale[bot]]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.