Design: Using WASM to compile full project

[JakeTrock]

I saw the tanks demo a while back and wanted to create something like it, but being a 16-year old with no college education hasn't helped too much. I followed and successfully completed the installation instructions on the wasm site, but there doesn't seem to be a transition between compiling a friendly little "hello world" program and compiling a dauntingly large program like was done at quakejs.com. Is there something out there for me to learn how to compile pre-made projects?

source:
https://github.com/ioquake/ioq3

[kripken]

The emscripten building projects page has some docs on this.

But I agree we could have better docs - a tutorial for building something like Bullet or BananaBread might be useful.

[JakeTrock]

Ok, seems straightforward, but just confirm to me how I should run a makefile like this through emscripten?:
https://github.com/ioquake/ioq3/blob/master/Makefile

(is there any specific command for makefiles? something like this: ./emscripten {makefile} -s USE_SDL=2 -s LEGACY_GL_EMULATION=1 -o q3.html

[kripken]

A Makefile is usually executed by the make command, and for emscripten, you should run

emmake make

(emmake sets up some env vars so that make can find emcc etc.)

[JakeTrock]

Alright, before I tried it out with emscripten, I tried it with GCC, and it worked flawlessly. Sadly, emscripten outputted this list of errors(if it matters, I use ubuntu 18.04 as a daily driver and for emscripten/GCC):
https://pastebin.com/kZLn42b8

(are my errors because my files are not "llvm bitcode"? how do I convert them?)

[kripken]

You can't convert them - you need to get the build commands to use emcc instead of gcc, and them emcc will emit bitcode for you.

Was emcc invoked? If you built with gcc in that directory already, it might think the (non-bitcode, gcc-emitted) files are already there, and not rebuild.

To make sure emcc is being invoked, you can run make as make VERBOSE=1 which will print out each command that is run - make sure emcc is in those commands, and not gcc or something else.

[JakeTrock]

Here's what happened when I ran it with verbose on:
https://pastebin.com/reM44qfx
How do I switch to EMCC?

[kripken]

"is up to date" means it is not rebuilding it, so it's leaving the non-bitcode versions there.

Starting with a new directory and copy of the code, and using the commands in that link (emconfigure ./configure, emmake make) should make it use emcc. (However, some makefiles ignore the env vars that emconfigure/emmake set, in which case, you may need to edit the makefile.)

Btw, a pull request with improvements for the docs on that link would be very welcome, anything that would have made it better for you would be great as it would help other people later.

[JakeTrock]

How do I properly configure with ./emconfigure? I typed "emconfigure ./configure" and the command line threw this error: https://pastebin.com/uqHg80EC

[kripken]

Does ./configure work without emconfigure? emconfigure is just a "wrapper" around a command that should already work.

One possibility is maybe it's called ./configure.sh in that project, or something else.

[kripken]

Another possibility is you may need to run that command in bash, emconfigure bash ./configure.sh or such.

[JakeTrock]

It seems I may not even have that directory. What a brainlet I am.
https://pastebin.com/3rRit3EK

[kripken]

If there's no configure, then just running the make step may work. However, if that project doesn't have an emscripten port already, there may be some work involved to get that set up, and proper APIs used, etc. May be worth asking them if there is a port, or a port in progress. It may be easier to use the quakejs repo https://github.com/inolen/quakejs

[JakeTrock]

Here's my results using the command inolen supplied in the repo:

https://pastebin.com/xFnNvWQL

Command used:
make PLATFORM=js EMSCRIPTEN=../blah blah blah

[kripken]

Based on that output, it doesn't seem to be using emscripten tools (LD build/release-js-x86_64/oa_ded.x86_64, and also errors like sys_main.c:(.text+0x6aa): are not emitted by us).

There may be a problem in the Makefile, options seem to be to either debug it or write a new one.

[JakeTrock]

I'm so sorry, that was the wrong paste... I was trying out a different repo and copied it. Here's the correct paste:
https://pastebin.com/2VvtQB2a

BTW
Thanks so much for the continued help :). I'm writing a tutorial for entry-level users to compile full projects in emscripten, with only minor experience in the GCC environment and linux filesystem commands

[kripken]

Ok, the Runtime.generateStructInfo is not a function is the problem then - that's an old API that emscripten no longer supports. It would need to be replaced, or the code that uses it be changed to no longer need it. Also see this recent discussion: https://github.com/kripken/emscripten/issues/7408#issuecomment-434512284

[kripken]

Sadly, figuring that generateStructInfo stuff out is probably not trivial. Using it at all in the quakejs port is pretty sophisticated, I didn't realize they did that. Perhaps the author of that port is interested to update it?

[JakeTrock]

Here's the runtime of the repo Inolen(the person who made qjs) forked his repo from:
https://pastebin.com/eTUwb27Q
(I'm pretty sure it may be easier to fix the structinfo stuff, optimization doesn't matter to me so I may run it with the autotool for simplicities sake)

[JakeTrock]

Or maybe could I somehow use an older version of emscripten?

[kripken]

You can use an older repo, yes - there are tags for each version in the emscripten repos. So you could get and build an old one manually. If you can find out which version was known to work with that project it might be a workaround.

[JakeTrock]

are there any little hints I can find in the repo that would indicate which version to use? The creator doesn't seem to respond to anything.

[kripken]

Looking at the dates of the commits might help, and comparing those to dates of version numbers in the main emscripten repo. There's a good chance those would be fairly close.

[JakeTrock]

The last update to it was 5 years ago, making the build 1.8.0 right?
BTW: found this on archive.org, this should be your readme:
https://web.archive.org/web/20130612181545/https://github.com/kripken/emscripten/wiki/Tutorial
It's a comprehensive overview in a short span, which is awesome(even though some version numbers/compatibility issues would need to be amended).

[JakeTrock]

Okay, I installed emscripten 1.7.9, anything else I need to do?

Console output: https://pastebin.com/EAXAeUdy

[JakeTrock]

It won't compile with just 1.7.9, do I need a different node and clang as well, or should I give up? I've been at this for 5 months and haven't gotten an ounce of progress

[kripken]

Node should be fine either way, but emscripten does depend on a specific version of LLVM and clang. So you would need to build emscripten-fastcomp and emscripten-fastcomp-clang of that version, and use that.

[JakeTrock]

Is there a list of which versions of LLVM and Clang that are compatible with emscripten 1.7.8?

[kripken]

It's the same version number in the 3 repos. (You can't use plain upstream LLVM 3.5 etc., you need to use the LLVM and clang in those emscripten-fastcomp* repos.)

[JakeTrock]

I don't understand... sorry

[kripken]

Which part?

There are 3 main emscripten repos,

https://github.com/kripken/emscripten/
https://github.com/kripken/emscripten-fastcomp/
https://github.com/kripken/emscripten-fastcomp-clang/

All three have the same versions. For example here is 1.38.15 in each:

https://github.com/kripken/emscripten/releases/tag/1.38.15
https://github.com/kripken/emscripten-fastcomp/releases/tag/1.38.15
https://github.com/kripken/emscripten-fastcomp-clang/releases/tag/1.38.15

If you get the same version/tag in all 3 repos, they are guaranteed to work properly together.

[JakeTrock]

oh, thank you! I was only using the command ./emsdk list --old and was confused.
But for some reason, only the emscripten for 1.7.9 existed, what clang and fastcomp do I use for those?
https://github.com/kripken/emscripten/releases/tag/1.7.9
https://github.com/kripken/emscripten-fastcomp/releases/tag/1.7.9
https://github.com/kripken/emscripten-fastcomp-clang/releases/tag/1.7.9

[JakeTrock]

clang only goes to v 11 and em-fastcomp only goes to v11 as well
https://github.com/kripken/emscripten-fastcomp-clang/releases?after=1.14.1
https://github.com/kripken/emscripten-fastcomp/releases?after=1.14.1

[kripken]

Oh, I guess we only started the version number at 1.11.1 or so. Which is about when we started to use fastcomp and fastcomp-clang.

So I think earlier versions did not use that, and just used upstream LLVM+clang. The version is probably 3.4, 3.5 or such - it should say in tools/shared.py if you search for llvm_version or something similar. Or maybe archived versions of the docs can help better.

[JakeTrock]

would this work in my windows 7 vm?
https://kripken.github.io/emscripten-site/docs/getting_started/downloads.html#archived-nsis-windows-sdk-releases

(is everything packaged together in it?)
Where are the archived tools?

[a2kas]

I have been trying to build ioq3 project for month, but still no success. According to https://github.com/inolen/quakejs/issues/14 you can build it with emscripten-1.12.0 and clang-3.2-64bit, unfortunately on linux clang-3.2-64bit is not supported and you will not see it in the list, because it does not have linux_url in .manifest file. One developer said to me that he successfully built project on OS X with latest emscripten version, but i have not tried yet.

[JakeTrock]

just wait until you join the 5-month club. btw, I'm going to try out 11.1 I believe that's the oldest emscripten goes.
https://github.com/kripken/emscripten-fastcomp-clang/releases/tag/1.11.1
https://github.com/kripken/emscripten-fastcomp/releases/tag/1.11.1
https://github.com/kripken/emscripten/releases/tag/1.11.1

[a2kas]

@hokuco let me know, if you successed

[JakeTrock]

I give up.

[kripken]

Sorry this didn't work out. Sometimes it can be hard to get a large and complex project building when it hasn't been active for years and the original author is not available. Starting with smaller stuff might be more fun for you.

[JakeTrock]

@kripken I will continue work on the beginner tutorial at issue #1249

[Ashaaradhya]

I'm so sorry, that was the wrong paste... I was trying out a different repo and copied it. Here's the correct paste:
https://pastebin.com/2VvtQB2a

BTW
Thanks so much for the continued help :). I'm writing a tutorial for entry-level users to compile full projects in emscripten, with only minor experience in the GCC environment and linux filesystem commands
Hi so have you written the tutorial ? If so could you please post the link?

[JakeTrock]

It's so much easier to use now. I was 16 or so when I was trying to make the tutorial, so that may have hindered me a bit. If you'd like, I can help you via email, but I realized the problem with tutorials was that they are very specific, and the uses of emscripten are so broad.

[Ashaaradhya]

@JakeTrock Thank you very much.. Yes , I am still struggling with it.. I have a project ( in c ) which as a whole needs to be converted to wasm. Tutorial does talk about 'emconfigure ./configure' and 'emmake make' but I was not quite able to understand on how to use them or even create configure file for that matter. Request you to email me on asha.p.[email protected] with any information that you have . Thanks in Advance

[JakeTrock]

if it's your own project, it's likely that you need to write a makefile/utilize emcc