Emscripten: Allow emcc settings to be specified in a file

[curiousdannii]

If you have a lot of emcc settings it can get unwieldy. It would be great if they could be specified in a file. This would also help with the quotes problem.

Questions:

1. Only the -s settings, or all settings? If all, how are they organised?
2. Loaded automatically like many other tools (emscripten.yml?) or require the file to be passed as an option?
3. File format? JSON? YAML? JSON is simple, YAML would allow you to have comments, both to describe what you're doing, as well as to more easily comment out options (currently messy if you use a makefile as it doesn't like comments in the middle of commands.)

[kripken]

Good idea. I'd suggest just the -s options (the others have no quoting issues) and using JSON for them for simplicity (we already have code for loading/storing such settings files, it's how we send the settings to the JS compiler portion). That would be a simple and useful feature to add I think.

[curiousdannii]

I really like the possibility of being able to easily comment out settings, which wouldn't be an option for JSON...

Another possibility would be an actual JS file, just like settings.js.

[dschuff]

Clang itself recently grew the ability to use a config file, so another option would be to use its format: https://github.com/llvm-mirror/clang/blob/master/docs/UsersManual.rst#configuration-files

[kripken]

@curiousdannii yeah, JSON + comments seems like the minimal useful thing. sad that's not an existing standard format...

[floooh]

I found that there's already support for reading one or several response file(s) in emcc.py (https://github.com/kripken/emscripten/blob/003a28776006e40b0148cef5a73fded37bab066c/emcc.py#L331), usually this is used to workaround the command line length limit on Windows, but it looks like it can be used as a general way to feed command line options into emcc through a file, and it would have the advantage that it just extends the incoming command line arguments and doesn't require special handling further down.

Just throwing this in here, don't know if it's a viable solution for the original poster :)

Other then that I would also vote for using the new clang configuration-file standard, that way there would be less different cases to worry about for multi-platform builds.

[curiousdannii]

I didn't know about that option. Will have to take a look at it.

Clang isn't involved when the settings are read, nor are most of the settings passed through to clang, so I don't think it's format has much advantage for us. If we weren't going to go for something like YAML it would make more sense to use something Python knows how to load, like configparser. But even then we'd have to manually parse any options which take arrays.

If we did go with YAML, solving #6069 would be pretty simple because we could just take the CLI options and turn them into a YAML file and then parse that. YAML is very forgiving so all the different quote options wouldn't matter. (Yes I'm fond of YAML :P)

[kripken]

Does python have support for yaml? Or would we need to write our own parser?

[curiousdannii]

There are a couple of good quality libraries.

[floooh]

I'm using PyYAML in my projects: https://github.com/floooh/fips/blob/master/yaml/PKG-INFO. However seeing how well writing JSON config files in Visual Studio Code works (they use some JSON variant which allows comments and have Intellisense which catches errors like that dreaded trailing comma), I would probably try to JSON in new projects. YAML is a bit overkill IMHO, much more complex to parse than JSON.

[kripken]

Thinking some more, it would be good if the settings file format supports the -s notation as a subset. That is, it should be easy to move code from -s in a Makefile into a settings file. I read that recent yaml versions almost but not entirely support json as a subset, so that might be worth verifying if we want yaml.

Another option here aside from yaml and json is what we currently have in settings.js and .emscripten, code that is both valid python and javascript, i.e., stuff like

KEY = 123;
KEY = ["str1", "str2"]; // comments are ok

In practice we can just eval it in emcc.py, so it would accept any valid python (but in our examples etc. we'd just show code that is also valid javascript).

[curiousdannii]

Is settings.js actually evaled in Python, not a JS shell? Cool!

[kripken]

We use it (by evalling) in both Python and JS :)

[stale[bot]]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

[curiousdannii]

This would still be good to do.

[sbc100]

In the short term we already support putting emcc args in the file. For example:

$ cat args.txt
 cat args.txt 
-c
-s ASSERTIONS=2
$ emcc @args.txt main.c

In the long term reading them from a file is only part of the solution. We also want a nicer way to express lists of string and deal with quotations more elegantly.

My proposal for how to achieve this second part is here: https://github.com/emscripten-core/emscripten/pull/new/s_parsing_strict

So I think the original issue here is already solved but there is a separate issue of nice way to specify lists which would not just be for file-based settings but also for command line settings. For example:

emcc -s EXPORTED_FUNCTIONS=foo,bar

is much nicer than:

emcc -s EXPORTED_FUNCTIONS="['foo', 'bar']"

What do you think? Should I leave this issue open or open a new one?

[curiousdannii]

Oh nice, I didn't know you could use @filename generally like that.

I don't mind whether you use this issue or make a new one.

[sbc100]

Opened: https://github.com/emscripten-core/emscripten/issues/9460