Meson: Change CLI to a git like interface.

[behlec]

Change the command line interface to a git like interface.

This means that all invocations of meson have the form meson cmd [options]. This would remove the possibility to call meson builddir to initialize a build in builddir. In return a new command foo would be added such that meson foo [dir1] [dir2] has the same semantics as currently meson [dir1] [dir2].

Rational: Improve user experience.
It would allow to detect if the user calls meson with a misspelled command. If a user calls meson with a misspelled command meson will assume it is a directory. This will either result in a strange error message or meson will initialize a new build directory. While both error cases are not severe, they can be annoying.

The drawback is that the old invocation of meson does not work anymore. However this requires only adding a command at the calls the rest of the invocation stays the same. While this might break some scripts the error message would be clear.

I do believe that this change would remove more pain in the long run than it causes in short term.

[becm]

It would be a good idea to always treat the first directory argument as as source.
meson init <src> [<dest>] with dest defaulting to current directory.

• clear meaning of positional arguments
• when run in source directory adding . as placeholder is passable overhead
• command run only once, longer argument count is not a problem
• The mode detection is broken anyway; create subdirectory if called with misspelled operation command (configur, tsed, …)

[becm]

A more pressing improvement would be a shorthand (set, conf, …) for the configure operation and also dropping the superfluous -D for assignments. This mode may be (the one most) frequently used and is definitly (one of) the longest to type.
The configure option does only handle one (positional) directory argument anyway (and prints a bad/wrong error message if called otherwise). Since it would normaly be run inside the build directory anyway this should be an extra argument option (-b <builddir>) .

If ninja reconfigure were able to detect the need to actually run it could be called on/before every meson configure to incorporate possible changes.
Was there a design rationale to execute a reconfigure unconditionally?

[jukkalaurila]

I'm also interested in improving this. I wrote up a proposal on how to merge some of these ideas and handle some error cases more gracefully here:

https://docs.google.com/document/d/1XT7uMuHDyBIjAgEplDCp7dF8vYBrKjqWZx62VXviWqc/edit#

Thoughts? I can grant comment access to the doc to people on request if that's more convenient.

[becm]

Guessing operation modes depending on current directory content or position in my opinion has still more potential to cause confusion or even harm than the reduction of keystrokes this will ever be able to justify.

Especially accidently creating new build directories inside source trees can become a very extensive operation, more so with parts of the suggested synthax change:

• meant to do meson foo=bar in build directory
• create whole new build directory if executed in source (or error on read-only source tree)

I see in general far less problems with using to the 3 (4) argument format I suggested above.
If init is kept for template some other (fitting) name will have to do (mkbuild ?).

[becm]

The option parser is already far too forgiving and subcommand doc quality is at most so-so.
To unfairly use the faulty init help page here:
meson --type executable --version 1.8
Not to worry; configure even has wrong errors:
meson configure .. .

[behlec]

@jukkalaurila Thanks for the write up. That really helps as a basis for the discussion.

I agree with @becm that requiring a command is less error prone. By the waysetup is already a command that does the same as calling meson without a command. I.e. meson build is the same as meson setup build.
Doing something like initializing a builddir without an argument is something I would not do.

However, this gave me an idea. Maybe in the future we could support some form of configuration. Either system wide, or per build file, or a mix like git. Then people could define default options/arguments for commands or a default command for meson. For now I would like to get a uniform invocation to improve the feedback and reduce error possibilities. I do not think that meson is called that often that these shortcuts should be available by default.

I have to admit though, that I usually do not call meson myself but only through scripts. I usually compile a project with several compilers and several options and use a script to initialize builddirs with meson.

[jukkalaurila]

Yes, after thinking about it a bit I also think that creating a build directory by default is too magical and can be a source for confusion.

I wrote up a second version of my proposal, this time leaving out everything except the improved error checks and what that necessarily implies. This change is already hard enough without being mixed up with other syntax changes. I tried to work out all the possible details and suggest a transition plan. My first choice would be to use flags -s and -b to indicate source and build directories instead of positional parameters.

Please take a look and tell me what you think: https://docs.google.com/document/d/1EZEYd17Bqzxgoyl223Uhb5qTK6xSl6u1aQR8am_2e_A/edit#

[jukkalaurila]

A rough prototype implementation of this is available at: https://github.com/jukkalaurila/meson/tree/cli-error-checks

[behlec]

Thanks, that looks like great work. I will take a closer look and give you more feedback. So far I could only skim your changes.

On a first glance: in MesonApp.validate_core_dirs
Should the following lines not be removed?

elif self.has_build_file(ndir2):
    (src_dir, build_dir) = ndir2, ndir1

This switches the directories if only one contains a build file.

[jukkalaurila]

It's better to concentrate on agreeing exactly how it should work instead of the details of the prototype code. There are several unpolished parts in it and it exists just as a way to try out how the change would feel like. Once there is agreement on a direction from people here and the project lead, I can move on to finishing the code.

I think there are two basic directions we could take:

1. Switch to command line flags like in my main proposal and current prototype.
2. Require using "meson setup" instead of "meson" but leave everything else as is, with positional arguments.

Would you agree, or are there alternative proposals?

[jukkalaurila]

I updated the document a bit, adding a variant of the "always require subcommand" proposal: If you run "meson" in an empty directory and there is a meson.build file in the parent directory, then interpret that as "meson setup".

Prototype implementations:

• require flags: https://github.com/jukkalaurila/meson/tree/cli-error-checks-require-flags
• require "setup": https://github.com/jukkalaurila/meson/tree/cli-error-checks-require-setup

[jukkalaurila]

And the variant that requires "setup" or no arguments: https://github.com/jukkalaurila/meson/tree/cli-error-checks-require-setup-or-no-args
This one actually looks pretty nice.

[becm]

Sorry to allways nag arround since it's you who's doing actual work.
In my experience constraints are the only way to reduce failure conditions to a manageable level. Without explicit operation mode and source directory the user intention is alway up to interpretation resulting in confusing or even misleading error messages.

Clear preconditions for meson setup <src> [<dest>] before modifying filesystem:

• src is readable, dest is writable/creatable
• src must contain meson.build project definition
• dest must be "empty" (no initialized build dir) (unless called with --force?)

Some errors for the last prototype with (subdir) meson.build in upper level:

• the user ment to do something else (missed --help?) → try build
• currently in write-only filesystem → python backtrace
• meson.build is not a project definition → meson ERROR

One can try to catch them all but Niantic users will come up with new and creative ways.

To me meson setup feels closest to a checkout/clone operation so explicitly giving source and (optional) destination as positional arguments would be consistent with other tools in workflow (git, mercurial).
Core operation arguments normally are not burdened with flags (ln, cp, mv, rsync, …).

[jukkalaurila]

No worries, your feedback is very welcome. I agree that we can and should introduce precondition checks to verify we've understood user intent correctly. The current code already checks that:

1. The directory selected as the build directory either doesn't exist or is empty. (Yes, a --force flag would be very handy. I'm tired of typing "rm -r build" all the time.)
2. Exactly one of the given or implicitly selected directories contains meson.build. This eliminates ambiguity in which directory is the source and which the destination and means that in the single-argument form you can give either the source or the destination as the argument and default the other to the current directory without any chance of confusion.

There is no explicit check for source directory being readable or dest being writeable/creatable, but the former is pretty well covered by 2 and if you can't write to the destination directory then by definition you can't mess it up. That said, we could give better error messages for those cases if we check for them explicitly.

I fixed the message you get when running "meson" without arguments and the preconditions "meson.build in .. and current directory is empty" are not met, now it prints the normal command-line help. Fixed code at https://github.com/jukkalaurila/meson/tree/cli-error-checks-require-setup-or-no-args

Good point about positional arguments for core operations. However, I'm not convinced it makes sense to require specifying two arguments for the common case of creating a build directory under the current source directory, since the command "meson setup build" is unambiguous. The situation with ln, cp and mv is different since in them there is no reasonable default for either parameter, and in those commands very often neither parameter is the current directory. With Meson almost always one of the directories is the current directory.

In any case, I think we all agree that any of the three options would be an improvement over the current state. Personally I'm starting to be convinced that the "setup or no args" approach is best overall. Are there any problems with that that I've missed?

[jukkalaurila]

Added error checks for the cases where build directory isn't writable or its creation fails, and switched the branch to the mode where it accepts the legacy syntax but warns about its deprecation. Code at https://github.com/jukkalaurila/meson/tree/cli-error-checks-require-setup-or-no-args

[becm]

Having the 3rd (1st positional) argument always mean source will reduce documentation complexity, extra directory checking (souce/target) and resulting error handling. It is consistent with clone in git and mercurial (actually also with rsync and ln), with the current directory as (always some sort of) default target.
When called from a (bad) script with an empty target the source argument can not be misused as target accidentially or with malicious intent.
→ Only write to current directory or an explicit (mandatory 4th argument) location.

The shorthand meson setup . build or meson setup .. . could also be handled via a (static) shell alias and tailored closer to the user's habit without favouring either approach or any extra code path or detection logic in meson.

[jukkalaurila]

I implemented your fixed-order proposal here: https://github.com/jukkalaurila/meson/tree/cli-error-checks-setup-fixed-order but left the additional shortcut that "meson" will mean "meson setup .. ." if it makes sense and added it as alternative design D into the doc.

While I agree requiring a certain ordering would reduce implementation complexity, I'm not convinced it's significant enough to always require passing the arguments.

• The error checking code is just a few conditionals and is pretty much done. I'll be happy to implement more checks.
• I'm not quite sure I understand the problem with "an empty target". The source and target directories can be reliably distinguished by checking which one contains meson.build and which one is empty or nonexistent. There is never any need to write to a directory which isn't empty. Am I missing some subtle point here?
• Shell aliases can always be used to cover any syntax, but the default syntax still matters.

I am still open to any new information or proposals on how this all should work, but in the end some of these details are a matter of taste, reasonable people can disagree on what is the best approach and the differences aren't very large.

Can we agree that any of the proposals would be both possible to live with and an improvement over the current state? It might be best to simply let the project lead pick his preferred option fairly soon so that we can move on.

[jpakkane]

I've been a bit busy with other stuff so haven't yet gone through this properly. It would seem that few people have been using setup so maybe we could change that to the new and better syntax but have the default commandless mode behave the same as currently. Once we have that hammered out we can start printing deprecation warnings.

[jukkalaurila]

Can you be more specific about which "new and better syntax" do you mean? There are several options being considered and at this point they are starting to be fairly hammered already.

[jpakkane]

It was not about the syntax itself, but about how it is exposed. So something like:

meson setup <new and improved syntax>
meson <no-command old behaviour for backwards compatibility>

The reason we could do this is that setup as a command is not used by many (at least that I know of) but the no-command version is widely used and changing it would be hard.

[jukkalaurila]

Just writing down that in an IRC discussion we talked about a possible new option: Interpret the syntax "meson foobar" as "meson setup foobar" if and only if foobar is not a subcommand and it is an empty directory. If foobar is a subcommand, interpret it as subcommand even if the directory foobar exists.

[becm]

Bad script example (ideally on a badly managed headless build server):

• be in a source directory
• get tricked to an empty target (no " framing)
• no checks applied to source arg → use /tmp/<randomdir>
• repeated lead meson to fill up (essential) directories → DOS

Another reason to make this (critical) meson operation (crystal) clear is it aspires to be the 2nd command line tool to be learned for new developers. It's operation has the chance to closely resemble the 3rd (git). Doing something unexpected (exept printing an error/help message) just because we detected a source directory one level up or in another command position is not really something I'd feel comfortable to explain to students.

Widely used tools implicitly have an example function to their users. If they allow lax preconditions or possibly erronous application, someone will feel entitled to do so in the next security-relevant application with fallout hunting us during the next 3 decades.

And worst of all; we'd have to change the issue topic ☺.

[jukkalaurila]

That scenario sounds a bit far fetched. If you have malicious input being passed to a shell command line without proper quoting, you already have bigger problems than your build system creating directories.

That said, I think I do understand what you are trying to say about simplicity and unambiguity. Balancing syntactic sugar and syntactic salt is a matter of taste.

[astavale]

Meson is designed around out of source tree builds. I think this should be made explicit in any CLI. So:

meson setup <builddir>

The action (verb) is setup and the object (noun) of the action is the builddir.
Now in some cases the command may be run outside of the source tree so that would need to be an option. This could be meson setup <builddir> [<sourcedir>] but I'm drawn to making that an explicit option:

meson setup <builddir> --sourcedir <sourcedir>

This is similar to --cross-file, which is an explicit optional argument.

With the other Meson commands the source directory is read from the build directory. So they would follow a similar form, but wouldn't need the --sourcedir option:

meson configure <builddir>
meson test <builddir>

These two commands show/set the build options for the build directory and build/rebuild and run the test suite for that build.

I hope this is a useful simplification that allows new users of Meson to readily understand the concept of out of tree builds, while still maintaining the flexibility for existing users.

[becm]

The out-of-tree build can as well be used to argue <sourcedir> being more important when the workflow is centered around (multiple) build directories.
Requiring both in fixed order (meson setup <sourcedir> <builddir>) should result in least unexepected behaviour (do_xy <from> <to> like anybody else).

In most cases meson configure and test (like ninja) will be run in the build directory, so positional arguments seem kind of wasted on a singular directory option.

The test subcommand already uses this kind of argument for test selection.
meson test test_1 test_2 [--builddir <builddir>]

And I still have hopes to get primary arguments for configure to assign options ☺.
meson configure opt1=val1 opt2=val2 [--builddir <builddir>]
or (more git-y)
meson config opt1 val1 [--builddir <builddir>]

[jukkalaurila]

Note that you can always detect which one is the source and which one is the build directory based on their contents. There is never any ambiguity about that. So this argument is essentially about whether we should give an error message if you specify them in one of the orders or just accept the command. There is only one way to interpret it in any case.

[Salamandar]

I like that.