Cabal: RFC: Multiway flags

[phadej]

In some cases we want to have multiway flags, when there are more then two disjoin options:

flag base
  description: Base version
  values: v49 v48 v47 -- default would be: True False
  manual: False
  -- default: v49, picks the first value from `values` list

if flag(base) -- same as if(flag==v49), i.e. compares to first value
  build-depends: base==4.9.*
if flag(base==v48)
  build-depends: base==4.8.*, semigroups
if flag(base==v47)
  build-depends: base==4.7.*, semigroups, void

on command line

-fbase same as -fbase=v49, because v49 is the first value
-f-base fails because flag doesn't have exactly two options

Problems:

• How to specify per package flags, as that is probably what we would need to do (or is it possible atm?)

[23Skidoo]

Looks like a good idea in principle. /cc @kosmikus

[kosmikus]

I'm not opposed, nor does it seem to be particularly difficult to do.

But I'm generally not overly enthusiastic about the whole idea of automatic flags, and this seems to make it likely that they'll be more widely used. They always seem backwards to me. (A flag value is inferred from packages that are available in the environment, but we're writing it down the other way around.)

Would it not be better to consider something like "or-dependencies" instead?

Also, if we're going to change the way the flag mechanism works, and considering new constraints, it may be worth looking at the bigger picture, and actually agreeing on a number of overall design goals.

[hvr]

I remember this (or a variant of this) having been suggested/discussed in the past already, I can't seem to find the ticket though... :-/

@kosmikus as for or-dependencies, would there be a way to freeze the selected branch from the outside without having a flag to reference?

EDIT: Found it again, I think: https://github.com/haskell/cabal/issues/2033#issuecomment-51904547

[phadej]

I have no strong opinion, the or-dependencies would serve the purpose as well.

To answer @hvr concern: for uses I have in mind, we can allow only decidably disjoint ranges for or-dependencies, as above.

It's probably not hard to check, whether there are a lot of overlapping automatically selectable ranges on Hackage. My common sense says that those are probably mistakes. I cannot come with a good reason where we would like solver to arbitrarily switch flag selection with the same package index.

OTOH then we cannot add e.g. cpp-options: -DHAS_FEATURE declaration to be used by CPP in the sources, so one would need to replicate the logic somewhere. This is a minor nuance though.

[ezyang]

See my comment on the or-depends. I think if depend(base == 4.8.*) is better.

[phadej]

I try to restate this RFC.

The problem

We start to need to specify multi-way disjunctions of build-depends.
I'm not sure whether we need to support overlapping disjunctions.

Current solutions

Use automatic flags

_Bad approach_: use flag per version, i.e.

flag base47
  ...
flag base48
  ...
flag base49
  ...

As there is 8 configurations, but only three valid.

_better current approach_ is:

flag baseA
flag baseB

if flag(baseA) && flag(baseB)
  build-depends: base ==4.9.*
if flag(baseA) && !flag(baseB)
  build-depends: base ==4.8.*
if !flag(baseA) && flag(baseB)
  build-depends: base ==4.7.*
if !flag(baseA) && !flag(baseB)
  -- Non buildable configuration.
  build-depends: base <0

Proposed solutions

Multi-way-flags

As above

or-dependencies

https://github.com/haskell/cabal/issues/2033

  if flag(network-uri)
    build-depends:
      network >= 2.6 && network-uri >= 2.6 || network < 2.6

_TODO_ is there an example were we loose control (as mentioned in https://github.com/haskell/cabal/issues/2033#issuecomment-51904547)?

if-depends:

Less powerful than _or-dependencies_

if depend(network >= 2.6)
  build-depends: network-uri >= 2.6

Related

provides proposal: https://github.com/haskell/cabal/issues/3061

[dcoutts]

Oh, I should have said this in this ticket...

We've considered adding more direct support for "fail" conditions. This is probably doable but will be much much easier once we replace the parser and the .cabal AST.

There's two related ideas:

1. explicit failure, with error messages

if !(flag(curl) xor flag(http)) fail: one of curl or http must be used

This is more or less equivalent to the existing tricks to make the solution impossible in some condition, but with more obvious intention and better error messages.

1. multi-value enum flags

flag transport values: none, curl, http default: curl

This is interpreted as an enumeration, and one value must be set.
Then instead of all flags being bools, e.g --constraint=darcs +curl -http they can be enums --constraint=darcs transport=curl.

I also support the idea of a more direct way to express these "depends" style conditionals. I think I've written about this before and what it means in the current system.

Summary is, given something like:

build-depends: network
if package(network >= 2.6)
  build-depends: network-uri >= 2.6

This is construct is only valid if the package already clearly depends on network (as it does above).

It means:

flag _auto_1

build-depends: network
if flag(_auto_1)
  build-depends: network >= 2.6
  build-depends: network-uri >= 2.6
else
  build-depends: network < 2.6

whether it's handled like that in the solver, by translation, or directly is up to the implementation.

[dcoutts]

But I'm generally not overly enthusiastic about the whole idea of automatic flags, and this seems to make it likely that they'll be more widely used. They always seem backwards to me. (A flag value is inferred from packages that are available in the environment, but we're writing it down the other way around.)

_Some_ flag values are uniquely determined from packages that are available in the environment, and that's an important special case, but it's not true in general.

Would it not be better to consider something like "or-dependencies" instead?

We considered this when we added conditionals in the first place. It is essential that any or-decisions be under the (optional) control of the person building the package. Having them be controlled by flags with the option to infer the value is the solution we arrived at. The main intended use case was optional features in applications.

That said, there are cases where there is no real choice, like

build-depends: network
if flag(_auto_1)
  build-depends: network >= 2.6
  build-depends: network-uri >= 2.6
else
  build-depends: network < 2.6

because the choices of versions (which are already under the control of the builder) uniquely determine the flag choice. So in these cases it is fine to omit a builder-visible flag, and add special flagless syntax to support these cases.

Also, if we're going to change the way the flag mechanism works, and considering new constraints, it may be worth looking at the bigger picture, and actually agreeing on a number of overall design goals.

Yes. There's several related ideas here and we should think about what makes sense overall.

[ezyang]

Related #3742.

[ezyang]

As long as we're thinking big, let me put this out here: it was a mistake for the Cabal file format, as consumed by Cabal the build system, to be intended to be written by humans. Instead, there should have been an intermediate representation mediating between what humans write and what the dependency solver and the build system works with.

This would make it easier to evolve the user-facing syntax to support more "convenience" features (features that don't make the language more expressive, but make it more convenient to express certain things.) Multi-way flags are one, but there are many others, as the popularity of hpack suggests. As it stands now the existing package description format is a terrible one to transpile to.

[hvr]

@ezyang ...and we could call this intermediate representation 'Cabal Core' :laughing:

EDIT: However, related to the original proposal:

I kinda like multi-value enums, and we'd have the option at some point to extend those enums to have an "Ord instance" just like haskell enums would: "if flag(base>=v48)"

EDIT2: We should really add some elif sugar to Cabal 2.0 IMO to avoid those staircase if/else we currently have to endure... :-/

[phadej]

FWIW, adding elif to the parsec parser is trivial (maybe 5 lines + tests). With ReadP parser we are a bit out-of-luck...

[phadej]

I'm not pursuing this idea anymore/