Cabal: `cabal new-install --only-dependencies --enable-tests` doesn't work

[goldfirere]

The title says it all. If I want to install the dependencies of a project (but not the project itself), including the dependencies of its testsuite, how do I do that in the brave new world?

[quasicomputational]

It'll depend precisely what you want to do.

• If you want the dependencies and such installed for global ghci use, then new-install would need to be fixed. This is a legitimate bug, cc @typedrat:

$ cabal new-install  --only-dependencies --enable-test --enable-bench
cabal: --enable-tests was specified, but tests can't be enabled in a remote
package

• If you just want the dependencies built and put into the store so that subsequent builds don't need to build them, cabal new-build --only-dependencies --enable-tests will do what you want.

[goldfirere]

I want the former.

My usage pattern for GHC is probably highly unusual, in that I rarely work on discrete projects (other than GHC, which doesn't really use cabal). Instead, I'm usually compiling project-less .hs files, as I'm tinkering with types. So I like to accumulate a large glob of installed packages in my global database, allowing these project-less files to compile with ease. Yes, I know the hazards of a large global database (and I am bitten by these hazards from time to time), but it's still the right workflow for my usual needs.

Because my situation is unusual, I'm OK with needing to specify a lesser-used flag or twiddle some settings, but I'd be sad if I can't do what I like anymore.

In general, I'm always a bit saddened by the general trend in Haskell tooling (both cabal and stack) to seem to require projects -- it all seems to discourage fly-by-night experimentation in favor of Serious Daytime Work on Software That Does Things. Of course, SDWoSTDT is what makes the world go round, but we can't all be serious all the time.

[23Skidoo]

Maybe the new script feature of cabal new-run can work for you? See https://www.haskell.org/cabal/release/cabal-latest/doc/users-guide/nix-local-build.html#cabal-new-run

[goldfirere]

Interesting. I like the idea of specifying, right in my code, what its dependencies are. I might indeed use this.

I still see two shortcomings:

• What if I don't want to run, but just compile?
• I don't like anything to come between me and my ghc. I often pass exotic flags to GHC, and want that to remain easy. (cabal wiz-waz blurp --ghc-options="-x -y -z" is not considered easy, to me.)

Again, I know I'm an unusual consumer here... but I don't think I'm so unusual as to be unique, and so it seems there should be some way to do this.

[23Skidoo]

You can have a ghc-options field in the {- cabal: block too. Regarding the former, not really supported, though you can try putting -fno-code in your ghc-options. I guess we could make new-build or (currently non-existent) cabal typecheck work in script mode too...

[goldfirere]

I could also have a {-# OPTIONS_GHC ... #-} block. But I still don't like having some other software getting between me and my compiler -- I want to just call ghc directly. Today, this is possible because I can install system libraries. The chief advantage of a direct call is that I know, precisely, how GHC has been configured; this is also easier to reproduce across systems.

In any case, you proposed in your first response a way forward: just allow the command in the title.

[fgaz]

Installing without the tests + installing the test deps manually should work though

[goldfirere]

Of course it would.... but I'd prefer that cabal have this capability directly. Is this hard to implement? That would surprise me. In any case, it's a regression from the abilities of today's cabal.

[RyanGlScott]

I often find myself in similar "tinkering" scenarios as @goldfirere. For those scenarios, I often use ghci -package-db ~/.cabal/store/ghc-<ver>/package.db (where <ver> might be, say, 8.6.3) to fire up a repl with the latest versions of every package I've installed via new-build, new-install, new-repl -b, etc. Then, if I want to make more packages available in my package.db, it's just a matter of running the right new-* command.

Unfortunately, I don't yet have a convenient alias for what cabal old-install --enable-tests --only-dependencies used to accomplish. The most direct workaround I can think of is cabal get <package-name> && cd package-name-x.y.z && cabal new-build --only-dependencies --enable-tests, which is rather indirect. It would be good to make new-install --only-dependencies --enable-tests do the right thing.

[phadej]

new-build --only-dependencies will install deps in store, or am I missing something?

Sent from my iPhone

On 12 Feb 2019, at 14.17, Ryan Scott [e-mail] wrote:

I often find myself in similar "tinkering" scenarios as @goldfirere. For those scenarios, I often use ghci -package-db ~/.cabal/store/ghc-/package.db (where might be, say, 8.6.3) to fire up a repl with the latest versions of every package I've installed via new-build, new-install, new-repl -b, etc. Then, if I want to make more packages available in my package.db, it's just a matter of running the right new-* command.

Unfortunately, I don't yet have a convenient alias for what cabal install --enable-tests --only-dependencies used to accomplish. The most direct workaround I can think of is cabal get && cd package-name-x.y.z && cabal new-build --only-dependencies --enable-tests, which is rather indirect. It would be good to make new-install --only-dependencies --enable-tests do the right thing.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[RyanGlScott]

Correct. The thing that's missing is the ability to do this for any package that lives on Hackage by just specifying the name of the package. cabal new-install <package-name> gets you close, but it lacks a functional --enable-tests flag, as documented in this ticket.

(Note that cabal new-build --enable-tests <package-name> isn't possible, as new-build only works when your CWD is directly inside of a project.)

[phadej]

I'm confused.

Does cabal old-build install --only-dependencies --enable-tests package-on-hackage work? I.e. solves for dependencies of some package-on-hackage version and installs library and test dependencies?

---

One could make a wrapper of

cabal get package
cd package-x.y.z
cabal new-build --only-dependencies --enable-tests

but that fixes the version of package, so is different from cabal new-install --lib package. Latter is free to chose the version of package.

Maybe someone else could figure out the sane Cli UI for this, but I think trying to cater for an exceptional (as Richard mentions himself) use case.

---

I want to point out that working with ~/.cabal/store/ghc-<ver>/package.db directly, is picking with internals; instead one should use package environments: (you can use only one or few, it's up to you)
I'd also argue, that if anything related, we should improve environment management capabilities in cabal.

The above works ok-ish (not really: https://github.com/haskell/cabal/issues/5888)

% cabal new-install --lib --package-env=lensy lens   
Resolving dependencies...
Up to date
% ghci -package-env=lensy
GHCi, version 8.4.4: http://www.haskell.org/ghc/  :? for help
Loaded package environment from /home/ogre/.ghc/x86_64-linux-8.4.4/environments/lensy
Loaded GHCi configuration from /home/ogre/.ghci
λ> :m +Control.Lens
λ Control.Lens> 

[phadej]

Can @goldfirere confirm you try to run cabal new-install --only-dependencies --enable-test --enable-bench in a project directory. Does below command do what's requested

cabal new-build  --only-dependencies --enable-test --enable-bench

(note: new-install changed to new-build)

---

I think that @RyanGlScott's follow-up is asking for something different (and IMHO difficult). Note that

new-install --lib foo

is free to pick any version of foo. And that freedom is the difficult part with --enable-tests.

[RyanGlScott]

Does cabal old-build install --only-dependencies --enable-tests package-on-hackage work? I.e. solves for dependencies of _some_ package-on-hackage version and installs library and test dependencies?

It works in the sense that it installs the test dependencies, yes. Note that I don't really care if there are different build plans for the library and the test suite—I just want to make sure that there is _some_ build plan installed so that I'm then free to tinker afterwards. (I suspect @goldfirere is in the same boat here.)

I want to point out that working with ~/.cabal/store/ghc-<ver>/package.db directly, is picking with internals; instead one should use package environments: (you can use only one or few, it's up to you)
I'd also argue, that if anything related, we should improve environment management capabilities in cabal.

I agree that using a package database directly like this isn't the best state of affairs, but there's not really a better alternative. Package environments didn't appear to work beyond trivial examples when I last tried it (I'm not sure if #5888 is the direct reason for this, but it looks related). I'm just offering what works well for me at the moment.

It's also worth noting that this -package-db hack is also useful if you need to invoke ghc directly on a file-plus-some-dependencies (perhaps to produce an executable). There really isn't any cabal way to do this at the moment AFAICT. (I've tried cabal new-exec -- ghc, but that doesn't work.) For this use case, there's _really_ no alternative except -package-db.

[phadej]

1. We need more precise specification of target selection than we have now at https://cabal.readthedocs.io/en/latest/nix-local-build.html#cabal-new-build
2. We need to use it uniformly, as a concrete example: I'd expect something like new-install --only-dependencies pkg:test:spec to work. (I actually don't know when one needs --enable-tests and when doesn't when asking for a specific test component?)

[goldfirere]

Here is what I observe with the latest released cabal, in a project directory:

rae:10:25:40 ~/work/packages/singletons> cabal new-install --only-dependencies --enable-test --enable-bench
cabal: --enable-tests was specified, but tests can't be enabled in a remote
package

When I use new-build instead of new-install, something happens, but not what I want: cabal has installed all the necessary dependencies locally, in the package-local package database. I want these packages in my global package database, so that they are available when I try to compile a non-cabalized source file, independent of any project.

[phadej]

The packages are installed also in ~/.cabal/store/... database as Ryan pointed out above. cabal v2-build doesn't touch and won't touch global database, that's the point of v2-build.

Sent from my iPhone

On 12 Feb 2019, at 17.31, Richard Eisenberg [e-mail] wrote:

Here is what I observe with the latest released cabal, in a project directory:

rae:10:25:40 ~/work/packages/singletons> cabal new-install --only-dependencies --enable-test --enable-bench
cabal: --enable-tests was specified, but tests can't be enabled in a remote
package
When I use new-build instead of new-install, something happens, but not what I want: cabal has installed all the necessary dependencies locally, in the package-local package database. I want these packages in my global package database, so that they are available when I try to compile a non-cabalized source file, independent of any project.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[goldfirere]

Right -- I know that new-build doesn't touch the global database. That's why I want new-install to work in this scenario. I was following your suggestion in https://github.com/haskell/cabal/issues/5650#issuecomment-462781827, to see if maybe I misunderstood new-build.

Right now, I can say cabal install --only-dependencies --enable-tests to get cabal to globally install the dependencies of a package, including that package's tests. It seems that behavior is impossible to replicate with the new- commands. My understanding is that the new style is meant to be strictly more powerful than the old style, but it seems that this is not the case in this one use-case.

[phadej]

Note that new-install doesn't touch global database either. new-install --lib works by maintaining a package environment, so it appears that packages are in global database.

So yes, v2-build is less powerful, as it doesn't allow any modifications to global database. I argue you don't want that ever (except maybe for ghc-dev where you might tear the whole compiler version after a week, so you don't have time to run into Cabal Hell).

Sent from my iPhone

On 12 Feb 2019, at 17.39, Richard Eisenberg [e-mail] wrote:

Right -- I know that new-build doesn't touch the global database. That's why I want new-install to work in this scenario. I was following your suggestion in #5650 (comment), to see if maybe I misunderstood new-build.

Right now, I can say cabal install --only-dependencies --enable-tests to get cabal to globally install the dependencies of a package, including that package's tests. It seems that behavior is impossible to replicate with the new- commands. My understanding is that the new style is meant to be strictly more powerful than the old style, but it seems that this is not the case in this one use-case.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[phadej]

I suspect there are sub-issues which e.g. v2-repl (instead of raw ghci) could solve if we take a step back and think properly what we are trying to do, X, instead of concentrating on "operational" problem Y = "we need to populate specific package database". Similarly, writing a *.cabal file to accompany single .hs is not much asked, IMHO; especially with new improvements in the format, .cabal can be very terse; yet explicit about a lot of things, incl. dependencies.

Sent from my iPhone

On 12 Feb 2019, at 17.49, Oleg Grenrus [e-mail] wrote:

Note that new-install doesn't touch global database either. new-install --lib works by maintaining a package environment, so it appears that packages are in global database.

So yes, v2-build is less powerful, as it doesn't allow any modifications to global database. I argue you don't want that ever (except maybe for ghc-dev where you might tear the whole compiler version after a week, so you don't have time to run into Cabal Hell).

Sent from my iPhone

On 12 Feb 2019, at 17.39, Richard Eisenberg [e-mail] wrote:

Right -- I know that new-build doesn't touch the global database. That's why I want new-install to work in this scenario. I was following your suggestion in #5650 (comment), to see if maybe I misunderstood new-build.

Right now, I can say cabal install --only-dependencies --enable-tests to get cabal to globally install the dependencies of a package, including that package's tests. It seems that behavior is impossible to replicate with the new- commands. My understanding is that the new style is meant to be strictly more powerful than the old style, but it seems that this is not the case in this one use-case.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[goldfirere]

I agree about the focus on the "operational" problem. So I'll describe my use-case:

I frequently have a scrap of code that I wish to interact with. This happens both as I'm poking around with GHC, and when I'm teaching. (So my being a GHC dev isn't the only reason I want this.) I want to have a set of libraries available so that when I compile that scrap of code, it can import what it needs. When I teach, I organize my course into separate lectures, with the example code in a folder corresponding to the lecture. There is no .cabal file in sight. Sometimes, a student question during class requires that I add an import -- I want this just to work (as I'm live in front of my class), and not need to fiddle with a .cabal file to make it work.

I recognize that these are not common scenarios, but I also can't imagine I'm the only one who wants to compile Haskell files without writing a project.

The --only-dependencies --enable-tests desire comes from wanting to debug in GHCi embedded into emacs. I install the dependencies of a package into my global database, so my emacs-based GHCi session can find them. Then, I can compile the bits and pieces of my package easily, with nice editor integration. Only when that works do I try to build with cabal. And, while I'm tinkering, it's very nice to have lots of packages available, so I don't need to interrupt my thoughts about coding with looking up version numbers to add to a .cabal file.