Stack: Stack fails to build "mutually recursive" packages

[mitchellwrosen]

General summary/comments (optional)

I have a two-package project, where the library in package two depends on the library in package one.

However, there's also an executable component in package one that depends on library two. For some reason, plan construction fails, even when I try to build single components.

Steps to reproduce

1. git clone [email protected]:mitchellwrosen/stack-issue-1.git
2. stack build
   

   Expected

Build succeeds

Actual

Build fails with:

While constructing the build plan, the following exceptions were encountered:



Plan construction failed.

Stack version

Version 1.2.1, Git revision 8ebadf3d8614bd0390d8f94a3198eb47ec36c906 (dirty) (4086 commits) x86_64 hpack-0.14.1

Method of installation

Built from source

[mitchellwrosen]

Just following up on this - is it a bug, or working as intended?

[mgsloan]

Thanks a bunch for the report and repro! We do intend to support this usecase, not sure why it isn't working.

[mitchellwrosen]

No problem, and to clarify, is it actually a cycle that can't be built by
cabal? I wasn't sure - it's not actually two components that depend on each
other

On Sep 29, 2016 7:00 PM, "Michael Sloan" [email protected] wrote:

This is not intended, it's a result of some cleanups to how the output is
presented. Looks like we aren't detecting cycles correctly. Thanks a bunch
for the report and repro!

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/commercialhaskell/stack/issues/2583#issuecomment-250616363,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBlprUL8LNaRUuW7q3Rfiu73NnQVccfks5qvEMJgaJpZM4J61G3
.

[mgsloan]

Hey @mitchellwrosen, sorry for the delay! I have committed a fix to the error reporting. Now the output looks like

However, you are correct that this case is intended to work fine with stack. I wrote the code involved, and intended to address the problem soon. Marking it as a P1 bug. Thanks so much for the report and repro!

[mitchellwrosen]

Nice error message, and thanks!

[mgsloan]

We actually only support such cycles for test-suites and benchmarks. I believe this is a limitation of the Cabal library, but I'm not 100% sure what limitations we'd run into.

If I try using cabal-install, I get

mgsloan@computer:~/fpco/test-stack/stack-issue-1$ cabal install one/ two/
Warning: The package list for 'hackage.haskell.org' is 193.2 days old.
Run 'cabal update' to get the latest list of available packages.
Resolving dependencies...
cabal: internal error: could not construct a valid install plan.
The proposed (invalid) plan contained the following problems:
The following packages are involved in a dependency cycle one-0.0.0, two-0.0.0
Proposed plan:
Configured one-0.0.0 (.fake.one-0.0.0)
Configured two-0.0.0 (.fake.two-0.0.0)
PreExisting base-4.9.0.0 (base-4.9.0.0)
PreExisting ghc-prim-0.5.0.0 (ghc-prim-0.5.0.0)
PreExisting integer-gmp-1.0.0.1 (integer-gmp-1.0.0.1)
PreExisting rts-1.0 (rts)

[Blaisorblade]

It should be checked whether Cabal 2/new-build/Backpack support enables supporting this use-case, among other things.

[chris-martin]

Here's another example project, for what it's worth: https://github.com/chris-martin/stack-cycle-issue It's the same thing, though I ran into it with a test suite rather than with an executable.

Still getting the same ambiguous output:

❯ stack test

Error: While constructing the build plan, the following exceptions were encountered:



Plan construction failed.

[DanBurton]

Bump, just ran into this issue as well. @chris-martin's example code is an accurate reflection of what I encountered. Same error.

Interestingly, stack will happily build/test either package in isolation, but chokes when asked to do both.

$ stack test foo-a --dry-run # this is fine
$ stack test foo-b --dry-run # this is fine
$ stack test foo-a foo-b --dry-run # this is not fine

Error: While constructing the build plan, the following exceptions were
encountered:



Some different approaches to resolving this:
(...)
Plan construction failed.

$ stack --version
Version 1.9.0.1, Git revision a2489de02cb1c7d28b74c69bc66d2402268ccb0a (6123 commits) x86_64 hpack-0.31.0

[dbaynard]

I also encountered this with 1.9.3 — it isn't obvious why the DependencyCycle code isn’t kicking in, but the logs are particularly unhelpful, too.

[nh2]

Indeed, when you have two packages a and b, where a's test-suite depends on b and b's test-suite depends on a, which I call cross-dependencies, like this:

   a             b
library       library
   \            /
     \        / 
       \    / 
         \/
         /\
       /    \
     /        \
  a            b
tests        tests

and you do stack test then you get a very bad error message:

stack test

Error: While constructing the build plan, the following exceptions were encountered:



Some different approaches to resolving this:

  * Set 'allow-newer: true' to ignore all version constraints and build anyway.

  * Consider trying 'stack solver', which uses the cabal-install solver to attempt to
    find some working build configuration. This can be convenient when dealing with
    many complicated constraint errors, but results may be unpredictable.


Plan construction failed.

It says the following exceptions were encountered: but the section below it is just empty, not showing which exceptions it's talking about.

This is easy to reproduce by doing

• mkdir tmp && cd tmp
• stack new a && stack new b
• adding a build-depends: a to b's test suite, and the other way around
• mv a/stack.yaml ., adding - a and - b to packages:
• stack test

[nh2]

@qrilka and me tried:

When you have a cross-dependency of test and benchmark like this:

   a             b
library       library
   \            /
     \        / 
       \    / 
         \/
         /\
       /    \
     /        \
  a            b
test        benchmark

then it usually works when you use stack test or stack bench but that is only because these invocations run tests or benchmarks _only_.

As soon as you use stack build --bench --test, this fails with the above empty error message, too.

[nh2]

Thanks a bunch for the report and repro! We do intend to support this usecase, not sure why it isn't working.

@mgsloan Do you know if this was _ever_ working? (In other words, did _intend_ here mean that it was working at some point and then it suddenly stopped, or did we only always _want_ it to be working but we don't know if it did?)

[nh2]

I believe this is a limitation of the Cabal library

@mgsloan also, do you remember whether you believed this just for real library a <-> library b cycle dependencies, or also for cross dependencies like the above?

[nh2]

We also found out that if you have a "wedge" dependency like this:

   a   <------   b
library       library
                /
              / 
            / 
          /
         /
       /
     /
  a
benchmark

then this actually works:

% stack bench
a-0.1.0.0: unregistering (components added: bench:a-bench)
b-0.1.0.0: unregistering (missing dependencies: a)
a-0.1.0.0: configure
a-0.1.0.0: build
a-0.1.0.0: copy/register
b-0.1.0.0: configure (lib + bench)
b-0.1.0.0: build (lib + bench)
b-0.1.0.0: copy/register
a-0.1.0.0: configure (bench)
a-0.1.0.0: build (bench)
a-0.1.0.0: benchmarks  

As we can see, in this case stack first builds and registers libraries of a, then of b, and then runs a's benchmarks at the very end.

So the build of b is sandwiched between the library and benchmark of a as desired.

• [ ] Now we should check why it can't do the cross case with:

a: copy/register
b: copy/register
a: benchmarks
b: benchmarks

[qrilka]

it appears that master version at least shows something:

Error: While constructing the build plan, the following exceptions were encountered:

In the dependencies for b-0.1.0.0:
    a dependency cycle detected: a, b, a
needed since b is a build target.

and as a way to build Stackage snapshots with Stack for #4217 I'll need to address this issue as with all of the Stackage packages it's quite easy to get a cycle

[qrilka]

I've tried to add cycle resolution to Stack.Build.ConstructPlan in d1386fc8809cd7f68f201d1a0f87b69678a0ba3a using an ad-hoc solution but doing that doesn't appear to be an easy task without a major refactoring of that module. The current algorithm tries multi-step builds (installing library separately from tests and benchmarks) on 1 package level and resolving longer cycles requires extra bookkeeping and so is quite error-prone. E.g. the referenced attempt failed to run integration test multi-test:

    Configuring multi-test-suite-0.1.0.0...
    Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2: Encountered missing dependencies:
    cyclic -any

In any case we will need component-based build plans for Backpack support but that will come later in time.
So as a way to proceed with #4217 I will use https://github.com/commercialhaskell/stack/blob/06154682b33d7ddcb2e06b451231502fdb40773b/subs/curator/build-constraints.yaml#L4346 as a workaround and build plan resolution refactoring will appear in a later attempt.

[qrilka]

@nh2 even a "wedge" dependency could get into a problem if we'll take into account that tests (which probably get easier get to this situation than benchmarks) in a could use types from a lib which could be returned from b. So we get to the situation

     a   <----------------   b   
  library               ->library
     ^              ---/         
     |          ---/       -     
     |      ---/                 
     a   --/                     
    test 

which isn't a cycle by itself as the links are directed. But Stack uses a bit of a hack as https://github.com/haskell/cabal/issues/1575 is still not resolved after 5 years after initial filing and that hack results in multiple builds of a and Cabal starting with version 2 doesn't assume types in those builds being equal (and theoretically they could be different). This gives compilation error #3892 some extra details described in https://github.com/haskell/cabal/issues/5200