Stack: How can I prevent `stack install` to run if I just run `stack install --enable-library-profiling`?

[adinapoli]

Hey guys,

sorry for the not-terribly-informative title, I will try to expand a bit what I mean. I have a big project at work that I would like to profile, for example to chase space leaks. My frustration at the moment is that stack rebuilds my entire project (and ALL its dependencies!) if I accidentally do this:

stack install --enable-library-profiling
-- Go for coffee, wait for 160 packages to be unregistered & reinstalled
stack install -- did by accident, without thinking about it
-- Stack unregisters all my 160 deps & reinstall them

This makes the entire profiling business too risky and simply not viable for me. What am I doing wrong?

Ideally, I would like stack to act a bit more like cabal here: one I pay the initial price to install the libs with profiling enabled, then all the other subsequent stack install install the underlying libs with profiling enabled, so that I have piece of mind of not unregistering everything by accident.

Any way I can accomplish the following with ease?

Thank you very much!

Alfredo

[borsboom]

This sounds like a regression to me. Pretty sure it used to work the way you want it to. To be clear: are your 160 dependencies mostly in the Stackage snapshot?

[adinapoli]

@borsboom I would say yes; I have roughly 40 which are not being pulled from the stackage snapshots (i.e. they are separate lines in the extra-deps section of my stack.yaml), all the rest are.

I'm using stack Version 0.1.5.0 X86_64

[borsboom]

I'm not able to reproduce this so far. I can toggle --enable-library-profiling and it doesn't rebuild any of the snapshot dependencies (it only did so the very first time I enabled --enable-library-profiling). Note that rebuilding the extra-deps is expected when toggling --enable-library-profiling.

Do you have an example project that that demonstrates the problem?

[adinapoli]

@borsboom Ah, I'm afraid that might be the culprit then. I cannot share the repo as it's a proprietary project, but I can share the stack.yaml file, which looks beefy enough:

resolver: lts-2.16
system-ghc: false
install-ghc: true
flags:
    atlas:
      library-only: true
      monitoring: false
    lifted-async:
      monad-control-1: false
    binary-serialise-cbor:
      newtime15: false
packages:
  - '.'
extra-deps:
  - atlas-1.2.6.6
  - api-tools-0.6
  - binary-serialise-cbor-0.1.1.0
  - acid-state-0.12.4
  - amqp-0.12
  - annotated-wl-pprint-0.6.0
  - ekg-0.4.0.8
  - ekg-core-0.1.1.0
  - ekg-json-0.1.0.0
  - hackage-uploader-0.2.0.10
  - io-streams-1.3.1.0
  - ixset-typed-0.3
  - pbkdf-1.1.1.1
  - postgresql-simple-sop-0.1.0.7
  - regex-compat-tdfa-0.95.1.4
  - resource-pool-catchio-0.2.1.0
  - rncryptor-0.0.2.1
  - shelly-1.5.4.1
  - snap-0.14.0.6
  - snap-core-0.9.8.0
  - snap-cors-1.2.9
  - snap-loader-static-0.9.0.2
  - snaplet-acid-state-0.2.7
  - snaplet-postgresql-simple-0.6.0.2
  - statsd-datadog-0.2.0.0
  - string-conv-0.1
  - tasty-hspec-1.1
  - tasty-rerun-1.1.4
  - threads-supervisor-1.0.3.0
  - lens-4.3.3
  - text-1.1.1.3
  - primitive-0.5.4.0
  - safecopy-0.8.2
  - exceptions-0.6.1
  - monad-control-0.3.3.1
  - lifted-async-0.6.0.1
  - aeson-pretty-0.7
  - blaze-builder-0.3.3.4
  - blaze-builder-enumerator-0.2.0.6
  - blaze-html-0.7.1.0
  - blaze-markup-0.6.3.0
  - blaze-textual-0.2.0.9

Most of the deps are needed for atlas, another in-house package I have no control over. atlas itself counts 59 direct deps, some of them constrained (is not using stack and I cannot port to it), so I think it's easy to see how all these adds up to those 160 deps I need to rebuild every time.

Here is the silly question: I guess in such circumstance I'm out of luck and there is no way of preventing recompiling packages not in the snapshot?

[borsboom]

One workaround may be to use a custom snapshot.

[adinapoli]

That sounds promising, I will have a look into that.

I guess we can close this. I hope that this issue will be of some use to people tripping up on this such I did ;)

Thanks!

[heather-alexander]

I've run into this as well. I'll try the workaround suggested here.

Nonetheless, I think it would be great if Stack avoided unregistering a library being replaced until after the replacement is fully built (or rollback unregistration if the build is interrupted). Some atomicity would make it much less costly to make stupid mistakes with the tool.

I say this after about the 10th time accidentally using the wrong set of options to build a big library and frantically hitting Ctrl-C, only to find that the damage is already done...

[adinapoli]

I say this after about the 10th time accidentally using the wrong set of options to build a big library and frantically hitting Ctrl-C, only to find that the damage is already done...

Same here unfortunately. I guess a solution might be to have an option (toggable in the stack.yaml) which would refuse to do a stack install or a similar operation whenever the tool detects a former profiling build has been attempted. That at least would prevent the frustration deriving from the accidental typing!

[maxigit]

Why is this closed ? I have the same problem (not doing install, but just build). I'm not using snapshot but just ghc-7.8.3 resolver. It's a major problem if you try to do profiling. Not only, you can make mistake and just type stack build instead of stack build all options, it's also problematic when you come back to a project and have no idea, was what the last options you use to build it. Why can't stack just keep a different version for each package and each build configuration ?

[borsboom]

@maxigit: see #1132

[charles-cooper]

It would also be helpful if one could specify the profiling options in stack.yaml, so at least they could be defaulted to in a given project / subsequent runs with the same stack.yaml file.

[mgsloan]

@charles-cooper Hey! Yes, that's a good idea, which is tracked by #846

[charles-cooper]

@mgsloan Cool :) Would I be duplicating work if I started working on this? Or is somebody already working on it

[mgsloan]

Hey! Sorry for the delayed response, feel free to implement it! I've added a comment describing what I think would be good https://github.com/commercialhaskell/stack/issues/846#issuecomment-167667824