Stack: Avoid rebuilding "sandbox" when profiling options turned off

[s9gf4ult]

Hello, I have an issue. Steps to reproduce:

1. cd myproject
2. stack build --library-profiling --executable-profiling to build "sandbox" and myproject with profiling options
3. stack build

Expected behaviour:
Reconfigure myproject and rebuild it without profiling options like cabal clean && cabal configure && cabal build would.

Actual behaviour:
Stack rebuilds all "sandbox" without profiling options which takes a lot of time.

Use case: when testing myproject runtime error occures and developer want to see a stacktrace (which requires profiling options). He/she rebuilds project (with "sandbox") with profiling options, fixes issue and turns profiling options off (to speed up compilation).

[borsboom]

This might be the same issue as #1102.

[heather-alexander]

Looks the same to me. I have the same issue; I'm posting slightly more thoughts at #1102.

[s9gf4ult]

Yes, this issue describes the same case as in https://github.com/commercialhaskell/stack/issues/1102 bot the former is closed. Should I close this issue as well? And what is the funal answer? Will issue be fixed, or actual behaviour is considered correct?

[borsboom]

Have you tried stack-0.1.6.0? It now has settings that control when command-line flag changes trigger recompiles, and the defaults for those new settings are for it to default to _not_ rebuilding when command-line flags change.

[pakanek]

I have similar issue, but in my case stack rebuilds packages OpenGLRaw, sdl2, parsec3-numbers. And it does that sometimes when I run stack haddock or stack build (with or without --library-profiling --executable-profiling).
Version 0.1.6.0, Git revision e22271f5ce9afa2cb5be3bad9cafa392c623f85c (2313 commits) x86_64

$ stack haddock
OpenGLRaw-2.6.0.0: unregistering (rebuilding with haddocks)
parsec3-numbers-0.1.0: unregistering (rebuilding with haddocks)
ren-0.1.0.0: unregistering (missing dependencies: OpenGLRaw, parsec3-numbers, sdl2)
sdl2-2.0.0: unregistering (rebuilding with haddocks)
parsec3-numbers-0.1.0: configure
parsec3-numbers-0.1.0: build
OpenGLRaw-2.6.0.0: configure
...

$ stack build --executable-profiling --library-profiling
OpenGLRaw-2.6.0.0: unregistering (flags changed from [] to ["--enable-library-profiling","--enable-executable-profiling"])
parsec3-numbers-0.1.0: unregistering (flags changed from [] to ["--enable-library-profiling","--enable-executable-profiling"])
ren-0.1.0.0: unregistering (missing dependencies: OpenGLRaw, parsec3-numbers, sdl2)
sdl2-2.0.0: unregistering (flags changed from [] to ["--enable-library-profiling","--enable-executable-profiling"])
sdl2-2.0.0: configure
sdl2-2.0.0: build
parsec3-numbers-0.1.0: configure
...

stack.yaml:
...
extra-deps:
- sdl2-2.0.0
- OpenGLRaw-2.6.0.0
- parsec3-numbers-0.1.0
...

[saevarb]

Same exact issue as @pakanek, same exact version. I am in fact even building some of the same libraries(OpenGLRaw), which is taking forever on my machine, and it results in a horrible, really frustrating experience with profiling.

Why aren't copies of libraries with profiling and without kept around when switching between profiling and no profiling? Am I correct in assuming that would make the build times more tolerable?

Moreover, I don't remember having a similar issue when just using vanilla cabal. If I remember correctly, the issue with cabal and profiling was that you'd have to recompile everything _once_ once you enabled profiling, but not constantly, over and over again. Additionally, if you wanted only to profile your own code, you could just enable profiling for your own code.

This is basically exactly what I want in my case. I don't really care about the performance of gloss(which is what is being recompiled constantly), so I don't want to enable profiling for gloss.

In a vain attempt to fix this issue, I just did a huge refactor, such that all the code I want to
profile doesn't depend on gloss in any way anymore. I thought this would mean that --enable-library-profiling would enable profiling for only my own libraries in my cabal file, but after making this change
and enabling profiling.. well, I'm writing this as I listen to my laptop choke on recompiling gloss.

I am really not sure if the behavior I expect is unreasonable, but I would like to know if either a) it is possible to make this experience more tolerable with stack as-is or b) if it is something that should
be fixed(and if there is something I can do to help with that).

However, I do believe there is at least one real issue for which I can submit an actual issue if someone
can confirm I'm not just doing something wrong:

If you enable profiling for your project, build it, and then run stack ghci, it will go ahead and
recompile everything without profiling. This is obviously not desireable behavior, and I suspect it
may be related to / same as #1186.

EDIT: I just realized that the packages that I keep having to recompile every time are extra-deps. Is it correct that the behavior wrt. profiling is different for extra-deps?

[borsboom]

This is the current expected behaviour. My proposal in #1265 would fix it for extra-deps (by making them act just like snapshots), but wouldn't change the behaviour for regular local packages.

I _think_ it would be possible to fix for regular local packages as well. It should be possible to have profiling flags treated specially, and not trigger rebuilds unless actually necessary. The rules might go something like this (assuming no source code or other flags changed between builds):

| Past build enabled profiling | New build enables profiling | Rebuild? |
| :-: | :-: | :-: |
| No | No | No |
| No | Yes | Yes (with profiling) |
| Yes | No | No |
| Yes | Yes | No |

I haven't really thought about how library profiling vs. executable profiling would fit in here, but I think the basic principle would hold.

@snoyberg: does this seem feasible to you?

[snoyberg]

Yes, it's feasible. I think in some of the predecessor tools to Stack, we
implemented exactly this logic.

On Mon, Nov 2, 2015 at 3:05 PM, Emanuel Borsboom [email protected]
wrote:

This is the current expected behaviour. My proposal in #1265
https://github.com/commercialhaskell/stack/issues/1265 would fix it for
extra-deps (by making them act just like snapshots), but wouldn't change
the behaviour for regular local packages.

I _think_ it would be possible to fix for regular local packages as well.
It should be possible to have profiling flags treated specially, and not
trigger rebuilds unless actually necessary. The rules might go something
like this (assuming no source code or other flags changed between builds):
Past build enabled profiling New build enables profiling Rebuild? No No No
No Yes Yes (with profiling) Yes No No Yes Yes No

I haven't really thought about how library profiling vs. executable
profiling would fit in here, but I think the basic principle would hold.

@snoyberg https://github.com/snoyberg: does this seem feasible to you?

—
Reply to this email directly or view it on GitHub
https://github.com/commercialhaskell/stack/issues/1132#issuecomment-153186702
.

[ntc2]

Note that #1265 is now closed, but the way it was implemented means this issue is still not fixed. The reason given in #1265 for not caching extra-deps was that it could lead to too much disk usage. For my purposes, just having two caches -- indexed by flags -- would be sufficient: the most recent non-profiling build and the most-recent profiling build. A more general version could have a conflig setting for how many caches to keep, but two is enough for the common case of speeding up the change between profiling and non-profiling builds.

[ntc2]

This may be mentioned in one of the duplicate tickets, but I think the --work-dir option can be used to work around this issue. For example, if you use

stack --work-dir .stack-work-profiling --profile build

for profiling builds and

stack build

for non-profiling builds, then the builds don't conflict/invalidate each other.

[bitemyapp]

@ntc2 nice work-around! Thanks!

[snoyberg]

Looks like this will be resolved by #4032, which is currently under development. Closing this as a duplicate.