cabal new-build does not track files added via TH’s addDependentFile

[cocreature]

Steps to reproduce

• Download https://gist.github.com/cocreature/d07ef6f35f7bd9043db50794fe31e977.
• Build using cabal new-build
• Edit fileinput
• Try rebuilding using cabal new-build.

Expected behavior

cabal new-build rebuilds because fileinput has changed.

Actual behavior

cabal new-build exits without rebuilding after saying Up to date.

This issue affects both cabal-2.0 as well as the master branch.

[phadej]

One hack solution is to watch extra-source-files too. But that won't solve dynamic (e.g. what inline-c uses?) stuff.

[23Skidoo]

BTW, is there a way to force ghc --make always rerun a TH function? E.g. if I'm doing some kind of I/O at compile-time, like reading from an environment variable.

[phadej]

@23Skidoo that would force GHC to always build that module, that doesn't sound like a very good idea.

[23Skidoo]

Yep, but sometimes you want that in a build system. For example, Shake rules can be marked alwaysRerun.

Re: extra-source-files, that field is package-global and commonly used for things like README and changelog. So we'd need to allow per-component extra-source-files to avoid unnecessary recompilation.

[23Skidoo]

An alternative way to fix this is to get the list of files to be monitored from ghc --make via -ddump-module-graph (a new feature that is going to be in 8.4).

[phadej]

Does -ddump-module-graph dump addDependentFile & co stuff?

[23Skidoo]

@phadej If you click that link, you'll see that I asked that exact question there.

[cocreature]

Given that this feature will only be in GHC 8.4 I don’t think that’s a reasonable solution at this point.

[23Skidoo]

@cocreature With old versions we can simulate that feature with ghc -M (which will be slower because we'll have to do dependency resolution twice - once for ghc -M and once for ghc --make, but for legacy GHC versions we can afford to not care). BTW, ghc -M does support addDependentFile dependencies:

# DO NOT DELETE: Beginning of Haskell dependencies
Data/FileEmbed.so : Data/FileEmbed.hs
Main.so : Main.hs
Main.so : Data/FileEmbed.shi
# DO NOT DELETE: End of Haskell dependencies

[hvr]

@23Skidoo btw, we can avoid paying for the slow-legacy fallback when it's not needed, because we know when TH is needed (we already require TemplateHaskell to be declared in *-extensions in order to enable other logic specific to TH). So I think that's a workable fallback.

[hvr]

Unfortunately we not only missed the boat for GHC 8.4, but most likely also for GHC 8.6 to get supporting features into GHC... as https://phabricator.haskell.org/D3898 got stuck on a decision which design path to go down... :-/

PS: Screen-scraping GHC's dump-hi output would be an obvious but terrible hack for the obvious reasons and since it's not intended to be consumed by tooling its format is neither documented nor guaranteed to be correct and can change without notice.

[ygale]

Upvoting this issue. It is critical for larger web-dev shops that have people who are totally focused on front-end web UI development. They require instant reload; they don't have time to poke around at files manually every time to induce rebuilds.

We are a yesod shop, so we are using stack. For comparison, stack detects this by parsing dump-hi files. The syntax change in dump-hi files in GHC 8.4 caused a similar issue in stack for a time, now fixed.

[tom-audm]

Another upvote for this issue!

I think type:bug is a more accurate description than type: enhancement. It's a regression from cabal old-build's behavior, and can cause truly wtf moments when using e.g. gitrev (see my issue #5866 ). I'd say it actually breaks the gitrev package for practical usage.

[kaldonir]

We ran into the same issue today. Is there any progress here?

[phadej]

@kaldonir, no. As far as I know no-one is actively working on adding necessary features to GHC (e.g. https://github.com/ghc-proposals/ghc-proposals/pull/245 or something similar so the compiler could tell Cabal what extra dependencies exist).

Alternatively or additionally cabal could track extra-source-files, which is imprecise yet better than nothing. Again, I'm not aware anybody is working on that either.

[kmicklas]

@phadej Are you opposed to making cabal track extra-source-files as an intermediate step? This wouldn't help with gitrev but would solve this for I suspect most use cases, until we have a proper fix in GHC. I'm willing to implement this if there's support for it.

[phadej]

I'm not. It should be easy, I think it should go to the same place as https://github.com/haskell/cabal/commit/c2f4625f2feeda94a9174d927dc39a91c0059a7e

PR welcome!