Stackage: Metadata contains wrong version for LTS-13.16 for transformers and process packages

[phadej]

Versions on website (and .yaml in stackage-lts repository) say version transformers-0.5.5.0 and process-1.6.3.0

• https://www.stackage.org/lts-13.16/package/transformers-0.5.5.0
• https://www.stackage.org/lts-13.16/package/process-1.6.3.0

However, when I % stack --resolver=lts-13.16 exec bash and

bash-4.4$ ghc-pkg list
/home/ogre/.stack/programs/x86_64-linux/ghc-8.6.4/lib/ghc-8.6.4/package.conf.d
    Cabal-2.4.0.1
    array-0.5.3.0
    base-4.12.0.0
    binary-0.8.6.0
    bytestring-0.10.8.2
    containers-0.6.0.1
    deepseq-1.4.4.0
    directory-1.3.3.0
    filepath-1.4.2.1
    ghc-8.6.4
    ghc-boot-8.6.4
    ghc-boot-th-8.6.4
    ghc-compact-0.1.0.0
    ghc-heap-8.6.4
    ghc-prim-0.5.3
    ghci-8.6.4
    haskeline-0.7.4.3
    hpc-0.6.0.3
    integer-gmp-1.0.2.0
    libiserv-8.6.3
    mtl-2.2.2
    parsec-3.1.13.0
    pretty-1.1.3.6
    process-1.6.5.0
    rts-1.0
    stm-2.5.0.0
    template-haskell-2.14.0.0
    terminfo-0.4.1.2
    text-1.2.3.1
    time-1.8.0.2
    transformers-0.5.6.2
    unix-2.7.2.2
    xhtml-3000.2.2.1

Note: process-1.6.5.0 and transformers-0.5.6.2!

[angerman]

Note that this is not just 13.16, I believe this affects all lts snapshots for 8.6.4 since ghc-8.6.3.

Also note: process-1.6.3.0 has an upper bound on base with <4.12, and base from ghc-8.6 has version 4.12.

[snoyberg]

Is this affecting display only, or are there tools which are being affected by this? This is definitely a metadata bug, but we're in the process of moving away from listing global packages in these snapshot files anyway, which will more holistically address this.

[angerman]

@snoyberg this is affecting tools that rebuild packages.

[snoyberg]

Can you clarify further which tools are being affected? I'm trying to understand what kind of fix is needed here

[angerman]

Can you clarify further which tools are being affected? I'm trying to understand what kind of fix is needed here

Concretely it falls apart for us with our haskell.nix, as that rebuilds all packages and the build plan as provided by stackage contains the wrong process and transformers packages; which fail to build. Thus we have to graft those as extra deps onto each project to fix the versions.

[snoyberg]

I don't know exactly how the Nix code uses the global package information, but it might work to replace the package versions in the YAML files in the lts-haskell repo. You want to try sending a PR and see if that resolves the situation?

[angerman]

https://github.com/input-output-hk/haskell.nix just uses the translated expressions from https://github.com/input-output-hk/stackage.nix (which is essentially just a 1:1 translation).

E.g. lts-13.13 is this one: https://github.com/input-output-hk/stackage.nix/blob/master/lts-13.13.nix

We could probably add an additional overlay to fix up the versions. Or hack the expressions here: https://github.com/input-output-hk/stackage.nix/blob/master/lts-13.13.nix#L2087

I'd prefer to rely on stackage as the ground truth though.

[snoyberg]

I'd prefer to rely on stackage as the ground truth though.

Sending a PR to update the yaml files should allow that, unless I'm missing something

[angerman]

@snoyberg alright. Was't sure how immutable stackage was supposed to be.

[DanBurton]

If I understand correctly, the root cause of this metadata bug is that we updated our policy about allowing a minor ghc upgrade mid-lts, but neglected to catch this bug when upgrading our tools & processes to accommodate the new policy.

There is reasonable confidence that stack will be ok with the PRs so long as they only change versions of globally-installed packages. As for other downstream users, it may indeed disrupt someone who was assuming these files are never edited, but nix is the only other serious downstream consumer of these files that I'm aware of.

It may be wise to check content hashes and download from specific git revisions in order to guarantee reproducibility when using these files. Alternatively, if our downstream users are requesting 100% immutability of the files, we can consider that. On the one hand, immutability is a Good Thing. On the other hand, files which are forever buggy are Not a Good Thing.

[DanBurton]

The new format for stackage snapshots does not have this issue.

See, for example, https://github.com/commercialhaskell/stackage-snapshots/blob/master/lts/14/7.yaml