Dhall-haskell: "OSX release binaries" should be available

[justinwoo]

I get support requests for some projects I have that use Dhall from OSX users who don't realize they can use the linux releases. It would be good to upload the same content as -macos or something like that.

[justinwoo]

Sorry, now I've realized the preface here is silly since the binary needs some patching for OSX, it seems

[Gabriel439]

Yeah, I thought the Linux binaries didn't work on OS X. I use OS X as my development environment and they failed with:

$ ./bin/dhall --help
-bash: ./bin/dhall: cannot execute binary file

[adrianroe]

Can I echo that OSX binaries would indeed be helpful. I'm trying to go through the tutorials and both
brew install dhall-json
and
brew install dhall-json --HEAD
seem to install version 1.2.4 whereas the "getting started" guide assumes 1.2.5

[adrianroe]

Having a dhall-json --version would also be very helpful!

[justinwoo]

dhall-to-json --version already exists in 1.2.5, but not in 1.2.4.

You'll see that the dhall formula PR for homebrew is stuck here: https://github.com/Homebrew/homebrew-core/pull/34157

[Gabriel439]

In the short term, I think we should continue to lean on brew as the distribution mechanism for OS X because that involves the least ongoing maintenance work for all of us.

The thing I'm hoping for in the long-term is to build statically linked OS X and Windows binaries using Nix like we do for Linux. We would be able auto-generate Windows binaries if this work gets upstreamed into Nixpkgs:

https://github.com/input-output-hk/nix-hs-hello-windows

For OS X, I don't know how much more work it would take to get the statically linked Linux executable support from https://github.com/NixOS/nixpkgs/issues/43795 to also work for OS X (cc: @nh2). It seems like the main Linux-specific part is the use of musl instead of libc

[nh2]

@Gabriel439 I learned from @angerman that static linking can be a big pain on OSX because Apple makes it difficult to do (perhaps you can expand on that a bit here @angerman?).

[blast-hardcheese]

I missed the github notification due to being out of town, I've addressed the concerns in the homebrew PR -- hopefully that unblocks getting the dhall binary directly into homebrew.

I guessed at what amount of coverage was required as far as tests, considering dhall is self-contained and written in a safe language. If there are some other tests that should explicitly be added to ensure functionality I'm not thinking about are covered, please feel free to comment and I'll add them.

[f-f]

Right now we publish only Linux and Windows binaries in releases, while the brew build appears to be building the binary from scratch.

I would like to setup an npm package for Dhall, as I think npm is a decent binary distribution method too.
We distribute spago with it, and it seems to work fine: being able to just npm install dhall would definitely help a lot with the question "how do I install this tool" (that I seem to get pretty often), and it would work on Linux, Windows and macOS.

This would require some changes:

• adding a macOS-only Travis build to:
  
  
  
  • build the binaries
  
  
  • on release, package them and upload them to the github release
  
  
  • on release, publish a new version of the npm package that just downloads the binaries (example unpacking script)
  
  
• change the Appveyor build so that it also packs files in .tar.bz2 so the unpacking script is simpler

I'd be happy to make this changes (and maintain them) if we are fine with having an npm package

[Gabriel439]

@f-f: Why is brew building the binary from scratch? My understanding is that brew supports a binary cache for the executables it distributes so I want to make sure that this isn't due to a misconfiguration of some sort

I'm reluctant to use Travis, though, as we kept running into timeouts the last time we tried using it to build on OS X and there is no paid option to extend the timeout.

[f-f]

Why is brew building the binary from scratch? My understanding is that brew supports a binary cache for the executables it distributes so I want to make sure that this isn't due to a misconfiguration of some sort

Sorry what I meant here is that they compile the binary on their CI infrastructure instead of pulling it from our releases (because we don't distribute the binary). If you install with brew it will indeed download the binary from their artifacts cache

I'm reluctant to use Travis, though, as we kept running into timeouts the last time we tried using it to build on OS X and there is no paid option to extend the timeout.

We run the CI for spago (and this is relevant as we have dhall as a dependency) on Travis building binaries for Linux and macOS since 6 months..and the interesting catch with Travis+Haskell is that you'll run into timeouts, but only for Linux builds.
In fact, the macOS is quite quick once the cache is warm (e.g. here's the latest job from spago's master, which built in 6 mins), but the only way to not have the Linux build timeout was to precompile all dependencies in a Docker image and pull that image for every build. Since it's >2GB it takes ~10mins just for the docker pull, but it looks like they are happy to tradeoff CPU resources for bandwidth 😄

TL;DR: in my experience the timeouts on Travis happen only on Linux builds and the macOS builds are fine, so I think we won't have this problem here

[Gabriel439]

So if I remember correctly the Travis timeouts we had were for OS X specifically, but we can try again if you want

[f-f]

Cool, I'll try this then; worst case we get back to the current situation 🙂

[joneshf]

Any updates?

[f-f]

@joneshf yep sorry, I have a branch locally that needs some polishing, I'll push it in the next days

[joneshf]

Coolio!

[jhenahan]

Just discovered @justinwoo's easy-dhall-nix, so I'm also very interested in helping with the Darwin release situation if there's anything I can do.

[Gabriel439]

Note that most of the dhall packages are available via brew and should be up to date. The exception is the new formula for dhall-bash, which is still waiting on review: https://github.com/Homebrew/homebrew-core/pull/41390. Also, now that I know how to update brew formulae I can do so upon each release.

However, I'd still welcome CI support for OS X executables regardless.

[Gabriel439]

@jhenahan: You can now obtain pre-built executables from the release page: https://github.com/dhall-lang/dhall-haskell/releases