Dhall-haskell: Move HsYAML-based code in dhall-json to a separate package dhall-yaml

[sjakobi]

1379 requests support for YAML with non-Text keys. To enable this, we have to translate Dhall to YAML directly, instead of going via JSON. At that point the HsYAML-based YAML-generation and parsing code doesn't have a justification to live in dhall-json anymore. Instead it should be moved to a separate package dhall-yaml.

The aeson-yaml-based version of dhall-to-yaml could continue to live in dhall-json.

As an additional benefit, we can get rid of the gpl flag in dhall-json.

[sjakobi]

The aeson-yaml-based version of dhall-to-yaml could continue to live in dhall-json.

I'm wondering whether we should rename this executable then, so it doesn't get confused with the one from dhall-yaml. dhall-to-yaml-via-json? dhall-to-yaml-light?

Or maybe we can completely get rid of this executable and recommend that users chain dhall-to-json with the json-to-yaml executable from aeson-yaml?

[neongreen]

Why would we want to recommend dhall-to-json | json-to-yaml instead of dhall-to-yaml in the first place, though?

[sjakobi]

Why would we want to recommend dhall-to-json | json-to-yaml instead of dhall-to-yaml in the first place, though?

dhall-yaml will have a GPL license. Some people might not want to use it for that reason.

[Gabriel439]

We definitely should keep the dhall-to-yaml executable. Removing the dhall-to-yaml executable from the dhall-json package would probably be too disruptive at this point

The suggestion I would make is to call the new one dhall-to-yaml-ng (where -ng is short for "next-gen"), to indicate that it's more featureful because it is the next generation of the tool

[sjakobi]

The suggestion I would make is to call the new one dhall-to-yaml-ng (where -ng is short for "next-gen"), to indicate that it's more featureful because it is the next generation of the tool

IMHO the ng suffix doesn't communicate very clearly that it offers more features. It could also be misunderstood to indicate that the "classic" dhall-to-yaml is "old" or would be superseded by the "new" version. I might be overthinking this… – anyways, I propose dhall-to-yaml-fancy! ;)

I assume that we want dhall-to-yaml-{ng,fancy} to provide a superset of the functionality of "classic" dhall-to-yaml. I wonder how to best ensure that the CLIs and their behaviours stay compatible.

We could have a small dhall-yaml-common package that contains common options and option parsers for both tools. We could also have a common test-suite for both tools, either in dhall-json or dhall-yaml or another package dhall-yaml-tests.

Thoughts?

[Gabriel439]

My understanding is that the whole reason for the switch to HsYAML is that it's supposed to supersede the old version. From my point of view it's a feature if the new executable supersedes the old one since we can eventually stop maintaining the old one instead of having to develop both in parallel. If we have to implement the same functionality twice indefinitely then it would be a drag on development.

[sjakobi]

Ah, I had believed that some users wouldn't want to use a GPL-licensed dhall-to-yaml, but if that's not so, all the better.

So the idea is to keep the aeson-yaml-based dhall-to-yaml around for a bit to smoothen the transition, and then remove it?

[patrickmn]

Raising hand as at least one user that would still need to be able to generate YAML, both via library and binary, without GPL-licensed components.

I like the idea of splitting the YAML stuff into its own package centered around HsYAML, and keeping the dhall-json binary around. Since I'm maintaining aeson-yaml and that library takes over after the conversion to an Aeson value, hopefully there is very little maintenance burden?

What about dhall-to-yaml-plus? :)

[DrSensor]

I agree with making HsYAML implementation in a separate package dhall-yaml. However, I still want the binary name to be the same to avoid confusion. So package dhall-json will have both dhall-to-{json,yaml} and when a user install dhall-yaml, existing dhall-to-yaml and yaml-to-dhall will be overwritten.

• dhall-json is tools that converts Dhall to JSON and it's derivative (YAML, etc) or vice versa. Which mean the derivative must be compatible with JSON convention (e.g key must be string)
• dhall-yaml is tools that converts Dhall to YAML or vice versa.

[Gabriel439]

@patrickmn: Alright, in that case we can support the old dhall-to-yaml, too.

I still think -ng is the most common suffix used in the wild when there is a new executable that overlaps substantially with an old one. You can find several examples here:

https://askubuntu.com/questions/46596/what-does-the-ng-suffix-mean-in-package-names

@DrSensor: I don't think it's wise to have two Haskell packages providing the same executable name (especially when they are closely related packages). I've been burned by this before

There are several specific issues with doing this:

• All documentation that mentions dhall-to-yaml needs to "fully qualify" which package the executable came from to avoid user confusion (and old documentation will break since it won't mention which one)

• Users who install the dhall-yaml package will unintentionally override dhall-to-yaml when they also install dhall-json. They are highly likely to make this mistake since most people who work with YAML also work with JSON, too. Then they will be surprised why dhall-to-yaml starts behaving subtly differently

• Users will need to resolve executable conflicts in provisioning tools when both packages need to be installed. For example, if I install both pkgs.dhall-json and pkgs.dhall-yaml on NixOS it will flag the conflict in installed executable names and ask me to specify which one I meant as a warning, and it will randomly select one or the other by default (and that's actually the best case!). Other provisioning tools are likely to fail silently or in weird ways

[sjakobi]

I've started to work on this.