Dhall-haskell: Converting (Expr Src X) to (Map key value)

[Michael-Kateregga]

Does anyone know how I can convert type (Expr Src X) to (Map key value) in haskell. I tried to follow #505 but the toMap suggested by @Gabriel439 requires a [KeyValue a] instead.

[f-f]

@Michael-Kateregga we do something like that in spacchetti and spacchetti-cli: basically the idea is that you match on the Expr Src X to be a RecordLit, and then to convert to a Map k v you check that all the keys are of type k and all the values of type v

[Michael-Kateregga]

@f-f Thanks for sharing. Which version of ghc are you using. I notice the Pragma DerivingStrategies is not supported in ghc ver 8.0.2?

[f-f]

@Michael-Kateregga we're on GHC 8.4. DerivingStrategies is needed only for this line, so it doesn't affect the behaviour of the example (it's basically there just for nicer debugging)

[Michael-Kateregga]

@f-f Oh I see but I will have to upgrade my ghc regardless. By the way I also get an error at line 128 i.e.

• Couldn't match type ‘containers-0.5.7.1:Data.Map.Base.Map
                         Text Package’
                 with ‘Map Text Package’
  NB: ‘Map’
        is defined in ‘Data.Map.Internal’ in package ‘containers-0.6.0.1’
      ‘containers-0.5.7.1:Data.Map.Base.Map’
        is defined in ‘Data.Map.Base’ in package ‘containers-0.5.7.1’
  Expected type: Dhall.Map.Map Text Package -> Map Text Package
    Actual type: Dhall.Map.Map Text Package
                 -> containers-0.5.7.1:Data.Map.Base.Map Text Package

[jneira]

That seems a pretty usual conversion, could be added to Dhall main module as a Dhall.Type Map?

[Gabriel439]

Yeah, it seems like an Interpret instance for Map would work. In other words:

instance Interpret value => Interpret (Map Text value) where
    ...

I'd definitely support adding that. @Michael-Kateregga: Does that address your use case?

[Michael-Kateregga]

@jneira @Gabriel439 I will try adding the instance and see.

[Gabriel439]

@Michael-Kateregga: So, looking at this more closely, it doesn't seem like there is a way to convert a RecordLit to a Haskell Map via Interpret. Interpret requires you to specify the expected type which would change as you're adding or removing record fields.

I think probably the best we can do in the foreseeable future is to complete support for toMap so that you can still:

• define a record in Dhall
• convert that Dhall record to a Dhall association list using toMap
• convert that Dhall association list to a Haskell association list using Interpret
• convert the Haskell association list to a Haskell Map using Data.Map.fromList or something similar

[jneira]

And what about map :: [String] -> Type a -> Type (Map String a)?
Caller should choose the restricted set of keys.

[f-f]

Or how about changing the expected :: Expr Src X to expected :: Expr Src X -> Expr Src X in the definition of Type? Would that make sense at all?
It would allow the Haskell bindings to be "dependent", and make possible to implement the Interpret (Map Text a) instance

[sellout]

@f-f Can’t you already do that via Interpret a => Interpret (Map String a)?

[sellout]

Oh, read earlier on the thread 😅

[jneira]

@f_f i had thought in that alternative, it would be nice to find other uses cases though. Anyway i would try to implement the version with explicit keys to let users being more strict

[f-f]

@jneira here's an implementation of the map you proposed above (note that it doesn't even need the Type a in the signature, as an Interpret constraint suffices - though we need the list of keys, since we don't get the expression in expected):

{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RankNTypes #-}
{-# LANGUAGE ScopedTypeVariables #-}

module Test where

import           Data.Map                              (Map)
import qualified Data.Map                              as Map
import           Data.Text                             (Text)
import qualified Data.Text                             as Text
import qualified Dhall
import qualified Dhall.Core                            as Dhall.Core
import qualified Dhall.Map

map :: forall a. Dhall.Interpret a => [Text] -> Dhall.Type (Map Text a)
map keys = Dhall.Type extractOut expectedOut
  where
    extractOut (Dhall.Core.RecordLit fields) = Dhall.Map.toMap
      <$> Dhall.Map.traverseWithKey toValue fields
    extractOut _ = Nothing

    expectedOut = Dhall.Core.Record
      $ Dhall.Map.fromList
      $ fmap (\k -> (k, Dhall.expected valueType)) keys

    valueType :: Dhall.Type a
    valueType = Dhall.auto

    toValue _label value = do
      let annot = Dhall.Core.Annot value $ Dhall.expected valueType
      let _typ = Dhall.TypeCheck.typeOf annot
      Dhall.extract Dhall.auto $ Dhall.Core.normalize $ annot

This should work (typechecks but I didn't test it), but as you said one has to know the keys in advance, which kind of takes away the whole point of using Map (I mean, if I know the keys I might as well just make a record type)

A great use case for this is what we have in spacchetti-cli, where we get from the user a Record with arbitrary keys, and we want a Map from that.
We currently extract it manually, but it's not fun since it's nested (contained in a parent record), so we have to manually handle also the conversion of the parent record (instead of getting the Interpret instance for free for it), so this approach doesn't scale well (more code to write as our configuration type grows); if we could derive the instance instead, we'd save all that code

[Gabriel439]

Maybe what we can do is change the expected field to Either Text (Expr Src X) where Left is a generic error message if decoding fails. i.e.: "The decoded value was not a record"

[jneira]

Jumm, @Gabriel439 do you mean a modification over @f-f proposal expected :: Expr Src X -> Either Text (Expr Src X)?
Mixing all a little bit i've sketched:

data Type a = Type
    { extract  :: Expr Src X -> Maybe a
    , expected :: Expr Src X -> Either Text (Expr Src X)
    } deriving (Functor)

map :: Type a -> Type (Data.Map.Map Text a)
map (Type extractIn expectedIn) = Type extractOut expectedOut
  where extractOut (Dhall.Core.RecordLit fields) =
          Dhall.Map.toMap <$> traverse extractIn fields
        extractOut _ = Nothing

        expectedOut (Dhall.Core.RecordLit fields) =
          Dhall.Core.Record <$> traverse expectedIn fields
        expectedOut _ = Left "The decoded value was not a record"

instance Interpret a => Interpret (Data.Map.Map Text a) where
     autoWith opts = map (autoWith opts)

But the signature of the two functions Type are closely related, right? Not sure it it asks for a redesign

[Gabriel439]

@Michael-Kateregga: If we were to fix https://github.com/dhall-lang/dhall-haskell/issues/673 then would you still need this issue to be fixed?

[Michael-Kateregga]

@Michael-Kateregga: If we were to fix #673 then would you still need this issue to be fixed?

@Gabriel439 Currently makeHaskellType is vital to have but I'm sure at a later stage I or someone else will need Expr Src X -> Map k v

[f-f]

Yeah, I just noticed that this Expr Src X -> Map k v already popped up in #505 (which is almost a duplicate of this)

[sjakobi]

Does the Interpret (Map k v) instance or the map-Type added in https://github.com/dhall-lang/dhall-haskell/commit/d172e4b677ba7141eade7468cd7911dace9c024f fix this issue?

You can convert any toMap-expression now.

[Gabriel439]

@sjakobi: I think it would, by virtue of using extract (auto @(Map k v))

[sjakobi]

Cool. I'll close this then. Feel free to reopen if there's anything left to address.