Dhall-haskell: Decoding a dhall Type Expression in haskell

[Michael-Kateregga]

I know decoding a dhall literal file in haskell we use input with the expected output type specified. What about a dhall type file?

I tried using inputExpr but I am not sure how I can convert the resulting (Expr Src X) into haskell's data structure.

[sellout]

You can use input the same way on types as well (all Dhall files contain expressions), but you will need a Haskell type that represents Dhall types. So, inputExpr can do that (although Expr represents both types _and_ terms). What is the Haskell type you’re trying to convert to?

[Michael-Kateregga]

@sellout I used RecordType in haskell to create the corresponding type as so:

Dhall

let person = {name : Text, address : Text, age : Natural} in person

Haskell

data Person = Person {name :: String, address :: String, age :: Natural} deriving (Interpret, Generic, Show)

person :: Type Person
person = 
   record
       ( Person <$> field "name" string
                      <*> field "address" string
                      <*> field "age" natural
       )

Trying to decode the dhall in haskell

Since the dhall file is just a type file and not a literal, I don't think we can decode it like below:

main = do 
  x <- input genericAuto "./myDhall_file"  :: IO (Type Person)
  ...

[sellout]

So, to use input to generate a Type Person, you need a Type (Type Person), no?

typePerson :: Dhall.Type (Dhall.Type Person)
person = Dhall.Type
  { extract = \_ -> Just person, -- but this doesn’t use the read expression at all
  , expected = Dhall.Core.Const Dhall.Core.Type
  }

I’m still not sure exactly what you’re trying to accomplish with this, but if what you want is to be able to read Dhall types to use in the expected field of your Dhall.Type values, that’s what I created staticDhallExpression for.

And apologies if I’m just totally missing your goal.

[Gabriel439]

@Michael-Kateregga: Just to clarify: are you trying to auto-generate the Haskell type definition from the Dhall type definition?

Or in other words, are you trying to replace this line of Haskell code:

data Person = Person {name :: String, address :: String, age :: Natural} deriving (Interpret, Generic, Show)

... with some Template Haskell that reads in the Dhall type description at compile time and generates the Person datatype definition from that file? Like this:

makeHaskellType "./personType.dhall"

[Michael-Kateregga]

@sellout thanks @Gabriel439 that's exactly what I need, you got it!

[Michael-Kateregga]

@sellout I have two questions on your comment:

• How does staticDhallExpression :: Text -> Q Exp relate to Person. In other words how can I Q Exp -> Person in haskell?

• Dhall's (Type a) works with literals (i.e. a) . In my case I am trying to decode a record type in haskell not a record literal. My understanding is that Person is a type of a literal in

person :: Dhall.Type (Dhall.Type Person)
person = Dhall.Type
{ extract = _ -> Just person, -- but this doesn’t use the read expression at all
, expected = Dhall.Core.Const Dhall.Core.Type
}

That's why I was working with RecordType as opposed to data in haskell. Hope this makes sense.

By the way my actual problem involves decoding a nested record type (e.g. {a : {x : Text, y : Text, z : Text }, b : {x : Text, y : Text, z : Text }, c : {x : Text, y : Text, z : Text }} and I will have to refer to each field type as need be.

@Gabriel439 Why do we have to define types in both dhall and haskell via data and RecordType. Won't it be cool if we had makeHaskellType "./personType.dhall" that gives you the corresponding type in haskell?

[Gabriel439]

@Michael-Kateregga: We can totally have a makeHaskellType function. It just takes a bit of time to author such a function

[Michael-Kateregga]

@Michael-Kateregga: We can totally have a makeHaskellType function. It just takes a bit of time to author such a function

Oh okay I see, I will try and work with what @sellout recommended i.e. staticDhallExpression but I am not sure how one can extract individual field types from such a structure.

[Michael-Kateregga]

@sellout Could please provide a use case of staticDhallExpression in haskell.

[sellout]

@Michael-Kateregga I linked to an example in the docs earlier in this thread – https://hackage.haskell.org/package/dhall-1.18.0/docs/Dhall-TH.html

Let me know if that isn’t clear enough, so I can improve the docs.

Also, I’m not opposed to something like makeHaskellType existing, but binding your Haskell ASTs to serialized data format is something to be wary of. My own preference is to _statically_ generate that kind of code once for the initial version, then maintain it manually – that way you are free to make independent changes on either side, only modifying the conversion. But at least dynamically generating the AST from the Dhall is better than the other way around. Dynamically generating Dhall from the AST can result in surprising your users with breakage.

staticDhallExpression is intended to get the compiler to force you to keep your conversions up-to-date.

[Michael-Kateregga]

@sellout I tried to follow the example on hackage but I really don't get it. I would want to understand how I would for instance attribute a haskell data structure whose type corresponds to r1 from a dhall type expression: {r1 = {a : Text, b : Natural}, r2 = {a : Double, b : Natural}} using staticDhallExpression.

[sellout]

Ok, I wrote a bunch of stuff, then I re-read this thread and deleted everything I wrote, because I don’t think I understand what you are trying to accomplish.

So, let’s say you used inputExpr as you mentioned above, now, in Haskell, you have a Dhall.Core.Expr Src X that represents a Dhall type that looks like { name : Text, address : Text, age : Natural }. What do you want to do with that Expr?

It _looks_ like maybe you want to use it to generate a Dhall.Type Person for you. This is what makeHaskellType _would_ do for you. Also note that by using deriving Interpret, you already have a value of type Dhall.Type Person available.

Finally, my own approach is to do

person :: Dhall.Type Person
person = Dhall.Type
  (\case
      D.RecordLit fields ->
        Person
        <$> (extract text =<< Map.lookup "name" fields)
        <$> (extract text =<< Map.lookup "address" fields)
        <$> (extract natural =<< Map.lookup "age" fields)
      _ -> Nothing)
  $(staticDhallExpression "./path/to/Person.dhall")

which requires you to manually maintain the conversion between the Haskell and Dhall types (a feature, IMO), but ensures the expected Dhall type matches the type you’ve actually defined in Dhall.

That example code is the same as in the docs, except that I converted a record type instead of a union type – so I have a feeling it’s not any more helpful to you. So I feel like I must be misunderstanding your goal.

[Michael-Kateregga]

@sellout this is clear now thank you. My concern was why I had to define a data type in haskell before I could import the dhall file but with staticDhallExpression there is no need. I will use this until I have a look at makeHaskellType

[Gabriel439]

@sellout: Actually, what I'm proposing is that makeHaskellType would be a Template Haskell function that takes a datatype name and Dhall source code for a Dhall type as input and would generate a Haskell data type definition for Person. In other words, this Haskell code:

{-# LANGUAGE TemplateHaskell #-}

Dhall.TH.makeHaskellType "Person" "{ foo :: Bool, bar :: Text }"

... and would generate this code:

data Person = Person { foo :: Bool, bar :: Text }
    deriving (Generic, Inject, Interpret)
````

The way I interpret this request is to use Dhall in a manner similar to protobuf or thrift.  You use Dhall as an IDL to specify various "message" types and generate bindings in several languages to that IDL so that they can communicate with one another using data structures idiomatic to each respective language.

Normally you generate the the bindings in each language using a command-line-tool that is a code generator (i.e. `protoc`, for protobuf, for example), but in Haskell you have the luxury of doing so within the language using Template Haskell.

For example, the protobuf tutorial uses these types for its running example:

```proto
// search.proto

syntax = "proto3";

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
}

message SearchResponse {
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}

The analogous Dhall file would be:

-- ./search.dhall

    let SearchRequest =
          { query : Text, page_number : Natural, result_per_page : Natural }

in  let Result = { url : Text, title : Text, snippets : List Text }

in  let SearchResponse = List Result

in  { SearchRequest =
        SearchRequest
    , Result =
        Result
    , SearchResponse =
        SearchResponse
    }

... and what I think @Michael-Kateregga wants to do is to be able to write:

makeHaskellType "SearchRequest" "(./search.dhall).SearchRequest"

makeHaskellType "Result" ("./search.dhall).Result"

makeHaskellType "SearchResponse" "(./search.dhall).SearchResponse"

... and that will generate the matching Haskell types and their Interpret/Inject instances.

More generally, what I think @Michael-Kateregga wants is for Dhall to be the source of truth for the what the types are rather than the Haskell code being the source of truth. The most common reason for this is that in a polyglot environment you don't want any one language to be the source of truth for a type that is shared between services in multiple languages.

@Michael-Kateregga: Correct me if my explanation is wrong and I misunderstood the request

[sellout]

@Gabriel439 I agree that having Dhall as the source of truth is better than having Haskell as the source of truth. staticDhallExpression is intended to do just that – treat the Dhall type as the source of truth.

However, I don’t believe that the Haskell types or conversions between the Dhall and Haskell types should be dynamically generated. As new features are added or versions are released, one or the other side of the types change, or you need to support multiple variants of the Dhall type or whatever, and so the types you want to use in your code are no longer perfectly described by the Dhall types. Keeping the mapping between the two explicit makes it possible to manage this. (And using staticDhallExpression gets the compiler to nudge you when you don’t manage it.)

We use protobuffers in Haskell at work, and the first thing we do upon receiving a protobuf message is convert the generated Haskell types to domain Haskell types. Granted, the problems are bigger with protos than with Dhall. E.g., since the types are so much weaker – everything in a proto is Maybe – validating required fields at the system boundary is important. Using generated proto types directly is a disaster before you even make a modification. In Dhall, it feels fine until you make that first change to your Haskell types (or make a change to the Dhall that can be handled without a change to the Haskell types), and then you start replacing makeHaskellType with hand-written Dhall.Type values.

However, I _believe_ you can use TH to simply statically generate the file, which is how I‘d recommend using makeHaskellType – avoid hand-writing the initial conversions – just generate them automatically, but then maintain them manually as they change.

[Gabriel439]

@sellout: Yeah, I agree that you sometimes don't want to directly use the generated type and you want to convert into your own type. What I meant is that it's easier to perform such a conversion if the starting type is already a strongly-typed Haskell record instead of a weakly-typed Expr abstract syntax tree.

[Michael-Kateregga]

@sellout: Actually, what I'm proposing is that makeHaskellType would be a Template Haskell function that takes a datatype name and Dhall source code for a Dhall type as input and would generate a Haskell data type definition for Person. In other words, this Haskell code:

{-# LANGUAGE TemplateHaskell #-}

Dhall.TH.makeHaskellType "Person" "{ foo :: Bool, bar :: Text }"

... and would generate this code:

data Person = Person { foo :: Bool, bar :: Text }
    deriving (Generic, Inject, Interpret)

The way I interpret this request is to use Dhall in a manner similar to protobuf or thrift. You use Dhall as an IDL to specify various "message" types and generate bindings in several languages to that IDL so that they can communicate with one another using data structures idiomatic to each respective language.

Normally you generate the the bindings in each language using a command-line-tool that is a code generator (i.e. protoc, for protobuf, for example), but in Haskell you have the luxury of doing so within the language using Template Haskell.

For example, the protobuf tutorial uses these types for its running example:

// search.proto

syntax = "proto3";

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
}

message SearchResponse {
  repeated Result results = 1;
}

message Result {
  string url = 1;
  string title = 2;
  repeated string snippets = 3;
}

The analogous Dhall file would be:

-- ./search.dhall

    let SearchRequest =
          { query : Text, page_number : Natural, result_per_page : Natural }

in  let Result = { url : Text, title : Text, snippets : List Text }

in  let SearchResponse = List Result

in  { SearchRequest =
        SearchRequest
    , Result =
        Result
    , SearchResponse =
        SearchResponse
    }

... and what I think @Michael-Kateregga wants to do is to be able to write:

makeHaskellType "SearchRequest" "(./search.dhall).SearchRequest"

makeHaskellType "Result" ("./search.dhall).Result"

makeHaskellType "SearchResponse" "(./search.dhall).SearchResponse"

... and that will generate the matching Haskell types and their Interpret/Inject instances.

More generally, what I think @Michael-Kateregga wants is for Dhall to be the source of truth for the what the types are rather than the Haskell code being the source of truth. The most common reason for this is that in a polyglot environment you don't want any one language to be the source of truth for a type that is shared between services in multiple languages.

@Michael-Kateregga: Correct me if my explanation is wrong and I misunderstood the request

@Gabriel439 This is exactly what I have in mind.

[Michael-Kateregga]

Ok, I wrote a bunch of stuff, then I re-read this thread and deleted everything I wrote, because I don’t think I understand what you are trying to accomplish.

So, let’s say you used inputExpr as you mentioned above, now, in Haskell, you have a Dhall.Core.Expr Src X that represents a Dhall type that looks like { name : Text, address : Text, age : Natural }. What do you want to do with that Expr?

It _looks_ like maybe you want to use it to generate a Dhall.Type Person for you. This is what makeHaskellType _would_ do for you. Also note that by using deriving Interpret, you already have a value of type Dhall.Type Person available.

Finally, my own approach is to do

person :: Dhall.Type Person
person = Dhall.Type
  (\case
      D.RecordLit fields ->
        Person
        <$> (extract text =<< Map.lookup "name" fields)
        <$> (extract text =<< Map.lookup "address" fields)
        <$> (extract natural =<< Map.lookup "age" fields)
      _ -> Nothing)
  $(staticDhallExpression "./path/to/Person.dhall")

which requires you to manually maintain the conversion between the Haskell and Dhall types (a feature, IMO), but ensures the expected Dhall type matches the type you’ve actually defined in Dhall.

That example code is the same as in the docs, except that I converted a record type instead of a union type – so I have a feeling it’s not any more helpful to you. So I feel like I must be misunderstanding your goal.

Are you not missing something in the code because I get a parse error at "\case" i.e.

error: parse error on input ‘case’
|
261 | (\case
| ^^^^

[Michael-Kateregga]

@sellout I have also noticed that staticDhallExpression is of type Q Exp but your pattern match expects Expr Src X. Am I missing something?

[sellout]

@Michael-Kateregga Ok, there are a couple issues – here’s a hopefully fixed one:

person :: Dhall.Type Person
person = Dhall.Type
  (\exp -> case exp of
      D.RecordLit fields ->
        Person
        <$> (extract text =<< Map.lookup "name" fields)
        <*> (extract text =<< Map.lookup "address" fields)
        <*> (extract natural =<< Map.lookup "age" fields)
      _ -> Nothing)
  $(staticDhallExpression "./path/to/Person.dhall")

• \case is enabled by the LambdaCase pragma, otherwise you need to do \x -> case x of
• I copy-pasted <$> and didn’t replace it with <*>
• Q Exp is correct – Exp is the type of a _Haskell_ AST, and Q is the uh quotation(?) monad. They’re used in Template Haskell. the $() expects a value of that type and replaces it with the generated haskell expression, which _does_ have type Expr Src X.

[Michael-Kateregga]

person :: Dhall.Type Person person = Dhall.Type (exp -> case exp of D.RecordLit fields -> Person <$> (extract text =<< Map.lookup "name" fields) <> (extract text =<< Map.lookup "address" fields) <> (extract natural =<< Map.lookup "age" fields) _ -> Nothing) $(staticDhallExpression "./path/to/Person.dhall")

The bug is now fixed. Thanks @sellout this approach requires Person to be defined before via data Person = Person {name :: String, address :: String, age :: Natural} deriving (Generic, Interpret, Show). I can work with this for now as I wait to test the foreseen makeHaskellType

[Michael-Kateregga]

@sellout sorry for my nagging but I have one more question: for a dhall record literal we can extract a field using record.field is there a way we can extract a field type say Natural for age from Type Person analogously?

[Michael-Kateregga]

@sellout sorry for my nagging but I have one more question: for a dhall record literal we can extract a field using record.field is there a way we can extract a field type say Natural for age from Type Person analogously?

@sellout I just realised this could do the job: (<$) :: a -> Type b -> Type a

[sjakobi]

https://github.com/dhall-lang/dhall-haskell/pull/1620 might close this?!

[sjakobi]

I'll optimistically close this issue under the assumption that the makeHaskellTypeFromUnion TH function added in #1620 was why this issue was still open.

Please re-open if there's more to do here! :)