Dhall-lang: Synchronization between specification versions and tests

[jneira]

Opening the issue to not lose the discussion in #324
There @Gabriel439 proposed keep the hashes updated to changes in master version, including in it the incoming new version of the standard (currently it would be 5.0.0)

My context is the setup of a test suite for dhall-eta, a eta wrapper over dhall-haskell to implement a java friendly api over it. As it uses the 1.19.1 dhall-haskell released version and current dhall-lang master has breaking changes i am using the tag v4.0.0 to run the tests against.

Some parsing tests are failing cause dhall-haskell is not fully compliant. I understand the reasons and hope that some dhall-haskell version will fail correctly (he!)

Unfortunately some of the import tests are failing cause hashes are not in sync (i think):

Importing fieldOrderA:                                      FAIL
        Exception: user error (The import failed but it had to success. Dhall exception: 
        Error: Import integrity check failed
        Expected hash:
        ↳ 72791c3846cef2ec49baabe6a5d38ca25301ed30b45754dfa1c6b06bab8faaf6
        Actual hash:
        ↳ 4a7866b88389e18cf481b525544fd7903325252faf3a86c8fdc981298c788a9b
        )

Importing issue553B:                                        FAIL
        Exception: user error (The import failed but it had to success. Dhall exception: 
        Error: Import integrity check failed
        Expected hash:
        ↳ ef4cce5b6c440b2409f9ba86e48fb788b7ccb757569a713492654f23209cb19b
        Actual hash:
        ↳ 6fb582c043889dd5a4c97176f8a58d2633252b5374cb71e288b93bc3757f9643
        )

• Checking the tests files it seems the hashes are for version 3.0.0 of the spec instead 4.0.0:

D:\ws\eta\dhall\dhall-lang\tests\import\success>git status
HEAD detached at v4.0.0
nothing to commit, working tree clean

D:\ws\eta\dhall\dhall-lang\tests\import\success>dhall-1.19.1 < fieldOrderA.dhall

↳ ./../data/fieldOrder/1.dhall sha256:72791c3846cef2ec49baabe6a5d38ca25301ed30
b45754dfa1c6b06bab8faaf6
This decoded version is not supported: 3.0.0

D:\ws\eta\dhall\dhall-lang\tests\import\success>dhall-1.18.0 < fieldOrderA.dhall

{ example0 = { bar = True, foo = 1 }, example1 = { bar = True, foo = 1 } }

D:\ws\eta\dhall\dhall-lang\tests\import\success>dhall-1.19.1 < issue553B.dhall

 [1;31mError [0m: Import integrity check failed
Expected hash:
↳ ef4cce5b6c440b2409f9ba86e48fb788b7ccb757569a713492654f23209cb19b
Actual hash:
↳ 6fb582c043889dd5a4c97176f8a58d2633252b5374cb71e288b93bc3757f9643

D:\ws\eta\dhall\dhall-lang\tests\import\success>dhall-1.18.0 < issue553B.dhall
{ A = λ(A : {}) → < A = A | B : {} >, B = λ(B : {}) → < B = B | A : {} > }
 > }

Would be possible/desirable (forcing) update the current v4.0.0 with the correct hashes? Would be the correct ones that are generated by the released dhall-1.19.1?

[f-f]

@jneira I'm not sure how you're pulling in dhall-lang, but would it work to keep somewhere a branch that points to the 4.0.0 tag + a cherry-pick of https://github.com/dhall-lang/dhall-lang/commit/7c4f273af9d004de5ff89c30e0a1c59dfafecff6 (which is the commit with the fixes for the hashes)?

[jneira]

Yeah, i've just do it force pushing tag v4.0.0 in my fork and checking out it instead dhall-lang.
Not sure if keeping the tag here with the tests not synchronized could impact other users though. :thinking:

[jneira]

Oh i've just realized that the released file already contains the actual tests and maybe should be updated too. I know force pushing tags/releases is not a good praxis so maybe could be created another tag v4.0.1 with the hashes fixed?

[Gabriel439]

How about we cut another release for 5.0.0 after fixing the hashes? It's been almost 30 days since the last release anyway

[jneira]

I think the tests are already in sync for last version 5.0.0 so closing this