Dotty: Community build

[felixmulder]

Todo List:

• [x] Infrastructure for community build : @olafurpg
• [x] ScalaCheck : @allanrenucci (stuck on this assertion error in typer.Implicits)
• [x] ScalaTest : @olafurpg
• [x] Squants : @olafurpg
• [x] Algebra : @olafurpg
• [ ] [json4s](https://github.com/json4s/json4s) : @liufengyun (fixed: #2218 #2250 #2239)
• [ ] [twitter/util](https://index.scala-lang.org/twitter/util): @liufengyun
• [x] [scopt/scopt](https://index.scala-lang.org/scopt/scopt): @allanrenucci
• [x] Better-Files : @allanrenucci
• [ ] [paiges](https://github.com/typelevel/paiges)
• [ ] [scalariform](https://github.com/scala-ide/scalariform)
• [x] [scala-parser-combinators](https://github.com/scala/scala-parser-combinators)
• [x] [scala-xml](https://github.com/scala/scala-xml)
• [ ] [jardiff](https://github.com/scala/scala-java8-compat)
• [ ] [sbinary](https://github.com/sbt/sbinary/)
• [ ] [log4s](https://github.com/Log4s/log4s) (note: scala-reflect macros can be replaced with inline def https://github.com/Log4s/log4s/blob/1155339023744cb6d2f9809cfff8ea5b57c59868/src/main/scala/org/log4s/Logger.scala#L88-L101)
• [ ] [http4s/blaze](https://github.com/http4s/blaze) (note: depends on log4s)

Repo: https://github.com/lampepfl/dotty-community-build

[olafurpg]

@liufengyun

• Which repo will the community build be hosted?
• Are there any planned candidate projects for the community build?

I'm happy to help out with this issue.

[felixmulder]

I can reiterate what was said at the meeting:

• Which repo will the community build be hosted?

lampepfl/dotty-community-build

• Are there any planned candidate projects for the community build?

We should include the ones already compiling (and working):

• Better files
• Scalacheck
• Scalatest (only tried compilation)

[DarkDimius]

Which repo will the community build be hosted?

lampepfl/dotty-community-build

That's not how I undestood what was discussed in the meeting.
The way I understood it:

• forks of projects in community build are maintained in organisation "dotty-staging"
• git submodules that refer to this projects are stored in Dotty repo itself.
• (wasn't discussed, but I infered it and propose to do it this way) we have a jUnit test that builds every of those projects, which is only run by the nightly CI.

[felixmulder]

Keeping it on staging is fine IMO, but the other point I don't agree with.

I propose the structure to be something like:

.
├── dotty (submodule @ latest master)
├── better-files (submodule)
└── scalatest (submodule)

Then you do publishLocal on dotty and use it to compile the other submodules.

Making a fork of dotty which contains submodules sounds messy.

[olafurpg]

@felixmulder Where is the latest work on compiling scalatest+scalacheck+better-files with dotty?

@DarkDimius Can you invite me and @liufengyun to the dotty-staging organization with permission to create new repositories?

[OlivierBlanvillain]

@olafurpg scalatest is here: https://github.com/OlivierBlanvillain/scalatest/commits/dotty

[olafurpg]

I made some progress today on the infrastructure side, see here https://github.com/lampepfl/dotty-community-build/pull/1

This sets up a build where @OlivierBlanvillain's fork of scalatest is compiled against the latest NIGHTLY release of Dotty. We can then add more projects to the build by adding new submodules.

[olafurpg]

Scalatest + infrastructure is up! @felixmulder Can you assign me please to this issue so I can check out those boxes.

I've started working on scalacheck now https://github.com/dotty-staging/scalacheck/pull/1

[felixmulder]

I cannot - @DarkDimius or @Densh has to add you to the dotty team, then you can self assign.

[olafurpg]

It seems better-files on master branch does not compile with 2.11 since November 2011. It uses uses BufferedIterator.headOption. I'm gonna work put better-files on hold until this gets fixed, better-files is incubating for inclusion in the Scala platform so it will need to support 2.11 according to @jvican.

I'm stuck on an assertion error in typer.Implicits for compiling scalacheck. See stack trace, could someone with more Dotty experience take a look at that? (ps. I asked same question on gitter)

I'm gonna work on squants next.

[DarkDimius]

I cannot - @DarkDimius or @Densh has to add you to the dotty team, then you can self assign.

Done.

[olafurpg]

The community-build is taking shape now. Here are a few notes that might be helpful on how to add a new project, say project ORG/REPO, to the current infrastructure:

• fork ORG/REPO to dotty-staging/REPO
• clone dotty-staging/REPO, create branch "dotty", push dotty branch

• Open dotty-staging/REPO on Github: Settings > Branches > change main branch to "dotty"
  

• go to https://travis-ci.org/profile/dotty-staging and enable that repo, you may need to click on "Sync account" and reload to see the newly forked repo

• fork dotty-staging/REPO to your personal github account
• edit the build to use sbt-dotty and fix all compiler errors, example PRs:
  
  
  
  • https://github.com/dotty-staging/scalacheck/pull/1
  
  
  • https://github.com/dotty-staging/squants/pull/1
  
  
  • https://github.com/dotty-staging/algebra/pull/1
  
  
• Once the project complies OK with dotty, copy the .travis.yml from here: https://github.com/dotty-staging/squants/blob/37ea54761b5e80ddc0dfc273fc4bbad430f201f5/.travis.yml#L3 Adjust the compile task.
  
  
  
  • I'm only compiling projects for now, not running any tests. Just making sure that the projects type-check is a great start.
  
  
• Open PR to merge your changes into the "dotty" branch of dotty-staging/REPO
• Once PR is merged into dotty-staging/REPO, open a PR to lampepfl/dotty-community-build adding the new dotty-staging/REPO to the community build. Example PR: https://github.com/lampepfl/dotty-community-build/pull/3

A nice benefit with this setup is that build failures in lampepfl/dotty-community-build are easy to debug. For example, if only one repo fails the travis build page looks like this

Build logs for each community project are separated so it's easy to track down a failure in a single project: https://travis-ci.org/lampepfl/dotty-community-build/jobs/220623982#L1705

I'm sure this setup can be simplified/automated/improve on many ways, so any feedback is welcome.

[olafurpg]

BTW. If someone's interested, it might be worth investigating if https://github.com/storm-enroute/mecha can be used for this setup.

[felixmulder]

@olafurpg - since we're now on 2.12.3 we should add better-files to the community build! :tada:

[allanrenucci]

@olafurpg Is ScalaCheck still stuck on an assertion error with the latest nightly?

[allanrenucci]

https://github.com/pathikrit/better-files is blocked by #3168 and #3171

[allanrenucci]

ScalaCheck is blocked by two implicit resolution failures: https://travis-ci.org/dotty-staging/scalacheck/builds/280025029#L1596

I was not able to minimise the failures though

[smarter]

For the second one, see https://github.com/smarter/scalacheck/commits/fix/ambiguous-implicits

[allanrenucci]

scopt/scopt is blocked by #3189

[Blaisorblade]

Idea/question: many projects are hard to test because they use macros, so could we go to their Scala 2.9 version? We'd run into other incompatibilities (view bounds), but I suspect they're much more superficial.

[smarter]

I don't think porting Scala 2.9 code is going to be very useful. Ideally we want to compile libraries with their dependencies, so we need recent version of these dependencies. What we need is to figure out how to replace macros in projects on case by case basis, even if this means adding special cases in the compiler to ease transition (e.g. support for kind-projector syntax)

[smarter]

We also want to have our changes merged into the upstream library when reasonable, fixing an old version of the library wouldn't help for that.