Cats: Long term roadmap - a 2.12 binary compat Cats 2.0?

[kailuowang]

1.x will be maintained for at least a year and will be fully binary compat as promised.

How about we have a 2.0 that maintains binary compat with 1.x but this compatibility is only guaranteed on Scala 2.12.X. This would give us significantly more leeway in providing addition to the API. Right now the vast majority of bin breaking changes are simply new methods with default implementation to traits which is bin compat on Scala 2.12. Thanks to this leeway, the life cycle of 2.X could be potentially longer. And at least on Scala 2.12, it's a continuation of 1.X.

Then 3.0 will aim at Scala 2.13 with the possibly new encodings.

[johnynek]

I think this sounds like a good plan. That said, I don’t think we should gratuitously break APIs. Long lived APIs are valuable and reduce costs for our users. I hope we can maximize the cross section of source and binary compatibility for a long time.

[kailuowang]

I 100% agree we shouldn't gratuitously break APIs. The main incentive for this 2.0 proposal is to extend the binary compatibility beyond 1.x.

Another consideration is that Scala 2.11 is near EOL. In the 2.11.12 release note from Nov 9th

We strongly encourage you to upgrade to the latest stable version of Scala 2.12.x, as the 2.11.x series is no longer actively maintained.

I'm not sure how long we should be held back by 2.11 and 2.10. If our user community is moving towards 2.12, I think we should consider proceed with this 1.x bin compatible Cats 2.0 (and also possibly stop supporting 2.10 altogether.)

[johnynek]

Can’t we use the sonatype download statistics to get an idea how many
2.10/2.11 users we have?

Also, since spark has still not done a 2.12 release a lot of folks are
still on 2.11.

On Thu, Dec 28, 2017 at 18:32 Kai(luo) Wang [e-mail]
wrote:

I 100% agree we shouldn't gratuitously break APIs. The main incentive for
this 2.0 proposal is to extend the binary compatibility beyond 1.x.

Another consideration is that Scala 2.11 is near EOL. In the 2.11.12
release note https://github.com/scala/scala/releases/tag/v2.11.12 from
Nov 9th

We strongly encourage you to upgrade to the latest stable version of Scala
2.12.x, as the 2.11.x series is no longer actively maintained.

I'm not sure how long we should be held back by 2.11 and 2.10. If our user
community is moving towards 2.12, I think we should consider proceed with
this 1.x bin compatible Cats 2.0 (and also possibly stop supporting 2.10
altogether.)

—
You are receiving this because you commented.

Reply to this email directly, view it on GitHub
https://github.com/typelevel/cats/issues/2130#issuecomment-354396651,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEJdgetPQD4kYAPlo11xIo1tqEYy_7dks5tFGttgaJpZM4RO1HK
.

>

P. Oscar Boykin, Ph.D. | http://twitter.com/posco | http://pobox.com/~boykin

[kailuowang]

@johnynek sorry I didn't mean to suggest a 2.0 in the immediate future. Just want to start thinking about the long-term so that we can make short-term decisions accordingly and communicate to our ecosystem well in advance.
For the timeline of 2.0 I was really thinking Q3-Q4 2018, roughly one year after Scala 2.11 active maintenance stopped. We can gauge download statistics then.

[kailuowang]

New update to the idea:
For cats 2.0 we might consider support scala 2.12 and 2.13 only.
The incentive is that a user either uses the 2.12 only back compat cats 2.0, or if they are on 2.11 they can only use the back compat cats 1.x. Either way the back compat is guaranteed, users won’t run into diamond dependency hell.

[alexandru]

I don't have an opinion about the timeline, just a general desire for taking backwards compatibility seriously, more seriously than what we've been doing in the Scala ecosystem until now.

At the same time we can't guarantee binary compatibility for Scala < 2.12 because of traits, fixed partially in Scala 2.12 due to Java 8. That's simply a temporary limitation of the runtime for older Java versions that have been EOL-ed anyway. Keeping binary compatibility for Scala <= 2.11 is unreasonable.

[ghost]

re scala 2.10

The last release of Scala.js, 1.0.0-M2 was the last release that they will be doing...

...that will support:

• sbt 0.13
• building on JDK 6 and JDK 7
• Scala 2.10

So I would guess that the release of scala.js 1.0 and cats usage of it would provide a reasonable time to drop 2.10 for cats, too. I know it's not _necessary_ for cats to be in absolute sync with sjs, just an easy and reasonable strategy.

Then again, we may have another problem of... when do we drop scala.js 0.6.x in favour of 1.0.0

[ghost]

taking backwards compatibility seriously

scalfix with macro-compat style compatibility is ideal for this. cats can use the latest, and the rewrites fix the new code to work with the old scala/libs.

Once we add scala.js 1.0/0.6 as mentioned ^^, things get trickier as the release matrix also has to take into account that as well. ie scala 2.11/2.12 and scala.js 0.6/1.0 would require 4 released libs. AFAIK, sbt cannot do this type of cross building out of the box. But _if_ it did, then it would be relatively straight forward for us to set this up.

So my question here would be, assuming this functionality is not currently available, is it something we would want?

[LukaJCB]

Another challenge will be 2.13 and the new collections, not sure exactly how much will break for us, but it will very probably cause problems with cross builds, no?

[ghost]

From their blog :

We expect the majority of collection usage to be compatible between 2.12 and 2.13 and the majority of the remaining incompatibilities to be automatically fixable with ScalaFix.

I might guess that they currently plan a 2.12->2.13 one off rewrite, rather than a compile time 2.13->2.12 rewrite as a mentioned ^^. But this would hopefully be covered, and is "easier" in the sense that it's not another 3rd party lib to add to the release matrix

[ghost]

I guess I'm trying to sound out interest in doing more work for catz-cradle that covers these type of issues

[kailuowang]

Looks like that if we release against scalajs 1.0 we will have to drop Scala 2.10. So for this release we have to decide:

1. whether to bump the major version (but maintain bin comp with 1.x on scala 2.11 and 2.12)
2. if we don't bump, we are pretty much saying that we won't continue maintain a version with 2.10 and scalajs 0.6 since we won't have a version available for that branch except the bug fix version number.
3. if we bump, for how long we should continue maintain 1.x with scalajs 0.6, 2.0 (unless we can find a way to cross build with different versions, we will have to maintain two branches)

[mdedetrich]

@BennyHill

Once we add scala.js 1.0/0.6 as mentioned ^^, things get trickier as the release matrix also has to take into account that as well. ie scala 2.11/2.12 and scala.js 0.6/1.0 would require 4 released libs. AFAIK, sbt cannot do this type of cross building out of the box. But if it did, then it would be relatively straight forward for us to set this up.

This isn't going to be possible any time soon due to how SBT is designed, I have experienced this issue in working with scalajson (which is also designed to be extremely strict in terms of binary compatibility), see https://github.com/portable-scala/sbt-crossproject/issues/47 for more info (its basically a fundamental limitation in SBT) **

** @sjrd Did do scalajs-dyna lib plugin for SBT which does support cross compiling against scala.js versions, however this plugin only supports pure cross-building, i.e. you can't separate a build for JVM and a build for JS (it also doesn't work with scala-native)

[ghost]

@mdedetrich Thanks for the feedback, and yes, I agree that sbt won't support this anytime soon :worried:

But based on other work I've done here and there, it _might_ be possible to cobble something together . So I guess earlier, I was just trying to sound folk out, before spending too much time :wink:

[martin-g]

@BennyHill @mdedetrich Mill seems to support this already: https://github.com/lihaoyi/mill/blob/46c516893a641a283f11a46f5c45a5cf1b4af373/scalajslib/test/src/mill/scalajslib/HelloJSWorldTests.scala#L28

[mdedetrich]

@martin-g Yup, its just a question of whether its worth the effort to convert to mill (and whether or not it has all of the plugins that cats uses). Unfortunately for my project I can't use mill because its part of scala-platform.

I also suspect that a migration to mill would be non trivial, it has a different way of organizing builds compared to sbt

[alexandru]

@mdedetrich Thanks for the feedback, and yes, I agree that sbt won't support this anytime soon 😟

But based on other work I've done here and there, it might be possible to cobble something together . So I guess earlier, I was just trying to sound folk out, before spending too much time 😉

@BennyHill just one thought — if Cats supports both Scala.js versions, then that's going to be the expectation for the other Typelevel projects.

So if you cobble something together, maybe you can write an article, a gist or something to let the other project maintainers know what to do.

[sjrd]

The recommended way to cross-compile for Scala.js 0.6.x and 1.x is explained at the bottom of the 1.x milestone release notes. sbt-dynscalajs should be used only if the recommended setup does not suit you.

You could also use Mill, as mentioned, but I would wait until more projects start their development with Mill and kick more of its tires, before spending the effort of migrating an existing non-trivial project to it 😉

[DavidGregory084]

I can highly endorse Mill, but it's worth bearing in mind that cats makes quite extensive use of SBT plugins, for which replacements might not yet exist in Mill

[ghost]

@mdedetrich, @martin-g, @alexandru, @sjrd @DavidGregory084 - Many thanks for the comments!!

So for now, looks like it is worth moving _some_ of the discussion here to https://github.com/typelevel/cats/issues/2195 that, I think, pre-dates these later comments.

[dwijnand]

1. if we don't bump, we are pretty much saying that we won't continue maintain a version with 2.10 and scalajs 0.6 since we won't have a version available for that branch except the bug fix version number.

I think this one makes the most sense.

Stick to the 1.x version strategy, with the next 1.2.0 release only for Scala 2.12+ and Scala.js 1.x. The 1.1.x version series of Cats would be the last one to support Scala 2.10/2.11 and Scala.js 0.6, and only (serious) bug fixes be back-ported and released.

Dropping support doesn't mean the library in its current state is no longer available, it just means that future developments won't be available until you upgrade.

[kailuowang]

@dwijnand to be clear, I was not talking about dropping Scala 2.11 in 1.x, only 2.10 and scalajs 0.6.
With so many users stuck with 2.11 due to Spark and other reasons, I don't know if we want to cross 2.11 out that quick.
cats-core_2.10 is about 1% of the downloads now while cats-core_2.11 is more than 50%

[dwijnand]

I see, thanks for clarifying!

Then I say make 1.1.x is the last version series for 2.10. Keep releasing cats for 2.11 and 2.12 in 1.2+.

Then when it makes sense and/or you get tired of being held back by Scala 2.11, then the version series around that time, say 1.3.x, becomes the last series for Scala 2.11, and 1.4+ becomes Scala 2.12+.

I think it makes things simple, linear, and easier to maintain (in terms of less long living parallel branches - not in terms of trait encoding).