Scala-native: Thoughts on Scala Native Next Steps, May 4, 2020

[LeeTibbert]

@errikos @sjrd
Congratulations on publishing the scala next steps roadmap. It is obvious that a lot of thought and hard work went into it.

I am writing my thoughts down as a SN Issue because I think they need more permanence
that a gitter discussion allows. I also think that this venue is more appropriate than
Scala Contributors.

I have been trained to speak openly and truly in council and then support the eventual
decision. I am told that the late Buddhist monk Tich Naht Hanh advised: "Show up, listen well,
speak your truth, and let go of consequence".

There is obviously more work to be done than 1.n (where n asymptotically approaches 0) can
do in reasonable time. I suggest that a "critical path' look at required skill sets. To my
thinking, the two most critical skills are 1) providing multi-year strategy and 2) interest & skill
in compiler internals. There are a number of members of the SN community who can provide
other required skills, but they depend on those two to make progress.

1) To my thinking, by far and away most important task is to drain the PR queue, probably
oldest-to-newest. The merge is going to be a bear in the best case. If it is done in j-random order, it is going a total mess. The code should have honorable work-person quality but need not be
Cathedral perfect. A "two approving reviews by someone other than author" approach would
reduce the burden on the person doing the final merge.

There are currently 67 outstanding PRs. Some of those (15 or so) are Work In Progress (WIP).
Some of rest of them need current review but should be non-controversial.  That leaves
the hard cases.  Some of those may be obsolete by now: too much change.  I have
a couple which will take some review (or faith) but which will save developer time,
especially for Scala 2.12 support.

A month of focused community effort, with an appointed merge/triage person from
Scala Center would get us through this.  I am willing to be part of that effort, if 
coronavirus allows.

2) Once the PR queue is drained, publishing a 0.4.0-M3 containing just those changes
would show the world that the Scala Native project is indeed alive and deserving of
attention and support. In three weeks, it will be over a year since 0.4.0-M2.

While calendar time alone should not be a determining factor, a year in these days
of rolling six week releases is an eternity.  It is bad enough if new or existing 
users of Scala Native encounter defects, it is worse for them to be told "Oh, those
were fixed in a year old PR, you just wasted days". Hard to reverse a bad first
impression.

3) My thoughts about the Scala Native testing environment and its defects are well known,
much as I try to keep my grumbles in check.

The best that can be said about the current testing environment is that where it works, it does
its job. Outside of that it can most charitably be characterized as somewhere between 
fragile & actively hostile. The test environment has been known to suck up person-months of
effort.  Having changes to it block or delay the next release may not be a good thing to do.

a) Changing it to use the new scala-reflect results in no user visible benefit.  It is probably
    easier for me to volunteer to make this change that to debate its benefit/wisdom.  That work
    would postpone a __scala_.== fix which is visible to ZIO but does not require 
    compiler-internal skill.

 b)  I __strongly__ suggest that support for the current test ensemble (all the Suites) be 
     continued and Junit(sp?) support and tests be developed along side it. In the network world
     this is called "dual stack" (IPv4 & IPv6 both on the same machine).  There is a relatively
    large body of existing tests; stopping the world while 1.n people convert them to Junit
    is going to be a long pause.  Yes, I know the cost of two code bases: Technical Debt.

   Keeping the existing tests (yes, for all the imprecations I hurl at them) for a few release
  cycles would allow the next release to get out in economic time and give confidence
  that the next scala 2.12 release did not break things.  If any Legacy tests fail that 
  would trigger a re-write to Junit.  

  Once Junit support is available, that would put a marker/barier in the pipeline after
  which all new tests need be Junit (or scripted, I see no way around that need).

 For example, I am about to do a PR for `__scala_==`. That PR absolutely needs 
 unit-tests.  I am reluctant to write the, to my mind, essential, tests if I know/suspect
 that they will be rejected because they are not Junit.

[sjrd]

Hello @LeeTibbert. Thanks for writing down your thoughts. I'll try to answer as well as I can.

I also think that this venue is more appropriate than Scala Contributors.

Not sure what was wrong with Scala Contributors, but since you posted here, I will reply here. However please understand that eventually this thread in the GitHub issue tracker will be closed, because it's not an actionable issue. This will have nothing to do with the value of the content.

To my thinking, by far and away most important task is to drain the PR queue, probably oldest-to-newest.

I agree that we need to make an effort in shrinking the PR queue. We should do that as soon as possible, but at the same time not sooner. And in particular, to be able to comfortably and accurately review PRs, we first need to get comfortable with the codebase ourselves, and that's why I only reviewed "trivial" and new PRs so far. As my own knowledge progresses, I'll be able to cover more involved PRs.

Personally I am more inclined to do newest-to-oldest, as newer PRs have a better chance of still merging cleanly or rebasing with minimal effort.

Once the PR queue is drained, publishing a 0.4.0-M3 containing just those changes would show the world that the Scala Native project is indeed alive and deserving of attention and support. In three weeks, it will be over a year since 0.4.0-M2.

This is going to be a long time before a new M release. I think we should release a 0.4.0 (non milestone) before completing the PR queue. After that, I expect we'll release much more often. In Scala.js we release roughly every 2 months.

The best that can be said about the current testing environment is that where it works, it does its job. Outside of that it can most charitably be characterized as somewhere between fragile & actively hostile. The test environment has been known to suck up person-months of effort. Having changes to it block or delay the next release may not be a good thing to do.

At this point, now that reflection is in, adapting the current testing environment is mostly a matter of removing things rather than adding, which we should be able to do very quickly. It will allow us to get rid of the NativeTest configuration, which will make everything stabler and easier to grasp.

Personally, I have been used to a codebase where testing is the single-most important thing. I would not be comfortable leading a project with subpar testing environment.

Changing it to use the new scala-reflect results in no user visible benefit.

It will. First, the NativeTest configuration won't be needed anymore; this will make it easier for new users to get up to speed with Scala Native, as the experience in sbt will be the same as for other platforms. Second, this will allow testing frameworks to catch exceptions thrown during instantiation of test classes, again allowing the support to be on par with other platforms.

I strongly suggest that support for the current test ensemble (all the Suites) be continued and Junit(sp?) support and tests be developed along side it. In the network world this is called "dual stack" (IPv4 & IPv6 both on the same machine). There is a relatively large body of existing tests; stopping the world while 1.n people convert them to Junit is going to be a long pause. Yes, I know the cost of two code bases: Technical Debt.

I have no intention of stopping the world to port all the existing tests to JUnit. No need to worry about that.

As soon as JUnit support is in, new tests should be written in JUnit, but before that, I will not refuse to look at a PR just because it adds tests in the current testing framework.

It's also worth noting that there is a large body of existing tests in Scala.js that we can simply copy-paste as soon as we have JUnit support in Native. This will be a time-saver, not a drag. Some of the Scala Native tests today are already ported from Scala.js, but with manual work involved in adapting the testing framework. I would like to avoid further waste of time in that area.

---

On a more general note, even though I lack experience with Scala Native and its codebase, I have been in charge of Scala.js for 7 years. I have insight into what, in the history of Scala.js, was instrumental for its success. Having a testing infrastructure that allows testing frameworks to effortlessly cross-compile and exhibit the same semantics was instrumental. It was virtually the only user-visible change in Scala.js 0.6.0 wrt. 0.5.x, and 0.6.0 was a turning point in the rate of adoption of Scala.js.

[ekrich]

One thing I would like to point out is that changing the tests to support JUnit should be as easy as a couple of search and replaces. I recently did the opposite from JUnit to MUnit in no time at all. This might be better than piece meal changes but probably needs to happen after the PRs are merged for a 0.4.0 release. I do agree with the fresher PR approach. The JUnit conversion could happen right after the release.

Also, once we have JUnit then downstream projects can take advantage of that to have testing across JVM/Scala.js/Scala Native. This is huge and a good reason to release a 0.4.0.

I do think we should address some of the deprecations so I think this PR and/or friends would be a great idea. https://github.com/scala-native/scala-native/pull/1752 This could also be done after the 0.4.0 release too to avoid any potential regressions.

It would be nice to have better compiler flags for unused imports etc. to help build in some quality. I am not sure about something as strict as sbt-tpolecat but that does handle cross platform. I don't have any direct experience with that so I cannot recommend either way for this type of codebase.

[LeeTibbert]

@sjrd Thank you for your timely, supported, & detailed response. I feel that my concerns
have been heard and will be factored in.

[LeeTibbert]

Language, especially English, is slippery. By "factored in" I mean considered, I do not presume to mean adopted as-is.