Scala-js: Remove unused fields

[japgolly]

This is my understanding of how fullOptJS's optimiser works in regard to vals.

• lazy vals that are never referenced, are removed.
• vals that are never referenced, are kept because the instantiation function could have side-effects.

In scalajs-react, I recently changed a large number of vals to lazy vals (https://github.com/japgolly/scalajs-react/issues/248) and shaved 19k off the final output size of my gh-pages' JS. Great! But there are a few small problems with this:

1. Each value now needs an additional value to indicate whether the field has been initialised. (Right?) This is a (minor) waste of memory.
2. Each value now uses performs an additional check each time it's referenced. I don't know how well JS engines optimise this but I think it's safe to say this is a (minor) waste of cpu.
3. Each value creates more JS than required due to the lazy machinery. So in cases that fields aren't eliminated as dead-code, the JS size increases.

I think a good improvement here would be to provide the optimiser with hints. My suggestion -- and I imagine it would be quite easy to implement too -- is to add an annotation like @pure that can be used to annotate vals without effects that can be safely eliminated if unreferenced. So for example:

// This can be removed by the optimiser if unreferenced.
@pure
val blah = Widget("Blah")

// This cannot.
val blah = {
  println("Initialising...")
  Widget("Blah")
}

That way the JS for the vals stays small, and doesn't need any additional mem or cpu.

WDYT?

[sjrd]

Unfortunately, your analysis is slightly off, and your suggestion is relatedly not at all easy to implement. (In fact, it would be easier to implement something where the optimizer figures it out on its own!)

The long story begins ...

First, fullOpt has nothing to do with it. fastOpt does the same optimization. In fact even "noOpt" (with scalaJSOptimizerOptions ~= { _.withDisableOptimizer(true) } does it! It is part of the normal behavior of the linker and its reachability analysis.

The problem

Now, the reachability analysis does not know what "fields" are. All it cares about are classes, interfaces and methods. So why are lazy vals removed? Well, we have to look at what scalac gives us. Given:

class A {
  val x = 5
  lazy val y = 10
}

with the option -Xprint:mixin given to scalac, we can see what scalac gives to the Scala.js compiler:

class A extends Object {
  @volatile private[this] var bitmap$0: Boolean = false;
  private def y$lzycompute(): Int = {
    {
      A.this.synchronized({
        if (A.this.bitmap$0.unary_!())
          {
            A.this.y = 10;
            A.this.bitmap$0 = true;
            ()
          };
        scala.runtime.BoxedUnit.UNIT
      });
      ()
    };
    A.this.y
  };
  private[this] val x: Int = _;
  <stable> <accessor> def x(): Int = A.this.x;
  lazy private[this] var y: Int = _;
  <stable> <accessor> lazy def y(): Int = if (A.this.bitmap$0.unary_!())
    A.this.y$lzycompute()
  else
    A.this.y;
  def <init>(): helloworld.A = {
    A.super.<init>();
    A.this.x = 5;
    ()
  }
};

There are a few very interesting things to note.

1. Both x and y receive an _accessor_ method def x() and def y()
2. def x() simply returns x
3. However, def y() do some complicated magic to initialize y if it hasn't been yet. The initialization code of y *is in the method y$lzycompute()!
4. The initialization code of x _is in the constructor_

Now, consider what the reachability sees, and it becomes clear why it can get rid of the initialization of y. Since def y() is never called, y$lzycompute() is never called either, and the initialization code of y disappears.

For x, that does not work. def x() is never called, so it is eliminated. But the initialization code of x is still in the constructor, and the constructor must be kept! It doesn't matter that the field x will never be used; the analysis keeps the constructor as a whole, and always keeps all fields.

Then, the optimizer comes, and optimizes the constructor. It sees this.x = 5 and simply has to keep it that way, because fields are always kept.

Note that, if you look at fastOptJS' output, the _field_ y is kept too! But it is never initialized, contrary to x, because its initialization code was moved to the y$lzycompute() method and subsequently discarded.

What could be improved?

The blocker is, in a sense, not where we expect it. The main problem is that the optimizer, seeing this.x = 5, must always keep the instruction (even though 5 is pure), because the field is kept no matter what.

What we could improve would be to enhance the reachability analysis to be aware of fields. And, in particular, to keep track of who _reads_ a field (and possibly also who writes a field, but that's less important here; as long as it's _separate_). If a field is never read, that does _not_ mean it can be removed (because someone could write to it). But that means _writes_ to the field can be removed. Therefore, the optimizer would have the opportunity to rewrite this.x = 5 into just 5. It would follow that it could remove 5 altogether, in this case, because it knows that it does not have side-effects.

When can we have that?

In a major version. This change would require to put _more_ information in the so-called "Info" section of .sjsir files. And that cannot be done without breaking backward binary compatibility.

In theory, we could do it in a minor version. We could reconstruct the missing Info from older .sjsir files at deserialization time, since infos can constructed from Trees. But that would cause a major performance penalty while reading .sjsir files compiled by earlier versions of Scala.js.

[japgolly]

Heh, my suppositions didn't match up at all :stuck_out_tongue_closed_eyes:

Your proposition sounds awesome. The only feedback I have is at this part:

writes to the field can be removed. Therefore, the optimizer would have the opportunity to rewrite this.x = 5 into just 5. It would follow that it could remove 5 altogether, in this case, because it knows that it does not have side-effects.

How will the optimiser determine whether a tree has side-effects? Primitive literals like 5 are easy but what about cases like these:

case class A(i: Int)

// Will the optimiser know that this is elidable?
A(5);

and

class A(i: Int)
implicit class IntExt(private val i: Int) extends AnyVal {
  def toA = new A(i)
}

// Will the optimiser know that this is elidable?
5.toA;

I don't think Scala or Scala.JS have a purity analysis. We might still need some kind of @pure-like annotation or other means of informing the optimiser, no? I'm imagining something like this:

@pure class A(i: Int)
implicit class IntExt(private val i: Int) extends AnyVal {
  def toA = new A(i)
}

5.toA             // Start here
new IntExt(5).toA // Optimiser expands
IntExt$.toA(5)    // Optimiser expands
new A(5)          // Optimiser inlines
()                // Optimiser discovers @pure and elides

I don't know the implementation feasibility of any of this stuff so I'm just throwing around ideas. Ideally it would also scan the constructor of such a class to see if anything happens, then it could determine purity without any annotation.

_(On a related note, I like that scalac provides warnings about discarded pure values on some built in types, but I wish I could annotation types that I create so that they issue warnings too when they are discarded. People tend to have problems with one of the types (Callback) in scalajs-react that a discard-warning would solve - I was thinking a @pure annotation would be a good SLIP to raise. If it's needed/useful for this usecase as well, I'd be more motivated to create such a SLIP. WDYT?)_

[sjrd]

I don't think Scala or Scala.JS have a purity analysis.

Scala.js has a limited side-effect-free analysis in its optimizer. Currently it is mostly useful to get rid of useless LoadModule instructions (i.e., when you inline a method of an object, you often can get rid of loading the singleton instance; but only if its constructor is side-effect-free).

We can generalize this constructor analysis to classes in general (trivial), and then we can omit new calls to side-effect-free class constructors (then, transitively eliminating actual arguments if they are side-effect-free).

So your example above would work without any annotation, because A is "trivially" side-effect-free. From new A(5), it would therefore simplify to 5 then to ().

If, however, the class constructor is more complex and cannot be proven trivially side-effect-free, the new call would be kept. This would be true if it does any kind of JS interop, since JS stuff is opaque to the optimizer.

I am not really open to adding optimization "hints" that could in fact cause the optimizer to break code. What if the developer put @pure on a method that is not, in fact, pure. Then, depending on how "smart" the optimizer was, the side effects could be applied or not. At the moment, I think this is better left alone. It would require extensive study.

[sjrd]

Now that Infos are computed at link-time (#3078), this does not need to be in by 1.0.0.