Aurelia: [RFC] Synchronous binding and batching changes without the TaskQueue

[fkleuver]

💬 RFC

Provide a way to batch changes with synchronously bound properties.

🔦 Context

vCurrent - property bindings

In vCurrent we have the TaskQueue to batch changes, guard against infinite two-way updating recursion and the likes.

It's a clean and simple solution but it has limitations that surface especially in complex scenarios. It doesn't differentiate between different kinds of operations whereas some operations need to be processed before others (even if they were queued later).

It also adds a certain amount of mandatory boilerplate to classes:

export class MyViewModel {
  public changeCount = 0;

  @bindable()
  public someProp

  somePropChanged() {
    this.changeCount++;
  }

  someMethod() {
    this.someProp = 'a new value';
    // this.changeCount === 0 (change is not propagated yet)
    this.taskQueue.queueMicroTask(() => {
      // this.changeCount === 1
    });
  }

vNext - property bindings (synchronous)

In vNext we got rid of the TaskQueue and I replaced it with a Lifecycle class. (Don't worry, TaskQueue will be available in a compat package and integrated with the Lifecycle for backwards compatibility.)

Property bindings are completely synchronous by default; only DOM updates are still batched asynchronously. It simplifies a lot of things both for us and for end users. It also improves performance and reduces memory pressure due to much fewer function scopes being created. Taking the above example again:

export class MyViewModel {
  public changeCount = 0;

  @bindable()
  public someProp

  somePropChanged() {
    this.changeCount++;
  }

  attached() {
    this.someProp = 'a new value';
    // this.changeCount === 1 (somePropChanged is invoked immediately)
  }

vNext - collection observation

Sometimes you want binding to be asynchronous for performance reasons as well. The repeater listens to collection changes in an async manner (via promises):

<div repeat.for="item of items">${item}</div>

export class MyViewModel {
  public items = [];

  attached() {
    this.items.push(2);
    this.items.push(1);
    this.items.sort();
    // DOM is still empty, updates are not propagated yet
    Promise.resolve().then(() => {
      // after waiting one promise tick, the changes are now propagated to the repeater
      // all at once, instead of 3 times one by one - this is the vCurrent equivalent of `queueMicroTask`
    })
  }

The collection observer actually has two ways to subscribe to it:
synchronous

collectionObserver.subscribe({
  handleChange(method, ...args) {
    console.log(method, ...args);
  }
})
// collection === [1, 2]
collection.push(3);
// log: 'push', 3
collection.push(4);
// log: 'push', 4
collection.reverse();
// log: 'reverse'

batched
indexMap is an array that contains the old indices, and the current indices map to the current array item's positions. This allows us to efficiently reconstruct the changes in the repeater.

(for vCurrent there will be a backwards compat packages that translates the indexMap to the old splices)

collectionObserver.subscribeBatched({
  handleBatchedChange(indexMap) {
    console.log(indexMap);
  }
})
// collection === [1, 2] (indexMap === [0, 1])
collection.push(3); // indexMap === [0, 1, -2] (newly added items are marked by -2)
collection.push(4); // indexMap === [0, 1, -2, -2] 
collection.reverse(); // indexMap === [-2, -2, 1, 0]
await Promise.resolve();
// log: [-2, -2, 1, 0]

Now these collection observer APIs are still largely designed with internal purposes in mind (for efficiency) and we still need to come up with something that's nice to end users. That will come with vCurrent's BindingEngine equivalent for vNext.

vNext - property bindings (async / batched)

I needed to lay this context however to get to the question I have right now: what is a nice way for end users to force batching (sync or async) or normal bound properties? Imagine this:

@bindable({ handler: 'update' }) height
@bindable({ handler: 'update' }) width

update() {
  this.applyDimensions();
}

changeDimensions(newHeight, newWidth) {
  this.height = newHeight; // update is invoked immediately
  this.width = newWidth; // update is invoked again immediately
}

Now update (and whatever it does/calls) is invoked twice, due to synchronous binding. But you want to perform updates on multiple properties and only have the change handler invoked once, after all changes are done. Then what?

On to the examples, I have several potential solutions in mind.

💻 Examples

1. batchedHandler - for asynchronous batching:

@bindable({ batchedHandler: 'update' }) height
@bindable({ batchedHandler: 'update' }) width

update() {
  this.applyDimensions();
}

changeDimensions(newHeight, newWidth) {
  this.height = newHeight; // change is queued, update is not yet invoked
  this.width = newWidth; // change is queued, update is not yet invoked
  Promise.resolve().then(() => {
    // one promise tick later, update() has now been called, and only once instead of twice
  })
}

2. $batch.start() + $batch.end() - for synchronous batching

@bindable({ handler: 'update' }) height
@bindable({ handler: 'update' }) width

update() {
  this.applyDimensions();
}

changeDimensions(newHeight, newWidth) {
  this.$batch.start();
  this.height = newHeight; // change is queued, update is not yet invoked
  this.width = newWidth; // change is queued, update is not yet invoked
  this.$batch.end();
  // after $batch.end(), changes are synchronously propagated and the update() handler is invoked immediately
}

3. @batch decorator - for synchronously batching everything in a method

@bindable({ handler: 'update' }) height
@bindable({ handler: 'update' }) width

update() {
  this.applyDimensions();
}

@batch // under the hood, the method is wrapped in $batch.start() + $batch.end()
changeDimensions(newHeight, newWidth) {
  this.height = newHeight; // change is queued, update is not yet invoked
  this.width = newWidth; // change is queued, update is not yet invoked
  // changes are not yet propagated
}

someOtherMethod() {
  this.changeDimensions(100, 200);
  // after the method exits, changes are synchronously propagated and the update() handler is invoked immediately
}

Technically you could also apply this decorator on a class and have all methods batched.

4. Expose the observers for bypassing change handlers

Currently you can do this (knowing the internals, as these are not exposed via typings):

@bindable({ handler: 'update' }) height
@bindable({ handler: 'update' }) width

update() {
  this.applyDimensions();
}

changeDimensions(newHeight, newWidth) {
  this.$observers.height.currentValue = newHeight; // change the observer's internal state directly,
  // bypassing the change detection
  this.width = newWidth; // handler is invoked immediately, and the update() method will get the
  // latest updated value for height and width
}

There are various cases where you might want to do this, such as in unit tests to initialize a value without invoking change handlers. We could make this into an official API for updating bound values without invoking change handlers. The existing way of doing that might not be the best from an API perspective, but the idea stands.

---

Option 1 is the more declarative approach (and more consistent with vCurrent's way of doing things)
Option 2 is the more imperative approach where you have control over precisely when to propagate changes
Option 3 is another declarative approach that keeps things synchronous
Option 4 approaches the problem in a completely different way, giving you low-level control of deciding when to trigger change handlers

What does everyone think about these 4 flavors? Does it make sense, does it seem natural, or do I need to go back to the drawing board? :)

[EisenbergEffect]

I kind of like having $batch.start() and $batch.end() for fine grained control and then the @batch decorator built on top of that for convenience in the most common scenario.

[3cp]

Add note: @fkleuver and I talked about adding @debounce (and maybe @throttle) to aurelia vNext kernel to help controlling the timing of asynchronous batching.

[akircher]

I have trouble wrapping my head around all of this so feel free to point out any errors:

Do we really need a synchronous batching syntax? Could the above just be solved with something like:

//I believe this would batch the changes 
changeDimensions(newHeight, newWidth) {
   this.dimensions = {height:newHeight, width:newWidth}
}

I think the above would be simpler to explain to people than framework specific apis.

---

Also just bike shedding the syntax but I would rather have

@bindable({ handler: 'update' , batched:true})

than

@bindable({ batchedHandler: 'update' })

And would rather have

collectionObserver.subscribe({
  handleChange(method, ...args) {
    console.log(method, ...args);
  },
  handleBatchedChange(indexMap) {
    console.log(indexMap);
  }
})

than

collectionObserver.subscribeBatched({
  handleBatchedChange(indexMap) {
    console.log(indexMap);
  }
})
collectionObserver.subscribe({
  handleChange(method, ...args) {
    console.log(method, ...args);
  }
})

[akircher]

Also I can see the differences but this discussion does remind me of Angular 1's unpopular $apply() method

[fkleuver]

Thanks for your replies @akircher it's really helpful to know how end users see this. You have some good points.

Do we really need a synchronous batching syntax?

Most people would probably never need to know about it. However, in those situations where you want the squeeze the best possible performance out of many components and keep things synchronous for one reason or the other, this would offer a simple API to do that instead of having to hack your way around it.

It is inspired by MobX which does computed observation with a somewhat similar underlying batching mechanism (increase the depth on start, decrease it on end, and only carry out the operation when the depth is back to 0).

Let me emphasize that unlike angular's $apply method this is only one of the several ways in which you can take control over how changes are propagated. If you forget about these API's, everything still works just like it does in vCurrent from a superficial usage perspective.

@bindable({ handler: 'update' , batched:true})

That makes sense, I think I like that better too.

> collectionObserver.subscribe({
  handleChange(method, ...args) {
    console.log(method, ...args);
  },
  handleBatchedChange(indexMap) {
    console.log(indexMap);
  }
})

We can do that too :)

[dannyBies]

Imagine the following html:

<div if.bind="isEnabled" class="ui dropdown">
</div>

And the following ts:

export class MyViewModel {
  public isEnabled: boolean= false;

  attached() {
    this.isEnabled = true;
    $('.ui.dropdown').dropdown();
  }
}

Property bindings are completely synchronous by default; only DOM updates are still batched asynchronously.

Am I correct in saying that it is possible that the div is not rendered before .dropdown() is called on it?
If so, would I need to Promise.resolve() and call .dropdown() inside?

[fkleuver]

@dannyBies Short answer: no, you do not need to do that.
I left out some important details on how the lifecycle works, in order to keep this RFC to the point.

The batching of DOM updates is only noticeable when DOM mutations are performed outside of the normal lifecycle (e.g. in response to user interaction after everything already loaded).

attached really means attached: everything is guaranteed to be done and updated at that point, including the DOM.
Here's a summary of the process:

• First, dependencies are registered and the templates are compiled. Now Aurelia knows exactly what to render
• Then, rendering happens (it is called $hydrate). This populates components with bindings, child components, sets up bindables, etc.
• After rendering is done, the $bind lifecycle starts. This sets all initial values and initializes the observers to start listening for changes.
• When all binding is done, the lifecycle flushes any pending batched changes that may have happened in any component's binding hooks, inside change handlers, etc. This is recursive and might trigger more bind lifecycles if new components are added (e.g. new array items to a repeater) and keeps going until everything is clean and no more pending changes. So specifically during app startup, all async changes are actually forced to be synchronous.
• Before attach starts, all bound hooks are invoked (during that hook it's guaranteed that a component is fully bound and updated). Again, this is recursive and might trigger more flushes and binds.
• With all changes propagated, the $attach lifecycle starts recursing through all components and collects any pending mount operations (the act of actually adding nodes to the DOM). When all mount operations are collected, they are executed in the order they were queued and all DOM updates happen in one go.
• Only after all that is done: everything is bound, all changes propagated, all DOM nodes added, and there is nothing left to do, the attached callbacks are invoked.

I should emphasize I've spent a ridiculous amount of time working on that and throwing thousands of unit+integration tests at it. We wanted this to be 100% deterministic, reliable and predictable. There should never be any need for queueMicroTask-like stuff just to make sure all changes propagated :)

However, if you respond to some user input and update a property that is bound to the DOM, then you need to await a promise tick for that to be propagated (or you can call $lifecycle.processFlushQueue() but that's a thing for later and an API that's still in progress)

[dannyBies]

This sounds great, thanks for your detailed explanation!

[fkleuver]

@dannyBies It seems my brain had a very creative way of processing your code sample and did not register the fact that you had that if.bind initially set to false, only to set it to true during the attached callback.

So my entire comment still holds true apart for the one detail of your specific example: you do actually need to await a promise tick there. Because, as clarified in my comment, attached is when everything is considered done. Whatever happens after that (in terms of updating the DOM) gets batched again.

However, let me clarify with a few examples when it is or isn't needed.

Given your html:

<div if.bind="isEnabled" class="ui dropdown"></div>

created

export class MyViewModel {
  public isEnabled: boolean= false;
  created() {
    // right after hydration is done and before binding starts
    this.isEnabled = true;
  }
  attached() {
    // no need to wait, element is mounted
    $('.ui.dropdown').dropdown();
  }
}

binding

export class MyViewModel {
  public isEnabled: boolean= false;
  binding() {
    // right after own bindings are bound, but children are not bound yet
    this.isEnabled = true;
  }
  attached() {
    // no need to wait, element is mounted
    $('.ui.dropdown').dropdown();
  }
}

bound

export class MyViewModel {
  public isEnabled: boolean= false;
  bound() {
    // right after all children are bound, and bound() callbacks have been invoked on all parents
    this.isEnabled = true;
  }
  attached() {
    // no need to wait, element is mounted
    $('.ui.dropdown').dropdown();
  }
}

attaching

export class MyViewModel {
  public isEnabled: boolean= false;
  attaching() {
    // everything is bound now and we're about to start gathering mount operations;
    // last chance to mutate state and keep it synchronous, as well as last chance to register
    // async tasks such as enter-animations
    this.isEnabled = true;
  }
  attached() {
    // no need to wait, element is mounted
    $('.ui.dropdown').dropdown();
  }
}

attached + promise #1

export class MyViewModel {
  public isEnabled: boolean= false;
  attached() {
    // all bindings and changes have already been set in stone for the current
    // mount operations, so this one will have to wait for the next turn
    this.isEnabled = true;
    Promise.resolve().then(() => {
      $('.ui.dropdown').dropdown();
    });
  }
}

attached + promise #2

export class MyViewModel {
  public isEnabled: boolean= false;
  async attached() {
    // same as #1 but a different flavor
    this.isEnabled = true;
    await Promise.resolve();
    $('.ui.dropdown').dropdown();
  }
}

attached + force flush

export class MyViewModel {
  public isEnabled: boolean= false;
  attached() {
    this.isEnabled = true;
    // we're now actually forcing the if's bind+mount to happen in between
    // so it is synchronous this way
    this.$lifecycle.processFlushQueue();
    $('.ui.dropdown').dropdown();
  }
}

Note the extra hooks that vCurrent doesn't have. These hooks are specifically there for the purpose of giving you a "right time" to be doing various things. The "right time" to change a bound value would certainly not be attached, if that would mutate state that the next synchronous operation depended on. So that's what binding() (which is comparable to vCurrent's bind() in terms of timing) and bound() are generally for, but you can still take attaching() as well.

[dannyBies]

@fkleuver
That makes sense.
I used changing a property in attached as a simple example. The more realistic scenario would be custom component A with lots of child components in which the child component would affect component A's properties.

If I understand you correctly as long as the child component does not have any logic in attached all the changes will be synchronous and be processed once component A's attached is invoked?

[fkleuver]

@dannyBies Correct.

A few rules of thumb apply (will be documented more extensively of course):

• If you need to make changes to the current component based on its currently bound values (only its own properties or those of its parent), use binding (similar to vCurrent's bind hook)
• If you need to make changes to the current component, its parent or its children based on bound values from any of those (that is, you simply need the whole component tree to be bound and up-to-date), use the bound hook (there is no such hook in vCurrent with this timing, but attached would be close to it, difference being that no DOM stuff has happened yet during bound)
• If you need stuff to happen before mounting (like queueing an animation), attaching is when you do that (again, no such hook in vCurrent). Or, if you need to do stuff after all bound hooks have been called, you can also do it here (but you'd want to try to minimize that due to the temporal coupling that will start trickling down)
• If you need everything to be done, e.g. initialize some 3rd party component on a ready element, attached is your friend

If you follow those rules of thumb, there will never be a need to explicitly flush or await ticks during the lifecycles.
When you mutate stuff outside of the lifecycles (e.g. in response to a user click you set a value of an if.bind to true), the same idea applies as when you do this during attached() as explained in previous comment.
The components themselves are still updated synchronously (so the actual if TemplateController will have the value of true), but the DOM updates happen next tick.