Apollo-client: Architecting for performance

[lorensr]

Goal: 10ms blocks

When you are in a browser environment, in order to have a smooth UI, main-thread JS should be kept to 10ms per frame. While the view layer is usually the larger culprit, we saw with minimongo (eg https://github.com/meteor/meteor/issues/6835) that updating the data store can easily exceed that 10ms budget.

Apollo work

Our JS work includes:

1. performing client-side query
2. sending query to server (should be quick)
3. performing client-side simulated mutations
4. receiving data and updating the store

Which of these might take over 10ms?

Solutions

1. Code speed improvements
2. Breaking work into estimated 10ms blocks and calling requestAnimationFrame between blocks. A proper system for this would be client-wide and include the view layer, so the per-frame work can be totaled / shared / turns can be taken. Related: meteor-kernel.
3. Doing work in a Worker (which does not block UI)

1 and 3 would be best. Workers have separate memory, so if work on an in-memory store were in a Worker, the store would also have to be in the Worker, and you'd need to pass messages between the Worker and the main thread for each query or change notification. Both have access to the same persistent stores (IndexedDB and localStorage), but they don't support change notifications, which you'd need for reactivity.

cc @mitar

[mitar]

What about using service workers? So that client-side database could really be a service running in the background?

[mitar]

(This can then be nicely turned into offline use as well.)

[stubailo]

@lorensr is there some way to run perf tests on this library without trying to guess at which parts are slow?

[mquandalle]

An important optimization is to avoid recomputing the UI (virtual DOM or other) when the state hasn’t changed, which is a problem a state management library like MobX is good at solving (see Dan Abramov tweet on that and related detailed explanations).

For its internal state management, the Apollo client uses Redux which isn’t as efficient by default but has other advantages (good community support, replayable state transitions, dev tools, etc). The good news is that the Redux community is aware of MobX-like dependency management and is working on the abstractions that would bring that to Redux as well (cf. Reselect selectors). I’m confident that fined-grained dependency tracking, including a notion of a “derived value” and memoization, will be standardized and recommended in the future Redux. If not, a possibility we also have is to swap the internal Apollo-client store as we discussed a bit in #63 but I would prefer to see some convergence between Redux and MobX than having to build a store abstraction that would support multiple backends under the hood as proposed in #92.

[stubailo]

There are some very low-hanging things we can do to make sure that we only recompute queries whose dependencies have changed, I think - that's why I want to have real profiling rather than guessing at what will make things faster, because I think we can get _very_ far without even thinking about any of the stuff mentioned above.

[lorensr]

Reactive specificity

An important optimization is to avoid recomputing the UI

Oh, I was thinking of that as part of the larger-culprit view layer I mentioned, since in Blaze that's decided by which reactive deps you use inside helpers. Thanks, wasn't familiar with how that's different w/ Redux.

low-hanging things we can do to make sure that we only recompute queries whose dependencies have changed

That would be a bigger deal – matching the reactive specificity of Tracker & Minimongo.

Service workers

I was actually thinking about them when I left the Web out of Workers ☺️ It would be fantastic to not only not need to rehydrate the store on subsequent page loads, but not even need to fetch the changes since last load, because the background service had been receiving them. Ideally we put the store in:

1. service worker when supported (chrome, FF, android)
2. web worker when no service workers (IE, safari, iOS)
3. main thread when no web workers (old IE, old android, or outside browser)

Measuring performance

You could use the Timeline or Profiles tab of Chrome Devtools for one-off. For repeatable tests that you can benchmark on different browsers, you can use the performance api, which is even supported on latest ios safari:

And have a test suite that times operations with moderate and large amounts of test data. A good slowest target might be mid-range Android phones.

[mitar]

An important optimization is to avoid recomputing the UI (virtual DOM or other) when the state hasn’t changed

One optimization people are doing here is that they make sure that they can use referential equality to know when a change in state happened. The idea is that you have immutable state objects and if state changes, you provide a new object with a new reference. Then a very fast comparing objects with === tells you if something changed and not, and you do not have to do a deep content comparison.

Maybe this is something Meteor can embrace.

[helfer]

Maybe this is something Meteor can embrace.

We're hoping to do that for Minimongo eventually, and I think it would make sense to go in that direction from the start with Apollo.

[lorensr]

Related info from JS community:

Google decided to support putting your Angular 2 app inside a Web Worker: (thanks @Urigo for pointing that out)

When you choose to run your application in a WebWorker angular runs both your application's logic and the majority of the core angular framework in a WebWorker. By offloading as much code as possible to the WebWorker we keep the UI thread free to handle events, manipulate the DOM, and run animations. This provides a better framerate and UX for applications.

https://github.com/angular/angular/blob/master/modules/angular2/docs/web_workers/web_workers.md

Facebook doesn't want to put React inside a Worker (instead waiting for Compositor Workers to come out), but they're "actively exploring" putting Relay in a Worker:

https://github.com/facebook/react/issues/3092#issuecomment-183154290

Also in Meteor:

https://github.com/meteor/meteor/issues/6222

[stubailo]

Very interesting! Actually I wonder if there is just a way to run all Redux reducers in a worker, that would help a lot. Then it's just about getting the query results to the UI.

[lorensr]

At the GraphQL meetup, Relay guy said:

• they use a mix of OO, functional, mutability, immutability, caching, memoization, depending on the use case.
• "queries are immutable objects, so when we diff them against the store, we get a new object, but the new result is referencing items in the original query" similar to immutable.js. "creating fewer JS objects reduces pressure on the GC, which improves performance"
  
• didn't use immutable.js for the store because it worsened their perf tests by 30+%
• they schedule differently the different pieces of work that relay performs:
  
• future performance work includes static queries, response streaming, integrated & configurable GC, and expressive mutations
• re: relay in web worker or in native code, they're thinking that instead of moving work off the main thread, they won't do that work at runtime at all – move the vast majority of processing to static build time and have a very minimal step that runs at run time. The problem with web workers wasn't that they didn't reduce main thread work, but that they had a higher memory load, since they had two copies of relay running. And elsewhere in the talk, he mentioned that Facebook has the requirement of running well on resource-constrained low-to-midrange mobile phones.

[stubailo]

These are good notes, but I want to make sure we don't assume that these optimizations are necessary or appropriate for Apollo until we have some way to make those decisions in a data-driven way.

[lorensr]

Yeah, it was great how they were able to try immutablejs and just measure :)

[lorensr]

Slides about Relay 2, which includes a few performance things. Starting here:

https://speakerdeck.com/wincent/relay-2-simpler-faster-more-predictable?slide=29

• aggregation into batches
• caching
• efficient shouldComponentUpdate

"Big bets":

• persisted queries
• everything static
• RelayConnection:

optimizations:

Improving time-to-interaction

"a world without diffing":

[scf4]

How is Apollo performance now? immutability and shouldcomponentupdate etc

[helfer]

@scf4 we haven't gotten around to doing any profiling yet. If you'd like to help out and set up a benchmark or something, that would be great!

[smolinari]

This would be a cool thing to take on for the contributer week this week. I am too inexperienced, otherwise I'd jump on it. I know the topic of a performant architecture is going to be a major USP for Apollo at some point, if it isn't already.

Scott

[stubailo]

I think a really cool way to work on this if anyone in the thread is still interested is to see if we can modify broadcastQueries to not always run all queries at once, but instead yield in between queries when we're going to take too much time.

[helfer]

Closing up old issues. Let's track this under #1409

[sammkj]

@stubailo @helfer, will apollo-client have support for web-worker? I'm having a few problems trying to put apollo-client in web worker. Specifically how react-apollo uses apollo-client, fetching data has to be on the main thread and ideally I would like it happen on the worker thread.

[helfer]

Hi @sammkj we haven't thought too much about it yet, but we'd definitely like to support that use-case. Can you tell us how you would envision using Apollo with a web worker?

[sammkj]

Hi @helfer , I wrote a slightly longer than expected response. Hope you don't mind.

I have created a package called workux to facilitate Redux and web worker integration. Check it out here.

Redux & Web Worker Architecture

Similar to createStore in Redux, Workux's createProxyStore creates a proxy store that exposes same methods like getState, dispatch, etc. but with different underlying logic. This proxy store uses worker messaging protocol aka postMessage and onmessage to dispatch actions and receive Redux store updates respectively. User can also specify enhancers / middlewares that need browser APIs or DOM in createProxyStore. For example, react-router-redux uses history module, which will not be available on the worker thread, in its middleware. To add a note, All of these happen on the main thread.

Workux also provides a method createWorkerStore to attach worker messaging protocol to Redux store. So when Redux store gets updated, worker store will postMessage to proxy store. Similarly, when worker store receives an action from proxy store, it will dispatch the action to Redux store.

Problem with Apollo client

I have to initialize Apollo client on the main thread so that react-apollo can use it. With that being said, XHR requests still happens on the main thread. This kind of defeats the purpose of moving everything to web worker. Preferably, I would like to have web worker handle XHR requests, store updates, etc., leaving just the UI and animations on the main thread. One way to solve this is to create an Apollo proxy just like what I did with Redux. But that would be a lot of work and potentially lots of rough edges.

An Oversimplified Proposal

Instead of having an Apollo Client instance, split it up to smaller modules like createApolloMiddleware, createApolloReducer and createApolloClient. XHR requests will happen in createApolloMiddleware as it is now. createApolloClient will dispatch actions and listen to store updates. That way I can put createApolloReducer and createApolloMiddleware in web worker and createApolloClient on the main thread.

[jimthedev]

Since this is basically a perf optimization I wonder if there is a way to architect apollo to allow people to use https://github.com/developit/workerize-loader for xhr. This way if you don't provide an implementation via the loader then the default on main thread implementation would be used.