Absinthe: QUESTION - Dataloader vs Preload join table

[ivawzh]

I would want to know your opinions on supporting eager load in Absinthe.

With Dataloader pattern, you avoid N+1 queries on a single layer query. But not on a nested query like user { followers { bestFriends { parents } } }. And eager load should be able to handle that.

I know in NodeJs space, there are some tools like Join Monster can achieve that (but it's an API to SQL persistence pattern, skipping ORM). Wondering if Absinthe would consider supporting join table features.

[benwilson512]

Hey! This is a great question.

Absinthe itself provides two built in pieces of functionality that help address this issue and others like it, and then there are supplementary libraries that use this functionality to provide more complete solutions.

Absinthe provides:

1. Batching
2. Flattened tree information

Batching

Batching is the ability to lazily execute fields together. So for example if you have:

{
  users {
    followers { name }
  }
}

When execution gets to the followers field instead of executing it, the field places information about what followers the field needs in an accumulator. This happens each time we get to the followers field under an individual user. When we've done that for all users and reached the end of the document all of that accumulated information is passed to a batching function, which can then look up all followers at once.

A second pass is then done on the document, filling in the followers into the tree where they belong, and execution carries on from there.

Batching as provided by Absinthe itself is a generic mechanism, equally applicable to both File system operations as well as database ones. The Absinthe.Ecto https://github.com/absinthe-graphql/absinthe_ecto project provides some helpful functions for doing this with a database. Your field is as simple as:

field :followers, list_of(:user), resolve: assoc(:followers)

Flattened Tree Information

The other thing that absinthe provides all resolution functions is a flattened view of the tree beneath the current node. Flattened here means that you don't have to worry about fragments, they've already been inlined.

You can use this information to do something like join monster and load everything eagerly. In fact, someone already has!

From here

This discussion has already been on going actually over on the Absinthe.Ecto project here https://github.com/absinthe-graphql/absinthe_ecto/issues/4. I'm actually gonna close this issue because I think it will be simpler to keep the discussion in one place.

I highly recommend reading the contents of that issue as it does a good job of summarizing where things stand now.

[tlvenn]

I have also opened an issue on the join monster project to discuss in more details on the drawbacks of using joins aka Flattened Tree Information:

https://github.com/stems/join-monster/issues/68

Imho, in most cases, the batching approach is recommended.

[acarl005]

@tlvenn Join Monster v1.0.0 now supports batching as an alternative to JOINs. Batching can be used entirely, or in conjuction with JOINs to find more optimal query plans