Graphql-dotnet: DataLoader documentation

[PrimeHydra]

Super excited about the DataLoader #539, but can't find anywhere how to use it. I'm new to GraphQL.NET so learning by squinting at the tests is sub-optimal. Thanks!

[johnrutherford]

I can work on this. @joemcbride, where should this go? In the wiki?

[joemcbride]

Docs are markdown files found in the docs folder. https://github.com/graphql-dotnet/graphql-dotnet/tree/master/docs

[Pets]

@johnrutherford Did you had the time to work on the documentation ? :)
Like @PrimeHydra I'm super excited about this feature... but it's unclear for me how to work with it.
Thank you!

[johnrutherford]

I've started on it, but I'm not done yet. Haven't had much time to work on it. I'll try to finish it this week.

You might be able to figure it out from looking at the test project:
https://github.com/graphql-dotnet/graphql-dotnet/tree/master/src/GraphQL.DataLoader.Tests

The IDataLoaderContextAccessor and DataLoaderDocumentListener need to be configured in your DI container. Then you need to add the DataLoaderDocumentListener instance as a listener on the DocumentExecuter. Then check out the OrderType and UserType to see how to get a loader instance and use it in a resolver.

[sommarlov]

I've been using this and its fantastic so far. Great work!

While waiting for the documentation, I have a question. Is there a way to do .LoadAsync() on a list of Ids? Lets say for example (looking at the unit test Types) that User already has the Order ids. In this case I would like to load the orders by a list of order ids instead of the User id.

I hope my explanation makes sense, thanks!

[MakaBuka]

I've just investigated implementation of DataLoader and have a few questions:
1) is it correct, that now, dataloader just cache similar queries? lets say get user (id: 1) {name} more that once

2) I haven't found solution for n+1 issue (classic case: we have 3 requests in our resolver - user(id:1){name}, user(id:2){name}, user(id:3){name} - it should be combined in the end to sth like this - users(id in (1,2,3)) {name}

Could u please advice how we can resolve issue that IS described above?
Thx, D.

[johnrutherford]

Ok, I've finally completed a first draft of the DataLoader documentation. I'd welcome any feedback on how it could be improved or made clearer.

@sommarlov That makes sense. There's no way to do that currently, but it could be added.

@MakaBuka The DataLoader does query batching as well as caching. Check out the documentation and let me know if it explains well enough how to use it to batch queries.

[sommarlov]

@johnrutherford
I think the documentation looks good! Just some code examples that uses tabs instead of spaces.

I would love to see this added. :)

[MakaBuka]

@johnrutherford thx for prompt reply
I'll double check docs , but yeah, it would be nice to have more examples.

[benmccallum]

Looks good. Maybe it's worth referencing "all" the examples in the unit tests?

The use of a cache key reminds me of a POC I did once where I managed to write a cache helper that wouldn't require a cache key, just the expression of the operation you wanted to cache the results of, and it'd figure out the cache key based on that. May not work here, but would save a dev needing to supply it. Only downside would be if someone is doing GetUserById and GetUserByIdAsync for some reason, the cache keys wouldn't overlap. Perf would take a small hit too I guess with expression Compile() and Invoke().

[acronoah]

I am unclear as to where to add this code:

var listener = Services.GetRequiredService<DataLoaderDocumentListener>();

var executer = new DocumentExecuter();
var result = executer.ExecuteAsync(opts => {

    ...

    opts.Listeners.Add(listener);
});

[acronoah]

I ended up modifying GraphQLHttpMiddleware.cs to add the listener.

https://github.com/Conning/graphql-dotnet-server/commit/56398c9e1e2693652e056d9adf867c1a961d34d4

[johnrutherford]

That works. I plan on adding support to the middleware, but haven’t had time.

[benmccallum]

@acronoah, thanks for that. Definitely the solution at the moment. Would be good for this to be supported without having to modify a framework class, but will have to do for now. Is this something that should fall in the server repo @pekkah ?

@johnrutherford, a question I've got right now is, how can I pass additional arguments to my repository/service/data getter implementation? Right now you're restricted to Func<IEnumerable<TKey>, Task<Dictionary<TKey, T>>>, is there any way we can add some overloads with T1, T2, T3? Or some generic clean way of allowing additional arguments? Would obviously need to involve this in the cache key generation but it'd be super helpful. In my case I need to pass paging arguments, but also the list of field names requested by the client, so it's a scenario that's going to be very common.

[johnrutherford]

@benmccallum, you can pass additional arguments using closures. For example:

int page = ...
var loader = accessor.Context.GetOrAddLoader<IEnumerable<User>>($"GetAllUsers[{page}]", 
    () => store.GetAllUsersAsync(page));

return loader.LoadAsync();

string[] fields = ...
var loader = accessor.Context.GetOrAddBatchLoader<int, User>($"GetUsersById[{String.Join(",", fields)}]", 
    (ids) => store.GetUsersByIdAsync(ids, fields));

return loader.LoadAsync(context.Source.UserId);

As for automatically generating the cache key, I thought about that a lot when I wrote the DataLoader code. I felt that with "magically" generating the cache key, there was the potential to cause bugs that would be very difficult to track down and fix. I decided it would be best to leave it completely in control of the developer. If there is a bug because the same cache key was used, it's very easy to understand the problem and fix it.

[benmccallum]

Thanks @johnrutherford! My brain was obviously too tired at that moment to think of using a closure. Thanks!

With the cache key, I get ya and understand.

[joemcbride]

Closing as there are now docs and a link to a blog series. https://graphql-dotnet.github.io/docs/guides/dataloader

[acelot]

Sorry for necroposting. But can someone explain how to use DataLoader with builtin AddGraphQL() helper? How to add listener to executer options as it described in docs examples?

var listener = Services.GetRequiredService<DataLoaderDocumentListener>();

var executer = new DocumentExecuter();
var result = executer.ExecuteAsync(opts => {
    ...
    opts.Listeners.Add(listener);
});

[johnrutherford]

@acelot, just do this:

services.AddGraphQL()
    .AddDataLoader();  // Add required services for DataLoader support

That will automatically wire up everything required for the DataLoader. You don't need to do it manually.

[STRATZ-Ken]

@johnrutherford I assume you mean in ConfigureServices to add that, and it does not exist.

[johnrutherford]

@STRATZ-Ken You need to use the GraphQL.Server package.
https://github.com/graphql-dotnet/server

Here is a sample project:
https://github.com/graphql-dotnet/server/blob/develop/samples/Samples.Server/Startup.cs#L42

[hishaamn]

Do we still need to add the Listener (opts.Listeners.Add(listener)) or does the services.AddGraphQL().AddDataLoader(); already adds the listener to the execution options? Because right now, my IDataLoaderContextAccessor is null.

The docs are a bit ambiguous

[shcyiza]

@acelot, just do this:

services.AddGraphQL()
    .AddDataLoader();  // Add required services for DataLoader support

That will automatically wire up everything required for the DataLoader. You don't need to do it manually.

This should be in the documentation! I would have saved a day :p

[sungam3r]

PRs are welcome.