Graphql-js: Managing concurrency of async resolvers

[lukehorvat]

I'm attempting to build a GraphQL interface for the Trello API:

type Query {
  getCards: [Card]!
}

type Card {
  id: ID!
  name: String!
  comments: [Comment]!
}

type Comment {
  id: ID!
  text: String!
}

Getting all cards is one HTTP request to the Trello API, and getting the comments for all cards is another n requests (where n is the number of cards):

class Query {
  getCards() {
    return trello.fetchCards().map(card => new Card(card)); // http://bluebirdjs.com/docs/api/promise.map.html
  }
}

class Card {
  constructor(data) {
    this.id = data.id;
    this.name = data.name;
  }

  comments() {
    return trello.fetchCardComments(this.id).map(comment => new Comment(comment));
  }
}

class Comment {
  constructor(data) {
    this.id = data.id;
    this.text = data.text;
  }
}

I'm encountering rate limit issues when calling getCards, because GraphQL tries to resolve every card's comments at the same time, leading to too many requests being sent to Trello at the same time. So is there a GraphQL way that I can manage the concurrency of async resolvers, so that comments are resolved in a slower, serial manner i.e. one card at a time? It should solve my rate limit problem.

Changing my schema to the following fixes things, but I lose the clean code I had before:

class Query {
  getCards() {
    return trello.fetchCards().map(card => (
      trello.fetchCardComments(card.id).then(comments => (
        Object.assign(card, { comments })
      ))
    ), {
      concurrency: 1 // http://bluebirdjs.com/docs/api/promise.map.html#map-option-concurrency
    }).map(card => new Card(card));
  }
}

class Card {
  constructor(data) {
    this.id = data.id;
    this.name = data.name;
    this.comments = data.comments.map(comment => new Comment(comment));
  }
}

[robzhu]

Hey @lukehorvat, this is a good question and one I imagine others will run into as it becomes more common to wrap REST APIs with GraphQL. As you mentioned, GraphQL tries to resolve everything in parallel, which can exceed rate limits on the underlying REST API. In your resolvers, rather than calling into the REST API directly, you can create a proxy that enforces some sort of queue-like behavior to limit the outgoing API requests while returning promises so that GraphQL is "tricked" into thinking everything is happening at once. Here's a gist of how it might work:

https://gist.github.com/robzhu/495e5a6a78805b2162adc3470ea939aa

Once you're past the rate limit, you may encounter the problem of asking for the same data repeatedly. For that, check out https://github.com/facebook/dataloader.

[lukehorvat]

@robzhu Thanks, that makes sense. I already used a similar proxy approach for caching, so I don't know why it didn't occur to me to just do the same thing for rate limiting... :stuck_out_tongue_closed_eyes:

[lukehorvat]

I ended up using this proxy approach to throttle requests in my graphql-trello package. Just thought I'd reference it here in case the code helps someone who comes across this thread...

[Xample]

I would simply use a semaphore to wrap the fetch method:

https://www.npmjs.com/package/await-semaphore

[felixfbecker]

Unfortunately it is not sufficient to limit concurrency _within_ the resolver. For _very_ large numbers of total fields (i.e. resolver calls), Node's event loop itself can get overwhelmed and crash the process. graphql-js should have some logic in place to at least not exhaust Node's internal resources with resolver execution.

RangeError: Value undefined out of range for undefined options property undefined
    at Set.add (<anonymous>)
    at AsyncHook.init (internal/inspector_async_hook.js:19:25)
    at PromiseWrap.emitInitNative (internal/async_hooks.js:134:43)
    at Promise.then (<anonymous>)
    at promiseForObject (node_modules/graphql/jsutils/promiseForObject.js:20:41)
    at executeFields (node_modules/graphql/execution/execute.js:294:40)
    at collectAndExecuteSubfields (node_modules/graphql/execution/execute.js:713:10)
    at completeObjectValue (node_modules/graphql/execution/execute.js:703:10)
    at completeValue (node_modules/graphql/execution/execute.js:591:12)
    at node_modules/graphql/execution/execute.js:492:16

Relevant SO question: https://stackoverflow.com/questions/58674238/did-my-javascript-run-out-of-asyncids-rangeerror-in-inspector-async-hook-js