Apollo-server: Apollo Server Slow Performance when resolving large data

[sergelerner]

When resolving large data I notice a very slow performance, from the moment of returning the result from my resolver to the client.

I assume apollo-server iterates over my result and checks the types... either way, the operation takes too long.

In my product I have to return large amount of data all at once, since its being used, all at once, to draw a chart in the UI. There is no pagination option for me where I can slice the data.

I suspect the slowness coming from apollo-server and not my resolver object creation.

Note, that I log the time the resolver takes to create the object, its fast, and not the bottle neck.

Later operations performed by apollo-server, which I dont know how to measure, takes a-lot of time.

Now, I have a version, where I return a custom scalar type JSON, the response, is much much faster. But I really prefer to return my Series type.

I measure the difference between the two types (Series and JSON) by looking at the network panel.

when AMOUNT is set to 500, and the type is Series, it takes ~1.5s (that is seconds)

when AMOUNT is set to 500, and the type is JSON, it takes ~150ms (fast!)

when AMOUNT is set to 1000, and the type is Series, its very slow...

when AMOUNT is set to 10000, and the type is Series, I'm getting JavaScript heap out of memory (which is unfortunately what we experience in our product)

---

I've also compared apollo-server performance to express-graphql, the later works faster, yet still not as fast as returning a custom scalar JSON.

when AMOUNT is set to 500, apollo-server, network takes 1.5s

when AMOUNT is set to 500, express-graphql, network takes 800ms

when AMOUNT is set to 1000, apollo-server, network takes 5.4s

when AMOUNT is set to 1000, express-graphql, network takes 3.4s

---

The Stack:

"dependencies": {
  "apollo-server": "^2.6.1",
  "graphql": "^14.3.1",
  "graphql-type-json": "^0.3.0",
  "lodash": "^4.17.11"
}

---

The Code:

const _ = require("lodash");
const { performance } = require("perf_hooks");
const { ApolloServer, gql } = require("apollo-server");
const GraphQLJSON = require('graphql-type-json');

// The GraphQL schema
const typeDefs = gql`
  scalar JSON

  type Unit {
    name: String!
    value: String!
  }

  type Group {
    name: String!
    values: [Unit!]!
  }

  type Series {
    data: [Group!]!
    keys: [Unit!]!
    hack: String
  }

  type Query {
    complex: Series
  }
`;

const AMOUNT = 500;

// A map of functions which return data for the schema.
const resolvers = {
  Query: {
    complex: () => {
      let before = performance.now();

      const result = {
        data: _.times(AMOUNT, () => ({
          name: "a",
          values: _.times(AMOUNT, () => (
            {
              name: "a",
              value: "a"
            }
          )),
        })),
        keys: _.times(AMOUNT, () => ({
          name: "a",
          value: "a"
        }))
      };

      let after = performance.now() - before;

      console.log("resolver took: ", after);

      return result
    }
  }
};

const server = new ApolloServer({
  typeDefs,
  resolvers: _.assign({ JSON: GraphQLJSON }, resolvers),
});

server.listen().then(({ url }) => {
  console.log(`🚀 Server ready at ${url}`);
});

---

The gql Query for the Playground (for type Series):

query {
  complex {
    data {
      name
      values {
        name
        value
      }
    }
    keys {
      name
      value
    }
  }
}

---

The gql Query for the Playground (for custom scalar type JSON):

query {
  complex
}

Here is a working example:

https://codesandbox.io/s/apollo-server-performance-issue-i7fk7

Any leads/ideas would be highly appreciated!

[sergelerner]

FYI, if anyone have same issue, I got my answer here on SO

https://stackoverflow.com/questions/56416447/apollo-server-slow-performance-when-resolving-large-data

"The benefits that GraphQL offers (validation, sub-selection, etc.) inevitably incur some overhead as they require additional processing of the data you're returning...."

After this, my only comment here is that express-graphql had better performance then apollo-server from what I saw. Not sure why.

[martijnwalraven]

Closing this because the issue does not seem actionable.

[radojko]

Did you notice Amount 500 JSON have 6MB and 1M lines and AMOUNT 1000 JSON have 23.9MB and 4M lines :)

[xdebbie]

Reliving an old issue here, I am also having performance issues with a large dataset on GraphQL, my JSON file is ~9 MB and hosted on MongoDB - I use GraphQL to query the data from the server. I have like 266,000 lines and 20,000+ data entries, still struggling to find a way to load my search engine faster...

[andrewphillipo]

There is a solution here - you could only perform the validations at the data loader level - for example if resolving with a data loader we can return 12 different objects but those objects might fill in/resolve 1000 different rows.

In this example the validation might happen 12 times or 1000 times. Currently it happens 1000 times. It worries me that you have no interest in fixing this very common use case.