Subscriptions-transport-ws: Memory leaks in duplicate subscriptions

[damour]

Return from iterator on server never called in duplicate subscriptions.
Caused by bug in client https://github.com/apollographql/subscriptions-transport-ws/issues/295#issuecomment-339323133 (see Issue 2)

POC test code:

it('should return from iterator in duplicate subscriptions', (done) => {
    server = createServer(notFoundRequestListener);
    server.listen(SERVER_EXECUTOR_TESTS_PORT);

    let iteratorsCreated = 0;
    let returnCallCount = 0;

    const executeWithAsyncIterable = () => {
      iteratorsCreated++;
      let called = false;

      return {
        next() {
          if (called === true) {
            return Promise.resolve({value: undefined, done: true});
          }

          called = true;

          return Promise.resolve({value: {data: {testString: 'value'}}, done: false});
        },
        return() {
          returnCallCount++;
          return Promise.resolve({value: undefined, done: true});
        },
        throw(e: Error) {
          return Promise.reject(e);
        },
        [$$asyncIterator]() {
          return this;
        },
      };
    };

    SubscriptionServer.create({
      schema,
      execute: executeWithAsyncIterable,
    }, {
      server,
      path: '/',
    });

    const wsClient = new WebSocket(`ws://localhost:${SERVER_EXECUTOR_TESTS_PORT}/`, GRAPHQL_WS);

    wsClient.on('open', () => {
      wsClient.send('{"type":"connection_init","payload":{}}', () => {
        wsClient.send('{"id":"1","type":"start","payload":{"query":"query { testString }","variables":null}}');
        wsClient.send('{"id":"1","type":"start","payload":{"query":"query { testString }","variables":null}}', () => {
          wsClient.close();
        });
      });
    });

    setTimeout(() => {
      expect(iteratorsCreated).to.eq(2);
      expect(returnCallCount).to.eq(2);
      done();
    }, 200);
  });

Result: actual returnCallCount = 1.

It causes memory leaks because unsubscribe will never be called:
https://github.com/davidyaha/graphql-redis-subscriptions/blob/master/src/pubsub-async-iterator.ts#L50
https://github.com/davidyaha/graphql-redis-subscriptions/blob/master/src/pubsub-async-iterator.ts#L107

[ValdoGhafoor]

Is this still relevant ?
My server has a very fast growing memory leak when I enable subscriptions that makes them completly unusable in production

[OmgImAlexis]

I seem to have hit the same issue.

[Shahor]

@ValdoGhafoor Maybe I'm wrong, but I too believed I had memory leaks when really it was just the node process being overwhelmed by the work it had to execute.

Basically the throughput was just too high and since node has few threads (4 by default) to send data through the sockets it ends up blocking the event loop while its sending the data out.

Since the event loop is blocked, the tasks to run pile up in memory and therefore it behaves like a memory leak.

You can confirm this by triggering the issue and then stopping messages being sent through the pubsub. If the memory gets stable, you're hitting the same issue :)

You can use this option to increase the number of threads available to send network, which might help with your issue.

I ended up reworking this whole part of my stack though to have control on the throughput in entry.