Apollo-server: Proposal: Replace integrations with an HTTP transport.

[abernix]

Introduction

⚠️🌐 Substantial Change Alert! 🌐⚠️

This document outlines the intent of some pattern changes to the way that Apollo Server is initialized which aims to:

• Allow a more natural HTTP framework integration pattern (e.g. with Express, Koa, etc.) by moving away from hyper-specific HTTP-server framework packages which utilize [applyMiddleware] (e.g. apollo-server-express, apollo-server-koa).
• Simplify the method in which GraphQL “speaks” over HTTP by keeping HTTP-specific concerns in a single code-path within a new HTTP transport which will map GraphQL requests and responses to corresponding HTTP requests and responses (e.g. headers, body and status codes).
• Introduce the concept of HTTP adapters which convert the body, status code and headers of an integrations' incoming request (which are frequently instances of Node.js' IncomingMessage) into the shape that the HTTP transport expects (See HTTPGraphQLRequest, below). In the outbound direction, these adapters convert the body, status code and headers from the object that the HTTP transport receives (see HTTPGraphQLResponse, below) into that which the framework expects (frequently instances of Node.js' ServerResponse).
• Provide a default HTTP handler which works for Node.js, Express, Koa and many other integrations which support the (req, res, next) request handler signature and hides away the details of the aforementioned HTTP adapters and HTTP transport.
• Create stronger typing for transport-related context properties, like http.
• Reduce the substantial maintenance overhead within the Apollo Server repository which are associated with the above.

At a high-level, this suggests implementors to:

• Move away from using applyMiddleware, opting instead for patterns appropriate for with their HTTP framework (e.g. Express, Koa, Hapi).

  These HTTP frameworks should be able to evolve at their own pace without the close coupling of their major versions with Apollo Server's own. Currently, because of our close coupling, Apollo Server is responsible for (too) many updates to those frameworks and their typings because they're baked into our integrations.

• Provide file-upload middleware, if necessary.

  This has the advantage of allowing different GraphQL upload integrations and, more importantly, the versions of those integrations (e.g. graphql-upload) which has been previously baked into Apollo Server and out of the user's control.

• Provide their own "health check" implementation, if necessary.

  Health checks are more complicated and application-specific than the limited implementation which we've previously provided (which was near nothing).

• Provide/choose GraphQL user interface (i.e. GraphQL Playground, GraphiQL, etc.), if necessary.

  It should be easier to remove the interface entirely, change it to a custom version, or replace it with one which supports offline support. Many organizations have decided that a particular interface is better for them, and it should be easy to switch these out, or leave them out entirely (if using a third-party tool, like Postman or Insomnia).

Note: While most of what this proposal addresses is specific to HTTP, the same motivation and patterns apply to other data exchange protocols (e.g. WebSockets), allowing transports to be implemented and tweaked outside of Apollo Server.

This work speaks directly to items that we've noted in the Apollo Server 3.0 Roadmap. Specifically, it helps with two of the roadmap's bullet-points:

• Improving integration with existing frameworks: While many developers will choose Express as their HTTP framework, it's simply not reasonable to expect everyone to use Express. The framework you choose (or rather, a specific version of that framework and its typings) should not be baked directly into Apollo Server (even if we decide to still test against it). This is particularly important as these frameworks release new major versions, but also as they patch bug fixes and security updates in sub-major releases. Put simply, Apollo Server should not need to be updated each time one of the integrations it hosts releases a new version.
• Plug-ability and extensibility: You should be free to plug in any middleware to your server as you'd like. If that means removing GraphQL Playground and replacing it with GraphiQL, or removing it altogether, that should be a simple and natural ergonomic behavior that's natural to the way your framework implements middleware.

Please read on for additional details!

Background

GraphQL requests and responses are most often exchanged over HTTP or WebSockets. As part of Apollo Server's processing of a GraphQL request and generating a GraphQL response, it must transform an incoming request — be it presented as HTTP, a WebSocket stream, or otherwise — into a shape that it can understand. For example, extracting the query and variables from a WebSocket stream or parsing them out of the HTTP request body. Additionally, it is in this phase that it has the opportunity to extract protocol-specific properties, like HTTP headers.

After processing the request and executing the operation, Apollo Server must turn the resulting GraphQL response into a format that confines with the desires of the transport the request came in on. For example, when transporting over HTTP, adding the correct Content-type headers, generating an appropriate status code (e.g. 200, 401, 500), etc.

Currently, Apollo Server has a number of so-called _integrations_ that allow it to do these various data-structure mappings to corresponding expectations of HTTP server frameworks (e.g. Express, Koa, etc.) and FaaS implementations (e.g. AWS Lambda, Google Cloud Functions, Azure Functions, etc.) .

Notably, the following eight integrations within the Apollo Server repository alone:

• apollo-server-azure-functions
• apollo-server-cloud-functions
• apollo-server-express
• apollo-server-fastify
• apollo-server-hapi
• apollo-server-koa
• apollo-server-lambda
• apollo-server-micro

Each one of these packages exists to define the aforementioned protocol mapping in a _similar_, though slightly different way.

Consider this incoming request scenario which leaves out some steps while still, I think, highlighting unnecessary complexity in our current approach:

1. The apollo-server-express package uses the cors package to prepare the response with the appropriate CORS Access-Control-* headers based on the settings provided on the cors option of the the applyMiddleware method. The apollo-server-lambda package uses a similar, albeit different configuration interface, but sets the same headers directly onto the response object.
2. The apollo-server-express package checks to see if req.method is equal to GET , whereas the apollo-server-lambda checks if event.httpMethod is GET.
3. _If the previous step results in a match_, the apollo-server-express package then checks to see if it's a user making the request via a browser by checking if the Accepts header contains text/html by using the accepts package to check req (IncomingMessage). The apollo-server-lamdba package makes a similar check, but merely uses String.prototype.includes on the event.headers.Accept property to check for the presence of text/html.
4. _If the previous step results in a match,_ the apollo-server-express and apollo-server-lambda package both return the result of renderPlaygroundPage, which returns the HTML for rendering GraphQL Playground. Prior to returning the (HTML) content though, the Express integration first sets the Content-type response header to text/html using res.setHeader, while the Lambda integration sets the statusCode property on the result context directly.
5. If we've made it this far, then we've determined that it's not a request for GraphQL Playground, and both the Express and Lambda integrations go through a similar set of checks on the headers in order to determine how to process the GraphQL request itself.
6. ... more duplicative steps occur.

If you see where this is going, you'll note that with a more abstracted HTTP implementation, much of this duplication could be consolidated. While the above simply contrasts the behavior in apollo-server-lambda with that of apollo-server-express, further examples of this duplication are prevalent in the integrations for other "integrations" — for example, our Micro integration, Koa integration, and Fastify integration all do mostly identical, though (frustratingly/) subtly different steps.

HTTP Transport (and Interfaces)

The Apollo Server repository would be the home of a new HTTP transport, and potentially others — to be determined as we build it out and demonstrate the need. WebSockets is certainly a strong contender.

HTTP framework-specific handlers (when not naturally compatible with these interfaces) would use the result of the transport to map to their own data structures An HTTP transport would be responsible for translating an HTTP request's data structures into the data structures necessary for GraphQL execution within Apollo Server. Most notably, that means feeding the request pipeline with the query, the operationName, the variables and the extensions. Additionally, the GraphQL request context should have access to strongly-typed transport-specific properties (e.g. method, url), which would be definitively undefined on non-HTTP requests.

To demonstrate this data structure mapping, some interfaces serve as good examples:

import { IncomingHttpHeaders, OutgoingHttpHeaders } from 'http';

/**
 * This represents the data structure that the HTTP transport needs from the
 * incoming request. These properties are made available in different ways
 * though generally available from the `req` (`IncomingMessage`) directly.
 * The handler's responsibility would be to ensure this data structure is
 * available, if its default structure is incompatible.
 */
interface HTTPGraphQLRequest {
  method: string;
  headers: IncomingHttpHeaders;
  /**
   * Each HTTP frameworks has a standardized way of doing body-parsing that
   * yields an object.  In practice, this `body` would have `query`,
   * `variables`, `extensions`, etc.
   */
  body: object;
  url: string;
}

/**
 * When processing an HTTP request, these will be made available on the `http`
 * property of the `GraphQLRequest`, which represents on `GraphQLRequestContext`
 * as `request` — not at all unlike they already are today, though now with
 * an implementation of that structure provided by the transport.
 * See: https://git.io/fjb44.
 */
interface HTTPGraphQLContext {
  readonly headers: {
    [key: string]: string | undefined;
  };
  readonly method: string;
}

/**
 * This interface is defined and populated by the HTTP transport based on the
 * `GraphQLResponse` returned from execution.  e.g. The status code might change,
 * within the transport itself, depending on `GraphQLResponse`'s `errors`.
 * See https://git.io/fjb4R for the current `GraphQLResponse` interface.
 */
interface HTTPGraphQLResponse {
  statusCode: number; // e.g. 200
  statusMessage?: string; // e.g. The "Forbidden" in "403 Forbidden".
  /**
   * The body is returned from execution as an `AsyncIterable` to support
   * multi-part responses (future functionality) like subscriptions,
   * `@defer` and `@stream` support, etc.
   */
  body: AsyncIterable<GraphQLResponse>;
  headers: OutgoingHttpHeaders;
}

Sending the right messages

While the above should handle the core bits of mapping to HTTP, we will certainly need to expand GraphQLResponse so it can provide the correct hints to the transport when it needs to make more complicated decisions.

For example, rather than directly setting a cacheControl header inside of Apollo Server, we should provide the transport with a maxAge calculation that it can use as necessary. For the HTTP transport, this still means setting a Cache-Control header, but for other transports (e.g. WebSockets) it might need to react differently or not at all.

API pattern changes to ApolloServer

The following sections demonstrate some bits that we should change in order to facilitate this decoupling.

While these changes might seem to move away from simpler patterns, I feel they enable GraphQL server users to have a more clear understanding of what their server is doing. Armed with that understanding — which we will provide whenever possible in the form of documentation and useful error messages — users will be better equipped to scale their server and the server will be easier to grow with.

"Getting Started" (Previously apollo-server)

The current getting started experience with Apollo Server runs an express server internally. Even though it has implications for the future that server, this hidden usage doesn't really make it clear to the user that we've made that choice for them. For example, it may not align with organizational requirements to run a particular framework flavor (e.g. Koa or Hapi). It's also just a larger dependency than the built-in http.createServer server that comes with Node.js!

This pattern should make it much more clear what server you're using and make it easier to scale out of a _Getting Started_ experience as an application grows (e.g. when non-GraphQL middleware needs to come into the picture):

Before

const { ApolloServer } = require('apollo-server');

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

server.listen(3000)
  .then(({ url }) => console.log(`🚀 Server running at ${url}`));

After

This utilizes the default HTTP transport internally and has no dependencies on Express.
See notes below for limitations of the default implementation which are different than the
current _Getting Started_ experience.

const { ApolloServer, httpHandler } = require('@apollo/server');
const { createServer } = require('http'); // Native Node.js module!

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

createServer(httpHandler(server))
  .listen(3000, () => console.log(`🚀 Server running at ${url}`));

With this particularly simple usage, there are some default restrictions which _might_ require adopting a more full-fledged HTTP framework in order to change:

• CORS is super restrictive (same domain).
• GraphQL uploads are separated from the project core and don't function out of the box.
• WebSockets, nor subscriptions, are enabled by default. For a _Getting Started_ experience, this is quite likely okay but the need for WebSockets is mostly associated with _subscriptions_, which are getting attention separately as part of the 3.x release line. I believe we should be able to implement subscriptions using the default HTTP transport without WebSockets.
• Body size limit is not enforced.

An HTTP framework is still not mandatory though, even with all of these options, since packages like compose-middleware work well with http.createServer too, allowing the composition/chaining of various middleware into a single handler. For example, we might consider providing suggestions like this one, passing in various middleware like cors(), graphqlPlayground, json body-parsing, etc.).

Express (previously apollo-server-express)

The widely used apollo-server-express integration bakes in a number of other pieces of _batteries-included_ functionality, though Apollo Server doesn't build on top of that functionality and instead merely passes it through to underlying dependencies (which again, fall out of date if we fail to update them regularly — which we have tried not to, but it takes a lot of work and extra versioning noise!):

Before

const { ApolloServer } = require('apollo-server-express');
const express = require('express');

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

const app = express();

server.applyMiddleware({
  app: app,
  path: '/graphql',
  cors: { origin: 'mydomain.com' },
  bodyParserConfig: { limit: '5MB' },
});

After

In my experience, this _after_ experience ends up being a lot more what you'd find in a typical server which has literally any other purpose other than serving GraphQL. You'll note that we're merely passing options through to the underlying dependencies (i.e. cors, body-parser):

const { ApolloServer, httpHandler } = require('@apollo/server');
const playground = require('@apollographql/graphql-playground-middleware-express');

// All three of these are likely already employed by any Node.js server which
// does anything besides process GraphQL.
const express = require('express');
const cors = require('cors');
const { json } = require('body-parser');

const server = new ApolloServer({
  typeDefs,
  resolvers,
});

const app = express();

app.use('/graphql',
  cors({ origin: 'mydomain.com' }),
  json({ limit: '10mb' }),
  playground(),
  httpHandler(server));

Even if there are more imports here, this _After_ scenario isn't actually introducing more dependencies since all of those are used internally already. Furthermore, when someone wants to "eject" from our behavior, they generally need to move to a pattern more similar to the above anyhow.

Batteries previously included?

Apollo Server 2.x adopted several “batteries included”-like functionalities which complicate integrations and cloud up HTTP transports including:

• Body parsing (i.e. deserializing the body and HTTP query parameters into native objects)
• Adding CORS headers
• Health check middleware
• Upload middleware

This section intends to shed light on what those are and what they (often, did not) do.

The first two bullet points above (body parsing and CORS) are implemented by utilizing popular third party packages (in Express, at least; and often manually in other frameworks).

The third (health checks) is merely a user-definable function which is just as well served passed directly to the framework itself.

The last is a third-party upload implementation which is well documented on its own and has previously needed to be updated independent of Apollo Server, but has been unfortunately at the mercy of our versioning.

Body Parser

In the apollo-server-express integration, Apollo Server automatically implements the body-parser package to parse the incoming HTTP request body. In the wild, most any server which does anything besides behave as a GraphQL server will likely have body-parsing functionality in place already. Though this can be implemented as:

import { json } from 'body-parser';
app.use(json({ /* define a limit, maybe */ }));

Other frameworks have their own approach here, like Koa's koa-bodyparser and Hapi's built-in route.options.payload.parse (default!).

CORS

In order to properly restrict requests to specific domains, CORS should be a consideration of any web server. By automatically using the cors package in its default configuration, Apollo Server allows the user to completely forget about CORS and just allow their server to participate in cross-domain messaging. That's great for getting started, but the default configuration of cors could be less secure than not enabling CORS at all since the defaults (of cors) sets origin: * and the corresponding Access-Control-* headers to permit all common HTTP verbs (e.g. GET, POST, DELETE, and so on.), allowing Apollo Server to serve any GraphQL request from any domain.

It’s arguable whether or not we’re doing much in the way of a favor to the user here, particularly since the code necessary for a user to put such permissive CORS behavior in place on their own is as simple as:

import cors from 'cors';
app.use(cors()); // Before Apollo Server

Similarly, Koa offers @koa/cors and Hapi has its route.options.cors. FaaS implementations often require you simply set the headers directly (which is all these other packages do).

While some teams might be tuned into important configuration like CORS, educating users of the importance of this option by making it more explicit and providing them the correct documentation to make the right decisions for their application seems of the utmost importance.

Health check

Right now, Apollo Server supplies a health-check middleware. If onHealthCheck is not defined — which it isn’t by default — Apollo Server merely returns a healthy status code response with {status: 'pass'} body — irregardless of whether Apollo Server is actually configured properly. If a user also defines the onHealthCheck method on their ApolloServer constructor, it will utilize that in order to quantify whether the server is healthy or not, though it doesn't receive the actual Apollo Server context, so it's abilities are limited

This same health-check middleware could be implemented in user-code with:

app.use('/.well-known/apollo/server-health',
    (_req, res) => res.json({ status: 'pass' });

There are other ways to do health checks, and even just providing a resolver which returns a similar value would be more accurate.

File uploads

Currently, Apollo Server automatically applies a third-party upload middleware called graphql-upload to provide support for file upload support via GraphQL fields that are typed as a special Upload scalar.

In order to replicate this behavior, the user would need to manually add the scalar Upload type (currently done automatically) to their type definitions and also add the graphqlUploadExpress middleware before their Apollo Server middleware:

import { graphqlUploadExpress } from 'graphql-upload';

app.use('/graphql', graphqlUploadExpress({ maxFileSize: 10000000, maxFiles: 10 }))

As noted above, this implementation is well documented on the graphql-upload GitHub repository. Having uploads enabled by default isn't necessary for most users and, in practice, there are often other ways that users choose to handle uploads.

Closing

HTTP isn't the only way that GraphQL execution can take place and even looking at just HTTP, the constantly growing number of frameworks has made it increasingly hard for Apollo Server to welcome those into the Apollo Server repository directly. I haven't personally seen it, but Amazon Lambdas, in addition to being triggered by API Gateway, can also be triggered by SQS, SMS, S3 uploads, and more. In practice, whether WebSockets or something more exotic, we've seen users needing to dig too deeply into Apollo Server's core in order to make this work, often sacrificing other Apollo Server features and functionality in the process.

Keeping HTTP-specific properties out of Apollo Server core will help other transports succeed but allow Apollo Server to focus on areas where it can provide greater value, such as caching, performance, observability, and different execution techniques. Additionally, the maintenance overhead and upkeep to inch all the integrations forward has been difficult, at best.

By defining this separation more concretely, I hope to make it easier for us to not need to selectively choose what frameworks belong in the core repository or not, since this data structure mapping should enable Apollo Server to co-exist with whatever framework flavor. In Apollo Server 3.x, we'll do our best to provide interfaces with maximum compatibility and minimal coupling to provide a better one-size-fits-all solution. This might mean providing an async handler that accepts a similar (req: IncomingMessage, res: ServerResponse) that could be used more ubiquitously, but should stop short of overly definitive implementations don't give access to necessary escape hatches.

I'm very excited about the opportunities this change will afford us, even if it does change the getting started experience. We're throwing around some other ideas to continue to keep that step easy (maybe easier than before!), but it's of the utmost importance that we provide flexibility to larger organizations as they grow out of that default experience.

I believe this is a step in that direction and feedback is very much appreciated.

[abernix]

Very much motivated by continued demand for a better story, as exhibited by countless emotes on https://github.com/apollographql/apollo-server/issues/1308, https://github.com/apollographql/apollo-server/pull/2244, https://github.com/apollographql/apollo-server/issues/1646, https://github.com/apollographql/apollo-server/issues/1117 and likely many more!

[alfaproject]

I know you've mentioned subscriptions over HTTP without web sockets, but is there any other document about this? This is super important in a serverless architecture like AWS APIGateway web sockets.

We did our own implementation hacking quite a lot of Apollo to have the desire results and while it works quite well, I'm really sad and not proud of how it looks like. It really is difficult to work around Apollo opinionated transport like expecting a long running server to always be available. ):

[harlandduman]

This is a great proposal that is super clearly explained. 100% behind each part of it.

While these changes might seem to move away from simpler patterns, I feel they enable GraphQL server users to have a more clear understanding of what their server is doing

These changes would make the starting experiences a tiny bit less easy, but I actually think it ends up being much simpler, since there is a lot less "magic" happening under the hood that inevitably becomes important to understand.

In my experience, this after experience ends up being a lot more what you'd find in a typical server which has literally any other purpose other than serving GraphQL

Spot on 💯

[nfantone]

☝️ Same here. Great work putting that together @abernix 👏. Current API design feels unnecessary restrictive and robs users from the freedom of applying their own custom solutions and patterns.

These changes would make the starting experiences a tiny bit less easy,

@harlandduman My two cents here? This needn't be so. There could always be an extra export/package that may serve as a canned/just-add-water solution - similarly to what's being exported now, without restricting everyone's experience.

[acao]

@alfaproject here is the IncrementalDelivery proposal, which is still a working draft in the transport spec.

@abernix this initiative is really exciting!

[alfaproject]

We work 100% with lambda even for websockets using a Serverless approach and while I'm happy to see progress with traditional long-lived HTTP servers, usually the implementations don't help us that much in that regard. I believe the solution for @defer and @stream with an AWS serverless architecture will have to serve the partial responses via websockets.

Happy to see progress and thank you for the heads up but I guess we'll have to keep waiting for the Apollo server protocol abstraction to start working on something useful for us.

[acao]

@alfaproject here is a demo of the IncrementalDelivery spec with netlify functions via lambda.

@robrichard and I took a look, and either netlify's proxy or the aws api gateway just waits til the last chunk to send all the payloads. hoping removing content-length fixes that.

streams are meant to be short lived and finite, so IMHO websockets don't make sense for this case, especially on lambda.

however, there is talk of another, non-competing websockets transport spec being submitted for @defer and @stream, so if you'd like to advance a new spec for that, @n1ru4l is a great person to talk to about that

note: if you have comments on the spec please open them as issues or PRs for the transport spec body, so we can iterate and submit an RFC to the IETF this year (hopefully)

[glasser]

We are still excited about the idea of replacing integrations with an HTTP transport in a future major version bump of Apollo Server. Our current plan for Apollo Server 3 is mostly focused on the story of "fewer hard-coded dependencies on specific versions of third-party software", and some of the things in this issue like decoupling from graphql-upload will be in Apollo Server 3. In the interest of getting those changes out in the world sooner rather than later, we're no longer targeting AS3 for the full HTTP transport refactor project, so I'm removing it from the AS3 milestone.