Postgraphile: load schema and type cast from file instead of introspection

[jmparsons]

I'm submitting a ...

• [ ] bug report
• [x] feature request
• [x] question

PostGraphQL version:

ANY

My database is accessed by lambda and in order to keep the execution time down I want to package the schema.

This is costly per endpoint of course because it introspects on each call - I'm using the query within node with pg context passed in:

const schema = await createPostGraphQLSchema(conn, this.schemaName);
const result = await withPostGraphQLContext({ pgPool }, async context =>
  graphql(schema, query, null, { ...context }, variables, operationName)
);

I'd like to pre-build the schema and somehow load and type cast map it to a schema:

// const schema = await createPostGraphQLSchema(conn, this.schemaName);
const schema = await loadExternalSchema('./schema.json');
const result = await withPostGraphQLContext({ pgPool }, async context =>
  graphql(schema, query, null, { ...context }, variables, operationName)
);

I've tried the cli output of introspectionQuery in gql and was able to combine it with buildSchema from the graphql package, but it doesn't recognize the query commands 'Cannot read property...'.

The new graphile-build is a bit over my head. I've been looking through the tests, but I don't see any load schema from previous introspected saved output file and type cast it to a GraphQLSchema, only introspect with inflections on an actual database.

Is there an easy way to save and load the schema then type casting it from a file?

[benjie]

There's some support for this in v3 where you can pass a PgCatalog object into postgraphql and it will use that rather then running introspection; but I cannot remember the incantation to get the PgCatalog out of postgraphql in the first place.

v4 does not yet have support for this, but you can track progress via https://github.com/graphile/graphile-build/issues/118

[benjie]

Cache introspection results

[evangow]

I'm just getting started with postgraphql (and programming in general, so this might just sound crazy!), but I'm interested in deploying it with RDS and Lambda as well.

After reading through the docs and some of the other issues submitted, it seems like one approach might be to make it sort of like create-react-app, where you can run a build process with webpack to compile the code (in this case a graphQL schema). This build process might also allow you to define custom types/queries/mutations which are then compiled along with the postgres schema in the build process.

This would enable a couple of use cases I am thinking about:

• you could hand authetication over to the graphql API instead of defining it in postgres (which seems like it can be a pain to manage after reading through some comments)
• you could add custom mutations for doing things like signing S3 urls and returning those to the user, so they can upload the file to S3 from the client, then the user would call a mutation for actually storing the S3 object key in the database (this mutation would be an auto-generated one from postgres's schema)
• you could run the build process and just upload the output file to lambda
• an "eject" script for postgraphql. In the event that postgraphql can no longer support all of your use cases, you can just run "eject" and it will run the build process for your graphql api and clean-up the source folder and expose whatever underlying postgres loaders you need (e.g. sequelize, etc.).

Not sure if this totally makes sense, but basically the gist is instead of directly exposing the database to the user through graphql, you're exposing the database to a graphql API build process. I understand the point of postgraphql is essentially to expose the database via a graphql API, but I think there are lots of use cases that this inhibits, and if you expose it to a build process, you can keep the simplicity of getting the graphql API up and running, but also enable more control over the graphql api if you want it.

Not sure if this makes sense or sounds crazy/un-doable since I'm a noob.

[benjie]

Definitely an interesting idea, but not really compatible with the way this project has been built (at least not v3/v4). It’s something I’ve been considering though; but in the mean time GraphQL extensions can be performed through plugins or schemas can be merged through graphql-weaver.

Personally I think RLS is a good fit for simple business logic and I think everyone should be using it even if they have further authorisation at other layers. That said, it can be hard for complex business logic to balance performance needs with security - sometimes it does warrant moving it up a level. This is currently out of scope of the project (though I do have some ideas on it I’d like to experiment with in future).

[evangow]

That makes sense.

[jmparsons]

@evangow For what it's worth we decided to keep introspection on our lambdas. I did get it working with v3, but the advancements elsewhere in v4 made it faster for our queries. We use a keep warm library because our endpoints talk directly to graphql independently rather than to one single endpoint.

[evangow]

Thanks for the heads up @jmparsons!

[benjie]

Support for this is now out in v4: https://github.com/postgraphql/postgraphql/releases/tag/v4.0.0-alpha2.28

Please test this and let me know how you get on.

Warning: I haven't written any actual software tests for this feature, so if anyone has time to send me a PR with some tests in, that would be fantastic! (Plus it should give you some confidence that the feature won't accidentally break in future releases!)

[approots]

I'm probably missing something, but with cloud hosting like Lambda, we don't use the file system so the new --write-cache <path> and --read-cache <path> won't work? I wonder if postgraphile could return the schema after introspection so the user could cache it in, for example, Redis? Then also allow the user to pass the schema back in to postgraphile?

[benjie]

It's possible to replace how the caching works by passing the relevant options into postgraphile-core, but you'll have to determine how to do that from the code:

https://github.com/graphile/graphile-build/blob/master/packages/postgraphile-core/src/index.js

However personally I'd just commit the cache to version control assuming your migration system is solid; or even write it to the lambda filesystem and then push to/from Redis/S3/whatever.

Note that this feature is still experimental and I've barely used it myself so if you can/can't get it working do please let us know. And contributing to the docs would be extremely welcome - there's so much for me to do!

[approots]

Thanks. Appreciate the tips and all your work on Postgraphile!

Library usage (using Express) of "writeCache" and "readCache" options worked for me. Didn't seem to improve the first request time though.

Introspection seems to be done on when the server starts, as opposed to first request, but first request always takes just over 100 msec and subsequent requests are about 20 msec.

[benjie]

Could just be JIT warmup time