Apollo-tooling: Codegen is re-writting all generated types as many times as there are included graphql files

[zapo]

Intended outcome:

Running apollo client:codegen --includes="app/ui/**/*.gql" ... should be writing generated types only once.

Actual outcome:

In the following code extracted from codegen.ts command it seems like apollo codegen writes the types one time but also hooks onproject.onDiagnostics to rewrite them as many time as there are "diagnostics".

  project.onDiagnostics(({ uri }) => {
    write();
  });

  const writtenFiles = write();

When building a new project in our current setup trying to debug very slow codegen I realized by console.log'ing uri in the above diagnostic hook that we end up having as many "diagnostics" as there are gql files included (not sure what "diagnostics" are used for in apollo projects).
This ends up writing n+1 times all the generated types and the whole process takes up to 5min on our project instead of a few seconds (when commenting out the project.onDiagnostics hook or the invalidate() project calls that are responsible of filling diagnostics set).

How to reproduce the issue:

Here is the command we use (local schema file) and we aim to generate __types__ folders alongside the queries:

apollo client:codegen --includes="app/ui/**/*.gql" --localSchemaFile=./schema.json --target=typescript --globalTypesFile=app/ui/__types__/graphql.ts __types__

I guess that the diagnostics hook is there for a reason and may be necessary in different setups (remote schema? apollo server?). If that's the case, could we provide a way to toggle off that hook in a setup like ours?

Thank you for your help!

Versions

This code is found in latest apollo version 2.8.2 and master.

[zapo]

An other related weird issue with the onDiagnostics hook is that it persists SIGINT (Ctrl-C) and keeps writing things on disk until it went through all diagnostics.

[zapo]

I realize onDiagnostics is used for watch mode (which calls onFileChange which append to diagnostics and rewrite as a result) and dropping it breaks it. I'll update the related PR accordingly.

[eltonio450]

this is a huge problem for large projects heavily relying on fragments :(. Basically, the generation never ends, preventing it from using it in our CI.

@zapo what do you think we could do ? Is there a drawback to use your PR ?

Thank you :)

[zapo]

Hey @eltonio450 we've been using my fork for about month at work without any issue. However note that I've only tested it in our setup which is basically running against a local schema.json. We run it in our CI once or in watch mode for devs.

You can try publish your own or try out my published fork https://www.npmjs.com/package/zipolupu-apollo/v/2.6.3 (based on 2.6.2 and only with that one patch).

[eltonio450]

Thanks !

[nchase]

This is an issue for our team as well. Codegen takes ~120 seconds now, used to take ~2 seconds.

[virtue3]

we codegen both flow and typescript types. The flow ones take especially long. almost 2 minutes. I would like to be able to include type generation on precommit/push hooks but can't do that with the way things are currently :(

[frankdugan3]

  ✔ Loading Apollo Project
  ✔ Generating query files with 'typescript' target - wrote 154 files
Done in 543.56s.

[TSMMark]

Generating takes more than *25 minutes now — this is true pain.

Is a solution known?

[nchase]

Generating takes more than 15 minutes now — this is true pain.

Is a solution known?

@TSMMark: People in this thread have mentioned a workaround that removes the code that is problematic – all you need to do is remove these lines:

https://github.com/apollographql/apollo-tooling/blob/master/packages/apollo/src/commands/client/codegen.ts#L214-L216

@zapo posted a fork above – you can use that fork, create your own fork, or use something like patch-package to fix your apollo dependency 😄

[kevinmook]

Any update on this? Running codegen currently takes 20 minutes which is untenable since this is part of our build process (we verify our types against production before deploying). With the fix in that PR, it takes only a few seconds.

Since it sounds like the problem with https://github.com/apollographql/apollo-tooling/pull/1183 is that it breaks in watch mode - would it be possible to disable diagnostics when not in watch mode? I realize that's not ideal, but this has been a problem since April and it'd be nice to have a way out other than a fork.

[TSMMark]

I have seen our codegen duration problem disappear by upgrading the package a couple weeks ago. Not sure which version fixed it unfortunately, but if you're not on the latest of packages related to apollo codegen you might want to try upgrading.

[kevinmook]

Yep, it looks like we were using an old version and just verified the latest version is super quick now. Thank you!

[eltonio450]

Same. I think we can close this issue

[zapo]

Yes this was fixed :+1: