Graphql-flutter: Further modularization of this project

[klavs]

This project is by far the most popular client-side GraphQL effort in Dart, but it seems to always be in a bad spot maintenance-wise. Some alternative/complementary projects have popped up recently.

I've been working on a GraphQL parser, which is already used by this project, and an AST-based Link layer which is already used by Artemis (@comigor) and gql_client (@smkhalsa).

To take the pressure off of this project and to optimize community's effort in various directions, I'd like to propose a further modularization of this project.
Here's a proposed list of modules:

• Link module (could use package:gql_link for that)
• Cache/Store module (@smkhalsa has done some interesting work on this)
• Client module (next iteration of package:graphql)
• Flutter module (next iteration of package:graphql_flutter)
• Codegen module (me, @comigor and @micimize have worked in this direction)

Modularization would enable independent iterations and breaking changes without necessarily braking the client itself. Also, multiple other projects may benefit from the Link module and Cache/Store module.

[mainawycliffe]

I am open to that. We currently lack active maintainers, time for me has become very difficult to come by and I took a new job and we are not using flutter, which makes it difficult to follow to up on flutter and this project in general for me. I have a few issues I have not found time to finish yet and I can not guarantee I will be able to work on them to the conclusion. On top of that, we also need more contributors and anyone interested to help in maintaining this wonderful project.

I am pining this issue.

[comigor]

I'm also up to having the good Artemis bits (on multiple file processing (with future work on middlewares), fragments globbing, custom scalars and parsers) integrated with this project (which lacks query typing for now).

[klavs]

FYI, I've labeled the link-related issues https://github.com/zino-app/graphql-flutter/labels/link

[eusdima]

Just as @mainawycliffe said, not working with the two core technologies graphql and flutter leads to a huge problem in terms of effort to context-switch for this project. Now that I am back working with @HofmannZ , we will push more effort into this project since we will work with the App that started this library.

I would definitely like to start again, but I need to start from scratch and read the documentation of graphql, apollo and all the cool tech that is making this library work.

@klavs If you have to explain what needs to be done to achieve this modularization, I would love to help.

[klavs]

@eusdima, It's really all about setting good API boundaries.

With the Link Module it's quite easy. It receives a GraphQL document and eventually returns the result. I've already implemented an AST-based Link API in gql_link. As you can see in the https://github.com/zino-app/graphql-flutter/pull/564, it removes any need for the Client itself to be concerned about the transport. Of course, all those issues will need to be solved externally, but in a more focused manner.
I've created a tracking-issue https://github.com/gql-dart/gql/issues/57 to list what's missing if graphql_flutter switches today. Those should be implemented while still looking at the existing issues (https://github.com/zino-app/graphql-flutter/labels/link) related to Link layer.

For the Cache Module, I think @micimize has the best mental model of how the API should be organized. Then that module should be implemented to it's own package.

When these APIs are well defined, at least a naive implementation of a Cache should be implemented which does the job. Meanwhile, the community has a stable Cache API interface to build their own Cache for their needs, relieving this project from the need to implement every fancy Cache possible.

To summarize:

• [x] Define a Link API (already done in package:gql_link)
• [x] Remove concrete Links from package:graphql (done in https://github.com/zino-app/graphql-flutter/pull/564)
• [x] Use only AST-based operations/documents in package:graphql (done in https://github.com/zino-app/graphql-flutter/pull/565)
• [ ] Implement all existing Links as separate packages (tracked by https://github.com/gql-dart/gql/issues/57)
• [ ] Fix known issues with Links (tracked at https://github.com/zino-app/graphql-flutter/labels/link)
• [ ] Write documentation for every Link
• [x] Define a Cache API (probably, with AST-based operations)
• [ ] Remove concrete Cache implementation from package:graphql
• [ ] Move the existing cache implementation to a separate package(s)
• [ ] Implement a normalized cache
• [x] Support optimistic updates in the normalized cache
• [ ] Fix known issues with Cache (tracked at https://github.com/zino-app/graphql-flutter/labels/cache)
• [ ] Write documentation for every Cache

_This might not be a totally acurate TODO list._

[jifalops]

Hey I have written a private graphQL cache. I haven't tested this one in months but I recall it wasn't saving often enough and the entire cache was a .txt file with a format that seemed like it wouldn't work. What's the status of cache packages?

The cache I wrote is basically a hashmap of query+variables => response. Each endpoint+auth has its own directory, and each cache entry is a file whose name is a hash of the query+variables. The contents of the file also includes a timestamp of the response and the full query and variables used for easier debugging. GraphQL responses are written to a temporary file when they are received, checksummed for errors, then renamed, which is an atomic operation in almost every OS. It is maybe a bit overkill but it is mission critical for us.

I want to have your client use my cache, but I'm not sure it's as simple as implementing the interface. Do you think it's a good fit?

[micimize]

@jifalops take a look at the v4 cache changes. I need to rework the HiveStore api, but in general the caching layer should be much more stable, and implementing your own store should be easier.

That said, a lot of the code in graphql_flutter revolves around the normalized cache and optimistic responses, and in v4 I simplified the library to focus more on those things (i.e. there is now a single GraphQLCache). You could still implement it and override all the NormalizingDataProxy methods to work with your caching solution and not be normalization dependent, but you might be better off composing lower-level gql libraries into a custom client.

Alternatively, if the point of your filesystem cache is a reliability layer, maybe it could all be implemented as gql_links, and you could also have multiple clients. It's a pretty strange setup that I don't fully grok so you'll probably want to do some digging into our codebase and gql's libraries to figure out the best way to integrate/if it makes sense at all

[micimize]

The most impactful thing here was the switch to the gql_link system. I toyed with some deeper modularization ideas, but in the end think it is a better DX for graphql/graphql_flutter to be treatable as a kitchen sink. That doesn't mean it wouldn't be better for its composite parts to be separate packages, but I just don't think it would be useful/feasible to, say, remove the concrete implementation of the core cache from the library.

The cache could be a standalone library that is re-exported like what I've done with links, but I'm not sure how much sense that makes either – as I discovered while trying to outsource store to ferry_store, it is hard to decouple the manager and the cache implementations.

Anyways, the tl;dr here is that I'm descoping further modularization than what's already been done thus far on v4 with links.