Graphql-flutter: Mutations

[ghost]

This is a really needed package.

I played around with it and got to talking to a golang dB on mobile that exposes the dB over graphql over JSON RPC.
It was just a quick and dirty smoke test.

Do you have a roadmap or anything as I can see what else needs to be done to your client and help

[HofmannZ]

We also believe the package is much needed. That's why we released it in such an early stage.

Internally we do have a small roadmap:

• [x] Cache
• [ ] Optimistic UI updates
• [ ] Client state management
• [ ] Support for Apollo server query batching

It would be great to hear some of your thoughts cache. We are currently looking to create a dart implementation of dataloader to handle the cache, but we might actually go with redux to align our project more with the react-apollo project.

Also it might be a good idea to get the I will update the package later today to include our roadmap.

[ghost]

I think its important that you distinguish between what Client and Server.
In your roadmap list that makes it a bit confusing because i dont knwo if you talking cleint or server.

I can give my own Point of view:
I have a graphql server on all mobiles because i just compile the golang one as a flutter plugin and expose it over JSON RPC to Flutter main.

here is the golang graphql server everyone uses:
https://github.com/graphql-go/graphql

you can compile it to run as a flutter plugin, by adding some extra stuff in go and dart handle Method Channel stuff. I can link to it later if your curious. But i am guessing you just want everything in dart ?

The golang code can then have a DB and do sync with the server. SO you have full offline support and sync with the server.
The main thing though is thats it just golang and its the exact same code that runs on the server. SO the One running on the Mobiles is like a cache DB. Mutations hit it and are stored if the mobile is off the network for example..

SO at least for me its really just about have a standard Dart graphQl client.

Now i dont need reflux at all. Why because i have a full DB and i have a way to do PUB SUB between the flutter Views and the DB. SO the Flutter views just have to subscribe to topic changes from the pub sub.
This is one of the reasons i went down this path. I kind of hate all the scaffolding for reflux.
I still need to listen for changes on a topic and still have to do some RX to update a view when a message comes in about a change. But it feels more portable and clenaer for me.

SO why GraphQL ? Because its really an abstraction of CQRS and PUB SUB into a standard and thats a good thing i guess :)

Geez i hope this is making sense...

[HofmannZ]

@gedw99 Interesting, it's the first time I hear about a full GraphQL server running on a mobile client.

It might actually work at a small scale, where you can actually store most data on the device and only need GraphQL to sync data between clients (like you explained.) With larger projects an the other hand, where storing all the data on the device is not an option (think about Facebook or GitHub for example.) it would quickly become an anti pattern to do so.

I don't think we will be taking the approach you explained.

Our implementation of caching would just save previously completed queries into a store (we're still figuring out what kind one store would be best.) With any subsequent queries we can then avoid the server altogether and serve the data strait from the cache.

With this system in place we can start implementing optimistic UI updates. After dispatching the mutation to the server we update the cache with optimistic results, the client can display these results until it receives the actual results form the server. At witch point we can update the cache a second time removing the optimistic result and replacing it with the result returned by the server.

Because were already interacting with a store we can just swell put local state in there, hence local state management.

This feels like the most logical direction of the library for us, hope that explains our reasoning. Cheers ✌🏼

[chimon2000]

I am a huge fan of react-apollo and I would love to see this client rival apollo-link such that adding caching, local state management, and other middleware is a breeze.

[HofmannZ]

@chimon2000 Yeah good suggestion! We might split the library into multiple smaler packages, but first we need to finish some basic features.

[chimon2000]

I would love to contribute to this in any way I can.

[HofmannZ]

@chimon2000 We're working on the cache right now, see PR #7. It would be really helpful if you can do a review.

[ghost]

@HofmannZ
what your building is almost exactly what i did. I guess i did not explain myself too well.

I guess the ONYL difference is that the code is the same on the client as it is on the server. The client just has less data. I think that might answer some of the aspects you brought up.

A key point is that each client only has a subset of the data, and they only have the data they are allowed to have ( based on the RBAC security rules). SO each client can now do offline editting, and sync back to the server. The best way to hold this in your head to use Google Drive as an example.
Google Drive is folders (namespaces), and you can share a folder of folder.
As a result of sharing another person now has access to that file / folder.
So its would magically be on their mobile in the "cache".

Now you need two more things

1. The DB is a Bucket of buckets ( not relational).
2. All things in the "DB" are versioned. Just a simple rev counter.

With that you can do intelligent offline editing and catch up when you back on line.

Lastly, you can think of clients on client on clients. Its tutles all the way down.
So you can have a tree of servers and clients and if any get partitioned its fine and they will all cache up.

SO it very very much scales. And it takes the load of your Origin Servers hugly.

Lastly you can do Materialised Views too now. Which is what front end coders call a ViewModel right.
Its all the same. If you have an RX or a Pub SUb anywhere your can get a mutation coming in and wire up lots of Materialised Vies that are saved into the DB / Cache at any level in the tree of servers.

Alot of this ironically is related to NDN
https://en.wikipedia.org/wiki/Named_data_networking

They call it the Internet 3.0. Ok thats a bit Lofty, but its really the idea of data always being pushed outwards from your origin servers to outer edge servers ( and client, which are exactly the same).
Alot of these concepts are already in big .5 mill enterprise system from Oracle, Tibco etc, but no one every says this in the documentation. Its all obfuscated in new terms..

Graphql Can sit on top of all this.

anyway, i will give your code a go and come along for the ride :)

I just pulled the code here and loving it.. Its refreshing to see this approach in Flutter.

[HofmannZ]

@gedw99 Oh that's interesting, we're thinking about running a lightweight GraphQL interpeter on the client. So we can actually decode the queries using the schema. After it's decoded we can save it the the cache. That is what you mean right? (Also this would also make local state management using queries possible.)

If you're up for it you can start a draft, we're interested in working on it ✌🏼

[ghost]

@HofmannZ

Responding to your last statement ...

Mhh interesting. I get it i think - Your basically doing reflection.. So, then you not going to need to construct structs at Design time ? Flutter is cool with data bind to runtime generated structs ?

I built mine in golang for a client. I Can help buid something similar here as i also want flutter integration. All backend stuff is golang though not flutter. Dont want a VM anywhere on servers.

[HofmannZ]

@gedw99 We essentially want to create a fancy tree based cache(log n read/write), but for that we need to know the types in the query response. That in turn requires us to parse the respons using the GraphQL schema.

Right now the cache is really primitive so it would be a good addition to this project.

[ghost]

Thanks @hofmanZ for that.

So you do or don't need to codegen the dart types that match the response from queries ?
Please let me know as that's crucial.

Regarding caching the query response. I have seen a few different approaches to this area. Apollo are working it out still imho.

But one thing with query response caching and subscriptions is that they are coupled and dataloaders with subscriptions into scale imho.

So i guues you have to decide first if subscription are essential to your project.

You cache parsing thing will work I think. But might be quite hard to reason about for Devs writing code on top of your architecture me thinks.

The approach I have seen work is really different from yours. I don't think your going to like it. But each mutation runs through the server and produces events that then modify a viewmodel / materialised view. To update the client cache you have to have a message queue and publish the CRUD operations that occurred on that viewmodel.
Those CRUD operations for a ViewModel are published to a topic on the MQ. Each client session is subscribed to that view model topic and so gets the view model crud events send in order to it.
Told you it's completely different to what you doing. This can scale to 1 m transaction a second on a basic server and can be clustered to multi data center using the right MQ and easily get to 15 million per second.

In this above architecture it's using event sourcing with graphql on top.

You can also decrease latency ore by using grpc for the networking layer and stuffing the data inside. Compared to JSON I reckon you can get an order of magnitude decrease in latency from server to client. This is dgraph do it I think.

In the end the clients are just dumb view engines which I like. They make mutations that hit the server and stuff happens and any view the client has in flutter page gets updated and it just draws the update. Scoped model should do it. You just check the CRUD event getting pushed to the view and update appropriately. I have grossly over simplified here btw !!!! This stuff is freaking hard.
The thing I hate about graphql is that a query can have so many responses. So you end up with resolvers, primed dataloaders and some magic to do subscriptions.

If the potential permutations of a query are more like a materialised view in a db then it's much easier. It's more like rest with a resource I'd mapping to a view model on the server and some pagination through that.
Now you can move the view model to the client and index it and page through it. No joins needed because it's already demoralised.

https://en.m.wikipedia.org/wiki/Denormalization

I know this is not exactly good news or maybe it is but that's my take on this so far as just another blind leading / following the blind.

The other thing that makes it makes to make progress in this area is the crappy way I and others describe the write flow and read flow in a systems architecture. Anyways ..