Webiny-js: Feature request: Appsync (backed by dynamodb) as datastore

[dburbridge]

This looks like a great project, my only concern is around mongodb which seems as a requirement to be the one thing that prevents this being a completly serverles - AWS documentDB requires you to manage (expensive) instances for example.

Could AWS appsync (backed by dynamodb) be used instead? Its based on Apollo iirc.

[doitadrian]

Hi @dburbridge, thank you for the question!

Webiny currently offers a few different apps out of the box (eg. Page Builder, File Manager, etc.), and they were all built using MongoDB as a primary database. But we are aware that not all apps should be built with MongoDB since some databases might work better for specific use cases. And that's why we don't force anyone to use it. You can actually build your own apps with any database, e.g. MySQL or DynamoDB, and utilize any additional tools you like, e.g. an ORM like Sequelize.

Our default apps are also built using our own library called Commodo (it definitely falls into the ORMs category) which can actually be used with different databases. So, you could even take it to the next level and try to write a new driver for the database of your choice, which would actually enable you to use Webiny with a different database. At this point, I'm not sure if all apps would be supported, but I'm sure you'd be able to get the base working, and from there move on with creating your own services.

If this is something of your interest, I may be able to assist you. I admit I didn't have a chance to work with DynamoDB, but I'm sure I would be able to help you with the driver implementation and everything related.

Regarding AppSync, it's a great product, but we don't support it currently. All of our services are built using the base Apollo Server Lambda library, which basically provides all of the GraphQL related needs.

I'm sorry if I left something unanswered. Feel free to let me know if you have more questions.

[everyonce]

@doitadrian Could you give a few more pointers toward how we could go about swapping mongo with dynamo on commodo? I took an initial look, but i don't want to flounder around if there's only a couple places that would need to be changed. I'm guessing the changes needed would be more substantial if we wanted to switch to something like mysql/postgres (for serverless aws, etc)?

[doitadrian]

Hi @everyonce, thanks for the question!

As a matter of fact, before we decided to fully go with MongoDB as our primary database for our base apps, we were using MySQL (we were targeting Aurora Serverless), and it actually worked the same way as it is now with MongoDB. The only difference was the tables management - meaning we had to create our tables before doing anything, execute ALTER scripts when changing tables and other standard MySQL stuff.

So yeah, in the same fashion, I do believe you can use commodo with DynamoDB. It's just a matter of writing a storage driver. That being said, I must mention that we didn't try DynamoDB yet and that we don't know if Webiny CMS is going to work with it in an optimal way.

But if you're willing to try, I'd be glad to assist you with it and point you in the right direction.

For now, I can give you a few references that'll help you write the storage driver, and once we have that sorted out, we can continue with the next steps (using the driver with Webiny CMS, maybe creating additional SLS components if needed, ...).

I'll write a short guide in the next comment.

[doitadrian]

First of all, commodo is located in a separate (mono)repo: https://github.com/webiny/commodo.

You can start by cloning it, and installing all dependencies by running yarn in the root of your project directory.

Once you have that ready, you can try running the tests, by typing yarn test. If all pass, you're good to go.

So, now that we have our project set up, I suggest you open packages/fields-storage-mongodb, this is the code of our current MongoDB driver.

The main file is MongoDbDriver.js, which basically just implements basic methods like save, delete, find, findOne, etc. Your DynamoDB driver must implement all of these.

I think it might be easier and faster if you'd just make a copy of the fields-storage-mongodb, and work on that by changing/adding/removing necessary parts. You'd be even able to adapt already existing tests, so you'd have a decent amount of test coverage too. But you can write them on your own if you want too.

This is basically all that you need to get started. So if you're interested, feel free to give it a try, and send me additional questions on the way if you'll have any.

BTW I believe we might start adding more commodo docs soon.

@everyonce

[everyonce]

excellent. I'll let you know as i make progress!

[doitadrian]

Hi @everyonce,

just wanted to check if you need any help, feel free to ping if you have any questions.

I also wanted to let you know that we discussed multi-database support internally, and we will be making some moves that will fully enable this. So we should be able to adapt all base apps.

Anyways, we would be glad to see Webiny working DynamoDB, especially because we received a considerable amount of interest from other people.

[everyonce]

does that actually implement appsync or dynamo? Just wondering why this is closed - it seems like there's still work to be done (which I hope to get to soon)

[Pavel910]

@everyonce nope, it doesn't. It just gives you an interface to build upon. Sorry, added the close keyword by mistake, just wanted to reference this issue :)

[doitadrian]

@everyonce We recently created a community on Gitter (https://gitter.im/Webiny/webiny-js), so you can also catch us there too if you bump into any problems.

[everyonce]

thanks guys!

[VividWombat]

I'm familiar with react/dynamodb and deploying with serverless. will do some code and see how it works. happy to share if its of interest. I agree that if you want to promote using the app on AWS, it should be using something like dynamodb

interesting project. good luck with it

[doitadrian]

Thank you @VividWombat.

@everyonce and I started already with something, we'll see how it goes.

Maybe we should create a separate branch/PR for this so everybody can see the status of this, I would say often asked, feature?

[everyonce]

I'm on board with any help anyone is willing to offer :-) - my progress has been slow just trying to understand the way the driver piece is abstracted, which makes sense.

[doitadrian]

I created a new issue for the DynamoDB support - #662.

So we can continue or discussion there, in a dedicated issue.

@everyonce @VividWombat @Pavel910 @dburbridge