Webiny-js: Is there any way to disable the acceleration option in webiny:files? It is causing my deployment to error out.

[chanced]

This is:

• Question

Specifications

• version: 3.2.5
• OS: macOS Catalina (10.15)
• Browser: Not applicable.

Expected Behavior

webiny deploy-api deploys api

Actual Behavior

webiny deploy-api errors out at webiny:files Setting acceleration to "true" for bucket webiny-files-xxxx.

Steps to Reproduce the Problem

1. Have an account that will not allow for S3 Acceleration.
2. webiny deploy-api --env=dev --debug
3. profit 😭

Detailed Description

For some reason, my account can not set S3 acceleration to true. I honestly don't know why that may be. Is there anyway to disable the need for it until I contact AWS support?

webiny:files Setting acceleration to "true" for bucket webiny-files-xxxx. +241ms
An error occurred during deployment of files
AccessDenied: Access Denied

Possible Solution

[Pavel910]

Hi @chanced, we had this exact issue last week. The problem was not the acceleration at all; we investigated it and it just started working all by itself. Later we found out there were some disruptions in AWS services.

So the best thing to try is just leave it for a few hours, then try deploying again. I’m leaving this issue open so we can stay in touch. Let us know how it goes.

Cheers!

[chanced]

No go on the attempt. I'm pretty sure in my case the issue is that I can't enable Accelerate. I wasn't able to enable it on the webiny generated bucket or one I manually created.

I have a ticket in with AWS.

[Pavel910]

@chanced just wondering, what AWS credentials are you using? Is it Full Admin user, or do you have a custom set or permissions on your user? That could be the reason 🤔

[chanced]

I should have Full Admin. My account doesn't belong to an organization and I haven't limited my access in any way. Not sure what it's going on about. I've had this account since AWS was 3 services.

[Pavel910]

What region are you deploying to? Never had this error before; would love to know the reason so we can include it in our docs.

[chanced]

us-east-1. I found a few other people asking about it on Stack and forums so I'm not alone in it. They didn't have solutions though.

[doitadrian]

I tried an hour ago, and it didn't work. I'm not sure what's exactly wrong here, because I believe I changed the bucket name, turned acceleration off (directly hard-coded the flag to false), and it was still failing.

After 10-15minutes of fiddling, it just started working again, even with the acceleration enabled. ¯_(ツ)_/¯

I would love to be more precise with my findings, but unfortunately, it's still a mystery kind of for me too.

Can you please try again @chanced, just to see if this started working on your side as well?

Thank you for your time and patience! 🍺

[chanced]

Negative. I'm still waiting on AWS support to see why I can't enable it.

[chanced]

FWIW, you guys are awesome for this support 😄

I was debating whether or not to roll my own CMS to accompany an app I'm building but you guys have built what looks like an awesome project. I'm picking up graphql so I can use webiny.

This is completely off topic but have you guys considered type-graph to cut down on the boilerplate? I don't know how well it'll meld with commodo yet as I haven't gotten far enough along in either to make that assessment but it seems incredibly promising.

I think the biggest challenge for it is that the commodo's field decelerations are through the HOC. I haven't thought through it enough to determine how easy it would be to accomplish without needing to create a second variant of the model's class. I'm going to run into the same problem with Mongoose. Typegoose solves that though. TypeORM plays incredibly well with it too.

[doitadrian]

Thanks a lot for your thoughts regarding the GraphQL/models @chanced!

Although type-graph definitely looks interesting, for now, we're gonna stick with the current approach, because we're unfortunately just super busy all over the place. But I'm sure If the community will demand changes in this "department", we will consider it.

By the way, Commodo is very flexible. With additional higher-order-functions that would wrap existing models, I don't see why we couldn't achieve something similar in the future.

Alright, when you get a chance, just let us know how's your deployment proceeding. Hopefully, we'll get this thing resolved soon!

[chanced]

Oh I'm sure you guys are! You're doing a good job; building a flexible CMS isn't easy by any stretch of the imagination. There are so many things that could be done, providing what seems like an endless amount of work.

Hopefully with traction, the community will alleviate the backlog through extensions/plugins. So long as the core is solid and extensible enough, that'll come. I'm sure that'll pick up a lot once you introduce a store.

Commodo looks incredibly interesting and flexible. It may be possible to do it really cleanly through reflection.

[chanced]

Where did you manually overwrite the config option to turn off Acceleration? I think I found it last night but I wasn't certain. Picking up graphql is taking enough time that I haven't gone back to investigating it.

[doitadrian]

Unfortunately, you cannot do it via serverless.yaml at the moment, you'll have to make a change directly in the node_modules/@webiny/serverless-files/serverless.js:46.

Try to add accelerated: false, just so we know if it makes any difference. If so, well, I think we'll just disable the acceleration (or at least expose a config param to do it), since as you also mentioned, a couple of users also stumbled upon this issue.

If you'll be testing this, please try with a different bucket name.

And BTW, please change the name of the bucket once you'll be trying this. It can be found at /api/.env.json:3 (S3_BUCKET).

[Pavel910]

@chanced just to add to @doitadrian answer regarding type-graph and other tools you mentioned:

Schema-from-model generation is ok if you have a simple CRUD application and a known app data model, and not too many access patterns; however, it's a very bad practice to map GraphQL schema to an exact data model as a 1:1 replica. More often than not, GraphQL API is only exposing a small subset of fields to the end users, and very-very often, there are tons of custom resolvers that have nothing to do with an actual data model (3rd party API communication, etc.).

We went through many iterations of Webiny, and at the beginning we had this 1:1 API-to-Model map, it did not end well. So from now on, we handle Schema and Data model separately - on purpose, to have far greater control over things and allow easier refactoring, changes of logic, etc.

Webiny is not a CRUD generator - it is a development platform which can handle huge apps with a very diverse business logic and different ways of handling data (each microservice can have its own ORM library if you really want it to). The only thing we care about is that you register a GraphQL plugin to expose your schema - how you handle resolvers internally is completely up to you.

Hopefully you'll enjoy working with Webiny :)
Cheers!

[chanced]

@doitadrian Thanks man! I did a search across the project for 'accelerate' and did't find it. I'm still waiting on AWS support. It's apparently not an account level issue though.

@Pavel910 I didn't mean to make it seem like I was questioning your design decisions. Sometimes things pop up after the initial analysis of offerings. I just wanted to bring it to your attention incase you guys hadn't seen it.

Having said that, unless I'm missing something, Type Graph doesn't corral you into a 1:1 mappings. You still have to write resolvers, which may or may not pertain to a specific domain model. Moreover, it also allows for inheritance and interface mappings. It can handle more complexity than crud operations, like subscriptions (I had no idea that graphql had a notion of such, which is cool.

The ORM doesn't matter or where the data comes from so long as the data gets converted over to the decorated class at some point. For typegoose, they use a middleware that converts all data. Typegoose doesn't alter Mongoose's functions so to get it back into the decorated class, they convert them all.

Obviously you guys need to build what works best for you and it's clearly working as you've made some awesome headway. I just figured I'd present it to you guys for you to consider. I figured if anything, it may give you ideas for future refactorings.

[Pavel910]

@chanced absolutely man, we love to get info about tools out there, there's so much going on in the industry it's impossible to keep up with everything. Thanks for sharing! 💯

Our setup allows you to use the existing apps as-is and develop your new apps using whatever tools you like. All you have to do is follow some simple rules to connect your service with the rest of the system (our system of plugins). It was designed that way for exactly that reason - you can't possibly predict what new tools will pop up or what new trends of development will emerge in the future. So it's all good, as long as it gets the job done 🚀

[doitadrian]

Thanks for the feedback @chanced.

I'm sure @Pavel910 will respond shortly, but I just wanted to add that if our users will still want to achieve something similar, I believe they'll be able to do create a custom, let's call it, withGraphQL higher-order-function, and do whatever they want it to do.

We'll see what the community will demand over time.

Oh, and BTW, we do have some nice scaffolding tools in the works, so a good portion of work will be automated with it I believe.

[doitadrian]

Hey @chanced,

Just wanted to let you know that we've released a fix for the Access Denied error that you, and a couple of other users have been experiencing over the last few weeks.

If it's still of interest to you, check my response here, where I explaing what needs to be done in order to resolve the issue.

Cheers 🍻

[chanced]

@doitadrian I'm sorry, I meant to update this. Amazon got back to me and fixed the issue on my account. The problem is on their side.

[doitadrian]

Very interesting.

Nevertheless, thanks for letting us know 🙂

[chanced]

I also meant to let you guys know that Typegoose + type-graph came with some serious overhead. One of my queries needs to return 8k+ results. It was taking 8 - 14 seconds to iterate over everything.

There were a few bottlenecks, namely Typegoose doesn't alter Mongoose so in order to get an object in Typegoose/type-graph, the results first have to be iterated over for Mongoose, then again to get the type-graph/typegoose annotated instances. It was made worse by the middleware type-graph used as an example because it checks subdocs as well.

[Pavel910]

@chanced Wow! Great info, thanks for sharing ❤️