Amplify-js: Storage.put, Storage.get with protected images give 403 error out of the box on localhost
Describe the bug
When developing locally, Storage.put throws a 403 error out of the box with minimal setup from auth, api and storage categories. If you set the bucket permissions to public read/write access, Storage.put works as expected (though I shouldn't have to do this and is unsafe). When trying to retrieve that same file, even with public read/write access, it returns 403 and I am unable to load the image.
Minimal reproduction repository: https://bitbucket.org/mayteio/amplify-storage-issue/src/master/
My current solution is to use amplify mock storage - do I always have to do this?
Amplify CLI Version
@aws-amplify/cli 4.13.4
To Reproduce
I've created a minimal reproduction of the issue:
git clone [email protected]:mayteio/amplify-storage-issue.git
cd amplify-storage-issue
yarn
amplify init
amplify push
yarn start
- sign up for an account in the app when it opens in the browser
- after confirmation, attempt to upload an image using
Storage.putand you get 403 instantly. * That's the first issue *.
Log into your console, visit your s3 bucket permissions -> ACL -> public read/write -> Save.
Now upload goes smoothly. Give your image a title and hit save. Success, your console should show something like this:
{
"data": {
"createImage": {
"id": "089bb095-6ace-447b-b86e-2e1a022b36cf",
"title": "Title",
"file": [
{
"key": "2.032109714444997Screen Shot 2020-02-24 at 9.48.35 am.png",
"identityId": "ap-southeast-2:2201d300-5134-4538-83ec-911d281d3eab"
}
],
"owner": "harley"
}
}
}
In, ImagesList.js, I use the S3Image component provided to display it. Now, despite setting the ACL to public read/write I get a 403 Error. Same issue if I write my own implementation.
const CustomS3Image = ({ file }) => {
const [image, setImage] = useState();
useEffect(() => {
Storage.get(file.key, {
level: 'protected',
identityId: file.identityId,
})
.then(setImage)
.catch(err => console.log(err));
}, [file]);
return image && <img src={image} alt="" />;
};
Expected behavior
Storage.put should work out of the box on localhost for development. I have to deploy it to s3 bucket hosting to get it to work. This is a terrible feedback loop.
Screenshots
Localhost failing:
Working on deployed s3 bucket:
Desktop (please complete the following information):
- macOS 10.15.3
- node 13.8.0
Additional context
Given Amplify's promise is easy development, I find it hilarious I have to jump through so many hoops to get something so fundamental working. Pretty frustrating. Or that I have to store identityId in dyanmodb to retrieve protected images... but that's another story.
Questions for the amplify team this leaves me with
- Is it safe to store the user's `identityId next to the S3 key, as we need it for retrieval?
- Is there any additional setup after running
amplify add storagethat I have to complete in the console or elsewhere? What am I missing here?
Related issues
mayteio
All 91 comments
@mayteio Did you enable triggers for Cognito (should be prompted on init with your repro steps)? If so which triggers did you enable?
nikhname
on 26 Feb 2020
Yes, just the post confirmation trigger with addUserToGroup lambda.
mayteio
on 26 Feb 2020
@mayteio I was not able to reproduce this using your project, Storage.put and listImages worked out of the box. Could you please review the IAM permissions for your account?
https://aws.amazon.com/iam/features/manage-permissions/
nikhname
on 28 Feb 2020
Thanks for getting back to me @nikhname, was that on localhost or when deployed? Because it works fine when deployed, just not testing locally.
mayteio
on 29 Feb 2020
I can reproduce this even if I add hosting and deploy the react app ...
I am using @auth(rules: [{ allow: groups, groups: ["Admin"], operations: [create update delete] }]) and I get a 403 on upload to S3.
What's even more concerning is that if the user that I am logged in with is not part of the Admin group then the mutation fails but the upload is successful ... yikes.
Even with the most minimalistic auth resource added to the model i.e. { allow: owner }, I get a 403 when uploading.
amuresia
on 2 Mar 2020
Hi @mayteio
I tried with your above instructions to reproduce the error on localhost but its working fine for me.
Both Storage.put and Storage.get are working fine on localhost, I can see the images also as you are displaying them side by side with the image name also.
If you are still stuck ,pls let me know.
akshbhu
on 6 Mar 2020
Yes, I was able to reproduce this today. It's still an issue. I have to disable auth on my graphql model for the Storage.put to be successful.
amuresia
on 19 Mar 2020
Came across the same issue again on another project. This is definitely reproducible across projects.
mayteio
on 27 Mar 2020
I am getting a 403 when using Storage.put as well, which started when I upgraded to latest versions of amplify-js...
ajpaulingalls
on 3 Apr 2020
Also having this issue, happened when upgrading recently.
lukepushlabs
on 6 Apr 2020
Hi @lukepushlabs @mayteio @ajpaulingalls
Can you tell me the steps like which to which versions of amplify-cli and amplify-js you upgraded to so that I can reproduce the above error.
akshbhu
on 6 Apr 2020
I dropped my details into this bug in amplify-js:
https://github.com/aws-amplify/amplify-js/issues/5273
I'm guessing it has more to do with the amplify-js changes than amplify-cli. However, this may be different from the original posters problem...
ajpaulingalls
on 6 Apr 2020
I also randomly started getting 403 on Storage.put for images/blobs a few days ago running [email protected]
I could still upload a text string using Storage.put and it would be going into the right bucket but failed for images.
Just downgraded to 2.1.1 and it is now working fine again.
wadamomo
on 10 Apr 2020
I am having the same 403 forbidden issue as reported by @amuresia. My graphql schema models all have static Cognito group authorization (@auth(rules: [{ allow: groups, groups: ["Admin", "Engineer", ...]})). The S3 storage was created by using Amplify Cli "amplify add storage". The access restriction is by individual groups and a lambda trigger is also added:
? Restrict access by? Individual Groups
? Select groups: Admin, FieldEngineer, Manufactory, Engineer
? What kind of access do you want for Admin users? create/update, read, delete
? What kind of access do you want for FieldEngineer users? create/update, read
? What kind of access do you want for Manufactory users? create/update, read
? What kind of access do you want for Engineer users? create/update, read, delete
? Do you want to add a Lambda Trigger for your S3 Bucket? Yes
? Select from the following options Create a new function
Successfully added resource S3Triggerxxxxx locally
I loaded some images into the S3 bucket. The React client calls Storage.get(key) and it returns the signed URL without error (return code 200). However, the signed URL is set to the src of HTML tag and it receives 403 forbidden error.
I have tried to change the S3 restrict access by "Auth users only", the problem is the same.
I also changed the React code by directly using S3Image from 'aws-amplify-react', it has the same 403 forbidden error.
My amplify cli version is 4.18.0.
jingxizhang
on 17 Apr 2020
For what it is worth, I am facing this issue too.
I get the errors for public and private files too.
ahansson89
on 20 Apr 2020
Version 3.0.8 is working for me.
jsgphd
on 20 Apr 2020
Version 3.0.8 is working for me.
3.0.8 Doesn't work for me. Getting 403.
spyboost
on 23 Apr 2020
I found the problem: the Storage.get() inserts "/public/" in between S3 bucket endpoint and the S3 key in the returned pre-signed url. For an image file when using this pre-signed url as the image src you will get the 403 forbidden error (it should return 404 file not found error, instead).
I tried to work around the problem by removing the "/public/" from the presigned url. However, I got an error "SignatureDoesNotMatch" return because it is a signed url I cannot change it.
If I create a "public" folder in my S3 bucket and move my image files into the "public" folder then it works.
Is there anyway to configure the Store sub-module not insert the "/public/" in the returned url?
jingxizhang
on 25 Apr 2020
@jingxizhang - I think this is how it works by design. Those access levels are documented here: https://docs.amplify.aws/lib/storage/configureaccess/q/platform/js
So everything goes to /public by default.
I don't know if it's possible to configure it differently.
spyboost
on 25 Apr 2020
@spyboost Thank you for the explaining.
I tried to change the Storage level using "public", "private" or empty string, I got different results.
The code snippet:
import AmplifyConfig from '../aws-exports'
const {
aws_user_files_s3_bucket,
aws_user_files_s3_bucket_region,
aws_cognito_identity_pool_id
} = AmplifyConfig
Storage.configure({
bucket: aws_user_files_s3_bucket,
level: "private",
region: aws_user_files_s3_bucket_region,
identityPoolId: aws_cognito_identity_pool_id
})
When I use "public" or empty string for the level the resulting url is s3Endpoint + "/public" + s3Key, as expected.
However, when I use "private" for the level the resulting url is s3Endpoint + "/private/us-west-2%3Af07b89c6-280c-4a05-9c4b-f76ba6746eb0" + s3Key. Not sure what is the numbers following "private/us-west-2" is. I could not find the document for it.
Can you help me for that?
jingxizhang
on 25 Apr 2020
@jingxizhang - unfortunately, I'm not an expert and learning this new stuff myself. My understanding is that this looks like the user id from Cognito.
spyboost
on 25 Apr 2020
That’s the identityId of the user who uploaded the file. Also by design. When you retrieve it you need to pass in the identityId. I store this against my S3Object in dynamodb as it’s the only way to reliably retrieve it. No idea if that’s a security flaw or not.
mayteio
on 25 Apr 2020
@jingxizhang - also the whole configuration can be simplified to just calling
Amplify.configure(awsExports);
From the docs (https://docs.amplify.aws/lib/storage/download/q/platform/js)
If you use aws-exports.js file, Storage is already configured when you call Amplify.configure(awsconfig).
spyboost
on 25 Apr 2020
@mayteio - so the upload is working for you with 3.0.8? You're lucky. In my case I can't even upload to /protected :(
spyboost
on 26 Apr 2020
Yes, but only when it is hosted. Not from localhost. Have you tried deploying it and testing? Or try using amplify mock to save yourself some time.
mayteio
on 26 Apr 2020
@mayteio nope local doesn't work for me even with mock. I think there's a separate github issue for it as it's trying to send to https which fails locally, although I'm not even sure right now as I got lost tracking this problem from many different angles.
This was original issue I found:
https://github.com/aws-amplify/amplify-js/issues/5378
None of the mentioned issues gave me an answer.
Then I found myself in the dead end of this maze.
Something more related to local mock issue:
https://github.com/aws-amplify/amplify-js/issues/5320
spyboost
on 26 Apr 2020
@mayteio Thank you.
How do I get the identityId of the user? I checked the Cognito user pool, the logged in user's sub is different from the number in the url.
I would like all the users are able to read the image files in the S3, which level I should user? According to AWS amplify document. The "protected" level should allow all users to read the file but only the owner can write to the file. However, I checked with "protected" level, Storage.get will automatically insert a different identity number in the url. However, can the user to access the file shared (read only)?
jingxizhang
on 26 Apr 2020
I believe from Auth.getCurrentCredentials. It’s in. The id token from cognito. Yes if any user has the identityId of the uploader they can see protected files.
mayteio
on 26 Apr 2020
How can user B get the user A's identityId who is the uploader?
jingxizhang
on 26 Apr 2020
Even if user B get the uploader's identityId, how can Storage to set to user uploader's identityId in the generated pre-signed url?
jingxizhang
on 26 Apr 2020
@jingxizhang - this appears to be a very huge torture.
https://github.com/aws-amplify/amplify-js/issues/2203
spyboost
on 26 Apr 2020
As mentioned, you can store it against an object at upload time to dynamodb.
type S3Object {
key: String
level: String
identityId: String
}
Then
Storage.get(key, {level, identityId})
I agree it’s torture but it’s the way they’ve built it.
mayteio
on 26 Apr 2020
@mayteio - thanks!
In my case, I have many images uploaded from other project (the IoT Core project) which security model is not through Cognito but using X.509 certificate (so there is no uploader's identityId). I am not sure how to share those images.
jingxizhang
on 26 Apr 2020
@akshbhu what do you need here to move this one forward? There is a “not reproducible” tag however plenty of people have reproduced it with instructions.
mayteio
on 26 Apr 2020
@jingxizhang no problem. I am not sure either without having my laptop. Maybe open another issue as that is unrelated to this one.
mayteio
on 26 Apr 2020
@mayteio - there might be a way to not store this in dynamodb, depending on how your graphql entities are stored/retrieved, of course. For example, in a hypothetical scenario, if you have a new cool dating app and return a list of user profiles, each item UserProfile already has the id in response which then is used to query the data from Storage. Just wanted to double check I'm correct in my assumption.
spyboost
on 26 Apr 2020
It would have their sub (cognito pool Id) but not their identityId from the identity pool. This is why you have to store it as it’s generated during authentication not account creation. You could store it once against the user in attributes and update it every time they log in. My solution above is the cleanest and easiest I have found though.
mayteio
on 26 Apr 2020
@mayteio thanks for clarification. At this point I'm confused even more. I'm looking at the response from currentAuthenticatedUser() and I don't understand what property is the identityId. There's no such field. And there are a lot of other properties that look like a potential id, but none of the values are being used to send data for Storage.Put after looking at the request in chrome console.
spyboost
on 26 Apr 2020
No worries. currentCredentials as I said above, not currentAuthenticatedUser. usage example
Auth.currentCredentials().then(({identityId}) => console.log(identityId))
Please open another issue if you still can’t, trying to keep this one on track.
mayteio
on 26 Apr 2020
Our project has just run into this issue too. We're attempting to downgrade to an older version to see if that helps, but I would love an update from @akshbhu or someone from AWS to at least find out if this is being investigated yet, or if this is still in the backlog to look at.
Being transparent, this is quite a frustrating bug to run into, affecting our velocity, because we're not doing anything weird or complicated.
danrivett
on 8 May 2020
Hi @danrivett
We are looking in this issue.I tried reproducing it earlier but no luck. But it seems it occurring frequently.
Can you provide the steps i.e like categories you added so that I can reproduce it ?
akshbhu
on 8 May 2020
@akshbhu Yeah a developer on my team reported this to me today and couldn't figure out what's going on with receiving a 403.
I'm going to attempt to recreate it myself later today and will report back with repro steps hopefully.
danrivett
on 8 May 2020
Hi, same issue for me. I tried @aws-amplify/storage v3.1.7, v3.1.8 and v3.1.9.
storage was added with these settings:
? Please select from one of the below mentioned services: Content (Images, audio, video, etc.)
? Restrict access by? Auth/Guest Users
? Who should have access: Auth and guest users
? What kind of access do you want for Authenticated users? create/update, read, delete
? What kind of access do you want for Guest users? create/update, read, delete
? Do you want to add a Lambda Trigger for your S3 Bucket? No
amplify status:
| Category | Resource name | Operation | Provider plugin |
| -------- | ------------------------- | --------- | ----------------- |
| Function | | No Change | awscloudformation |
| Function | | No Change | awscloudformation |
| Auth | | No Change | awscloudformation |
| Auth | | No Change | awscloudformation |
| Api | | No Change | awscloudformation |
| Api | | No Change | awscloudformation |
| Storage | | No Change | awscloudformation |
debug log:
[DEBUG] 34:14.647 Credentials - credentials not changed and not expired, directly return
[DEBUG] 34:14.651 AWSS3Provider - set credentials for storage Object {
"accessKeyId": "XXX",
"authenticated": true,
"identityId": "XXX",
"secretAccessKey": "XXX",
"sessionToken": "XXX",
}
[DEBUG] 34:14.660 AWSS3Provider - put XXX.jpg to public/XXX.jpg
[DEBUG] 34:15.779 axios-http-handler Event {
"isTrusted": false,
"lengthComputable": true,
"loaded": 32768,
"total": 1303178,
}
[DEBUG] 34:15.796 axios-http-handler Event {
"isTrusted": false,
"lengthComputable": true,
"loaded": 196608,
"total": 1303178,
}
[DEBUG] 34:20.797 axios-http-handler Event {
"isTrusted": false,
"lengthComputable": true,
"loaded": 393216,
"total": 1303178,
}
[DEBUG] 34:23.146 axios-http-handler Event {
"isTrusted": false,
"lengthComputable": true,
"loaded": 1179648,
"total": 1303178,
}
[DEBUG] 34:23.404 axios-http-handler Event {
"isTrusted": false,
"lengthComputable": true,
"loaded": 1303178,
"total": 1303178,
}
[ERROR] 34:23.880 axios-http-handler, [Error: Request failed with status code 403]
- node_modules/expo/build/environment/muteWarnings.fx.js:27:24 in error
* [native code]:null in error
- node_modules/@aws-amplify/core/lib-esm/Logger/ConsoleLogger.js:96:8 in prototype._log
* http://127.0.0.1:19001/node_modules/expo/AppEntry.bundle?platform=ios&dev=true&minify=false&hot=false:124334:22 in error
- node_modules/@aws-amplify/storage/lib-esm/providers/axios-http-handler.js:90:23 in axios.request.then._catch$argument_0
- node_modules/promise/setimmediate/core.js:37:14 in tryCallOne
- node_modules/promise/setimmediate/core.js:123:25 in setImmediate$argument_0
- node_modules/react-native/Libraries/Core/Timers/JSTimers.js:146:14 in _callTimer
- node_modules/react-native/Libraries/Core/Timers/JSTimers.js:194:17 in _callImmediatesPass
- node_modules/react-native/Libraries/Core/Timers/JSTimers.js:458:30 in callImmediates
* [native code]:null in callImmediates
- node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:407:6 in __callImmediates
- node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:143:6 in __guard$argument_0
- node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:384:10 in __guard
- node_modules/react-native/Libraries/BatchedBridge/MessageQueue.js:142:17 in __guard$argument_0
* [native code]:null in flushedQueue
* [native code]:null in callFunctionReturnFlushedQueue
code in React Native app:
import { Storage } from '@aws-amplify/storage';
await Storage.put('XXX', blob, {
contentType: blob.type,
});
mtakac
on 8 May 2020
Hi folks - diving into this more, it seems like it's a JS library issue. I'm transferring this issue to the JS repo. cc @sammartinez
renebrandel
on 9 May 2020
Great, can you please explain what you found and why you transferred it?
mayteio
on 9 May 2020
Hi @mayteio
Storage.put() with blob is not working with amplify-js version 3.x.
akshbhu
on 9 May 2020
I'm still investigating, but I'm also finding the following isn't working for me:
await Storage.put('test.txt', 'Protected Content', {
level: 'protected',
contentType: 'text/plain',
});
So I don't think this particular issue is related to blobs. I'm getting a 403 for that too.
What I'm finding odd, and perhaps it's correct, but seems suspicious is the path it's trying to PUT to:
[DEBUG] 16:28.624 AWSS3Provider - put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt
Does that path look correct? The IAM policy has protected/${cognito-identity.amazonaws.com:sub}/* but that value is not the sub of the current user id. Here's more debug logging you can see the log of the current user which shows the sub as f0bc9559-a0e1-408f-9ad0-45f5b379f98d but that's not what is being used in the S3 path.
[DEBUG] 15:54.81 AuthClass - Succeed to get the user session CognitoUserSession {
"accessToken": CognitoAccessToken {
"jwtToken": "***",
"payload": Object {
"auth_time": 1588979621,
"client_id": "36dq980j6m4mdmldjkdfa706eh",
"cognito:groups": Array [
"Workers",
],
"event_id": "cb73749a-74f9-4cc3-9b30-1f10f41349d3",
"exp": 1588983221,
"iat": 1588979621,
"iss": "https://cognito-idp.us-west-2.amazonaws.com/us-west-2_qzYQxS5Ab",
"jti": "125cd2c9-53bd-447a-9edd-e65f7ebbc716",
"scope": "aws.cognito.signin.user.admin",
"sub": "f0bc9559-a0e1-408f-9ad0-45f5b379f98d",
"token_use": "access",
"username": "f0bc9559-a0e1-408f-9ad0-45f5b379f98d",
},
},
"clockDrift": 0,
"idToken": ...,
"refreshToken": ...,
}
DAN: Current credentials: {"accessKeyId":"***","secretAccessKey":"***","sessionToken":"***","expiration":"2020-05-09T00:15:53.000Z","identityId":"us-west-2:2723a679-65cb-440d-a3bb-155b7c209241","authenticated":true}
...
[DEBUG] 16:28.613 AWSS3Provider - set credentials for storage Object {
"accessKeyId": "***",
"authenticated": true,
"identityId": "us-west-2:2723a679-65cb-440d-a3bb-155b7c209241",
"secretAccessKey": "***",
"sessionToken": "***",
}
...
[DEBUG] 16:28.624 AWSS3Provider - put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt
And without wanting to put salt in the wounds, one of our biggest pain points with Amplify to date has been its opaqueness because when errors occur, and they of course they do, it's very hard to know why.
I think it's also very hard for you guys at AWS too, you have to figure out if there's a genuine problem here or whether it's just a user error and so largely noise.
It would be great if when errors occur we could turn a setting on for Amplify to capture diagnostic information and send it for review. I've turned on Amplify debugging logging which is the best I know of, but it contains a lot of sensitive info which I can't really wholesale copy and paste here, nor is it helpful to other non-AWS subscribers.
If there are better ways of submitting logs including sensitive info that may be helpful for debugging, please advise. I think this would be especially helpful for errors that aren't recreatable by yourselves like this one appears to be.
danrivett
on 9 May 2020
Hi @danrivett
The IAM policy has protected
/${cognito-identity.amazonaws.com:sub}/*but that value is not the sub of the current user id.
Yes you are correct, this is the Cognito IdentityId which is different from the User sub which you get from user Pool access tokens.
Technically, you wouldn't need Cognito user-pools to authenticate against S3.
put test.txt to protected/us-west-2:2723a679-65cb-440d-a3bb-155b7c209241/test.txt
So this PUT request should work as it is using Identity ID.
I think your issue is a bit different from the one above , you can open an issue here .
Regarding submitting issues you can mail you amplify folder zip to [email protected]. Please DO NOT post sensitive information on public page. We can reproduce this issue using your amplify folder.
akshbhu
on 9 May 2020
@akshbhu thinking about this more, sending sensitive information via email isn't much more secure to be honest - there's a reason why banks never do this as totally vulnerable in transit. The amplify directory contains authentication data so I'm a bit uncomfortable sending it via email even if it is non-prod.
For prod it would be a non-starter so there must be a better mechanism to send you the amplify directory? Does AWS have an upload page that I can upload the zip via rather than sending it via email?
danrivett
on 9 May 2020
+1 with the issue similar with @danrivett where the policy is for /${cognito-identity.amazonaws.com:sub}/* but the request actually sends to identityId instead.
This is a strange behavior as everything was working fine 3 days ago but some recent updates from the amplify package might have caused this issue.
drixta
on 11 May 2020
There seems to be a lot of different and unrelated contexts in this issue which makes it hard to debug and pin point the root cause. I'll try to classify the contexts here in this issue, if your issue is not categorized here, let me know.
OP's (@mayteio) issue seems to be testing the Storage category in dev environment and getting 403 errors. The issue seems to be with s3 bucket permissions since OP is able to change the permissions in console and get it working. Also to note is that using amplify mock works for OP, which bypass the AWS S3 service (again pointing to permissions issue). Since this issue was created and reported before [email protected] was launched, it seems completely unrelated to amplify-js library issue. Tagging @akshbhu to look into this issue.
Issue reported by @ajpaulingalls on Apr 2nd and later by @lukepushlabs , @wadamomo was an amplify-js issue https://github.com/aws-amplify/amplify-js/issues/5273 which has been resolved. Please let us know by creating another issue in amplify-js if you are still facing issues related to https://github.com/aws-amplify/amplify-js/issues/5273
@jingxizhang I believe you are facing issues due to understanding and I hope other community members were able to help you out.
@spyboost I'm not sure I completely understood the issue you are facing, if you can post your error messages and your code snippets (using the latest amplify-js library), it will be really helpful. To confirm, there is an outstanding issue in amplify-js https://github.com/aws-amplify/amplify-js/issues/5320 preventing the use of amplify mock. We are looking to release the fix this week.
@mtakac can you post your complete app sample where you can reproduce the issue in the react native? Most importantly I'm interested in looking at how you are retrieving the blob.
@danrivett @drixta, are you facing issues only with protected paths of with private as well? IdentityId in the upload path for both protected and private paths has been there for a long time https://github.com/aws-amplify/amplify-js/blob/e9887cee066b7cde991aefb8c632b8cb45f7a996/packages/storage/src/providers/AWSS3Provider.ts#L478-L483. Can you also post what you see in the network request and response in the dev tools? You can also paste the debug logs here after removing any sensitive information
Amplifiyer
on 11 May 2020
@danrivett @drixta I just tried reproducing this issue in react and I was not able to. Can you verify that the user is signed in before attempting to PUT files to protected/private folders?
Amplifiyer
on 11 May 2020
Hi @mayteio and @amuresia
I think the issue is tied to the access policies for the users in a admin group. You are assuming that the group will inherit all the policies tied to an auth IAM role by default? That's not the case actually.
I think you have to do amplify update storage and select the operations you want a certain group to have access to and that's when the IAM role associated with the user pool group will be updated.
Let me know if that works for you?
akshbhu
on 11 May 2020
@danrivett @drixta I just tried reproducing this issue in react and I was not able to. Can you verify that the user is signed in before attempting to PUT files to protected/private folders?
@Amplifiyer If you look at the debug logging I copied in an earlier commit above you'll see I am logged in and the AWSS3Provider has picked those up and logged them out, so I think we're good there.
I'm currently working on getting more debug info from the serverside via CloudTrail, but if you have any other ideas to get better diagnostics, please let me know.
My current theory is it's trying to use a Group the user is in for IAM access and so getting a 403 rather than owner authentication, but haven't got much further other than I deleted the Storage class and recreated it with just Authenticated users only (and not also with Group access) but I still get the same 403 error on PUT:
? Please select from one of the below mentioned services: Content (Images, audio, video, etc.)
? Please provide a friendly name for your resource that will be used to label this category in the project: personnel
? Please provide bucket name: ***
? Restrict access by? Auth/Guest Users
? Who should have access: Auth users only
? What kind of access do you want for Authenticated users? create/update, read, delete
? Do you want to add a Lambda Trigger for your S3 Bucket? No
Successfully added resource personnel locally
@danrivett thanks for conforming. Can you post the response from S3 service that you see in network requests in browser devtools and your minimal reproducible code that generates this error?
Amplifiyer
on 11 May 2020
@Amplifiyer if there's any way I can send you the CloudTrail logs securely? Sending via email isn't secure enough for credential info really.
I just tried both a protected and public PUT and neither work, I do believe based on the following CloudTrail log that _it may be because the Amplify JS library is attempting to authorize the request through role based authentication through a Group the user happens to be a member of rather than owner authentication_.
This is even though Groups aren't even being used to provide access (see my last comment), but even if the "Both" option was used there to allow for adminstrators to have access for example, owner based authentication should still work. But that is a tangent currently because I deliberately avoided configuring Group based access in the Storage category.
{
"eventVersion": "1.07",
"userIdentity": {
"type": "AssumedRole",
"principalId": "***:CognitoIdentityCredentials",
"arn": "arn:aws:sts::***:assumed-role/us-west-2_***-WorkersGroupRole/CognitoIdentityCredentials",
"accountId": "***",
"accessKeyId": "***",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "***",
"arn": "arn:aws:iam::***:role/us-west-2_***-WorkersGroupRole",
"accountId": "***",
"userName": "us-west-2_***-WorkersGroupRole"
},
"webIdFederationData": {
"federatedProvider": "cognito-identity.amazonaws.com",
"attributes": {
"cognito-identity.amazonaws.com:amr": "[\"authenticated\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***:CognitoSignIn:***\"]",
"cognito-identity.amazonaws.com:aud": "us-west-2:***",
"cognito-identity.amazonaws.com:sub": "us-west-2:***"
}
},
"attributes": {
"creationDate": "2020-05-11T21:24:28Z",
"mfaAuthenticated": "false"
}
}
},
"eventTime": "2020-05-11T21:24:38Z",
"eventSource": "s3.amazonaws.com",
"eventName": "PutObject",
"awsRegion": "us-west-2",
"sourceIPAddress": "***",
"userAgent": "[Expo/2.15.4.10229 CFNetwork/1125.2 Darwin/19.4.0]",
"errorCode": "AccessDenied",
"errorMessage": "Access Denied",
"requestParameters": {
"bucketName": "personnel***",
"Host": "personnel***.s3.us-west-2.amazonaws.com",
"key": "public/test-public.txt",
"x-id": "PutObject"
},
"responseElements": null,
"additionalEventData": {
"SignatureVersion": "SigV4",
"CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
"bytesTransferredIn": 0.0,
"AuthenticationMethod": "AuthHeader",
"x-amz-id-2": "***",
"bytesTransferredOut": 243.0
},
"requestID": "3DEDB3806C9AFB48",
"eventID": "ab3ad64f-13c8-46d1-a479-ef4e3a28975a",
"readOnly": false,
"resources": [
{
"type": "AWS::S3::Object",
"ARN": "arn:aws:s3:::personnel***/public/test-public.txt"
},
{
"accountId": "***",
"type": "AWS::S3::Bucket",
"ARN": "arn:aws:s3:::personnel***"
}
],
"eventType": "AwsApiCall",
"managementEvent": false,
"recipientAccountId": "***",
"eventCategory": "Data"
}
I am a layman here, but look under the sessionIssuer, the userName value is a Group role and not a user, could this be the problem?
I can send you the full event output to CloudTrail if you provide me a secure option.
danrivett
on 11 May 2020
@Amplifiyer Here is the example request header of the failed PUT request to S3
PUT /protected/us-east-2%3Adac20090-8bb4-40c2-8acf-3b7231a7028e/fullsize/anmrm1lec3ta4n7wtl6q7_00000.png?x-id=PutObject HTTP/1.1
Host: duoquserassets161347-dev.s3.us-east-2.amazonaws.com
Connection: keep-alive
Content-Length: 49451
x-amz-meta-albumid: bdb304da-2b7d-448c-b03f-a4fb8f471753
x-amz-meta-owner: 04b8e3c3-959c-490c-9984-32c9f89e9396
x-amz-user-agent: aws-sdk-js-v3-@aws-sdk/client-s3/1.0.0-beta.5 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36 aws-amplify/3.2.5 js
authorization: AWS4-HMAC-SHA256 Credential=ASIAVI3LPOATCRGJ6Q4V/20200511/us-east-2/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-albumid;x-amz-meta-owner;x-amz-security-token;x-amz-user-agent, Signature=8ed6b33e5deb118687be5c34648ce4356dadd927af756d7940863a5d1f72accd
Content-Type: image/jpeg
x-amz-content-sha256: UNSIGNED-PAYLOAD
Accept: application/json, text/plain, /
x-amz-security-token: IQoJb3JpZ2luX2VjEO3//////////wEaCXVzLWVhc3QtMiJHMEUCIQCldEd3ccb2NuIkbIgZ7XJRIuQZ6Phm4cCuf9yI20UMQQIgGkbMvosrPEjuBtncnFLybt3mlSHGVRXDVcnKu8f09Kgq/gMINxAAGgwzNjI2MTMyNzI2MTQiDFPrvwq62wQLUdxVByrbAxwyvp++6XsXosbG1zB0LR2a1illknAKLNOg6NUAV8CeBAixD1QcqWYozJLqxD8GS6CyVa44ys4bPjdX2GZnJdoeUDmfvHerQ2foOOnTViFdlA0IKXoSbSPNO8z1OhrvgRUepTV1KaLbVLp63ySnUiQL16JXWP6j/WYXI18I8E5AWiPG3cY0k84LdgjfvX0wPLr9cR08pqgDrmbnxajJzeo8tdj4ntCMNOZzyNlnjVcpex79CTdWpd3IDeL9OCDsBpCfrFZKKInbZAHqQBcpBgEQfgRVHsiZj5+Hy0rLz2iXpIWOgjG+7CYTSFtchLp9WkxWC0YN5Ah/1kcCemvL0E1ZHmAe0IABI5lQN7G1B50XNDyPO2dtEFLoMuar+KD+OeZeGT9mBKtdFDyagvArkBxo4NenJ/Q96xP/3o24yThFTMcsKMaXwZXVPE/yPesySLhhUDEVR7q/ATQ3vB0DC6QIEa0euy1izkTu1PPW31/+DunttQVo8BuD5iE8rEY8jAoc9yMJPHbMhVOTVC7X4OjPcB4HQidLDujYcJcWyTU1B4MPfy45J/2S0nbTNfvcCg+K/cOx7lASLts3WgG7fKHsma8pURtsIS7uQ0kh4iGZsur5xqRK3zAsWKswtIfn9QU6ywJ4jlIFTMLvWpDVvwovV7u+eDMh8sDGwwYgodFYj6ZuLV1qeeoXAqclbcTu30aUAvVmZacB0y7KuGbbBgQ0DfEI657eYdimDtsBO6k1g34OyXkfNbCdl9qYWxQ2mWEOdPVuF6H4UE+pKQq8ondDdgn4RxHLLXBPX7phsa6bBmYwxAnmJF6D2rtRC0k19TGPnIbcbfeIHLr99pwZ79O7rUYS6hNePiXx3q6ys76DHB/t6ix3Vx1EanU68+KTwBcalUNRaFn3dzU712z37gBvuYzo+ziBKAT3It229xuYpf2SeW7YM0Dtp+mKQPqvCdUlCuiiw+Zy3J/zyvSer8njVaSVK5GsGPN+NIaZFAh3GB3JCVV/XmcEp54dS1cB9vzpoWCeeSkeJcVnOTJCUYUvlsdMQcHJDoKrJRlY9r82pct4hDicrm2dD4YOHBvm
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36
x-amz-date: 20200511T213811Z
Origin: http://localhost:3000
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://localhost:3000/setting/account/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,vi;q=0.8,zh-CN;q=0.7,zh;q=0.6,pt-BR;q=0.5,pt;q=0.4
drixta
on 11 May 2020
@danrivett, there currently isn't a way to securely upload your secrets and we shouldn't even need to. Can you put together a minimal reproducible steps with code that I can recreate on my end with my own AWS credentials to be able to debug this?
@drixta, thanks for this. Even more helpful would be the S3 response that resulted in 403 to see whether it's a AccessDenied error, SignatureMismatch or something else.
Thanks both for working with us to troubleshoot it.
Amplifiyer
on 12 May 2020
@danrivett, there currently isn't a way to securely upload your secrets and we shouldn't even need to. Can you put together a minimal reproducible steps with code that I can recreate on my end with my own AWS credentials to be able to debug this?
@drixta, thanks for this. Even more helpful would be the S3 response that resulted in 403 to see whether it's a AccessDenied error, SignatureMismatch or something else.
Thanks both for working with us to troubleshoot it.
Response
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>EBB3F76F73B5EB97</RequestId><HostId>ANlr9OkaQy4UNpzmD4bUP6v5nhshvdorB02Z+Wwp/tVhr0WbtE8HslQK9VNLIiJ65KAGIXXL+LE=</HostId></Error>
Response header
HTTP/1.1 403 Forbidden
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, HEAD, PUT, POST, DELETE
Access-Control-Expose-Headers: x-amz-server-side-encryption, x-amz-request-id, x-amz-id-2, ETag
Access-Control-Max-Age: 3000
Vary: Origin, Access-Control-Request-Headers, Access-Control-Request-Method
x-amz-request-id: EBB3F76F73B5EB97
x-amz-id-2: ANlr9OkaQy4UNpzmD4bUP6v5nhshvdorB02Z+Wwp/tVhr0WbtE8HslQK9VNLIiJ65KAGIXXL+LE=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Mon, 11 May 2020 22:38:15 GMT
Connection: close
Server: AmazonS3
Thanks @drixta, this helps in narrow down the problem. Can you also provide the below
Can you put together a minimal reproducible steps with code that I can recreate on my end with my own AWS credentials to be able to debug this?
Amplifiyer
on 12 May 2020
@Amplifiyer I have confirmed my issue is related to Cognito User Groups since I tried the same PUT request with a user that has no Group memberships and it succeeded. So the question is why is the fact a user is a member of a Group getting in the way here?
Here's my CloudTrail redacted log with no error reported this time:
{
"eventVersion": "1.07",
"userIdentity": {
"type": "AssumedRole",
"principalId": "***:CognitoIdentityCredentials",
"arn": "arn:aws:sts::***:assumed-role/amplify-personnel-dan-***-authRole/CognitoIdentityCredentials",
"accountId": "***",
"accessKeyId": "***",
"sessionContext": {
"sessionIssuer": {
"type": "Role",
"principalId": "***",
"arn": "arn:aws:iam::***:role/amplify-personnel-dan-***-authRole",
"accountId": "***",
"userName": "amplify-personnel-dan-***-authRole"
},
"webIdFederationData": {
"federatedProvider": "cognito-identity.amazonaws.com",
"attributes": {
"cognito-identity.amazonaws.com:amr": "[\"authenticated\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***\",\"cognito-idp.us-west-2.amazonaws.com/us-west-2_***:CognitoSignIn:***\"]",
"cognito-identity.amazonaws.com:aud": "us-west-2:***",
"cognito-identity.amazonaws.com:sub": "us-west-2:***"
}
},
"attributes": {
"creationDate": "2020-05-11T22:28:13Z",
"mfaAuthenticated": "false"
}
}
},
"eventTime": "2020-05-11T22:29:36Z",
"eventSource": "s3.amazonaws.com",
"eventName": "PutObject",
"awsRegion": "us-west-2",
"sourceIPAddress": "***",
"userAgent": "[Expo/2.15.4.10229 CFNetwork/1125.2 Darwin/19.4.0]",
"requestParameters": {
"bucketName": "personnel***-dan",
"Host": "personnel***-dan.s3.us-west-2.amazonaws.com",
"key": "public/test-public.txt",
"x-id": "PutObject"
},
"responseElements": null,
"additionalEventData": {
"SignatureVersion": "SigV4",
"CipherSuite": "ECDHE-RSA-AES128-GCM-SHA256",
"bytesTransferredIn": 5.0,
"AuthenticationMethod": "AuthHeader",
"x-amz-id-2": "***",
"bytesTransferredOut": 0.0
},
"requestID": "D40C14C33283A7EB",
"eventID": "2bb90f3c-6109-48e6-8c8d-c2b44f1e9266",
"readOnly": false,
"resources": [
{
"type": "AWS::S3::Object",
"ARN": "arn:aws:s3:::personnel***-dan/public/test-public.txt"
},
{
"accountId": "***",
"type": "AWS::S3::Bucket",
"ARN": "arn:aws:s3:::personnel***-dan"
}
],
"eventType": "AwsApiCall",
"managementEvent": false,
"recipientAccountId": "***",
"eventCategory": "Data"
}
Notice that the userName now ends in authRole?
Regarding recreating, it's very hard to create a simple recreate since I'm recreating using a React Native application connected to an Amplify provisioned backend.
I would suggest you may be able to recreate it yourself by:
- Adding the user you're authenticating with to a Cognito Group first
- Then reauthenticating again after a few minutes has passed
- (To allow the Group membership in Cognito to take effect since I don't know if this happens instantaneously or in an eventually-consistent manner)
- Then reattempting your PUT.
- Hopefully you'll get a 403 this time.
danrivett
on 12 May 2020
@Amplifiyer sure thing
Client side here is my upload code
const name = file.name;
const fileName = `fullsize/${generateRandomString()}_${name}`;
return await Storage.put(fileName, file, {
contentType: file.type,
level: 'protected',
metadata: {
albumid: albumId,
owner: ownerId,
},
});
The file here is just the image file
Here is the Protected Policy of the AuthRole for this bucket
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::duoquserassets161347-dev/protected/${cognito-identity.amazonaws.com:sub}/*"
],
"Effect": "Allow"
}
]
}
If possible I'd be happy to hop on a call with you guys to go through this issue
drixta
on 12 May 2020
Thanks @danrivett and @drixta, this helps a lot. Pinging @akshbhu here again as it does seem to be related to CLI provisioning of the resources. I'll also try to recreate it on my end.
Last but not least, can you folks try to downgrade amplify to V2 (and corresponding aws-amplify-react and aws-amplify-react-native to V3) and see if it helps?
Amplifiyer
on 12 May 2020
Thanks @danrivett and @drixta, this helps a lot. Pinging @akshbhu here again as it does seem to be related to CLI provisioning of the resources. I'll also try to recreate it on my end.
Last but not least, can you folks try to downgrade amplify to V2 (and corresponding aws-amplify-react and aws-amplify-react-native to V3) and see if it helps?
This issue didn't occur until 2 days ago, and I've only been using V3 of aws-amplify so far, so downgrading to V2 wouldn't solve this issue considering the amount of refactoring required for our project.
drixta
on 12 May 2020
Hi @drixta and @danrivett
Can you check S3 cloudformation for group policy for created user group. It should in form of <group-name>-group-policy. This policy enables users in that group to upload files to S3.
For that you have to do amplify update storage and select the operations you want a certain group to have access to and that's when the IAM role associated with the user pool group will be updated.
Let me know if that works for you?
akshbhu
on 12 May 2020
@akshbhu I think the first bug I'm seeing here is that cognito group membership should not even be considered by the Storage component when I do not configure the Storage component on the CLI to use Group authentication but instead just allow 'authorized' (non-guest) users:
? Restrict access by? Auth/Guest Users
? Who should have access: Auth users only
I did not choose 'Both' or Group access, so it should not care that the user happens to be part of a Cognito Group. So that's a bug from my understanding. Users may be part of groups for reasons outside of Amplify or certainly Storage permissions, and that should not break them here.
I would definitely appreciate if you could recreate this yourself as it should be very simple to recreate, and I think it will help in understanding where this first bug lies.
danrivett
on 13 May 2020
@akshbhu Secondly, after fixing the first bug we would actually like to configure Storage with the 'Both' option to allow administrators CRUD access to all the storage, viewers read-only access to all storage, and any other user not part of those groups only access to their own storage using the standard 'private', 'protected', and 'public' access levels the Storage component provides.
Maybe that's not how the Storage component works, and is a misunderstanding on my part, but those expectations are based on how group permissions work with API component and the @Auth annotation. For example my model is annotated as such:
type Worker @model
@auth(rules: [
{ allow: owner },
{ allow: groups, groups: ["Administrators"] },
{ allow: groups, groups: ["Viewers"], operations: [read] }
This provides:
- The owner with CRUD access but only to its own Worker
- Administrators with CRUD access for all
- Viewers with only read access to all (and CRUD access to its own Worker of course)
So I expect the Storage component to work in the same manner.
However even if it is designed to not work like that, if I don't configure any Storage permissions for a particular unrelated Cognito Group, then it should ignore it and just use the other configured Storage permissions, which in my case would be regular owner permissions since the user was neither in the 'Adminstrators' or 'Viewers' group.
This would mean that the user could do a private, protected or public PUT using its owner credentials. However that is not currently the case, it gets a 403 because it's trying to use a group membership that has not been configured in the Storage component and is completely unrelated to providing Storage permissions.
danrivett
on 13 May 2020
I second to @danrivett 's recommendation on user groups shall not be a deciding factor for accessing storage. I'd propose an additional step in amplify storage update:
After selecting Restrict access by? Auth/Guest Users, there should be a question of Do you wish to use the same kind access for all user groups? with default Yes. In the Yes case all user groups access permissions copies authenticated user permissions. In the No case a step by step guide should direct user to select auth permissions for each individual user group.
hello2all
on 13 May 2020
For those who wishes to use an Admin user group which has access to all other users' uploads, I found out a temporary solution by manually modifying storage config files.
We'd modify in total 3 files to grant proper permissions to the Admin user group:
- backend-config.json, under
./amplify/backend - s3-cloudformation-template.json, under
./amplify/backend/storage/YourStorageResourceName - storage-params.json, under
./amplify/backend/storage/YourStorageResourceName
Replace all YourAuthResourceNameHere in the following templates with your authentication id, typically this is the folder name under ./amplify/backend/auth/. And all YourStorageResourceNameHere with your storage resource name, typically this is the folder name under ./amplify/backend/storage.
In backend-config.json, add the following dependsOn section into your storage resource:
"storage": {
"YourStorageResourceNameHere": {
"service": "S3",
"providerPlugin": "awscloudformation",
"dependsOn": [
{
"category": "auth",
"resourceName": "YourAuthResourceNameHere",
"attributes": [
"UserPoolId"
]
},
{
"category": "auth",
"resourceName": "userPoolGroups",
"attributes": [
"AdminGroupRole"
]
}
]
}
},
In s3-cloudformation-template.json, append the following to Resources:
"AdminGroupPolicy": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "Admin-group-s3-policy",
"Roles": [
{
"Fn::Join": [
"",
[
{
"Ref": "authYourAuthResourceNameHereUserPoolId"
},
"-AdminGroupRole"
]
]
}
],
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:DeleteObject"
],
"Resource": [
{
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "S3Bucket"
},
"/*"
]
]
}
]
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": [
{
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "S3Bucket"
}
]
]
}
]
}
]
}
}
}
And finally, create a new file storage-params.json under ./amplify/backend/storage/YourStorageResourceName:
{
"groupPermissionMap": {
"Admin": [
"create/update",
"read",
"delete"
]
}
}
I seem to be experiencing the same thing with aws-amplify: 3.0.11, except I am using google federated logins and only experience a 403 when using a google account
I've had a look at the logs and it seems that identityId: undefined
chadfaurie
on 15 May 2020
Hi @chad-codeworkshop
I think your issue is a bit different from the one above , you can open an issue here so that we can easily track the progress.
akshbhu
on 15 May 2020
@akshbhu could we get an update on this, as I spent a lot of time identifying at least one if not 2 bugs about 10 days ago and written up above, and I would really appreciate if yourself or someone on your team could at least validate the first bug I mentioned (that if a user happens to be a member of a Cognito Group then even if Group permissions aren't configured for Storage access it still fails).
This is currently causing us pain because we have had to temporarily disable a post-sign-up lambda automatically assigning new users to appropriate groups for other reasons unrelated to Storage permissions.
Thanks.
danrivett
on 23 May 2020
PS - I didn't attempt what you suggested since the first bug scenario I mentioned does not use Groups to configure permissions for Storage, and in the second bug I mentioned after that, I expect that Group permissions would work similarly to API group permissions where owner permissions would still be available even if the user is not part of a group with additional permissions, or those additional permissions are more restrictive.
Example: If a user is part of a Viewers group which provides read only access, I would expect that means they have read only access to data from all users and not just their own, but they would not give up write access for their own data.
Let me know if I've misunderstood that, but even so, this is a separate concern from the first bug I raised which is recreatable even when group permissions are not configured for Storage.
Thanks.
danrivett
on 23 May 2020
Hi @danrivett
Thank you for your patience and apologies for the delay. We are on top of this and will circle back to you with updates.
akshbhu
on 25 May 2020
Hi @danrivett
I think the first bug I'm seeing here is that cognito group membership should not even be considered by the Storage component when I do not configure the Storage component on the CLI to use Group authentication but instead just allow 'authorized' (non-guest) users:
It doesn't work like this. If you have added user to the cognito user group and you must set IAM policy on the bucket to peform crud actions on S3.You can do this amplify update storage.
If a user is part of a group , it uses groupRole rather than AuthRole.
The request is failing because the storage doesn't have permissions assigned for the group to perform actions on it.
akshbhu
on 28 May 2020
If a user is part of a group , it uses groupRole rather than AuthRole.
Is this somewhere in the documentation? (If not, could it be added?)
I also interpreted the CLI prompts to imply that setting up "Authorized users" would include all authorized users, regardless of Group - especially when I did not set up Group authentication. It is not clear from the prompts that a group role will replace an auth role.
taylornewton
on 28 May 2020
Hi @taylornewton
Thanks for reporting this. We have taken up this task in our queue to improve the documentation so that this confusion can be avoided.
Let me know if anyone is still stuck on this issue.
akshbhu
on 29 May 2020
It doesn't work like this. If you have added user to the cognito user group and you must set IAM policy on the bucket to peform crud actions on S3.
@akshbhu I feel like this is a pretty disappointing design limitation, and I would really ask if it has to remain like this, or if this can be improved.
The reason being is Amplify isn't necessarily the centre of the universe when it comes to user permissions and so users can be added to groups for other reasons outside of Amplify permissions, and certainly outside of Amplify Storage permissions, and now if that happens it will break Amplify Storage since those groups won't have been configured for storage access because they aren't being used for storage access.
I'm not sure how much work it would be to update Amplify Storage to not use group permissions in any circumstance if configured on the CLI to just use authorized user permissions, but I would strongly encourage assessing and considering making that change, because as it currently stands it's very unintuitive and problematic for the fact that Cognito User Pools aren't necessarily exclusively accessed by Amplify.
danrivett
on 29 May 2020
@danrivett Sorry for the confusion here. To be clear, is the ask out here that if a user is in a Cognito userpool group - then there should be an option in the CLI for the group to just inherit all the auth permissions?
We could assign the auth role to the groups - but the nuance out there would be that then you won't be able to have different rule-set for that group and it would always be tied to the auth role permissions - which is fine for your use-case I believe. This is a great potential advanced option which we could include in the CLI and I would urge you to open up a feature-request in the CLI repo - https://github.com/aws-amplify/amplify-cli for it and we'll look into prioritizing it.
Our design on this was based on customer feedback last year around different rules & grouping - but I understand your request here and would take it back to the team to discuss further.
kaustavghosh06
on 29 May 2020
@kaustavghosh06 I think the main confusion and issue I have is the CLI already asks the following question:
? Restrict access by? (Use arrow keys)
❯ Auth/Guest Users
Individual Groups
Both
And the options are for either restrict by Auth/Guest Users, or by Groups, or by both.
I would expect that if just Auth/Guests were selected, it would be immaterial what group membership a user is in because I explicitly didn't select the Groups option in the CLI config.
Let me know if I'm misunderstanding something there.
As a follow-on and _separate_ issue, the interesting question then becomes what does "Both" mean for that question in that context?
I had assumed it would allow access to be configured both at the user level and the group level and so a user would have a super set of privileges based on owner privileges and group membership privileges - in the same way the API component handles this using the @auth annotation (see my previous comment) but I'm not sure that's true, so I'm trying to understand what Both means in this case.
But as I said, this is a secondary issue - the primary pain point I have is when I configure Storage to restrict access by just Auth/Guest users, I really do not expect Cognito User Group membership to have any effect since they could be part of groups for reasons outside of Amplify configuration.
Let me know if any of that doesn't make sense, as I've only just started using Amplify Storage in the last 2 weeks and so it's still unfamiliar to me. Thank you for reviewing.
danrivett
on 29 May 2020
@danrivett Ah, I see your confusion here. I'll dive into some details out here for explaining this behavior.
TLDR - Groups do not inherit permissions from the auth permissions/role. Groups have their own set of permissions which are completely independent of the auth and guest users permissions.
Explanation: When we add a Cognito user pool group using the CLI - each group gets an IAM role attached to it with the least set of permissions (empty to begin with) which is independent of the role assumed by the auth and guests users. Unless you explicitly add permissions for that group for a given category (using the amplify update category flow) the permissions don't get get propagated to the group permissions.
One of the things we could expose in the CLI is the ability for customers to inherit the permissions from the auth permissions/role and add it to the group role - as I've proposed out here as well - https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-635728670
The downside to this is that now the groups will always have all the permissions tied to the auth user - which some customers might not want - hence I would consider this as an advanced option and not the default.
Regarding the options out here:
? Restrict access by? (Use arrow keys)
❯ Auth/Guest Users
Individual Groups
Both
If you choose Auth/Guest users you'll be presented with options to select from the CRUD permissions for the Auth and Guest users only - the Group roles would still have no permissions. If you select "Individual Groups" - you'll be presented with options to select from the CRUD permissions for each individual group. So you can provide say "Read" permissions to one of the groups and just "Create" permissions to another group. When you select "Both" - the CLI walks you through configuring the permissions for Auth, Guest Users as well as Individual Groups (so it's basically a combination of the first and second option).
Regarding the primary pain point which you've mentioned out here - if a user gets added to a group outside of Amplify and if you want all the permissions to be inherited from the auth user- then the group IAM role has to also be configured to have permissions to perform all the actions of auth users or if you want to add any custom permissions for the group - you can use the custom policy functionality which we expose. You can read more about it here - https://docs.amplify.aws/cli/auth/groups#group-access-controls
I also have an explanation to your comment out here - https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-628014869
However even if it is designed to not work like that, if I don't configure any Storage permissions for a particular unrelated Cognito Group, then it should ignore it and just use the other configured Storage permissions, which in my case would be regular owner permissions since the user was neither in the 'Adminstrators' or 'Viewers' group
The reason for this again is because Storage category uses IAM roles for its permissions and if the corresponding group role doesn't have permissions configured to inherit permissions from the auth role - then the users in the group wouldn't be able to perform the operations which the auth users are able to perform. Exposing the advanced option which I've proposed above - would solve for this issue as well.
Also regarding comparison with @auth, let's take your example in the comment https://github.com/aws-amplify/amplify-js/issues/5729#issuecomment-628014869 and see how you can achieve this with storage category permissions
@Auth annotation. For example my model is annotated as such:
type Worker @model
@auth(rules: [
{ allow: owner },
{ allow: groups, groups: ["Administrators"] },
{ allow: groups, groups: ["Viewers"], operations: [read] }
This provides:
- The owner with CRUD access but only to its own Worker
If the user uploads a file using the private level of protection from the client, the the user has options to perform CRUD given CRUD permissions are configured for the auth users in the CLI flow.
Administrators with CRUD access for all
It's not currently possible for a auth users or any groups to perform CRUD on all the files i.e all files stored in protected/, private/ and public/. Only the files stored using public protection can be read/updated/deleted by auth or groups depending on the CRUD permissions configured.
Viewers with only read access to all (and CRUD access to its own Worker of course)
In this case the viewers group can be configured to have only "Read" operations and you can use the protected/ level to upload the files from the client
As you see above - the permissions for the files stored using the Storage category rely both on the client-level protections(public/private/protected) as well as the backend configurations (CRUD operations) set by the CLI unlike the API level protections which is strictly backend configurations - making it a bit difficult to draw a 1:1 corollary. At the same time, I would say our team is discussing this more in depth to see if we can make this flow simpler and more intuitive and similar to the api flows.
kaustavghosh06
on 29 May 2020
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
![stale[bot] picture](https://avatars.githubusercontent.com/in/1724?v=4&s=40)
stale[bot]
on 28 Jun 2020
@kaustavghosh06 thanks for your detailed reply a few weeks ago. I did read it, but needed to digest it more and then 2 other projects I'm involved in decided to take my time away, so only just circling back to this issue.
I've just re-read your comments. The last part about the comparison between Auth permissions vs Storage permissions makes sense to me as they use different constructs with the latter using private/protected/public access controls. That being said I do think it would be useful to support cross-cutting permissions so that CSRs, moderators, administrators etc can have access permissions across users that other users wouldn't have - e.g. they could write to other users protected files and read their private files etc.
But it's the first point you explained in detail that still confuses me, and I apologise for that.
The thing that I am struggling to understand is if I've configured Amplify Storage to not restrict access by Cognito User Groups, e.g I select option 1:
? Restrict access by? (Use arrow keys)
❯ Auth/Guest Users
Individual Groups
Both
Then I don't understand why a user's group membership is even looked at in regard to Amplify Storage determining permissions. As I said before other actors in the system completely separate from Amplify Storage assign groups to users which have no relevance to Amplify Storage and therefore I wouldn't want to merge those concerns with having to add Amplify Storage permissions if another system actor assigns a user to a group.
An an example, consider the scenario where users are not in any groups and I choose to restrict Amplify Storage access just by Auth/Guest Users, then if another backend system decides to assign users to a particular group for their own backend process role management, I'm struggling to understand why Amplify Storage should break in that case because those Groups don't happen to have been assigned Amplify Storage permissions. This is the scenario that happened to us.
I would think in my case as I explicitly didn't select "Individual Groups" for Amplify Storage access restrictions it should have no effect. I know you explained this before above, but please could you try again as I do think this is either a bug or a limitation that seems counter intuitive to me at least.
Many thanks.
danrivett
on 5 Jul 2020
Related to issues discussed in this thread, I want to explain a situation in our web app that is causing a 403 when using Amplify to upload a file to S3 for an authenticated user that belongs to a Cognito group that has a policy attached to it granting all access to the specific S3 bucket and folders.
I have designed a Serverless app using the guidelines defined at [http://serverless-stack.com]. This guide does not use the Amplify CLI for creating/generating/configuring the serverless architecture, but a similar architecture is manually defined/deployed and used. The client app is built using React and the amplify libraries are installed and included using npm. Amplify Auth, Storage, and API are all configured using Amplify.configure() as detailed on this page [https://serverless-stack.com/chapters/configure-aws-amplify.html].
We use Cognito User Pools, User Groups, and Identity Pools to manage all of our app's users. When a new user signs up for our service using the web app, the new user is created in Cognito with the default policy we have defined for new users. New users then have the option of creating a new Account or joining an existing Account. Accounts represent an organization or group and designed as a workspace for one or more users that need to share a set of resources such as images and other things associated with that account. When an account is created, a corresponding Cognito Group is created with the same name, a new Policy is defined and attached to the group allowing for access to certain S3 folders that will contain files accessible only by members of the group, and the Cognito user is then added to the Cognito group. Of course, any Cognitio user can belong to more than a single account (and therefore more than a single Cognito group).
When an authenticated user is logged in, they have no issues using Storage.vault.put() to add files to their own private folder in S3. However, when we attempt to use Storage.put() to add files to the specific group/account folder defined in the access policy in the Cognito group they belong to, then receive a 403 Access Denied.
I have checked to make sure the PUT storage request is generated properly and observed the request/response to make sure we are attempting to properly access the right S3 bucket and path (all good). So I am left scratching my head as to why the Cognito Group policy doesn't appear to be in effect for the given Cognito user who belongs to that Cognito group when attempting to access the specific resource in S3 allocated for that group when using Amplify.
Any suggestions?
Thank you.
GetJoot
on 21 Jul 2020
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
stale[bot]
on 22 Aug 2020
This issue has been automatically closed because of inactivity. Please open a new issue if are still encountering problems.
stale[bot]
on 29 Aug 2020
If a user is part of a group , it uses groupRole rather than AuthRole.
Is this somewhere in the documentation? (If not, could it be added?)
I also interpreted the CLI prompts to imply that setting up "Authorized users" would include all authorized users, regardless of Group - especially when I did not set up Group authentication. It is not clear from the prompts that a group role will replace an auth role.
@taylornewton is there a way to retrospectively change this in your amplify S3 config?
matt-d-webb
on 10 Sep 2020
If a user is part of a group , it uses groupRole rather than AuthRole.
Is this somewhere in the documentation? (If not, could it be added?)
I also interpreted the CLI prompts to imply that setting up "Authorized users" would include all authorized users, regardless of Group - especially when I did not set up Group authentication. It is not clear from the prompts that a group role will replace an auth role.@taylornewton is there a way to retrospectively change this in your amplify S3 config?
@matt-d-webb If I remember correctly, I manually updated the permissions in S3 to allow the Admin group role to have permissions to the bucket. I did not do anything with the Amplify config.
taylornewton
on 10 Sep 2020
@danrivett Ah, I see your confusion here. I'll dive into some details out here for explaining this behavior.
TLDR - Groups do not inherit permissions from the auth permissions/role. Groups have their own set of permissions which are completely independent of the auth and guest users permissions.
Explanation: When we add a Cognito user pool group using the CLI - each group gets an IAM role attached to it with the least set of permissions (empty to begin with) which is independent of the role assumed by the auth and guests users. Unless you explicitly add permissions for that group for a given category (using the amplify update category flow) the permissions don't get get propagated to the group permissions.
One of the things we could expose in the CLI is the ability for customers to inherit the permissions from the auth permissions/role and add it to the group role - as I've proposed out here as well - #5729 (comment)
The downside to this is that now the groups will always have all the permissions tied to the auth user - which some customers might not want - hence I would consider this as an advanced option and not the default.Regarding the options out here:
? Restrict access by? (Use arrow keys)
❯ Auth/Guest Users
Individual Groups
BothIf you choose Auth/Guest users you'll be presented with options to select from the CRUD permissions for the Auth and Guest users only - the Group roles would still have no permissions. If you select "Individual Groups" - you'll be presented with options to select from the CRUD permissions for each individual group. So you can provide say "Read" permissions to one of the groups and just "Create" permissions to another group. When you select "Both" - the CLI walks you through configuring the permissions for Auth, Guest Users as well as Individual Groups (so it's basically a combination of the first and second option).
Regarding the primary pain point which you've mentioned out here - if a user gets added to a group outside of Amplify and if you want all the permissions to be inherited from the auth user- then the group IAM role has to also be configured to have permissions to perform all the actions of auth users or if you want to add any custom permissions for the group - you can use the custom policy functionality which we expose. You can read more about it here - https://docs.amplify.aws/cli/auth/groups#group-access-controls
I also have an explanation to your comment out here - #5729 (comment)
However even if it is designed to not work like that, if I don't configure any Storage permissions for a particular unrelated Cognito Group, then it should ignore it and just use the other configured Storage permissions, which in my case would be regular owner permissions since the user was neither in the 'Adminstrators' or 'Viewers' groupThe reason for this again is because Storage category uses IAM roles for its permissions and if the corresponding group role doesn't have permissions configured to inherit permissions from the auth role - then the users in the group wouldn't be able to perform the operations which the auth users are able to perform. Exposing the advanced option which I've proposed above - would solve for this issue as well.
Also regarding comparison with @auth, let's take your example in the comment #5729 (comment) and see how you can achieve this with storage category permissions
@Authannotation. For example my model is annotated as such:type Worker @model @auth(rules: [ { allow: owner }, { allow: groups, groups: ["Administrators"] }, { allow: groups, groups: ["Viewers"], operations: [read] }This provides:
- The owner with CRUD access but only to its own Worker
If the user uploads a file using the private level of protection from the client, the the user has options to perform CRUD given CRUD permissions are configured for the auth users in the CLI flow.
- Administrators with CRUD access for all
It's not currently possible for a auth users or any groups to perform CRUD on all the files i.e all files stored in protected/, private/ and public/. Only the files stored using public protection can be read/updated/deleted by auth or groups depending on the CRUD permissions configured.
- Viewers with only read access to all (and CRUD access to its own Worker of course)
In this case the viewers group can be configured to have only "Read" operations and you can use the protected/ level to upload the files from the clientAs you see above - the permissions for the files stored using the Storage category rely both on the client-level protections(public/private/protected) as well as the backend configurations (CRUD operations) set by the CLI unlike the API level protections which is strictly backend configurations - making it a bit difficult to draw a 1:1 corollary. At the same time, I would say our team is discussing this more in depth to see if we can make this flow simpler and more intuitive and similar to the api flows.
Thanks, I think you need to make that clear in the documentation and in the cli that Group permissions will be required is users are part of a Group - and you can tell, particularly if and automatic trigger has been set to include new users to a group.
duncangroenewald
on 9 Oct 2020
We are trying to access protected assets(Assets created by someone else). We are getting
NoSuchKey: The specified key does not exist. We were able to access the same key using aws s3api cli.
Anyway to resolve this issue? We gave access to S3 bucket to the cognito user pool.
abhilashdonepudi
on 14 Oct 2020
Related issues
leantide
·
3Comments
ldgarcia
·
3Comments
ddemoll
·
3Comments
shinnapatthesix
·
3Comments
rygo6
·
3Comments
Most helpful comment
I am getting a 403 when using Storage.put as well, which started when I upgraded to latest versions of amplify-js...