Describe the bug
I am getting a 401 error when using a type with @auth { allow: owner } and trying to create a new record for that type, via a mutation, by using the Angular API service auto-generated code.
To Reproduce
Here is the entire schema that produced this error (the relevant type to look at is type PrivateTrack as the error is caused when calling the generated angular service function CreatePrivateTrack, but I am providing the entire schema for reference):
type Search
@model(subscriptions: null)
@auth(
rules: [
{ allow: public, provider: iam, operations: [create, update, read] }
]
)
@key(fields: ["normalizedQuery"])
{
normalizedQuery: String!
titles: String!
nTimes: Float!
nTimesInLast24h: Int!
updatedAt: AWSTimestamp!
titlesUpdatedAt: AWSTimestamp!
}
type PrivateTrack
@model
@auth(rules: [{ allow: owner } ])
{
id: ID!
path: String!
}
type PublicTrack
@model
@auth(
rules: [
{ allow: owner },
{ allow: public, provider: iam, operations: [read] }
]
)
{
id: ID!
path: String!
}
Steps to reproduce the behavior:
bash
amplify delete
amplify init
amplify add api (adding IAM and Cognito user pools here)
amplify push
Then I always needed a manual step -- to check the checkbox in Cognito, for the created Identity pool, that allows it to have UnAuthorized access. I always have to do that manual step...
Here's a transcript of the above sequence of operations (before the manual step described above):
```bash
(2037) dorontal@thing5: amplify init
...
✔ Successfully created initial AWS cloud resources for deployments.
✔ Initialized provider successfully.
Initialized your environment successfully.
...
(2038) dorontal@thing5: amplify add api
? Please select from one of the below mentioned services: GraphQL
? Provide API name: tracktunes
? Choose the default authorization type for the API IAM
? Do you want to configure advanced settings for the GraphQL API Yes, I want to make some additional changes.
? Configure additional auth types? Yes
? Choose the additional authorization types you want to configure for the API Amazon Cognito User Pool
Cognito UserPool configuration
Using service: Cognito, provided by: awscloudformation
The current configured provider is Amazon Cognito.
Do you want to use the default authentication and security configuration? Default configuration
Warning: you will not be able to edit these selections.
How do you want users to be able to sign in? Email
Do you want to configure advanced settings? Yes, I want to make some additional changes.
Warning: you will not be able to edit these selections.
What attributes are required for signing up? Email
Do you want to enable any of the following capabilities? Email Verification Link with Redirect
? Enter the URL that your users will be redirected to upon account confirmation: https://explore.tracktunes.org/#/app/registration-redirect
? Enter the subject for your custom account confirmation email: Welcome to Tracktunes music collaboration!
? Enter the body text for your custom account confirmation email (this will appear before the link URL): Please click the link below to confirm (and complete) your registration:
Succesfully added the Lambda function locally
? Do you want to edit your verification-link function now? No
Successfully added auth resource
? Configure conflict detection? No
? Do you have an annotated GraphQL schema? Yes
? Provide your schema file path: schema.graphql
GraphQL schema compiled successfully.
...
(2039) dorontal@thing5: amplify push
...
# everything succeeded in the push, without a single complaint
```
CreatePrivateTrack, I get a 401 "Unauthorized Error" response.Expected behavior
I expected not to get a 401 error, since this is the only user and since the user is enabled and confirmed and sincce { allow: owner } was used in the schema, this user should be permitted to create a PrivateTrack record via the above function call.
Code Snippet
When I try to use the code to create a new entry for a new track, via the mutation call CreatePrivateTrack I get a 401 error. Here's the function call that generates this 401 error:
this.awsAPI.CreatePrivateTrack({ path: 'test-path' }).then((res: any) => {
console.log('successfully created a private track!', res);
});
And here is the console error in the browser:
core.js:6014 ERROR Error: Uncaught (in promise): Object: {"data":{},"errors":[{"message":"Request failed with status code 401"}]}
In other words, getting a 401 error, even though I did verify in the Cognito Console that this user, with the username that was supplied in the function call matches the username and that this user is enabled and that this user's account status says 'CONFIRMED'.
Screenshots
If applicable, add screenshots to help explain your problem.
What is Configured?
If applicable, please provide what is configured for Amplify CLI:
See above.
Which resources do you have configured?
aws-exports file:const awsmobile = {
"aws_project_region": "us-east-1",
"aws_cognito_identity_pool_id": "us-east-1:5050e9e0-ba3e-42af-9ecc-5b164935d0e8",
"aws_cognito_region": "us-east-1",
"aws_user_pools_id": "us-east-1_O678oxx1g",
"aws_user_pools_web_client_id": "7pu8d730imn7bi114fsep1s5q7",
"oauth": {},
"aws_appsync_graphqlEndpoint": "https://r5hrmpqtbvephaic43zl2rm5uu.appsync-api.us-east-1.amazonaws.com/graphql",
"aws_appsync_region": "us-east-1",
"aws_appsync_authenticationType": "AWS_IAM"
};
If applicable, please provide your manual configuration example:
No manual configuration was used.
If applicable, provide more configuration data, for example for Amazon Cognito, run aws cognito-idp describe-user-pool --user-pool-id us-west-2_xxxxxx (Be sure to remove any sensitive data)
I ran the above command, it is not useful here.
Environment
(2041) dorontal@thing5: npx envinfo --system --binaries --browsers --npmPackages
npx: installed 1 in 0.983s
System:
OS: Linux 4.19 Debian GNU/Linux 10 (buster) 10 (buster)
CPU: (6) x64 Intel(R) Core(TM) i5-8400 CPU @ 2.80GHz
Memory: 4.11 GB / 11.56 GB
Container: Yes
Shell: 5.0.3 - /bin/bash
Binaries:
Node: 12.0.0 - /usr/local/bin/node
npm: 6.14.4 - /usr/local/bin/npm
Browsers:
Chrome: 81.0.4044.122
Firefox: 68.7.0esr
npmPackages:
@angular-devkit/build-angular: ~0.803.20 => 0.803.26
@angular/cli: ~8.3.23 => 8.3.26
@angular/common: ~8.2.14 => 8.2.14
@angular/compiler: ~8.2.14 => 8.2.14
@angular/compiler-cli: ~8.2.14 => 8.2.14
@angular/core: ~8.2.14 => 8.2.14
@angular/forms: ~8.2.14 => 8.2.14
@angular/language-service: ~8.2.14 => 8.2.14
@angular/platform-browser: ~8.2.14 => 8.2.14
@angular/platform-browser-dynamic: ~8.2.14 => 8.2.14
@angular/router: ~8.2.14 => 8.2.14
@aws-amplify/api: ^3.1.7 => 3.1.7
@aws-amplify/pubsub: ^3.0.8 => 3.0.8
@ionic-native/core: ^5.0.7 => 5.24.0
@ionic-native/splash-screen: ^5.0.0 => 5.24.0
@ionic-native/status-bar: ^5.0.0 => 5.24.0
@ionic/angular: ^5.0.0 => 5.0.7
@ionic/angular-toolkit: ^2.1.1 => 2.2.0
@types/jasmine: ^3.5.6 => 3.5.10
@types/jasminewd2: ^2.0.8 => 2.0.8
@types/node: ^13.7.4 => 13.13.2
aws-amplify: ^3.0.8 => 3.0.8
aws-amplify-angular: ^5.0.8 => 5.0.8
codelyzer: ^5.0.0 => 5.2.2
core-js: ^2.5.4 => 2.6.11
eslint: ^6.8.0 => 6.8.0
jasmine-core: ~3.4.0 => 3.4.0
jasmine-spec-reporter: ~4.2.1 => 4.2.1
karma: ^4.4.1 => 4.4.1
karma-chrome-launcher: ~2.2.0 => 2.2.0
karma-coverage-istanbul-reporter: ~2.0.1 => 2.0.6
karma-jasmine: ~2.0.1 => 2.0.1
karma-jasmine-html-reporter: ^1.4.0 => 1.5.3
protractor: ~5.4.0 => 5.4.4
rxjs: ~6.5.1 => 6.5.5
ts-node: ~7.0.0 => 7.0.1
tslib: ^1.9.0 => 1.11.1
tslint: ~5.15.0 => 5.15.0
typescript: ~3.4.3 => 3.4.5
zone.js: ~0.9.1 => 0.9.1
npmGlobalPackages:
@aws-amplify/cli: 4.18.1
@ionic/cli: 6.6.0
npm: 6.14.4
Smartphone (please complete the following information):
Not a smartphone - running in Chromium browser on desktop with the following version information Version 80.0.3987.162 (Developer Build) built on Debian 10.3, running on Debian 10.3 (64-bit)
I thought that perhaps the order in which we set up the authorization type matters, so I tried the above (without changing any of the code) amplify add api command differently: by choosing as the default authorization type (the first one specified) to be Cognito (in the above it's IAM) instead of IAM. Then I chose IAM for the second authorization type. Of course in both cases above and this one, I had to go in and set up the IAM identity pool to allow unauthorized access, manually, via the Cognito console. I did that.
All I did here is change the order of setting up the authorization type, but unfortunately, that did not let me get past the first stage of my app, which does a search and uses the IAM unauthorized access (a query on the Search type, which works when the default authorization type is IAM but does not work when the default authorization type is Cognito) - that produced an error "No current user".
The schema used here needs both IAM and Cognito -- it seems that those two do not work well together, regardless of which one of those two you choose as the default. If you choose IAM as the default, then you get Cognito user pool unauthorized 401 errors (as the above) and if you choose Cognito as the default, you get "no such user" when trying to access a type (Search) that does not need the Cognito user pool at all.
Ideally, it should not matter which authorization type is the default - right? Ideally, the above schema should work regardless of the order in which you set up authorization types via the amplify cli, right? I'd appreciate a pointer if I've missed something.
This is a blocking issue. I cannot continue to develop with Amplify until this is resolved. I hope I've missed something; I'm not sure if it's a bug on your end or not. Any pointers would be appreciated as we are seriously blocked by this now! Thanks!
See Issue aws-amplify/amplify-js#5540 for a more concise description of this problem.
@dorontal
When using owner authorization for a type you will have to use AMAZON_COGNITO_USER_POOLS as the auth_mode for that operation.
I closed the other issue, so keep communicating on this issue
@lorzafe - thanks for the response, but:
I am. I am using AMAZON_COGNITO_USER_POOLS for that operation.
The CreatePrivateTable is the operation (mutation).
The user (owner) is logged in, successfully, at the time that I call that mutation. Here is the console.log() output of the CognitoUser that's logged in that shows user is logged in successfully:
Auth: Signed in!
CognitoUser {username: "701e13a7-5dce-403e-a068-ff54218a636b", pool: CognitoUserPool, Session: null, client: Client, signInUserSession: CognitoUserSession, …}
username: "701e13a7-5dce-403e-a068-ff54218a636b"
pool: CognitoUserPool {userPoolId: "us-east-1_0WpvvX9eb", clientId: "dqclejttpbln500s1a4so4lg4", client: Client, advancedSecurityDataCollectionFlag: true, storage: Storage}
Session: null
client: Client {endpoint: "https://cognito-idp.us-east-1.amazonaws.com/"}
signInUserSession: CognitoUserSession {idToken: CognitoIdToken, refreshToken: CognitoRefreshToken, accessToken: CognitoAccessToken, clockDrift: 1}
authenticationFlowType: "USER_SRP_AUTH"
storage: Storage {CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4.701e13a7-5dce-403e-a068-ff54218a636b.refreshToken: "eyJjdHkiOiJKV1QiLCJlbmMiOiJBMjU2R0NNIiwiYWxnIjoiUl…lMo-DuIAhQKsYpsTXaeorWhSGY.unlDqP7WHjn0tXoukYmI7A", amplify-signin-with-hostedUI: "false", CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4.701e13a7-5dce-403e-a068-ff54218a636b.userData: "{"UserAttributes":[{"Name":"sub","Value":"701e13a7…Username":"701e13a7-5dce-403e-a068-ff54218a636b"}", CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4.701e13a7-5dce-403e-a068-ff54218a636b.idToken: "eyJraWQiOiJkWmkyWFlianBybno1SzVSSzJGcmoxQ1FDSm0zSk…wNULHzLkl02Nk3nPAl9_pERrjZjSg3QTF63eos58h6E971kpA", CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4.701e13a7-5dce-403e-a068-ff54218a636b.accessToken: "eyJraWQiOiJzT0N3TkJTNEt0eU1oNUczWk5yVjRUS1lqMW94Mk…X6HNCcwNYaSmjQKGJwNddLiUZreO5SWs6GrrHDcBu2cD2yh-w", …}
keyPrefix: "CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4"
userDataKey: "CognitoIdentityServiceProvider.dqclejttpbln500s1a4so4lg4.701e13a7-5dce-403e-a068-ff54218a636b.userData"
deviceKey: null
attributes: {sub: "701e13a7-5dce-403e-a068-ff54218a636b", email_verified: true, custom:bytesUsed: "0", custom:bytesQuota: "2000000", email: "[email protected]"}
preferredMFA: "NOMFA"
__proto__: Object
But I still get a 401 error when running the mutation on a type that has @auth(rules: [{ allow: owner } ]) as the only @auth directive attached to it, as you can see in the above schema (the type is PrivateTrack.
Did I miss something?
@dorontal can you check on your network request (on dev tools) and check Authorization header if it has a token there?
@elorzafe - okay, just did, and it does seem to have that header (though it is all lowercase: 'authorization'). There were several back and forth POST-OPTIONS-POST... in the network tab, but here is the one that was the last one and came back with an error:
Request URL: https://cl7elcp5znajhdxfaklaotatn4.appsync-api.us-east-1.amazonaws.com/graphql
Request Method: POST
Status Code: 401
Remote Address: 99.84.32.33:443
Referrer Policy: no-referrer-when-downgrade
access-control-allow-origin: *
access-control-expose-headers: x-amzn-RequestId,x-amzn-ErrorType,x-amz-user-agent,x-amzn-ErrorMessage,Date,x-amz-schema-version
content-length: 105
content-type: application/json;charset=UTF-8
date: Fri, 24 Apr 2020 20:58:00 GMT
status: 401
via: 1.1 a0845df335efaa79f84feeb1d7861c1a.cloudfront.net (CloudFront)
x-amz-cf-id: s-6MeHen7rw01Qx3zlKqokp4U7Gz7mA7fGdbHvVSS5XkiF1TITSUyw==
x-amz-cf-pop: EWR52-C4
x-amzn-errortype: UnauthorizedException
x-amzn-requestid: 0c762e3e-3aa7-4ed6-9c3d-fb967aaf390c
x-cache: Error from cloudfront
:authority: cl7elcp5znajhdxfaklaotatn4.appsync-api.us-east-1.amazonaws.com
:method: POST
:path: /graphql
:scheme: https
accept: application/json, text/plain, */*
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
authorization: AWS4-HMAC-SHA256 Credential=ASIA554YJA7WFKYHME72/20200424/us-east-1/appsync/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token;x-amz-user-agent, Signature=e59dead648fa05b654e029819829fedeaebcee88671d3a83092191aed05ef666
cache-control: no-cache
content-length: 305
content-type: application/json; charset=UTF-8
origin: http://localhost:4200
pragma: no-cache
referer: http://localhost:4200/
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: cross-site
user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 7_1_2 like Mac OS X) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 Mobile/11D257 Safari/9537.53
x-amz-date: 20200424T205759Z
x-amz-security-token: IQoJb3JpZ2luX2VjEFUaCXVzLWVhc3QtMSJGMEQCIAZtWShnG18F/nsfnp0ZkB3vkjO0lISCns22/1iEeTjIAiAEKwEP68gip1w9FtPzx0shCHdWoP9cycGu+HNcQIGXHCr+Awh+EAEaDDk1NzU2MDE5NTA1MiIMPkl1bk6lfbnBZU0CKtsDwJgFX4B6gc0vV9CsEeElKhX20Dm9558uUzknzELLrpX5kZwM9aebF7hMEDYPFhGwa7fSgGv6+v94QVZc6Ya7VcwoUVC44ifPuuPCHP4LZ9JOiTXO9tzUGGkWHLdpiZWDu9HGGdTqHSPncyCMDM2s2UzrPZxWoT3YB8MZM/oVcNwYzXWQK8f7PsbQI7CHUvLFtMSMkJPBVb5RtuAvi4Y6bQXqAcKpKiu6prFrGws2qb+jipmNTyIKISE+5u9mEz8JYi/CqX/WuLc0ZtbGawBx/BSjOHB5LjoRUzs9U96CetoVyhXZm7J0MIw4jTnB0qFiarswhjvZhTsbg2fvVCAQKux23F92UUb7BLnM1bw0T+n0FjNwpyTkv7KMOBDpJKba6rLTD6FiDo1dTkrtMXePMIpa6N5m7Rzr3SOMOumJ6nbN2XRolggcmQ0PRa0XKLp5VCRCVaHgyHW2wBzPUVhKDhwrpoI9oDIXNdHtQE1zhLqWf0CH1IzmKRb6fSdBUXqHlKeAxS8yengX3Zi7zsKpLAy310/T8nSofud7lzFR0XfIXscWmRRwDBMWVXqFEguQvwunMh4/j86Do1acrD+w+O8V9OBAgBt21R15UnE7W6/tD96WL82NILAgfjDXpY31BTrMAsM3X/YfJtIteqnVw6Iv3jQ6PuQxQW6TpmZITMqLUwilIbIRRAOIF/gsLtCboXAHR4X6C3CN1uAMQIXU+/AlA3l4ApQEXuRROyxvMhxlgf894E3PiWVLo4AK/34nsKiVvc1j7UAUvR0V/TXhSP7j4xwX3MDUlDco7o26gka3mOsrNMZzY9cuPw15p8QWolPbDvhVokKS1reghKOyod7k4JOScMSc9cre4sWML9p39gKhrd/mvWI9jpRbl/d8egyNBS5NRi/YfFIk2/HS8hkCJ3TVoFxNtDVCEDjes68TDDt96eLwDyAteveDBqRkGgZNVjPe0OgiBx52A5qKymLtV3+/cbbe2U1O+nYepIVlsaXAVXWgvVYYxmDrE/7oc2Cp+YcThESMQx4Ca1v3a3izNNCRJunvc/I4fiRgb2IRz9B32aK/4K8dE051WUtR
x-amz-user-agent: aws-amplify/3.2.3 js
{,…}
query: "mutation CreatePrivateTrack($input: CreatePrivateTrackInput!, $condition: ModelPrivateTrackConditionInput) {↵ createPrivateTrack(input: $input, condition: $condition) {↵ __typename↵ id↵ path↵ owner↵ }↵}↵"
variables: {input: {path: "test-dir/mp_20190927_seg_21_64.mp3"}}
@dorontal is not sending the token, is sending a sigV4 signing instead, that is why you are getting the 401.
Maybe is an issue with AmplifyService. Can you share the code you are using with API category?
Sure @elorzafe, Amplify generated a service (for Angular). That service is, in my code, referred to as this.awsAPI. Here is the call:
this.awsAPI.CreatePrivateTrack(
{
path: 'test-dir/' + fileName
}
).then((res: any) => {
console.log('successfully created a private track!', res);
});
I am not sure what you meant by "API category". Hopefully the above is what you were asking me to post. This is as much code as there is, really, it's pretty much all of it (which is the beauty of Amplify).
@elorzafe - perhaps the reason it is not sending the token has to do with the fact that there are multiple authorization types in this schema. Perhaps the Amplify service / code-generation did not properly pick the right Authorization header in this case and it's picking the Authorization header for the other authorization type (which was IAM)?
@dorontal
Yeah, is picking the default authorization mode. You can overwrite that on each query/mutation/subscription like this
// Creating a post is restricted to COGNITO USER POOLS
const createdTodo = await API.graphql({
query: queries.createTodo,
variables: {input: todoDetails},
authMode: 'AMAZON_COGNITO_USER_POOLS'
});
Thanks very much, @elorzafe!
This will be fixed in the future, right? I mean, the auto-generated Angular service will be able to supply the proper authMode: depending on the initial schema that was used to generate that code, and on the Mutation being used, right? If this is going to be fixed, then I prefer to just wait for it and work on other stuff. But if it takes Amazon too long to fix this bug, then I may very well use the workaround you describe and will then report over here about it. I appreciate very much that you've provided a temporary workaround!
If you have any clue as to when the auto-generated service code will be fixed, that would be useful.
@dorontal
I am happy that worked for you.
The other problem related to Angular service, it requires changes to be addressed on Amplify CLI. I will transfer this issue to that repo where they can handle this properly.
Hello @dorontal looking at your CLI workflow for the configuration I see that you made the default auth provider API IAM. The library uses the default mode of authentication, which in this case is AWS_IAM. Should you need to change the mode authorization you would need to specify that in the API.graphql operation.
Docs: https://docs.amplify.aws/lib/graphqlapi/authz/q/platform/js#using-amplify-graphql-client
I understand @SwaySway, but if I make COGNITO the default, then I can't use any of the IAM pubic access parts of the schema. It seems, and please tell me if I'm wrong, that it is not possible to use anything other than the default with the library, at the moment, right? I'm referring, specifically, to the auto-generated Angular service code. That code allows us to not have to use API.graphql directly, but instead it's a higher-level abstraction that is extremely convenient and is part of the main reasons for using Amplify in the first place.
When you need to make calls with a different auth provider you can specify that as such in the operation. If you need to go back to making calls with AWS_IAM by not specifying an authorization provider, the library for API would then default back to the auth mode you configured in the CLI.
Thanks @SwaySway, I understand, but this makes the auto-generated service unusable with more than the default auth mode. The auto-generated service for Angular is an abstraction above using API.graphql that does not require you to use API.graphql but you are only supposed to use the service functions instead (they use API.graphql behind the scenes). I would like to continue using the auto-generated service -- I don't think that service was meant to be used only with the default auth mode. It was meant to be used with multiple auth modes (or else the documentation is misleading, because that's what the documentation says). If it's possible to fix the auto-generated code, that would help a lot of developers who need to support both private and public access in their apps. Thanks!
@elorzafe and @SwaySway
I've just noticed that this exists: https://github.com/aws-amplify/amplify-js/issues
Since this issue is with the Angular code generation process of the service code, perhaps I should post this issue over there instead? In any case, I would very much appreciate help on getting this to the right people.
FYI, Tried it again today, after upgrading to Angular 9 with its Ivy renderer turned on by default and all new aws-amplify packages ("@aws-amplify/api": "^3.1.1" and "aws-amplify": "^3.0.11", which are the only two AWS/amplify packages used -- and the exact same bug is still there: a 401 error when the (auto-generated Angular service code) mutation is called with a logged-in user because the wrong Authorization header gets sent (see above discussion), which is probably due to there being multiple authorization types used (Cognito and IAM).
(Attn: @SwaySway and @ammarkarachi) Woohoo!! I found a super simple workaround that fixes the issue reported here, even better than using API.graphql (the API.graphql workaround would have required ignoring the auto-generated angular service code and rewriting that service with calls to API.graphql each of which specifies the auth mode - it would have worked but it would have taken numerous new lines of code. The following workaround is just one line of code).
The workaround is based on this issue and comment https://github.com/aws-amplify/amplify-cli/issues/1576#issuecomment-665338424 -- thank you @kwhitejr and @dabit3 !!
To work around this issue, I just added one line of code above the line that caused trouble. Added:
Amplify.configure({
aws_appsync_authenticationType: 'AMAZON_COGNITO_USER_POOLS'
});
before the call to
this.awsAPI.CreatePrivateTrack({ ...
Done! All works as expected now.
Most helpful comment
(Attn: @SwaySway and @ammarkarachi) Woohoo!! I found a super simple workaround that fixes the issue reported here, even better than using
API.graphql(theAPI.graphqlworkaround would have required ignoring the auto-generated angular service code and rewriting that service with calls toAPI.graphqleach of which specifies the auth mode - it would have worked but it would have taken numerous new lines of code. The following workaround is just one line of code).The workaround is based on this issue and comment https://github.com/aws-amplify/amplify-cli/issues/1576#issuecomment-665338424 -- thank you @kwhitejr and @dabit3 !!
To work around this issue, I just added one line of code above the line that caused trouble. Added:
before the call to
Done! All works as expected now.