Got: `beforeRequest` hook is not called before the actual request

[jameslshannon]

I had some AWS API integrations using Got v9.6.0 that were working fine until the Got dependency was updated to 10.0.4 by the server when all of a sudden the integrations broke. Despite the updated example code for aws4 integration using prefixUrl which I updated to, it still didn't work. Further debugging revealed that not only was aws4 rejecting the v10 options object (complaining it was type object and not string), even if I passed it as a string instead, the resultant options object from aws4 was completely mangled (properties concatenated, etc.) - hence why the integrations broke. As soon as I reverted my code and the Got dependency to 9.6.0 again, everything was back to working fine.

aws4 compatibility may therefore be worth looking at (I'm guessing this is commonly used) as well as the example shown at https://github.com/sindresorhus/got#aws - for now I'll therefore hold at 9.6.0

Thanks,

James

[szmarczak]

Could you post here the error that Got throws?

[jameslshannon]

It's not a specific Got error (I think it was something like "body, form and json are mutually exclusive" or something - because the v10 options passed to aws4 are mangled before Got processes them it's not surprising since most of the required options are missing due to incorrect concatenation of various options properties. It's easy to reproduce using the sample code for AWS.

[szmarczak]

Thanks, will try it out.

[szmarczak]

Can you try again with v10.2.0? Unfortunately I don't use AWS, so I can't help you further.

@reconbot Would you be able to look into this?

[jameslshannon]

Did some more testing with this today using 10.2.0. I can no longer reproduce the type/header mangling issue described above. However, it still doesn't play nicely with aws4.

To start with aws4 is dependent on both the path and hostname options which were previously set automatically when using baseUrl - the change to prefixUrl means that these both now need to be specified in addition to prefixUrl. Even with this change though, despite aws4 setting the AWS authentication headers in the options, I think the change to the json/body handling in v10 is causing issues with the aws4 signature calculation, resulting in either 403 or 400 errors with AWS requests.

I'm therefore having to stay on 9.6.0 and the sample code for aws4 integration should be commented as such.

[szmarczak]

You're right. After calling the beforeRequest hooks there's some body-normalization thing.

[reconbot]

cc @mhart I'm not able to take a look right now

[mhart]

Ay? Why cc me? aws4 hasn't changed anything 🤷‍♂

[szmarczak]

I believe that OP was using json option and the beforeRequest hook was called before body normalization. If anybody could point out if this is the faulty thing here, that would be great (I don't have an AWS account to test this). Anyway I'll try to fix this issue next week.