K6: AWS v4 signature authentication

[robingustafsson]

To better support users running tests against AWS infrastructure and to increase compatibility with Postman collections, we want to include the AWS v4 signature auth algorithm as one of the built-in auth options in k6.

The expected JS API would look something like this:

export default function() {
    let res = http.post("https://some.aws.amazon.com/api", null,
        { auth: {
            type: "awsv4",
            accessKey: _ENV.AWS_ACCESS_KEY,
            secretKey: _ENV.AWS_SECRET_KEY,
            service: "s3",
            region: "us-east-1"
         }
    });
}

The following “awsv4” auth options are optional:
service: as it can be derived from the URL host in most (all?) cases
region: as it can be derived from the URL host in most (all?) cases

This would require the following changes to k6:

• We’d need to handle auth request param option being either a string or an object
• We’d need to add handling of “awsv4” auth as part of request processing, probably around the place where we handle Digest auth today: https://github.com/loadimpact/k6/blob/master/lib/netext/httpext/request.go#L295
• We’d need to handle parsing and matching the service and region from the URL if that’s not included in the AWS Go SDK.

Resources:

• AWS Go SDK: https://github.com/aws/aws-sdk-go
• v4 signature code: https://github.com/aws/aws-sdk-go/blob/master/aws/signer/v4/v4.go
• We’d most likely use a static provider to pass in access key and secret key: https://github.com/aws/aws-sdk-go/blob/master/aws/credentials/static_provider.go

[na--]

The evaluation needed tag above is because I'm still of the opinion that if we're having fast crypto APIs exposed to the JS code, we might as well use them for something like this, instead of implementing it in Go as another auth method like basic, digest, and ntlm. The reasons for this are many:

• Unlike the current auth values, this is not a standard, it's just someone's API authentication... If we natively support this, should we support Google Cloud, MS Azure, DigitalOcean, ?
• Some of the current auth mechanisms cannot actually be implemented in JS, especially NTLM, while this can be - it will even likely be more flexible in JS.
• The current auth handling in the Go code isn't very pretty and likely needs refactoring. Injecting another custom method there will likely force the issue, making us spend a lot of extra time on this.
• Even if we implement it, it's likely to change - after all, the v4 is in the name, I don't want to have to deprecate this and add v5 when the time comes...
• It's currently easy to use JS code from remote modules by having import awsAuth from "github.com/loadimpact/k6-lib/awsauth.js" or something like that. And after we implement https://github.com/loadimpact/k6/issues/1037, we will even be standards compliant...

[na--]

Something to add to the list above: it seems like the AWSv4 signature authentication doesn't require anything more than HMAC, which we already support in k6 with the existing crypto functions. Sources:

• https://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-javascript
• https://github.com/mhart/aws4/blob/master/aws4.js
• https://github.com/loadimpact/k6/pull/952/files#diff-e1da18586b35daee5124f3ddfd634e07R411

So, it seems like we don't need browserified code or any of the new crypto primitives in order to implement a nice awsv4 helper JS module.

[alanbrent]

We have a need for AWSv4 signing in our infrastructure, and I have just put together a POC-grade patch that works.

I'm wondering if the k6 team is interested in shepherding this into main? I don't really have much golang chops, so I would definitely need some help with testing and any kind of style requirements!

Current working code (minus vendor changes): https://github.com/Tapjoy/k6/pull/1/commits/82cbee7e04e7407a2a3113f6abe938069d2a0c54

[MStoykov]

I have made (a long time ago) a small js library that does that and a small example using it with the S3 API. The code isn't really all that great as it was a POC reworking of https://github.com/mhart/aws4/blob/master/aws4.js using what is available in k6.

I would argue that this still shouldn't get inside k6, but should be a js library, but better written than mine. This code both doesn't need to be in k6 (which is in my opinion is a reason enough in the majority of cases) and will make the already not great HTTP handling code even more complex (even if it's a tiny bit more). Additionally, this adds the WHOLE aws-sdk for this one small thing, which also seems way too big a price.

[alanbrent]

@MStoykov Doing this in Javascript is not going to work for the amount of throughput we need to push for our load tests. The CPU and memory requirements are just too exorbitant. Indeed, there's already a Javascript-based suggested method for signing AWS requests, and it says right on the tin that it's CPU and memory heavy. When we tried it, we went from low 10^3 QPS to low 10^1 QPS. I am assuming this is due to inherent differences between JS vs. Go, but I'm open to alternative explanations.

TLDR as far as I can tell we need to generate these signatures in the k6 engine itself.

[alanbrent]

Additionally, this adds the WHOLE aws-sdk for this one small thing, which also seems way too big a price.

I agree that it results is a very big changeset, though I'm not sure what you mean by "price". I get that you want to keep dependencies small, and agree with the goal in principle. In the end, however, it seems like not a great reason to refuse a change.

AWS _is_ working on aws-sdk-go-v2, which looks like it will allow you to pull in _only_ the bits from the SDK that are relevant to your application. Unfortunately that SDK is currently in "Developer Preview" status, so I decided to use the _current_ canonical SDK.

All that said, if there are alternative Go libraries for doing this signing I would be happy to try using them instead.

[MStoykov]

My experiments with "my"(as it is just using the k6 crypto) lib with -u 200 -d 30s show that :

1. the usual http request takes ~160% CPU on my machine if no authinationcat
2. the patch provided by you takes ~185% CPU ( I needed to drop the body signing as I was doing gets)
3. the js only implementation takes ~280% CPU

So around 50% slower .. while this seems like a lot .. this is only for the AWS requests. I don't think a lot of people will be making enough AWS requests for this to matter so I am still of the opinion that this is better done with js ... ALSO ... the current crypto lib of k6 is not very optimized so it's likely this will get better once we get to actually putting some work with it, especially now that the JS VM that is used by k6 (goja) has better "binary" support

The comparison is definetely not apples to apples but IMO the difference isn't 10x in this case as we DO use k6/crypto for the heavy lifting, and given the posibility of improvements putting this in k6 isn't what I think should be done.

Code for the aws authentications:(yes the js lib one is not ... good :D or ... correct, but this is just PoC)

import http from "k6/http";
import { check } from "k6";
import aws from "https://gist.githubusercontent.com/MStoykov/38cc1293daa9080b11e26053589a6865/raw/9eee2c2f4af6d193d8e93719b6de2e09e451aa52/aws_k6.js";

function request(method, url, data, params) {
        options =  {};
      options.method = method;
      options.signSessionToken = true;
      options.doubleEscape = false;
      url =  aws.createPresignedURL(
              options.method,
              "test.k6.io",
              "/",
              "s3",
              "UNSIGNED-PAYLOAD",
              options
            );

    return http.request(method, url, data,params)
}

export default function () {
var res = request("GET", "https://test.k6.io/")
  check(res, {
    "status is 200": (r) => r.status == 200,
    "protocol is HTTP/2": (r) => r.proto == "HTTP/2.0",
  });
}

import http from "k6/http";
import { check } from "k6";

export default function () {
var res = http.request("GET", "https://test.k6.io/", null, {"aws_service": "s3", "auth": "awsv4"})
  check(res, {
    "status is 200": (r) => r.status == 200,
    "protocol is HTTP/2": (r) => r.proto == "HTTP/2.0",
  });
}

[MStoykov]

The code of the WIP JS implementation is here https://github.com/loadimpact/jslib.k6.io/pull/11 and is looking for people to test it as that turned out to be the hard part, given that you need to play with AWS console, which I am not particularly familiar with.

Also, me testing that it does something is not very useful if what it will be used for isn't tested so I am looking for people to pitch in and try it out :). Possibly with feedback on what they need more from it, apart from obviously better API for each AWS service