Bref: Support for HTTP API

[aknosis]

HTTP APIs are now generally available: https://aws.amazon.com/blogs/compute/building-better-apis-http-apis-now-generally-available/

I think we should implement HTTP APIs as an option during initialization and eventually migrate toward making them the default.

Reasons to migrate to HTTP APIs:

Faster

For the majority of use cases, HTTP APIs offers up to 60% reduction in latency

Cheaper

Using the pricing for us-east-1, the image shows a cost comparison for 100 million, 500 million, and 1 billion requests a month. Overall, HTTP APIs is at least 71% lower cost compared to API Gateway REST APIs.

$default Stage

HTTP APIs allow for a $default stage, this stage will point to the root domain (e.g. https://abc123.execute-api.us-east-2.amazonaws.com), this gets rid of all the issues relating to the state prefix.

The changes required are very simple (assuming default serverless.yml setup).

REST API:

    events:
      - http: 'ANY /'
      - http: 'ANY /{proxy+}'

HTTP API:

    events:
      - httpApi:
          path: '*'

Anecdotally, I have been running a bref app on an HTTP API for about a month now with no issues.

[mnapoli]

Hi, I completely agree with you. I have been using it as well in a side project and it's great.

The blog article does mention a difference in the payload though (https://aws.amazon.com/fr/blogs/compute/building-better-apis-http-apis-now-generally-available/), I think we need to add some tests and review if there is any impact to Bref before adding it in the documentation.

[aknosis]

I didn't notice the payload change since serverless defaults to the 1.0 version (
https://github.com/serverless/serverless/blob/master/lib/plugins/aws/package/compile/events/httpApi/index.js#L427).

Also not configurable to the 2.0 version yet (https://github.com/serverless/serverless/issues/7477), so once that is possible it would make sense to implement.

It also appears the response format may have to differ when using 2.0 format: https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html#http-api-develop-integrations-lambda.proxy-format

[mnapoli]

Oh that's interesting!

So if I recap to make sure we are on the same page:

• if I deploy an HTTP API (the new thing that is not REST API) with serverless, I get the payload v1.0
• AWS introduced a new payload v2.0 that can be used with the new HTTP API
• we can enable that new payload, but not with serverless yet

Do you know if the payload v2.0 applies also to the REST API? And if v1.0 is deprecated?

[aknosis]

My best guess is that 2.0 will not apply to REST API because of features that it has to support. HTTP API has less overall features and thus why it was able to have a simpler request / response format.

[aknosis]

Support for configuring payload 2.0 was merged:

provider:
  httpApi:
    payload: '2.0'

https://github.com/serverless/serverless/pull/7623

[larswww]

FYI this still errors:

provider:
  httpApi:
    payload: 2.0

adding single quotes around '2.0' makes it work. The guide/cheat sheet https://www.serverless.com/framework/docs/providers/aws/guide/serverless.yml/ shows without quotes

[mnapoli]

Thank you @larswww, I have opened https://github.com/serverless/serverless/pull/7772 to improve the documentation in Serverless.

[geoff-ellison]

With traditional REST API calls, bref passes context through in $_SERVER['LAMBDA_REQUEST_CONTEXT']
Does anyone know if bref is supposed to do the same when using httpApi? In my tests there are a lot of server variables missing, including:

• LAMBDA_REQUEST_CONTEXT
• LAMBDA_INVOCATION_CONTEXT
• REQUEST_URI
• REQUEST_METHOD
• REMOTE_ADDR
• PATH_INFO
• all HTTP_*

I've tried explicitly setting payload for both 1.0 and 2.0 but neither one provides these missing values. Is there anywhere I can get them from?
Thanks

[geoff-ellison]

Found the solution: use an explicit layer arn and payload 1.0.

provider:
    httpApi:
        payload: '1.0'

functions:
    api:
        handler: public/index.php
        description: ''
        layers:
            - arn:aws:lambda:ap-southeast-2:209497400698:layer:php-74-fpm:7

Leaving this here in case someone else runs into the same problem.
Cheers

With traditional REST API calls, bref passes context through in $_SERVER['LAMBDA_REQUEST_CONTEXT']
Does anyone know if bref is supposed to do the same when using httpApi? In my tests there are a lot of server variables missing, including:

• LAMBDA_REQUEST_CONTEXT
• LAMBDA_INVOCATION_CONTEXT
• REQUEST_URI
• REQUEST_METHOD
• REMOTE_ADDR
• PATH_INFO
• all HTTP_*

I've tried explicitly setting payload for both 1.0 and 2.0 but neither one provides these missing values. Is there anywhere I can get them from?
Thanks

[aknosis]

@geoff-ellison This doesn't seem correct. Is it possible there is an issue with the original layer version you were using?

Explicitly listing the layer vs using the plugin syntax - ${bref:layer.php-74} should have no effect other than the plugin would use the most recent version listed in layers.json. Note that the latest version is 10 and your serverless.yml is showing v4 (https://runtimes.bref.sh/?region=ap-southeast-2&version=0.5.25)

I'm using this layer arn:aws:lambda:us-east-2:209497400698:layer:php-74-fpm:9 combined with HTTP API and my $_SERVER (redacted) has a majority of those defined.

functions:
  api:
    handler: public/index.php
    timeout: 28
    layers:
      - ${bref:layer.php-74-fpm}
    events:
      - httpApi:
          path: '*'

    "AWS_LAMBDA_FUNCTION_VERSION": "$LATEST",
    "AWS_SESSION_TOKEN": "...",
    "AWS_LAMBDA_LOG_GROUP_NAME": "...",
    "LAMBDA_TASK_ROOT": "/var/task",
    "LD_LIBRARY_PATH": "/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib",
    "AWS_LAMBDA_RUNTIME_API": "127.0.0.1:9001",
    "AWS_LAMBDA_LOG_STREAM_NAME": "...",
    "AWS_XRAY_DAEMON_ADDRESS": "...",
    "AWS_LAMBDA_FUNCTION_NAME": "...",
    "PATH": "/usr/local/bin:/usr/bin/:/bin:/opt/bin",
    "AWS_DEFAULT_REGION": "...",
    "PWD": "/var/task",
    "LAMBDA_RUNTIME_DIR": "/var/runtime",
    "LANG": "en_US.UTF-8",
    "AWS_REGION": "...",
    "TZ": ":UTC",
    "SHLVL": "0",
    "_AWS_XRAY_DAEMON_ADDRESS": "169.254.79.2",
    "_AWS_XRAY_DAEMON_PORT": "2000",
    "AWS_XRAY_CONTEXT_MISSING": "LOG_ERROR",
    "_HANDLER": "public/index.php",
    "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": "1024",
    "USER": "sbx_user1051",
    "HOME": "/home/sbx_user1051",
    "CONTENT_LENGTH": "0",
    "CONTENT_TYPE": "application/x-www-form-urlencoded",
    "SERVER_PROTOCOL": "HTTP/1.1",
    "SERVER_NAME": "...",
    "SERVER_PORT": "443",
    "SERVER_ADDR": "127.0.0.1",
    "REMOTE_PORT": "443",
    "REMOTE_ADDR": "127.0.0.1",
    "SERVER_SOFTWARE": "bref",
    "SCRIPT_FILENAME": "/var/task/public/index.php",
    "REQUEST_URI": "/",
    "REQUEST_METHOD": "GET",
    "GATEWAY_INTERFACE": "FastCGI/1.0",
    "HTTP_ACCEPT_ENCODING": "gzip, deflate, br",
    "HTTP_ACCEPT": "*/*",
    "HTTP_X_FORWARDED_PROTO": "https",
    "HTTP_X_FORWARDED_PORT": "443",
    "HTTP_X_FORWARDED_FOR": "...",
    "HTTP_X_AMZN_TRACE_ID": "...",
    "HTTP_USER_AGENT": "PostmanRuntime/7.25.0",
    "HTTP_HOST": "...",
    "HTTP_CONTENT_LENGTH": "0",
    "LAMBDA_CONTEXT": "{...}",
    "LAMBDA_INVOCATION_CONTEXT": "{...}",
    "QUERY_STRING": "",
    "PATH_INFO": "/",
    "FCGI_ROLE": "RESPONDER",
    "PHP_SELF": "/",
    "REQUEST_TIME_FLOAT": ...,
    "REQUEST_TIME": ...,

[mnapoli]

Full support with the v2 payload format has been added in #604, I will tag a new release.

[geoff-ellison]

@geoff-ellison This doesn't seem correct. Is it possible there is an issue with the original layer version you were using?

Explicitly listing the layer vs using the plugin syntax - ${bref:layer.php-74} should have no effect [...]

Thanks @aknosis but the problem was the combination of the layer together with cognito-based authentication. The only circumstance where everything worked was an explicit bref layer together with payload v1. And then suddenly, I could use the latest (${bref:layer.php-74-fpm}) bref layer! I had no idea why it suddenly started working (with payload v1), but then today I received this email from AWS:

Hello,

We are reaching out to you because our data shows that you have been using JWT Authorizers as part of the following Amazon API Gateway HTTP APIs in the AP-SOUTHEAST-2 Region:

38*(redacted)**4h

Amazon API Gateway recently rolled out a change that may have affected one of your APIs. This change specifically modified the format in which API Gateway sends JWT claims as part of the payload.

The change started rolling out at June 1, 10:45 AM PDT and it was rolled back from all regions on June 3, 7:30 PM PDT.

We apologize for any issues this may have caused. To avoid issues like this, all future changes to JWT claims payload will be versioned using the Payload Format Version [1], so you can opt-in to use it at your convenience.

Should you have any questions or concerns, the AWS Support Team is available on the community forums and via AWS Support [2].

[1] https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html
[2] https://aws.amazon.com/support

Sincerely,
Amazon Web Services

Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210

The seemingly random outcomes now all make sense. The timing of 3 June fits exactly. So now when I get a chance I will attempt with payload v2 (and latest php 7.4 bref) and see what happens.