Serverless-application-model: API Property for Handle 404 and Handle 405

[beck3905]

API Gateway default behavior is to return a 403 - Forbidden response with a message of 'Missing Authentication Token' instead of a 404 - Not Found when the wrong resource path is provided and the same response instead of a 405 - Method Not Allowed when the wrong method is provided. Apparently this is by design, but is not desirable for many APIs.

I have developed a way to handle these situations to actually return a 404 and 405 where appropriate. I do this by using a /{proxy+} resource on the root of my API with a Lambda Proxy integration to Lambda that simply returns a 404 response. Similarly, on each of my defined paths I add an ANY method with a Lambda Proxy integration to a Lambda that simply returns a 405.

I currently do this using a swagger definition, but it gets quite verbose. I'm considering submitting a PR to SAM to add this capability to the transform similar to how the Cors property works on an AWS::Serverless::API.

Would anyone else be interested in this capability? Would this be an acceptable feature addition to the SAM transform?

[jlhood]

Hi @beck3905. Really appreciate the issue and thanks for the PR offer! We discussed this and while using your approach of /{proxy+} and ANY does create the behavior you want, there are downsides to this approach. Specifically that you would not be able to use /{proxy+} for any other purpose in your API. We think a better approach is to use API Gateway's Gateway Responses feature. SAM's next release will include support for easily specifying Gateway Responses on AWS::Serverless::Api resources. The advantage to using Gateway Responses is that it does not require a Lambda function and doesn't preclude using /{proxy+} for other purposes in your API.

That said, we tried to model the exact behavior you're looking for (404 and 405) using Gateway Responses and found some issues. Here is the closest we were able to get:

  ExampleApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: Prod
      GatewayResponses:
        MISSING_AUTHENTICATION_TOKEN:
          StatusCode: 404
          ResponseTemplates:
            "application/*": '{ "message": “Not found" }'

MISSING_AUTHENTICATION_TOKEN is a gateway response type defined by API Gateway. You can read more about the available gateway response types and their meanings here. Adding the above gateway response configuration changes the behavior of the API to return 404 in both the invalid resource and valid resource/invalid method case. However, if you are using authentication, it will also return 404 in the missing authentication token case.

While this behavior does not match your desired behavior exactly, we still think Gateway Responses is the better approach to use. We'll talk to the API Gateway team about your specific use case and see if they can add new gateway response types that will allow you to return different status codes for each individual case.

[beck3905]

@jlhood Thanks for your response. I looked into gateway responses too when I first came across this issue. I found that there was no way to determine which gateway response is triggered in which situations (i.e. incorrect resource triggers 404 instead of 403) and determined that this would mean that my API could not return 403 in valid forbidden circumstances.

The correct fix for this issue would be for API Gateway to return a 404 instead of 403 when an incorrect path is provided or a 405 instead of 403 when an incorrect method is provided. I have seen several forum posts over several years now of people complaining about this, but the response I have seen from folks at AWS is that it is by design that they return 403. I'd like to suggest to them that at least they could give us the option to return 404/405 instead of 403 in these situations, but 403 could be the default behavior.

I recognize that my solution is a workaround and not a true solution. There are certainly some limitations, but I think it addresses a common use case. The /{proxy+} resource would only be precluded on the root of the API. That is the only place necessary to add the 404 handler. Any /<resource>/{proxy+} would still be available.

To reiterate the solution:

• Add an ANY method to the root API that proxy's to a Lambda that returns a 404 response
• Add a /{proxy+}/ANY method to the root API that proxy's to a Lambda that returns a 404 response
• Add an ANY method to any declared path (including those generated by AWS::Serverless::Function) that proxy's to a Lambda that returns a 405 response

[jlhood]

@beck3905 Thanks for your feedback. I've passed it along to the Amazon API Gateway team. What would you envision the SAM syntax looking like if you added this feature? Could you give an example AWS::Serverless::Api demonstrating the developer experience for this?

[beck3905]

@jlhood I was thinking of something extremely simple. It could be 1 property or 2.

1 Property:

MyApi:
  Type: AWS::Serverless::Api
  Properties:
    Name: MyApi
    Stage: prod

    # optional parameter; might need more descriptive name; triggers transform to add /{proxy+}/ANY and /ANY integrations to Api root and add /ANY integrations to all defined paths
    HandlePathErrors: true

2 Properties:

MyApi:
  Type: AWS::Serverless::Api
  Properties:
    Name: MyApi
    Stage: prod

    # optional parameter; triggers transform to add /{proxy+}/ANY and /ANY integrations to Api root
    HandlePathNotFound: true

    # optional parameter; triggers transform to add /ANY integrations to all defined paths
    HandleMethodNotAllowed: true

[uhef]

As a note:

You can simplify @beck3905 solution a little bit by using mock integration instead of a lambda proxy integration that would return status code 404 / 405. Here's a snippet of OpenAPI definition to achieve this:

  '/{catchall+}':
    x-amazon-apigateway-any-method:
      consumes:
        - application/json
      produces:
        - application/json
      x-amazon-apigateway-integration:
        type: mock
        requestTemplates:
          application/json: |
            {
              "statusCode": 404
            }
        responses:
          "default":
            statusCode: "404"
            responseParameters: 
              method.response.header.Content-Type: "'application/json'"
            responseTemplates:
              application/json: |
                {}
      responses:
        404:
          description: Resource not found
          headers:
            Content-Type:
              type: string

[okigan]

Just waisted several hours just to find out the path was not correctly specified on the request as I was chasing an authentication issue as specified by the error message.

The workaround above with 'MISSING_AUTHENTICATION_TOKEN' creates (even) more confusion by masking true authentication errors and resource not found!

This is utterly perplexing why this would behave per HTTP standard: accessing a resource that does not exist shall return HTTP 404 (i.e. first check the paths/routes then perform authentication).

Don't know if this need to be solved on SAM level or api gateway itself, but this bizarre behavior should be removed.