Aws-sam-cli: Problem with OPTIONS requests

[charsleysa]

Description:

Trying to access my API from a test app in my browser and the OPTIONS request is being handled by SAM CLI API Gateway but it's not returning the configuration specified in my template file.

The weird part is that if I try to access a different endpoint from the same template it executes the lambda function (as expected since that's what SAM Local did).

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
Mac

Output of sam --version:
SAM CLI, version 0.3.0

Optional Debug logs:

> export AWS_SDK_LOAD_CONFIG=true && sam local start-api --docker-network myblueprint  "--debug-port" "5858"

2018-05-10 16:23:38 Mounting HealthCheckFunction at http://127.0.0.1:3000/healthcheck [OPTIONS, GET]
2018-05-10 16:23:38 Mounting EnsureUserFunction at http://127.0.0.1:3000/ensure_user [OPTIONS, PUT]
2018-05-10 16:23:38 Mounting GraphQLServerFunction at http://127.0.0.1:3000/graphql [GET, POST, OPTIONS]
2018-05-10 16:23:38 You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2018-05-10 16:23:38  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
2018-05-10 16:24:07 127.0.0.1 - - [10/May/2018 16:24:07] "OPTIONS /graphql HTTP/1.1" 200 -
2018-05-10 16:24:45 Invoking dist/healthcheck.handler (nodejs8.10)
2018-05-10 16:24:45 Found credentials in shared credentials file: ~/.aws/credentials

Fetching lambci/lambda:nodejs8.10 Docker container image......
2018-05-10 16:24:49 Mounting /Users/stefan/Projects/blueprint-api/packages/api as /var/task:ro inside runtime container
Debugger listening on ws://0.0.0.0:5858/698de0ac-e3af-4f27-b5dd-bf3caf4cbc58
For help see https://nodejs.org/en/docs/inspector
Debugger attached.
Debugger listening on ws://0.0.0.0:5858/698de0ac-e3af-4f27-b5dd-bf3caf4cbc58
For help see https://nodejs.org/en/docs/inspector
START RequestId: 2ee985d7-416b-182e-ec4a-79af497b0cc2 Version: $LATEST
2018-05-10T04:25:06.756Z        2ee985d7-416b-182e-ec4a-79af497b0cc2    ---TRIMMED ERROR---
END RequestId: 2ee985d7-416b-182e-ec4a-79af497b0cc2
REPORT RequestId: 2ee985d7-416b-182e-ec4a-79af497b0cc2  Duration: 2838.28 ms    Billed Duration: 2900 ms        Memory Size: 1024 MB    Max Memory Used: 52 MB
2018-05-10 16:25:03 No Content-Type given. Defaulting to 'application/json'.
2018-05-10 16:25:03 127.0.0.1 - - [10/May/2018 16:25:03] "OPTIONS /healthcheck HTTP/1.1" 503 -

Template file for reference

---
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: MyBlueprint API

Parameters:
  Environment:
    Type: String
    AllowedValues:
      - DEVELOPMENT
      - STAGING
      - PRODUCTION

# DEV SHOULD NEVER BE RUN OUTSIDE OF SAM LOCAL
Mappings:
  Config:
    'DEVELOPMENT':
      EnvironmentLowerCase: 'development'
      HostedZoneName: 'localhost'
      HostedZoneId: 'NONE'
      S3FilesBucket: 'blah'
    'STAGING':
      EnvironmentLowerCase: 'staging'
      HostedZoneName: '<REDACTED>'
      HostedZoneId: '<REDACTED>'
      S3FilesBucket: '<REDACTED>'
    'PRODUCTION':
      EnvironmentLowerCase: 'production'
      HostedZoneName: '<REDACTED>'
      HostedZoneId: '<REDACTED>'
      S3FilesBucket: '<REDACTED>'

Conditions:
  CreateS3FileProcessingResources:
    !Equals [!Ref 'AWS::Region', 'ap-southeast-2']

Globals:
  Function:
      Runtime: 'nodejs8.10'
      MemorySize: 1024
      Timeout: 5
      Tracing: PassThrough
      Environment:
        Variables:
          CODE_ENVIRONMENT: !Ref Environment

Resources:
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Action: 'sts:AssumeRole'
          Effect: Allow
          Principal:
            Service: 'lambda.amazonaws.com'
      Path: /
      ManagedPolicyArns:
      - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      - 'arn:aws:iam::aws:policy/AWSXrayWriteOnlyAccess'
      Policies:
      - PolicyName: s3
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Action:
            - 's3:*'
            Effect: Allow
            Resource: '*'
      - PolicyName: parameters
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Effect: Allow
            Action:
            - 'ssm:GetParameter'
            - 'ssm:GetParameters'
            - 'ssm:GetParametersByPath'
            Resource:
            - !Sub 'arn:aws:ssm:${AWS::Region}:${AWS::AccountId}:parameter/${Environment}/API/*'
          - Effect: Allow
            Action:
            - 'kms:Decrypt'
            Resource:
            - !Sub 'arn:aws:kms:${AWS::Region}:${AWS::AccountId}:key/*'

  MyBlueprintApi:
    Type: 'AWS::Serverless::Api'
    Properties:
      StageName: 'prod'
      EndpointConfiguration: 'REGIONAL'
      Cors:
        AllowMethods: "'GET,OPTIONS,POST,PUT'"
        AllowHeaders: "'Authorization,Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
        AllowOrigin: "'*'"
        MaxAge: "'86400'"
      DefinitionBody:
        swagger: '2.0'
        info:
          version: '0.0.1'
          title:
            'Fn::Sub':
            - 'myblueprint-${EnvironmentLowerCase}'
            - EnvironmentLowerCase: !FindInMap [Config, !Ref Environment, EnvironmentLowerCase]
        basePath: '/prod'
        schemes:
        - 'https'
        paths:
          '/graphql':
            get:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GraphQLServerFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_templates'
                httpMethod: 'POST'
                contentHandling: 'CONVERT_TO_TEXT'
                type: 'aws_proxy'
            post:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GraphQLServerFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_templates'
                httpMethod: 'POST'
                contentHandling: 'CONVERT_TO_TEXT'
                type: 'aws_proxy'
            options:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
                    Access-Control-Allow-Methods:
                      type: 'string'
                    Access-Control-Allow-Headers:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Methods: "'GET,OPTIONS,POST'"
                      method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                requestTemplates:
                  application/json: '{"statusCode": 200}'
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GraphQLServerFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_match'
                type: 'mock'
          '/ensure_user':
            put:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${EnsureUserFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_templates'
                httpMethod: 'POST'
                contentHandling: 'CONVERT_TO_TEXT'
                type: 'aws_proxy'
            options:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
                    Access-Control-Allow-Methods:
                      type: 'string'
                    Access-Control-Allow-Headers:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Methods: "'PUT,OPTIONS'"
                      method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                requestTemplates:
                  application/json: '{"statusCode": 200}'
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${EnsureUserFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_match'
                type: 'mock'
          '/healthcheck':
            get:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HealthCheckFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_templates'
                httpMethod: 'POST'
                contentHandling: 'CONVERT_TO_TEXT'
                type: 'aws_proxy'
            options:
              consumes:
              - 'application/json'
              produces:
              - 'application/json'
              responses:
                '200':
                  description: '200 response'
                  schema:
                    $ref: '#/definitions/Empty'
                  headers:
                    Access-Control-Allow-Origin:
                      type: 'string'
                    Access-Control-Allow-Methods:
                      type: 'string'
                    Access-Control-Allow-Headers:
                      type: 'string'
              x-amazon-apigateway-integration:
                responses:
                  default:
                    statusCode: '200'
                    responseParameters:
                      method.response.header.Access-Control-Allow-Methods: "'GET,OPTIONS'"
                      method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
                      method.response.header.Access-Control-Allow-Origin: "'*'"
                requestTemplates:
                  application/json: '{"statusCode": 200}'
                uri: !Sub 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${HealthCheckFunction.Arn}/invocations'
                passthroughBehavior: 'when_no_match'
                type: 'mock'
        definitions:
          Empty:
            type: 'object'
            title: 'Empty Schema'

  GraphQLServerFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: 'dist/graphql.handler'
      CodeUri: './packages/api/'
      Role: !GetAtt LambdaExecutionRole.Arn
      MemorySize: 1536
      Timeout: 30
      Events:
        GetResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/graphql'
            Method: get
        OptionsResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/graphql'
            Method: options
        PostResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/graphql'
            Method: post

  EnsureUserFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: 'dist/ensure_user.handler'
      CodeUri: './packages/api/'
      Role: !GetAtt LambdaExecutionRole.Arn
      Events:
        PutResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/ensure_user'
            Method: put
        OptionsResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/ensure_user'
            Method: options

  S3FileProcessingFunction:
    Type: 'AWS::Serverless::Function'
    Condition: 'CreateS3FileProcessingResources'
    Properties:
      Handler: 'dist/s3files.handler'
      CodeUri: './packages/api/'
      Role: !GetAtt LambdaExecutionRole.Arn
      Timeout: 30
# Cant use this because the bucket must be created in the same template
#      Events:
#        ObjectCreated:
#          Type: S3
#          Properties:
#            Bucket: !FindInMap [Config, !Ref Environment, S3FilesBucket]
#            Events: s3:ObjectCreated:*

  HealthCheckFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: 'dist/healthcheck.handler'
      CodeUri: './packages/api/'
      Role: !GetAtt LambdaExecutionRole.Arn
      Events:
        GetResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/healthcheck'
            Method: get
        OptionsResource:
          Type: Api
          Properties:
            RestApiId: !Ref MyBlueprintApi
            Path: '/healthcheck'
            Method: options

  # TODO: update this to use DNS validation when CF supports it
  ApiCertificate:
    Type: 'AWS::CertificateManager::Certificate'
    Properties:
      DomainName:
        'Fn::Sub':
        - 'api.${HostedZoneName}'
        - HostedZoneName: !FindInMap [Config, !Ref Environment, HostedZoneName]
      DomainValidationOptions:
      - DomainName:
          'Fn::Sub':
          - 'api.${HostedZoneName}'
          - HostedZoneName: !FindInMap [Config, !Ref Environment, HostedZoneName]
        ValidationDomain: 'myblueprint.cloud'

  CustomResourceLambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
        - Action: 'sts:AssumeRole'
          Effect: Allow
          Principal:
            Service: 'lambda.amazonaws.com'
      Path: /
      ManagedPolicyArns:
      - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'
      Policies:
      - PolicyName: ApiGateway
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
          - Action:
            - 'apigateway:*'
            Effect: Allow
            Resource: '*'

  DomainNameInfoCustomResourceFunction:
    Type: 'AWS::Lambda::Function'
    Properties:
      Handler: index.handler
      Role: !GetAtt CustomResourceLambdaExecutionRole.Arn
      Runtime: 'nodejs6.10'
      Timeout: 300
      Code:
        ZipFile: |
          const AWS = require('aws-sdk');
          const response = require('cfn-response');

          exports.handler = function(event, context) {
              const ApiGateway = new AWS.APIGateway();
              ApiGateway.getDomainName({
                  domainName: event.ResourceProperties.DomainName
              }, (err, data) => {
                  if (err != null) {
                      response.send(event, context, response.FAILED, undefined);
                  } else {
                      response.send(event, context, response.SUCCESS, {
                          DomainName: data.domainName,
                          RegionalDomainName: data.regionalDomainName,
                          RegionalHostedZoneId: data.regionalHostedZoneId,
                          DistributionDomainName: data.distributionDomainName,
                          DistributionHostedZoneId: data.distributionHostedZoneId
                      });
                  }
              });
          }

  ApiDomainName:
    Type: 'AWS::ApiGateway::DomainName'
    Properties:
      DomainName:
        'Fn::Sub':
        - 'api.${HostedZoneName}'
        - HostedZoneName: !FindInMap [Config, !Ref Environment, HostedZoneName]
      EndpointConfiguration:
        Types:
        - REGIONAL
      RegionalCertificateArn: !Ref ApiCertificate

  ApiBasePathMapping:
    Type: 'AWS::ApiGateway::BasePathMapping'
    DependsOn: [MyBlueprintApi, ApiDomainName]
    Properties:
      DomainName:
        'Fn::Sub':
        - 'api.${HostedZoneName}'
        - HostedZoneName: !FindInMap [Config, !Ref Environment, HostedZoneName]
      RestApiId: !Ref MyBlueprintApi
      Stage: 'prod'

  ApiDomainNameInfo:
    Type: 'Custom::DomainNameInfo'
    DependsOn: [ApiDomainName, ApiBasePathMapping]
    Properties:
      ServiceToken: !GetAtt DomainNameInfoCustomResourceFunction.Arn
      DomainName: !Ref ApiDomainName

  ApiHealthCheck:
    Type: 'AWS::Route53::HealthCheck'
    Properties:
      HealthCheckConfig:
        Port: 443
        Type: 'HTTPS_STR_MATCH'
        SearchString: 'ok'
        ResourcePath: '/prod/healthcheck'
        FullyQualifiedDomainName: !Sub '${MyBlueprintApi}.execute-api.${AWS::Region}.amazonaws.com'
        RequestInterval: 60
        FailureThreshold: 2
      HealthCheckTags:
      - Key: Name
        Value:
          'Fn::Sub':
          - 'api-regional-${EnvironmentLowerCase}-${Region}'
          - EnvironmentLowerCase: !FindInMap [Config, !Ref Environment, EnvironmentLowerCase]
            Region: !Ref 'AWS::Region'

  ApiRecordSet:
    Type: 'AWS::Route53::RecordSet'
    DependsOn: [ApiDomainNameInfo]
    Properties:
      HostedZoneId: !FindInMap [Config, !Ref Environment, HostedZoneId]
      Name:
        'Fn::Sub':
        - 'lbr-api.${HostedZoneName}'
        - HostedZoneName: !FindInMap [Config, !Ref Environment, HostedZoneName]
      ResourceRecords:
      - !GetAtt ApiDomainNameInfo.RegionalDomainName
      Region: !Ref 'AWS::Region'
      SetIdentifier: !Sub 'api-${AWS::Region}'
      HealthCheckId: !Ref ApiHealthCheck
      Type: CNAME
      TTL: 60

Outputs:
  RestAPIID:
    Description: Rest API ID
    Value: !Ref MyBlueprintApi
  ApiUrl:
    Description: URL of your API endpoint
    Value: !Ref ApiRecordSet
  HealthcheckApiUrl:
    Description: URL of your API health check endpoint
    Value: !Sub 'https://${MyBlueprintApi}.execute-api.${AWS::Region}.amazonaws.com/prod/healthcheck'

[hobotroid]

I'm having the same problem after upgrading AWS SAM to v0.3.0. My OPTIONS requests are no longer going through the handler I specify in my template file, yet they are returning 200. So I'm no longer able to specify allowed headers in the response as I could before the update.

[jfuss]

@charsleysa What do you mean by "but it's not returning the configuration specified in my template file"?

I did a quick test with a function defined through an Events structure for only options and see the data being displayed that was returned from the Local Lambda Function.

[charsleysa]

@jfuss well I saw some code related to CORS in the rewrite so I assumed there was support for CORS because for the /graphql endpoint in my template it wasn't executing my Lambda function during the OPTIONS request. (Refer to the debug details in my original post)

I have CORS configured in my template but the result being returned by the SAM CLI API Gateway does not contain any Headers which match my CORS configuration.

As mentioned above, this issue doesn't seem to affect all functions, in fact it seems to only affect the first function defined in the template.

All other functions defined in the template execute the Lambda when an OPTIONS request is received.

[jfuss]

@charsleysa How did you validate that the wrong local Lambda was being called? Did you see unexpected results or just looking at output? If you are looking at the output logs, what you are seeing is expected. Logs are printed before the log for the request. Flask logs the request after it is handled.

Not sure what you are referring to about code related to CORS. We don't currently parse any data for CORS, even though it is in the object: https://github.com/awslabs/aws-sam-cli/blob/develop/samcli/commands/local/lib/sam_api_provider.py#L443

Options works but only returns the data that is returned from the Local Lambda, which is expected. We currently do not support CORS (neither did version previous to 0.3.0). Please follow #323 for CORS support.

[charsleysa]

@jfuss thank you for confirming that there is no CORS support.

This means there is a bug in the execution of my Lambda as my handler is never executed.

If you take a look at the debug details from my original post you will see the following line: 2018-05-10 16:24:07 127.0.0.1 - - [10/May/2018 16:24:07] "OPTIONS /graphql HTTP/1.1" 200 -
There is no execution logs for my handler and I ran this in debug mode and my debugger never attached to any container so I can verify that my handler was never executed.

The response received by the browser was an empty 200 with no CORS Headers.

EDIT:
I tried again but with a different endpoint and you can see the execution logs for the OPTIONS request followed by 2018-05-10 16:25:03 127.0.0.1 - - [10/May/2018 16:25:03] "OPTIONS /healthcheck HTTP/1.1" 503 -
(disregard the 503 status as that was expected for that endpoint)

[hobotroid]

@charsleysa Have you had any luck with this issue? I'm still struggling with it--it's simply not obeying most of my OPTIONS rules, and responds with a generic 200. It's definitely a v0.3.0 issue.

[charsleysa]

@hobotroid still no luck. Haven't managed to figure out why some work and some don't, it feels random but it's not because it only affects specific functions but when I check their config I can't find anything that could be causing it.

This new version is definitely a pain as another feature that 0.2.x had (and seems like it was undocumented) is no longer available (the feature was passing CloudFormation parameter values to the start API command).

[hobotroid]

@charsleysa Yes, I spent hours trying to get all my OPTIONS rules to work. But at best I could get 75%. It has something to do with the path parsing, but I still can't figured out exactly what.

[jfuss]

@hobotroid Again there is no CORS support currently. OPTIONS behaves the same as GET or any other of the HTTP verbs. If this is not the case, please give us more details so we can reproduce. As it stands, I have not been able to observe any incorrect behavior, that is when I return a proxy response from a lambda through an OPTIONS verb, I get the correct response as the caller.

@charsleysa Make sure the debugger is setup correctly. If you are using VS code, you need to be setting the debugger to legacy, if I recall correctly. If you are seeing issues with the debugger, please file a separate issue.

As for the CloudFormation Parameter values, are you are needing/wanting to override values in the Env Vars? If so, you can still pass --env-vars to the command. If you are referring to something else, please cut an issue and explain the use-case

[charsleysa]

@jfuss I have the debugger setup correctly.

As I mentioned in my previous posts, when I make OPTIONS requests to my endpoints some of them execute and some of them don't which should not occur.

From my template, the /graphql endpoint should execute my function for an OPTIONS request but currently does not. The /healthcheck endpoint should execute my function for an OPTIONS request and currently does execute as expected and does connect to my debugger as expected.

I can see no reason why both endpoints are not responding the same as they should.

I will update the title of this issue to better reflect the problem.

[hobotroid]

Thanks @jfuss! Yeah, I'm simply trying to do custom responses to OPTIONS requests -- has nothing specifically to do with CORS.

I have multiple GET, POST, and PUT rules -- all work correctly. But their corresponding OPTIONS rules are what I'm having trouble with. It simply doesn't obey all of them. I'll try to pare it down to as few rules as possible so it's easily reproducible. I'm fairly certain it has something to do with the path parameters not matching correctly -- though I have no idea why they work fine with the non-OPTIONS rules.

[jfuss]

Finally, got the root cause here. This is a bug on our end and was pretty difficult to find.

I pulled the head of master locally (equivalent to 0.3.0) and after some long digging found this while inspecting the Flask self._app in service.py.

Pdb) p self._app.url_map
Map([<Rule '/healthcheck' (OPTIONS) -> /healthcheck>,
 <Rule '/ensure_user' (OPTIONS) -> /ensure_user>,
 <Rule '/ensure_user' (PUT, OPTIONS) -> /ensure_user>,
 <Rule '/healthcheck' (HEAD, OPTIONS, GET) -> /healthcheck>,
 <Rule '/graphql' (HEAD, OPTIONS, GET) -> /graphql>,
 <Rule '/graphql' (POST, OPTIONS) -> /graphql>,
 <Rule '/graphql' (OPTIONS) -> /graphql>])

You can see OPTIONS is getting added to every request but there are many rules for each endpoint. The many rules is fine (just caused by the way we define a route). The interesting thing is, why does OPTIONS get appended to every route. Well in Flask 0.8, a provide_automatic_options functionality was added and is on by default. When this is enabled, Flask checks to see if the list of methods pass into add_url_rule has an options method and adds it if it doesn't. The reason /graphql was only returning a 200 was because of this (Flask will not invoke the _request_handler if this is enabled).

There are a couple things, I think we need to do here:

1. This would have been addressed if we called add_url_rule once per endpoint, since for the functions that would have added OPTIONS would have worked as expected.
2. The first only hides the root problem. Since customers defines what endpoints/https verbs they want, we should be disabling this feature in Flask.

@charsleysa I apologize for not going deeper into this originally. The reason I didn't catch this in the first place was due to not completely keeping the template the same (I scoped this down to make it easier for me to understand but didn't keep the AWS::Serverless::Function's identical. The problem is limited to endpoints defined with OPTIONS and other HTTP Verbs. You may not see the issue because it is dependent on the order we add_url_rule (why it worked for /healthcheck but not /graphql.

Updating Labels to reflect current state.

[hobotroid]

Great find @jfuss! I was able to narrow down the bug to a single GET and a single corresponding OPTIONS request. It looks like you've got a good lead on the bug already, but just in case, here's the template.yml to duplicate:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Globals:
  Function:
    Runtime: nodejs6.10
    Timeout: 10
Resources:
  VerifyUserFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handlers/VerifyUser.handler
      Policies: AmazonDynamoDBFullAccess
      Events:
        VerifyUser:
          Type: Api
          Properties:
            Path: /users/verify/{token}
            Method: GET
  VerifyUserOptionsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handlers/Options.handler
      Policies: AmazonDynamoDBFullAccess
      Events:
        verifyUserOptions:
          Type: Api
          Properties:
            Path: /users/verify/{token}
            Method: OPTIONS

The OPTIONS request to /users/verify/{token}, in this case, does not go through its assigned handler.

[charsleysa]

@jfuss thanks for your work in finding the problem! That definitely was a hidden one.

@hobotroid thanks for your help as well!

[hobotroid]

@jfuss Out of curiosity, is there a workaround for this that I can use until the fix comes out? Some way to just force all OPTIONS requests to respond with specific headers (Access-Control-Allow-Methods, Access-Control-Allow-Origin, Access-Control-Allow-Headers)? I only ask because I'm hitting a wall at this point, since a handful of my requests simply don't work without the correct OPTIONS responses.

[pcx]

is there a workaround for this that I can use until the fix comes out?

+1. Reverting to an older version of aws-sam-local seems like the only option for now.

[jfuss]

Reopening. This will be addressed in #468

[jfuss]

This was released as apart of v0.4.0

Closing

[mohammedzamakhan]

I am using sam 0.5.0, and I am getting 403 Forbidden for OPTIONS call

2018-08-13 15:10:01 127.0.0.1 - - [13/Aug/2018 15:10:01] "OPTIONS /siteModel/dsmSurface HTTP/1.1" 403 -

[skix123]

Same here, getting 403 for OPTIONS call

SAM CLI, version 0.6.0

[meza]

I think the previous flask fix regressed. Getting automatic OPTIONS again in 0.6.1

[jfuss]

Re-opening.

@mohammedzamakhan, @skix123, or @meza:
Could one of you provide a template so we can reproduce this?

We haven't change anything with Flask or the start-api command recently, so looks like I never fully solved the root issue here.

[andres-lowrie]

Not sure if this is relevant or not but I landed on this issue from

https://github.com/awslabs/aws-sam-cli/pull/468

and I didn't know if this merits a new issue so I'm adding it here (apologizes if it's not relevant)

---

I was getting this issue when running sam local start-api:

[I] visitor-forms:be-forms* λ sam --version
SAM CLI, version 0.6.2
[I] visitor-forms:be-forms* λ sam local start-api --debug
2018-11-17 11:25:55 local start-api command is called
2018-11-17 11:25:55 No Parameters detected in the template
2018-11-17 11:25:55 2 resources found in the template
2018-11-17 11:25:55 Found Serverless function with name='VisitorFormsFunction' and CodeUri='.'
2018-11-17 11:25:55 Trying paths: ['/Users/andreslowrie/.docker/config.json', '/Users/andreslowrie/.dockercfg']
2018-11-17 11:25:55 Found file at path: /Users/andreslowrie/.docker/config.json
2018-11-17 11:25:55 Found 'auths' section
2018-11-17 11:25:55 Auth data for https://index.docker.io/v1/ is absent. Client might be using a credentials store instead.
2018-11-17 11:25:55 http://localhost:None "GET /v1.35/_ping HTTP/1.1" 200 2
2018-11-17 11:25:55 No Parameters detected in the template
2018-11-17 11:25:55 2 resources found in the template
2018-11-17 11:25:55 Detected Inline Swagger definition
2018-11-17 11:25:55 Lambda function integration not found in Swagger document at path='/' method='post'
2018-11-17 11:25:55 Found '0' APIs in resource 'ServerlessRestApi'
2018-11-17 11:25:55 Found '1' API Events in Serverless function with name 'VisitorFormsFunction'
2018-11-17 11:25:55 Removed duplicates from '0' Explicit APIs and '1' Implicit APIs to produce '1' APIs
2018-11-17 11:25:55 1 APIs found in the template
2018-11-17 11:25:55 Trying paths: ['/Users/andreslowrie/.docker/config.json', '/Users/andreslowrie/.dockercfg']
2018-11-17 11:25:55 Found file at path: /Users/andreslowrie/.docker/config.json
2018-11-17 11:25:55 Found 'auths' section
2018-11-17 11:25:55 Auth data for https://index.docker.io/v1/ is absent. Client might be using a credentials store instead.
Traceback (most recent call last):
  File "/usr/local/bin/sam", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args[1:], **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/samcli/commands/local/start_api/cli.py", line 56, in cli
    parameter_overrides)  # pragma: no cover
  File "/usr/local/lib/python2.7/site-packages/samcli/commands/local/start_api/cli.py", line 90, in do_cli
    service.start()
  File "/usr/local/lib/python2.7/site-packages/samcli/commands/local/lib/local_api_service.py", line 74, in start
    service.create()
  File "/usr/local/lib/python2.7/site-packages/samcli/local/apigw/local_apigw_service.py", line 81, in create
    provide_automatic_options=False)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 62, in wrapper_func
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 976, in add_url_rule
    rule = self.url_rule_class(rule, methods=methods, **options)
TypeError: __init__() got an unexpected keyword argument 'provide_automatic_options'

My template looks like this:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
    visitor-forms

    Handles the submission of the "forms" droplet in any site

Resources:

    VisitorFormsFunction:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: .
            Handler: index.handler
            Runtime: nodejs8.10
            Events:
                Submission:
                    Type: Api
                    Properties:
                        Path: /
                        Method: post
Outputs:

    VisitorFormsApi:
      Description: "API Gateway endpoint URL for Prod stage for VisitorForms function"
      Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/VisitorForms/"

    VisitorFormsFunction:
      Description: "VisitorForms Lambda Function ARN"
      Value: !GetAtt VisitorFormsFunction.Arn

    VisitorFormsFunctionIamRole:
      Description: "Implicit IAM Role created for VisitorForms function"
      Value: !GetAtt VisitorFormsFunctionRole.Arn

Even though my sam version matches last release, I figured I'll do the ol' reinstall to see if that fixes it, ... which it appeared to do so.

Looks like new version of Flask was pulled down

Installing collected packages: six, Flask
  Found existing installation: six 1.10.0
    Uninstalling six-1.10.0:
      Successfully uninstalled six-1.10.0
  Found existing installation: Flask 0.10.1
    Uninstalling Flask-0.10.1:
      Successfully uninstalled Flask-0.10.1
Successfully installed Flask-1.0.2 six-1.11.0

I thought this might help with issue
Again apologies if this is not relevant here.

[jfuss]

@andres-lowrie For whatever reason, the Flask version of 1.0.2 wasn't getting picked up. This is something wrong with the installation on your system. If you are currently using pip to install, I would uninstall the CLI (pip uninstall --user aws-sam-cli and pip3 uninstall --user aws-sam-cli if you have python3 as well) and move to our new installers. This will give you an isolated install of the CLI and ensure the correct versions of all dependencies are install correctly.

[andres-lowrie]

Ty.. will do

[rayhaanq]

Hi, I've got the same issue. Getting 403 for OPTIONS requests to my graphql endpoint for sam local. Is there any work around for this at the moment? CLI version 0.10.0
@jfuss

[cl0ckwork]

Hi, I've got the same issue. Getting 403 for OPTIONS requests to my graphql endpoint for sam local. Is there any work around for this at the moment? CLI version 0.10.0
@jfuss

@rayhaanq Give this a try in your cloudformation yaml

Globals:
  Api:
    # enable CORS; to make more specific, change the origin wildcard
    # to a particular domain name, e.g. "'www.example.com'"
    Cors:
      AllowMethods: "'OPTIONS,GET,POST,PUT,DELETE'"
      AllowHeaders: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key'"
      AllowOrigin: "'*'"

Resources:
   OptionsFunction:
    Type: AWS::Serverless::Function
    Properties:
# create a custom handler to return 200 and appropriate headers for your OPTIONS requests
      FunctionName: options-handler
      Handler: index.options #nodejs
      Events:
        Options:
          Type: Api
          Properties:
            Path: /{cors+}
            Method: OPTIONS
            Auth:
              Authorizer: NONE

[rayhaanq]

@cl0ckwork awesome, this worked. I just set up another function for options to return a success response. Thanks

[chau-bao-long]

Finally make it work after a long day suffer from this issue.

It's a good idea to write another lambda for options method which response successful pre-light check.

I didn't notice that AWS has mentioned it in their docs
https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-cors.html

[a-bx]

It's a good idea to write another lambda for options method which response successful pre-light check.

yeah but requests are too slow in local

[jfuss]

Tracking issue for CORS is #323. See that issue for more details.

The original request here was Flask responding to OPTIONS requests which was solved and release: https://github.com/awslabs/aws-sam-cli/issues/400#issuecomment-398394463

There was addition reports of Flask still responding here: https://github.com/awslabs/aws-sam-cli/issues/400#issuecomment-433442194 but no response from those parties.

Closing at this looks to be solved and other comments relate to CORS support, which has it's own issue

[gordonmleigh]

@jfuss I've still got this problem with sam 0.22.0 (from brew) and a CDK-generated template.

See my repro repo.

SAM shows that the lambda is not executed for an OPTIONS request, but is for a GET:

Mounting TestFunction22AD90FC at http://127.0.0.1:3000/{proxy+} [DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT]
Mounting TestFunction22AD90FC at http://127.0.0.1:3000/ [DELETE, GET, HEAD, OPTIONS, PATCH, POST, PUT]
You can now browse to the above endpoints to invoke your functions. You do not need to restart/reload SAM CLI while working on your functions, changes will be reflected instantly/automatically. You only need to restart SAM CLI if you update your AWS SAM template
2019-10-15 14:21:50  * Running on http://127.0.0.1:3000/ (Press CTRL+C to quit)
Invoking lambda.handler (nodejs10.x)
2019-10-15 14:21:55 Found credentials in shared credentials file: ~/.aws/credentials

Fetching lambci/lambda:nodejs10.x Docker container image......
Mounting /Users/gordonmleigh/Projects/scratch/sam-options-repro/lib as /var/task:ro,delegated inside runtime container
START RequestId: 9696c7d8-792d-11b4-8318-9996464153fc Version: $LATEST
END RequestId: 9696c7d8-792d-11b4-8318-9996464153fc
REPORT RequestId: 9696c7d8-792d-11b4-8318-9996464153fc  Duration: 18.30 ms      Billed Duration: 100 ms Memory Size: 128 MB     Max Memory Used: 42 MB
2019-10-15 14:21:58 127.0.0.1 - - [15/Oct/2019 14:21:58] "GET / HTTP/1.1" 200 -
2019-10-15 14:22:03 127.0.0.1 - - [15/Oct/2019 14:22:03] "OPTIONS / HTTP/1.1" 200 -

[gordonmleigh]

Though #1434 seems to reproduce it more easily. Should have checked for more recent issues first. I'll follow that issue.