Serverless-application-model: AWS::ApiGateway::UsagePlan cannot use a stage from AWS::Serverless::Api

[JasonQuinn]

I have a AWS::Serverless::Api defined as
``` MyApiResource:
Type: AWS::Serverless::Api
Properties:
StageName: test
DefinitionUri: swagger.json

and a AWS::ApiGateway::UsagePlan defined as 
```  UsagePlan:
    Type: AWS::ApiGateway::UsagePlan
    Properties:
      ApiStages:
        -
          ApiId: !Ref MyApiResource
          Stage: test
      Throttle:
        BurstLimit: 500
        RateLimit: 50
      UsagePlanName:
        Fn::Join: [
            "_",
            [
              "UsagePlan",
              !Ref "AWS::Region",
              !Ref "AWS::StackName"
            ]
          ]

If I run the cloud formation script it fails to create the UsagePlan with an error saying the stage test does not exist on the API.

If I instead comment out the usage plan and run the script and then run it again with the API and the usage plan enabled it will create the usage plan and link it to the stage properly.

[JasonQuinn]

So what i had to do to fix this was add DependsOn to the AWS::ApiGateway::UsagePlan as DependsOn: MyApiResourcetestStage not sure if there is a better way to do this.

[sanathkr]

DependsOn is the right solution. Your UsagePlan was being created before the Stage resource.

[dinvlad]

Alas, that still feels a bit hacky, since we're relying on the implicit naming scheme for the stage resource (which gets created by the Api resource), as opposed to referring to it directly. I.e. if MyApi resource has a StageName: MyStageName, then the DependsOn for a UsagePlan resource must list "MyApiMyStageNameStage" literal, otherwise it won't work. This naming scheme is not listed anywhere in the spec AFAIK, thus it may be unreliable (and not very readable).

[dinvlad]

@sanathkr, we're seeing the same problem for AWS::ApiGateway::BasePathMapping as well. Either AWS::Serverless::Api should wait until its implicit Stage resource is fully created (so other resources that depend on Api may use that stage), or we have to rely on that _undocumented_ Stage resource naming scheme with DependsOn (or just use AWS::ApiGateway::RestApi instead altogether for such cases).

[bargom]

The serverless approach is nice and clean but not much mature in the api keys/stages/usage plans area. It creates an extra "Stage" (named "Stage") even if we define our own stage, and the field is mandatory.

I couldn't find a permanent solution for the dirty stacks, and usage plans inside the cloudformation script, so I wrote a bash script that deleted the stage named "Stage" and creates api keys, usage plans then assigns keys to plans. Parsing JSON in the bash, clearing the stages, takes away the clean and neat solution of serverless in the Api gateway : (

Here is the script:

#!/usr/bin/env bash

echo "==============================="
echo "Get id of the api (set output param ApiId in CF)"
echo "==============================="
APIID=$(aws cloudformation describe-stacks \
--stack-name my-stack \
--query 'Stacks[0].Outputs[?OutputKey==`ApiId`].OutputValue' \
--output text)

echo "==============================="
echo "Delete the Stage env"
echo "API ID: ${APIID}"
echo "==============================="
aws apigateway delete-stage \
--rest-api-id ${APIID} \
--stage-name 'Stage'

echo "==============================="
echo "Create usage plan for stage name: v1"
echo "API ID: ${APIID}"
echo "==============================="
USAGEPLANJSON=$(aws apigateway create-usage-plan \
--name "my-usage-plan" \
--description "Usage plan for low usage of api" \
--api-stages apiId="${APIID}",stage='v1' \
--throttle burstLimit=2,rateLimit=1)
echo ${USAGEPLANJSON}

echo "==============================="
echo "Created usage plan for v1"
echo "API ID: ${APIID}"
echo "Getting Usage plan Id for:"
echo "==============================="
USAGEPLANID=$(echo ${USAGEPLANJSON} | python -c 'import sys, json
print(json.load(sys.stdin)["id"])')

echo "==============================="
echo "Creating api key for v1"
echo "==============================="
APIKEYJSON=`aws apigateway create-api-key \
--name 'my-api-key' \
--description 'Used for security of the admin api' \
--enabled`
echo ${APIKEYJSON}

echo "==============================="
echo "Created api key for v1"
echo "==============================="
APIKEYID=$(echo ${APIKEYJSON} | python -c 'import sys, json
print(json.load(sys.stdin)["id"])')

echo "==============================="
echo "Create usage plan key for v1"
echo "API ID: ${APIID}"
echo "USAGE PLAN ID: ${USAGEPLANID}"
echo "API KEY ID: ${APIKEYID}"
echo "==============================="
aws apigateway create-usage-plan-key \
--usage-plan-id ${USAGEPLANID} \
--key-type "API_KEY" \
--key-id ${APIKEYID}

[jfuss]

@bargom _DO NOT_ delete the Stage Stage outside of CloudFormation. You are creating more of a headache for yourself. When you go to deploy a change, CloudFormation will try and update the resource and fail since the resource does not exist. You should never be deleting resources that are controlled by CloudFormation outside of CloudFormation (console, script, etc).

Just leave Stage Stage alone. It isn't harming anything or anyone by being there. Yes, we know this happens. For more info, see #191.

[bargom]

I see your point. But in my opinion, "Stage"-stage is a security breach. I setup all my keys and environment (usage plans) for the stage I know. So, if I leave the "Stage" there, a developer can guess the endpoint, and can reach it freely. I don't want to leave any open points that I don't have control.

I think also deploying changes over existing API Gateway is a confusing way to deploy. I deploy the full stack with a version number and run it in parallel until everything is confirmed, then switch to new version and delete the old one. I believe if someone following this approach and using API keys, it is best to delete the uncontrolled "Stage".

Thanks for letting me know it causes problems in update. I agree modifying a stack after CF is not ideal, but until there is a solution in serverless framework for usage plans, api keys; I will follow this way.

[Nr18]

@bargom if you are using Lambda proxies you could manage the invocation of the lambda function from your "allowed" stages using the AWS::Lambda::Permission resource.

This way you could leave the Stage stage and it cannot be used.

LogicalResourceName:
  Type: AWS::Lambda::Permission
  Properties:
    Action: lambda:InvokeFunction
    FunctionName: arn:aws:lambda:region:account-id:function:function-name:alias-name
    Principal: apigateway.amazonaws.com
    SourceArn: arn:aws:execute-api:region:account-id:api-id/stage-name/*

[bargom]

@Nr18 that is a great idea. Some of my endpoints are using proxies, but most of them are connecting to lambdas w/o proxy, and some calls are passing parameters (and jwt tokens) to 3rd party API's to create a gateway to external application from internal apps.