Serverless-application-model: Using Variables in External Swaggers File Breaks Deployment

[andrewryan1906]

I've been using the examples to try and deploy with CloudFormation and kept getting an "Invalid ARN Found" message. I realized that the problem was that I had to use DefinitionBody and do an AWS::Include to inline my swagger file, vs use it externally.

This works.

But my problem now is, my swagger file has to be sitting up on S3 for me the CloudFormation stack to work. My Swagger file is part of my project and I'd like it to get built & deployment when I run aws cloudformation package/deploy commands.

So what's the best way to do this? How can I use inline Swagger (so I get the ARN variables) but also get CloudFormation to automatically deploy the file to S3 fo me?

[andrewryan1906]

Just a follow up.

I can use DefinitionBody to reference an external Swagger file, but if I do, I lose the CloudFormation transformation variables and I have to hardcode the ARN of my labmdas into my Swagger. This would mean I would need to manually create all of my lambda functions first, before I could use a SAM script, which is obviously less than ideal.

But I just realized that sam-local freaks out when I try to start it up and my SAM template is referencing a swagger file inline with DefinitionBody; so that really now is pushing me to use an external swagger file.

So my updated question is; how can I use an external swagger file and reference it with DefinitionUri, but not have to reference to my lambda function's ARN explicitly - and thus still get the dynamic creation of lambda functions on deploy?

[xdissent]

You can reference stage variables in the api integration uri:

# main template
Resources:
  Api:
    Type: AWS::Serverless::Api
    Properties:
      DefinitionUri: api.yml
      Variables:
        MyFn: !Ref MyFn
  MyFn:
    Type: AWS::Serverless::Function
    Properties:
      Events:
        Api:
          Type: Api
          Properties:
            Path: /my-fn
            Method: GET
            RestApiId: !Ref Api

# swagger
paths:
  /my-fn:
    get:
      x-amazon-apigateway-integration:
        httpMethod: POST
        type: aws_proxy
        uri: arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/arn:aws:lambda:us-east-1:00000000:function:${stageVariables.MyFn}/invocations

You still have to hardcode the region and account ID, but at least you don't have to know the fn arn ahead of time.

[michaelj-smith]

Oh, that's a nice trick, @xdissent . I wasn't aware that this was available in external swagger. It would be nice to get the Swagger example updated to show this approach.

@andrewryan1906 Based on your descriptions of what you want, these are the reasons that have led me to use inline swagger, using the DefinitionBody property. I don't have any problem with SAM Local and inline swagger, so I wouldn't give up on that yet.

[I159]

From my latest experience. Not stage variables neither pseudo parameters don't work in json formatted external Swagger files. Variables of ${} syntax don't substitute with values. At the same time if you need to declare api gateway policy than you forced to use json format because api gateway policy valid only in json format.

[ekcrisp]

@andrewryan1906 I just submit a PR that will allow swagger specs referenced using AWS::Include to reference local files

[gitorrin]

@andrewryan1906, I had the same issue with CloudFormation and having to reference API definition from S3. The file has to be pre-uploaded to S3, indeed.
I solved it by copying it from the Build step of my CodePipeline, as a command in the buildspec.

@xdissent , region and accountId don't have to be hard-coded. CloudFormation will pick up those, from the context it's currently executing.
Example:

MyLambda:
    Type: 'AWS::Serverless::Function'
    Properties:
      FunctionName: FunctionA
      Handler: handlerConfig
      Runtime: dotnetcore2.0
      CodeUri: 
        Bucket: !Ref BucketA
        Key: 'Folder1/deploy.zip'      
      MemorySize: 256
      Timeout: 59
      Role: !Ref myRole      
MyAPI:
    Type: "AWS::Serverless::Api"
    Properties:
      Name: MessageAPI      
      StageName: latest
      DefinitionBody: 
        Fn::Transform:
          Name: AWS::Include
          Parameters:            
            Location: 
              Fn::Sub: 's3://myBucket/MyAPI/swagger.yaml'

And here's a snippet from the swagger file:

x-amazon-apigateway-integration:

        uri: 
          Fn::Sub: 'arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyLambda.Arn}/invocations'

A full list of pseudo parameters reference is available here - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/pseudo-parameter-reference.html

[0xdevalias]

@ekcrisp I just submit a PR that will allow swagger specs referenced using AWS::Include to reference local files

Are you able to link to that PR by chance? I didn't see it when I just looked.

My interest is slightly different.. but I want to be able to break up my template file into 'logical sections', and use AWS::Include to combine it all back together, but I don't believe SAM will currently upload those included files to S3 (which this PR sounds like it would do) I will open another issue about this specifically and link back to here.

[hectorvs]

@gitorrin that's a good tip, but the reason it works is because you're using Fn::Transform.

That declaration will essentially download the referenced file from s3 and add it to the cloudformation template before processing. Therefore, the ${} substitutions will work.

This is good and all, but it forces you to upload the file manually to a bucket before deploying.

For this, the aws cli has a very convenient command.

aws cloudformation package
https://docs.aws.amazon.com/cli/latest/reference/cloudformation/package.html

When you run package, it will take a reference to a local file / folder, zip it up, upload it to an s3 bucket, and modify the template with the s3 uri it generates.

It supports a bunch of references, including DefinitionUri, however the Fn::Transform isn't supported.

I believe the stageVariables method is useful... I don't see why you couldn't pass the region as a value there as well 🤔

Edit:

Just read up on this issue a bit more on the repo and there's work to get something similar to this going :)

[ekcrisp]

@0xdevalias I actually submit a PR that covers the broader case that you mention. We had a similar need. Here is the PR, I'm waiting on a review, it uploads referenced documents to S3 for AWS::Include transforms anywhere in your template that reference a local file.

https://github.com/aws/aws-cli/pull/3454