Aws-sam-cli: Document how to use sam build in combination with package and deploy

[mwarkentin]

Describe your idea/feature/enhancement

There isn't a clear indication of how to use sam build in combination with sam package and sam deploy. Specifically, after using sam build, you do not need to pass the --template option to sam package as it will be detected automatically. If you do pass --template things get screwy with the CodeUri parameter as sam build wants you to point it at your src directory and sam package wants you to point it at an artifact to be uploaded to s3.

Proposal

Update the docs and help commands (maybe?) for the specific commands and in a more general overview of how the various pieces fit together.

Things to consider:

1. Will this require any updates to the SAM Spec: No

Additional Details

sam-dev slack discussion

Michael Warkentin [4:30 PM]
What’s the proper way to configure our template when we have a Python project w/ requirements.txt and want to use `sam build`, `sam package`, and `sam deploy`?

`sam build` seems to want us to point `CodeUri` at our src directory, but then for `sam package` we need to point it at the built artifact? Am I missing something?
The docs seem to reference a `Code` property that I thought might reference our src and then `CodeUri` could reference the artifact, but that doesn’t seem to be working: `samtranslator.model.exceptions.InvalidDocumentException: [InvalidResourceException('ZendeskOnDemand', 'property Code not defined for resource of type AWS::Serverless::Function')]`

Jason Kusar [4:32 PM]
the key to make it work properly is to not specify the --template parameter when you do sam package

Jacob Fuss [4:32 PM]
`sam package` will pick up the build artifacts. So if you use `sam build` with `sam package`, you don’t need to worry about it.

Jason Kusar [4:32 PM]
just let it auto-discover the one that was created by build
It's not clear from the docs.  I found out that trick quite by accident

Jacob Fuss [4:33 PM]
On the second part: `AWS::Serverless::Function` does not have a `Code` property it has a `CodeUri` property.
The `Code` Property is specific to `AWS::Lambda::Function`

Jason Kusar [4:34 PM]
CodeUri in your template points to your src directory.  sam build will create a new template with CodeUri pointed at the .aws-sam/build directory with the code and dependencies.  sam package will generate yet a third template with CodeUri pointing to the package in S3. (edited) 

Michael Warkentin [4:36 PM]
Thanks, will give that a shot :slightly_smiling_face:

Michael Warkentin [4:42 PM]
Any idea where the best place to document that would be? I could take a crack at a PR. Probably on the `sam package` docs?

Jason Kusar [4:43 PM]
I was just wondering the same. :slightly_smiling_face:
Did it work for you?

Jason Kusar [4:54 PM]
I think there needs to be some documentation that brings together how build, package, and deploy work together.  Perhaps that's more appropriate for the SAM docs inside the AWS documentation, but it seems to me that a lot of things are documented individually but not necessarily the interaction between them.

Jacob Fuss [4:58 PM]
@JasonMK or @Michael Warkentin could either of you capture this in a github issue. Adding it to the help text of package might be a good place to start but the documentation Jason describes would also be useful (which would be an update here: https://github.com/awsdocs/aws-sam-developer-guide).
awsdocs/aws-sam-developer-guide

You can capture it all in SAM CLI’s github

[damien-monni]

I needed a few hours and this issue before to understand why my NPM packages were not included when deploying. It would be great if this would be documented somewhere :wink:

[ba7op]

Most likely related.
I'm using SAM with python3.6 and my packages were not being included in the bundle uploaded to s3 when running sam package --debug --output-template-file packaged.yaml --s3-bucket $(BUCKET_NAME).
Luckily I found this issue (thanks!), tried removing the --template-file param and now everything works like a charm.
The current behavior is confusing but maybe also wrong?

[sanathkr]

With the latest SAM CLI release, this should not be an issue any more. We have made some improvements to make build, package and deploy consistent.

Following should work:

(NO need to pass template file to any command)
$ sam build

# No `sam package` needed. Deploy will autopackage. No template also needed
$ sam deploy --s3-bucket mybucket --stack-name mystack --capabilities CAPABILITY_IAM

[ba7op]

@sanathkr would I still be able to specify a package file to sam deploy? We use this to rollback:
sam deploy --template-file packaged-prev.yaml

[sanathkr]

Yes absolutely. The new experiences are fully backwards compatible.

[ba7op]

Great!
Thanks :)

[cognitiaclaeves]

Just wanted to give you a head's up that I ran into this issue with SAM CLI, version 0.47.0 (which led me to upgrade to 0.48.0, which did not fix my situation.)

I had a script that I was using which was using this:

sam package     --template-file template.yaml     --output-template-file packaged.dev.yaml     --s3-bucket dev-artifacts --s3-prefix devops-sam-lambda-deployments/adv-dev-pusher

It took me hours to trip on the right google key words to find this issue ... so I wouldn't say that 'this should not be an issue anymore' ...

[revolutionisme]

This issue actually wasted quite some of my time. There should be a warning or something when someone specifies the template file on 'package' or 'deploy' command, that the contents of build won't be used.

[mhemani-fuze]

This issue also caused some frustration, some clearer documentation around sam package with --template-file not including a project's dependencies would be greatly appreciated!

[jfuss]

How do we make this better and clearer? Passing in the value for into the command should indicate the command is going to operate on that. It seems like we are missing something, but not sure what.

[revolutionisme]

I believe the problem is that, as the same location could have more than 1 template, thus it's not so intuitive to use template file to build using sam build and then not specify the template file for the deploy process using sam deploy

Maybe one command which builds and deploys using a given template file would be great. But again this is just, my opinion.

[ronaldour]

I believe it should be a good idea to allow the sam package command to package the template using the artifacts produced by sam build without actually 'building' the functions (downloading the dependencies) again.
For example if I have a template with many functions and if I change something in the template that's not directly related with the functions code, for example a function description, I still have to run sam build in order for my changes to take effect and this increases the time by having to build everything again, it would be nice if I could run sam package instead and get the same result.
Maybe keeping track of the artifacts with some kind of metadata you can make the sam package use the artifacts generated by sam build and make it work even if using --template-file

[elenasch]

Agree with many of the above comments regarding usability of sam commands, and want to point out that the documentation issue _isn't fixed._ Nowhere in the package documentation does it say not to use --template-file param in order to pickup artifacts built by build command. Additionally the output from sam package --s3-bucket lambda-poc --s3-prefix dev --output-template-file packaged-template.yaml --debug command states

Execute the following command to deploy the packaged template
sam deploy --template-file <path to my output template>/packaged-template.yaml --stack-name <YOUR STACK NAME>