Aws-sam-cli: Specifying --template-file does not upload Python dependencies

[tcco]

Description:

When specifying the template file with --template-file with the sam package command, my python dependencies are not included in my zip that gets uploaded to s3, subsequently is not importable in my lambda function.

My zip when I allow the template file to be defaulted is 7.2MB but when specified is 17KB. It only includes my Flask application code but no dependencies. It is even viewable in the online code editing window of Lambda.

Steps to reproduce the issue:

1. Create another template say dev-template.yaml
2. Run sam package --template-file dev-template.yaml --output-template-file packaged.yaml --s3-bucket S3_BUCKET_NAME

Observed result:
View in s3 that artifact is not the same size or that lambda is unable to run. In this case, I have api gateway and it returns an internal server error. Along with that, I've rm -rf .aws-sam and run sam build - dev-template.yaml prior and observed the same result. Even after nuking the .aws-sam, all my dependencies are built locally but not packaged.

Expected result:
The .aws-sam/build/lambda-function-name folder would get fully uploaded, similar to when template file is not specified and run with default values.

Sam Version: 0.17.0
Python Version: 3.7.3

[jfuss]

@tcco When you are using build with package, you do not need to pass the template to the package command. SAM CLI will pick up the built template. Since you are specifying the template-file in the command, that is what is used to find where your function code is and upload to S3. So the behavior you are seeing is expected.

Closing as the behavior is expected and if you want the build template/code to be picked up, do not pass template-file to the package command

[tcco]

Thank you @jfuss. I started by adding the template file to the package command. I followed your guidance and it behaved as expected.

[mikelax]

We just ran in to this exact issue, and I have a tough time saying that behavior is working as expected. I would think at the very least there should be documentation added to the sam package page.

I'm happy to write up an issue to track the doc change update, but I don't 100% understand even why this is working as expected in the first place.

[jfuss]

@mikelax Are you passing the --template|--template-file parameter into the sam package command? If so, you are telling the command to use that specific template and will be pointing to the source and not built code. If you are using sam build, then remove the --template|--template-file parameter from the sam package command to allow the cli to auto discover the built template.

If that is not what is happening, cut a new issue.

[mikelax]

@jfuss thanks for the response. I understand what you are saying and we have everything working now as it should. My point was that from the SAM documentation, it's not clear what you explained here is what is actually happening.

Maybe just some updates to the cli documentation page for sam package for the template-file option.

[nlothian]

I got bitten by this today.

I _think_ it means that with no template parameter it uses the (generated, processed) template file from the .aws-sam/build directory, which references the code correctly. If you do pass this parameter it's extremely unclear how it is supposed to work.

One would think that doing sam build --template somefile.yaml then sam deploy --template-file somefile.yaml would work, but it doesn't.

That's pretty surprising behaviour! Also it's not at all clear how one would actually get a second template file to work? Is the idea you do the build with it, and then update the references within it to point to the built code?

[gideon-maina]

If you don't want to deal with all the package issues do this

1. create a codebuild role for the build stage
2. give that role the necessary permissions for Lamda, IAM, S3 and whatever else your function needs
3. then in your buildspec include sam build and sam deploy in the list of commands

N.B : This would mean that you remove the deploy stage in your pipeline if you have one , as the deployment takes place in the build phase.

[szabolcs-dekany]

I got bitten by this today.

I _think_ it means that with no template parameter it uses the (generated, processed) template file from the .aws-sam/build directory, which references the code correctly. If you do pass this parameter it's extremely unclear how it is supposed to work.

One would think that doing sam build --template somefile.yaml then sam deploy --template-file somefile.yaml would work, but it doesn't.

That's pretty surprising behaviour! Also it's not at all clear how one would actually get a second template file to work? Is the idea you do the build with it, and then update the references within it to point to the built code?

It is indeed a bit confusing to me.
How would you approach handling two different lambdas with the same code base? I Thought using two template files through building and deployment could be used for this purpose

[alexhung]

I lost at an hour today to this.

I used the -t template.dev.yml to deploy to different account/environment. I did not expect that SAM behaves differently since there is no documentation about this behavior.

Now I have to device another scalable way to maintain different VPC and ALB info for each environment.

BTW, this is for a python lambda. I have used the -t successfully with a nodejs lambda

[theholy7]

I was also caught by this today. Lost a few hours until I found this thread. Defo not obvious.

For the record, I was creating 2 templates based on a parametrized one where I was replacing variables with sed.

Then I was pointing sam package to the correct template, but I was running sam deploy before.

[kokospapa8]

I have stumbled upon this thread.
if you are using multiple templates, refer to this post
https://stackoverflow.com/questions/59815363/aws-sam-cli-ignoring-my-python-dependencies-during-build-package-and-deploy/59815472#59815472?newreg=f43dce66b39b417fbafb8b337673e2f4

sam build -t <your_manifest>.yaml
sam deploy -t .aws/build/template.yaml