Aws-sam-cli: watch Option for SAM Build command

[jfuss]

Describe your idea/feature/enhancement

When using sam build to build functions that will later be invoked through sam local [invoke|start-lambda|start-api], you need to run sam build before each invoke. The reason for this requirement is because the build command places artifacts (code + dependencies + updated template) into a build folder. The sam local [invoke|start-lambda|start-api] use these built templates (this has updated locations to the built function code) by default but there is no way to update the built code without running sam build again.

Proposal

For the initial implementation of sam build, the --watch option was out of scope. The proposal is to add support for this option to all customers to enable building when source code changes.

An alternative could be to make the sam local suite of commands build the function during invoke. The main concern with this is the speed at which you can now invoke (other build enhancements might be a requirement for this, speed, incremental builds, etc).

Workaround

All of the sam local command allow 'reloading' of the function code. This means for a given template, if you update the contents of a functions Code/CodeUri, SAM CLI will mount this updated content. So the current option for making interactions easier is to have two terminal windows open, one that builds and one that you local invoke with.

[alxx]

To me this only happens using Ruby once I've added a gem into the equation, with my functions having Gemfiles in their respective directories. Then I'm forced to build using --use-container parameter, which takes a lot of time; simply running sam build doesn't work because the app doesn't find the gems.

If I don't use gems at all, then I have live reloading; but once I need gems, everything only works by rebuilding inside a container. And since that takes about a minute, it makes SAM pretty much unusable for me. I can't build every time I make a small change, it just takes too long.

[AmillerSC]

Is there any progress on this? Having to build every time for the most minor of code changes is not ideal.

[sliedig]

I'd like to see a slightly different take on this proposal. Rather than having a continual build function through --watch option, only build functions that have changed since the last build. I often find myself having to rebuild functions that haven't been modified, or just making changes to the template file, making the feedback loop unnecessarily long.

[tamvm]

To me this only happens using Ruby once I've added a gem into the equation, with my functions having Gemfiles in their respective directories. Then I'm forced to build using --use-container parameter, which takes a lot of time; simply running sam build doesn't work because the app doesn't find the gems.

If I don't use gems at all, then I have live reloading; but once I need gems, everything only works by rebuilding inside a container. And since that takes about a minute, it makes SAM pretty much unusable for me. I can't build every time I make a small change, it just takes too long.

Hi @alxx,

So what is your workaround for that? Or can you share your experience on that?

Thank you,

[mims92]

I don't like to compare but Chalice as this feature if I'm not mistaken ;)

[justengland]

Assuming this is hard to build the watch option since this has been open for the better part of a year. It seems an easier option would be to use a docker volume to share your code rather than requiring a build, this way you do not have to have an actual watch.

[dgomesbr]

For a workaround, you're able to simply delete the .aws-sam folder at the root of your project and get back invoking your function.

ref: rm -rf .aws-sam/*at root.

[dz902]

Lacking of a watch mechanism is eating much of my time (and laptop battery). Would love to have this feature. Especially we are so fully-heartedly dedicate ourselves to serverless thus creating lots of APIs and functions that could really use some watching.

[serverhorror]

For a workaround, you're able to simply delete the .aws-sam folder at the root of your project and get back invoking your function.

ref: rm -rf .aws-sam/*at root.

That doesn't work for me. Running in python3.8 now and it doesn't pick up the packages in requirements.txt.

[codecounselor]

If you're using a Node runtime you may want to consider using Webpack to build bundles for each function instead of sam build. Then you can use webpack -w to have it watch for changes which sam local picks up.

I followed this very excellent write-up: https://dev.to/elthrasher/managing-multiple-functions-with-aws-sam-and-webpack-1581
I found https://www.npmjs.com/package/aws-sam-webpack-plugin also but preferred the simplicity of the former setup.

The relevant config:

webpack.config.js

module.exports = {
    entry: {
        functionA: path.join(__dirname, 'src/functionA.js'),
        functionB: path.join(__dirname, 'src/functionB.js'),
    },
    output: {
        libraryTarget: 'commonjs2',
        path: `${__dirname}/build`,
        filename: '[name].js',
    },
    mode: process.env.NODE_ENV === 'dev' ? 'development' : 'production',
    target: "node", // Let webpack know to generate a Node.js bundle
}

template.yaml

  PingFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: build/
      Handler: functionA.lambdaHandler
      Runtime: nodejs12.x

1. npx webpack -w
2. sam local start-api

[kamrankhanwastaken]

I just used nodemon for this. Going to put this here in case it would make anyone else's life easier:

npm i -g nodemon
nodemon --exec sam build

Run this in the root folder in one terminal and run sam local start-api in another. nodemon should build the application automatically on any code changes.

[eamarce]

Hey guys,

So, we created a small npm package based in nodemon (samwatch):

https://www.npmjs.com/package/samwatch

https://github.com/mxitgo/samwatch

The way it works is, it copies your js/json files as you save them from the source folder to the corresponding .aws-sam/build folder (as long as the names are the same for your lambda and the code uri in template.yaml).

That is useful to see your changes reflected on the fly if you are running sam local start-api, so you don't have to run sam build after every change.

On the other hand, if the corresponding file copy is not within the .aws-sam/build folder already, the package will trigger a sam build (unless you use n parameter).

The package has helped us get a hot reload feeling for our local lambda function development with sam.

I hope anyone can find it useful.

Thanks!

[butsjoh]

My usage is also with ruby code so i am using the live reloading as follows currently without actually using sam build. Not sure though if this is recommend or bad practice so would like to get feedback on it.

Let say i have the following folder structure with 2 functions:

lambda_example/
|--hello
||--app.rb
||--Gemfile
|--hello_other
||--app.rb
||--Gemfile
|--template.yaml

instead of using sam build i just do: (i do this from the root directory lambda_example)

# To install new gems for hello (only need to run this when you update the Gemfile or when you don't have a Gemfile.lock yet)
bundle install --gemfile=hello_world/Gemfile

# Now the following installs the gems into the vendor bundle inside the function directory
# similar what sam build does actually if you would look in the .aws-sam folder
bundle install --gemfile=hello_world/Gemfile --deployment --path vendor/bundle

Same thing then for the other function

bundle install --gemfile=hello_other/Gemfile
bundle install --gemfile=hello_other/Gemfile --deployment --path vendor/bundle

Now if you then issue sam local start-api and you make changes to the code then the live reloading stuff should be working without actually needing to build each time. Ofcourse this only probably works if you don't need to build native extensions for gems but esentially you only need to call the bundle install commands above if you change gem dependencies which should not happen that often.

The samwatch thing is cool but does not work for ruby cause it only monitors js or json files it seems. So maybe add an extra option in your cli @eamarce to specify which files to watch?

[paulmowat]

Now that warm containers has been delivered under #2383 . Is there anyway this can be prioritized? It would help make that dev experience so much better.

[anubhavmalik]

Its such a painful process running sam build even for a small change such as a console.log(). Any good workaround? The solutions above (webpack and samwatch) force a folder structure which I am not willing to do.

[mims92]

@anubhavmalik only thing is to modify the code in the build folder.

[dror-g]

such a pain... maybe instead of a "watch" option, which will still need to build on every change and slow down development, we could mount the source folder in the container?