Chalice: Setup environment variables when loading app.py

[JordonPhillips]

Chalice imports app.py during execution here. Naturally this results in all of the code at module level being executed. The problem is that we don't do anything to setup the environment beforehand. This results in an issue where local execution is different than remote execution.

By setting up the environment, we would be able to allow features such as #576 without any additional work. Stage-specific setup would be generally possible.

As an example, here's an app that I may want to deploy:

from chalice import Chalice
from chalice import CognitoUserPoolAuthorizer

app = Chalice(app_name='resources')
authorizer = CognitoUserPoolAuthorizer(
    'MyPool', provider_arns=[os.environ['USER_POOL_ARN']])


@app.route('/', methods=['GET'], authorizer=authorizer)
def index():
    return {'success': True}

With this config:

{
  "version": "2.0",
  "app_name": "resources",
  "stages": {
    "dev": {
      "api_gateway_stage": "api",
      "environment_variables": {
        "USER_POOL_ARN": "arn:aws:cognito-idp:us-west-2:...:userpool/name"
      }
    },
    "prod": {
      "api_gateway_stage": "prod-api",
      "environment_variables": {
        "USER_POOL_ARN": "arn:aws:cognito-idp:us-west-2:...:userpool/othername"
      }
    }

  },
  "profile": "tmp-chalice"
}

As a user, I would expect USER_POOL_ARN to be set based on which stage I'm deploying.

Additionally, you have issues like #573 that suffer from being run locally in a different environment. I'm not sure how we want to handle cases like that though, they may need to be separate cases.

[kadrach]

I have the same problem with chalice package. For example, take a Lambda function with an IAM role that grants read access to an s3 bucket:

import pandas as pd
from chalice import Chalice

app = Chalice(app_name='hello')
data = pd.read_csv('s3://mybucket/some_file.txt')  # no permissions here during `package`

@app.route('/')
def hello():
    # do something with data
    pass

The CI toolchain should not have these permissions. I can catch and ignore the exception, but I'd not want to do this in the actual environment. Is there a clever way to work around this?

[dmulter]

+1

[dmulter]

Until this is resolved, is there any way to determine the stage an app is in during initialization? I could create my own parameters per stage mechanism, but I need some form of api.xxx() to figure out the stage.

[leason]

I've verified this issue on Windows and Linux as well. Essentially what I'm trying to do is setup a DB connection and the Cognito configuration outside of a handler and base them on environment variables since that seems best practice in general. As a work around for now I'm just delaying the DB connection until the first handler call and using a singleton pattern. This lets the DB connection have access to environment variables setup from config.json and unblocks local mode. I don't have a good work around for local Cognito connections yet.

[abdullah353]

+1

[wollerman]

@JordonPhillips any progress on this? Thanks!

[wollerman]

For those still experiencing pain from environment variables not being set, I created a quick script that will set the appropriate stage environment variables before calling chalice deploy: https://gist.github.com/wollerman/7ec04fd29e13b1b47ef61ae71c61a4d7

It depends on jq being installed.

Usage is simply ./deploy.sh dev or a valid defined stage name.

[kgutwin]

Does PR #798 fix this?

[akravetz]

yes

From: Karl Gutwin [mailto:[email protected]]
Sent: Thursday, April 19, 2018 3:37 PM
To: aws/chalice [e-mail]
Cc: Alex Kravetz [e-mail]; Manual [e-mail]
Subject: Re: [aws/chalice] Setup environment variables when loading app.py (#584)

Does PR #798https://github.com/aws/chalice/pull/798 fix this?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/aws/chalice/issues/584#issuecomment-382855875, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AhDYJc8wYX5zsGb1G4eE1d1iSQeKIEvtks5tqOdbgaJpZM4QIUNC.

[aalvrz]

I am experiencing a similar issue that is probably related. If I try to load an environment variable like this:

app = Chalice(app_name='myapp')
app.debug = True

BUCKET_NAME = os.environ['BUCKET_NAME']
s3 = boto3.resource('s3', region_name=REGION)
bucket = s3.Bucket(BUCKET_NAME)

When running chalice deploy or chalice package I get the following error:

KeyError: 'BUCKET_NAME'

Yet I have correctly defined BUCKET_NAME in config.json.

Any ideas?

[wollerman]

@BigChief45 this is exactly the issue addressed by this thread. There is currently a PR out but no movement from the chalice team on it. The work around options are:
1) defaulting with os.getenv('envvar', 'default')
2) using my deploy script above
3) waiting for the PR to be merged
4) writing your own handling

[wollerman]

@JordonPhillips is there any way to know when #798 will be merged? Thanks.

[jamesls]

PR has been merged via https://github.com/aws/chalice/pull/833. This will go out in the next release.

[kadrach]

@jamesls #833 somewhat addresses the issue, but quoting @JordonPhillips:

Chalice imports app.py during execution here. Naturally this results in all of the code at module level being executed.

Guarding against initialization during packaging can be done with environment variables. Is there a recommended way of suppressing certain initialization during a package operation?

[dmulter]

Just FYI, the solution for this worked great for the way I did initialization. Unfortunately I have another use case that is still an issue. I don't use shell scripts for things like deployment, but instead use commands via invoke. When I perform commands/tasks, my app initialization still doesn't have access to env vars as it's not a local/stage environment. Not sure if there's a solution, but wanted to point out another problem area.