Chalice: sample for writing tests for chalice based app

[dimoMacke]

Hello,

could you please provide a best practice sample on how to best write tests for a chalice based app?
I looked through the tests folder, but I think most of the tests there are not necessary on my own code. I just want to write minimal code to test my own endpoints.

Ideally including how the tests can be run on AWS Codebuild / AWS Codepipeline before deployment with CloudFormation.

A short paragraph in the readme file would be great.

Best regards,
Dieter

[toshitanian]

I want to know what is the best practice of chalice testing too.

[jsanchezs]

+1

[troyswanson]

Yes please.

[wollerman]

+1. Even a simple update to the readme with a new tutorial section outlining an example test would be great.

Ideally, the new-project command would generate a tests folder with a test stub. This way everyone can start fresh with a stub against the hello world example.

[wollerman]

Looking at https://github.com/aws/chalice/blob/master/tests/unit/test_app.py#L208-L211 even something simple like this would be great. This obviously runs with a fixture, so need to set it up in a such a way that the main app.py is used instead.

[rajasimon]

@wollerman That's okay but it's very hard to follow and create such testcase. It will be great Chalice will support TestCase and expose to developers.

[variable]

I am writing my own pytests and wondering what the best ways it is to mock the app.current_requests, some guidelines would be great.

[wollerman]

+1 on potential best practices for mocking app.current_requests

Personally I did the following:
Because app should be available to all methods in app.py, my approach was to create a method get_request(). Then you can mock your get_request() to return whatever you need.

But it would be great to get official chalice feedback!

[rajasimon]

While waiting for the official annoncement. I have created similar workflow...

conftest.py

@fixture
def create_event():
    def create_event_inner(uri, method, path, content_type='application/json'):
        return {
            'requestContext': {
                'httpMethod': method,
                'resourcePath': uri,
            },
            'headers': {
                'Content-Type': content_type,
            },
            'pathParameters': path,
            'queryStringParameters': {},
            'body': "",
            'stageVariables': {},
        }
    return create_event_inner

test_app.py

import json
from pytest import fixture

from app import app


@fixture
def sample_app():
    return app


def assert_response_body_is(response, body):
    assert json.loads(response['body']) == body


def test_index_get(sample_app, create_event):
    event = create_event('/', 'GET', {})
    response = sample_app(event, context=None)
    assert_response_body_is(response, {'hello': 'world'})

Testing using the pytest

$ pytest test_app.py

================================================================================== test session starts ==================================================================================
platform linux -- Python 3.6.3, pytest-3.3.1, py-1.5.2, pluggy-0.6.0
rootdir: /home/simon/Nightybuild/webhook, inifile:
collected 1 item                                                                                                                                                                        

tests/test_app.py .                                                                                                                                                               [100%]

=============================================================================== 1 passed in 0.01 seconds ================================================================================

[syabro]

I've implemented a basic test using unittest.TestCase since we don't use py.test

#!/usr/bin/env python
import json
import unittest

from .app import app


class TestApiSchema(unittest.TestCase):

    @staticmethod
    def create_event(uri, method, path, content_type='application/json'):
        return {
            'requestContext': {
                'httpMethod': method,
                'resourcePath': uri,
            },
            'headers': {
                'Content-Type': content_type,
            },
            'pathParameters': path,
            'queryStringParameters': {},
            'body': "",
            'stageVariables': {},
        }

    @classmethod
    def get_app_response(cls, _app, uri, method, path, content_type='application/json', context=None):
        context = context or {}
        response = _app(cls.create_event(uri, method, path, content_type), context)
        response['body'] = json.loads(response['body'])
        return response

    def test_get_request(self):
        response = self.get_app_response(app, '/', 'GET', {})
        self.assertEqual(response['statusCode'], 200)


if __name__ == '__main__':
    unittest.main()

[nplutt]

I don't know if there is a preferred way to implement test Chalice apps but this is what I have been doing.

I've gone about implementing integration tests for my Chalice apps by creating an instance of the LocalGateway and using that to implement my tests. The local gateway is the same handler that Chalice uses to create the event object when running locally.

Below is a small example:

import json
from unittest import TestCase

from chalice.config import Config
from chalice.local import LocalGateway

from app import app


class TestApp(TestCase):
    def setUp(self):
        self.lg = LocalGateway(app, Config())

    def test_get_books(self):
        response = self.lg.handle_request(method='GET',
                                          path='/books',
                                          headers={},
                                          body='')

        assert response['statusCode'] == 200
        assert json.loads(response['body']) == [
            dict(book_id=1),
            dict(book_id=2)
        ]

    def test_post_book(self):
        body = dict(book_info='hello')
        response = self.lg.handle_request(method='POST',
                                          path='/books',
                                          headers={
                                              'Content-Type': 'application/json'
                                          },
                                          body=json.dumps(body))

        assert response['statusCode'] == 201
        assert response['headers'].get('Location') == '/books/3'

More thorough examples can be found in a repo that I created here.

[aalvrz]

@nplutt Looks really good.

Would really like some opinion from the Chalice team on this testing approach using the local gateway? @joguSD

[himanshugpt]

@nplutt how do you specify the env to target your tests?

[nplutt]

@himanshugpt, are you asking how to specify local environment variables?

[himanshugpt]

@nplutt Yes, that was my question but I found out the config examples to do it. Thanks.

[aalvrz]

@himanshugpt Can you share the examples you found?

[tmcfarlane]

Here are some unit tests I wrote for a lambda that triggers from an s3 event, all configured in chalice. The one caveat is that the app.handler() function takes a single event argument because, I imagine, chalice handles the rest behind the scenes. But I've found that if you just pass anything in as a second argument, where the context argument is normally accepted for lambda handler functions, then things will work during runtime but pylint will tell you that the function doesn't take 2 arguments.

I even tried adding a context argument to app.handler function but during runtime it then says the function requires 3 arguments. So some magic is happening in chalice here.

Here are the unit tests:

from unittest import TestCase
import app
import test.testdata as testdata
import boto3
import mock

class TestApp(TestCase):

    def setUp(self):
        pass

    @mock.patch('app.dynamodb')
    def test_index_data_happy_path(self, dynamoMock):

        # set up mocks
        tableMock = mock.Mock()
        tableMock.put_item.return_value = {}
        dynamoMock.Table.return_value = tableMock

        # setup fake data to send to function
        face_api_response = testdata.mock_response_from_face_api
        imageurl = 'https://garbageurl.com/upload/nothing.jpg'

        # call the function to test
        app.index_data(face_api_response, imageurl)

        # verifications
        tableMock.put_item.assert_called_once()

    @mock.patch('app.dynamodb')
    def test_index_data_put_item_fails(self, dynamoMock):

        # set up expected data
        expected_exception_message = 'Put item failed'

        # set up mocks
        tableMock = mock.Mock()
        tableMock.put_item.side_effect = Exception(expected_exception_message)
        dynamoMock.Table.return_value = tableMock

        # setup fake data to send to function
        face_api_response = testdata.mock_response_from_face_api
        imageurl = 'https://garbageurl.com/upload/nothing.jpg'

        # call the function to test w/ verifications
        with self.assertRaises(Exception) as e:
            app.index_data(face_api_response, imageurl)

        # verifications
        self.assertEqual(expected_exception_message, e.exception.args[0]) # exception arguments are stored in args

    @mock.patch('requests.post')
    @mock.patch('app.index_data')
    def test_handler_happy_path(self, mock_index_data, mock_requests_post):
        # the index_data function doesn't return anything and therefore only needs to be mocked for 
        # a thumbs up when its call

        # setup mock data
        mock_response = mock.Mock()
        mock_response.json.return_value = testdata.mock_response_from_face_api
        mock_requests_post.return_value = mock_response

        # setup fake data to send to function
        event = testdata.mock_incoming_event_from_s3

        # call the function to test
        app.handler(event, 'context') # ignore pylint error E1121 on app object here

    @mock.patch('requests.post')
    @mock.patch('app.index_data')
    def test_handler_call_to_face_api_fails(self, mock_index_data, mock_requests_post):
        # the index_data function doesn't return anything and therefore only needs to be mocked for 
        # a thumbs up when its call

        # set up expected data
        mock_response = mock.Mock()
        mock_response.json.return_value = testdata.mock_response_from_face_api
        mock_requests_post.return_value = mock_response
        expected_exception_message = 'Face API request failed'

        # setup mock data

        mock_requests_post.side_effect = Exception(expected_exception_message)

        # setup fake data to send to function
        event = testdata.mock_incoming_event_from_s3

        # call the function to test
        with self.assertRaises(Exception) as e:
            app.handler(event, 'context') # ignore pylint error E1121 on app object here

        # verification
        self.assertEqual(expected_exception_message, e.exception.args[0]) # exception arguments are stored in args

    @mock.patch('requests.post', return_value=testdata.mock_response_from_face_api)
    @mock.patch('app.index_data')
    def test_handler_index_data_fails(self, mock_index_data, mock_requests_post):

        # set up expected data
        expected_exception_message = 'Put item failed'

        # setup mock data
        mock_response = mock.Mock()
        mock_response.json.return_value = testdata.mock_response_from_face_api
        mock_requests_post.return_value = mock_response
        mock_index_data.side_effect = Exception(expected_exception_message)

        # setup fake data to send to function
        event = testdata.mock_incoming_event_from_s3

        # call the function to test
        with self.assertRaises(Exception) as e:
            app.handler(event, 'context') # ignore pylint error E1121 on app object here

        # verification
        self.assertEqual(expected_exception_message, e.exception.args[0]) # exception arguments are stored in args

As for environment variables, if your lambda needs environment variables then you should have them configured in your config.json but how does this help you for testing? The answer is that it doesn't. Since these unit tests don't do anything with chalice specifically it doesn't know to get the variables from your config.json.

The best way I've found is to include your environment variables in a __init__.py on the test level (aka same directory as test_app.py). This will tell python to load these environment variables before executing your test_app.py.

With that said, you have two options:

1. Be super cool and read your environment variables from .chalice/config.json
2. Be super uncool and load them manually

I'm very uncool so I did the latter option. Here's what it looks like:

import os

# setup dummy environment variables
os.environ['subscription_key'] = 'dummy_sub_key'
os.environ['index_table'] = 'dummy_table_name'

Hope this helps!

[himanshugpt]

@BigChief45

I added them in the config.json file under .chalice folder. I needed different bucket for every env.

{
  "stages": {
    "beta": {
      "api_gateway_stage": "api",
      "environment_variables": {
        "BUCKET_NAME": "bucket-y"
      }
    },
    "prod": {
      "api_gateway_stage": "api",
      "environment_variables": {
        "BUCKET_NAME": "bucket-x"
      }
    }
  },
  "version": "2.0",
  "app_name": "xyz"
}

[switch180]

@tmcfarlane @BigChief45 here's my function to load the environment variables from the config using only the stage. It uses chalice.config's Config to load the config file, and with some extra work you can have it load all the configs in the .chalice folder, such as 'config-dev.json' for the dev stage, but I haven't tried yet. The only thing I need from Config are the environment variables, and the class will load the stage appropriate vars with no extra work in my testing.

```import os
import json
from chalice.config import Config

chalice_config='.chalice/config.json'

def load_environ(chalice_stage, extra_environ=dict()):
with open(chalice_config, 'r') as config_json:
stage_variables = Config(
chalice_stage=chalice_stage,
config_from_disk=json.loads(config_json.read())).environment_variables
for k,v in stage_variables.items():
os.environ[k] = v
for k,v in extra_environ.items():
os.environ[k] = v
return os.environ

I return os.environ in case you're trying to run integration tests. If you take syabro's example above and then initialize app in the __init__ method in his TestApiSchema class then chalice will pickup the environment variables you loaded from the config json file(s) (as mentioned, if you use multiple config files then you need to figure out how to load them too).

class TestApp(TestCase):
chalice_stage='dev'
def __init__(self):
app._initialize(FakeChaliceConfig.load_environ(self.chalice_stage, {'EXTRA_ENV_VAR':'some_value'})

One limitation of this method is that you can only call _initialize once for some reason which means you can only set custom 'extra_environ' variables once. For this reason, I created a separate class called FakeChalice with syabro's code and my modification, and I create an instance each time I want to tweak the environment variables

import json
from app import app

class FakeChalice():
chalice_stage='dev'

def __init__(self, extra_environ=None):
    app._initialize(load_environ(self.chalice_stage, extra_environ))
@staticmethod
def create_event(uri, method, path, body, content_type):
    return {
        'requestContext': {
            'httpMethod': method,
            'resourcePath': uri,
        },
        'headers': {
            'Content-Type': content_type,
        },
        'pathParameters': path,
        'queryStringParameters': {},
        'body': body,
        'stageVariables': {},
    }
@classmethod
def get_app_response(self, uri, method, content_type='application/json', path={}, context={}, body=""):
    response = app(self.create_event(uri, method, path, body, content_type), context)
    response['body'] = json.loads(response['body'])
    return response

```

This is just my method - not sure what the best way to load env variables is.

[owenbrown]

+1
I'd also like guidance on writing unit tests.

[maxrodrigo]

I also took the LocalGateway approach, similar to what @nplutt does. Here's the basic class:

class TestClient(object):
    """Simulates requests to the local gateway
    Keyword Arguments:
        headers (dict): Default headers to set on every request (default: ``None``).
    """

    def __init__(self, app, headers=None):
        self.app = app
        self._default_headers = headers

    def request(self, method="GET", path="/", headers=None, body=None):
        """Simulates a request to the app local gateway.
        Performs a request against the application.
        :path: (str) The URL path to request (default: '/').
        :method: (str) An HTTP method to use in the request (default: GET).
        :headers: (dict) Additional headers to include in the request (default: ``None``).
        :body: (str) A string to send as the body of the request (default: ``None``).
        :returns: (dict) The result of the request.
        """
        if not path.startswith("/"):
            raise ValueError("path must start with '/'")

        valid_methods = ["GET", "HEAD", "POST", "PUT", "DELETE"]
        if method not in valid_methods:
            raise ValueError(f"method must be one of: {valid_methods}")

        body = body or ""
        headers = headers or {}

        gateway = LocalGateway(self.app, Config())
        response = gateway.handle_request(method, path, headers, body)
        return response

    def get(self, path="/", **kwargs):
        """Simulates a GET request to the local gateway"""
        return self.request("GET", path, **kwargs)

    def head(self, path="/", **kwargs):
        """Simulates a HEAD request to the local gateway"""
        return self.request("HEAD", path, **kwargs)

    def post(self, path="/", **kwargs):
        """Simulates a POST request to the local gateway"""
        return self.request("POST", path, **kwargs)

    def put(self, path="/", **kwargs):
        """Simulates a PUT request to the local gateway"""
        return self.request("PUT", path, **kwargs)

    def delete(self, path="/", **kwargs):
        """Simulates a DELETE request to the local gateway"""
        return self.request("DELETE", path, **kwargs)

Then I simply create a fixture to return the client.

@pytest.fixture(scope="module")
def client():
return TestClient(app)

It works for us but I don't know if that's the best way to do it.

[Sytten]

For those like me that use other triggers than HTTP, what I ended up doing is creating a service layer in chalicelib with all the business logic and only testing that instead of the controller. It's annoying though to not have a proper way to test it... I would have expected a test client like we have in flask to be available.

[ernestkoe]

@nplutt Yes, that was my question but I found out the config examples to do it. Thanks.

can you share what you've found? I am trying to do this, too.

[jrbeilke]

+1 something like https://docs.djangoproject.com/en/3.0/topics/testing/tools/#django.test.TestCase would be really handy for testing Chalice apps and looks like #1193 is a step in the right direction

[jamesls]

Hey everyone, I've tried to combine all of the feedback and suggestions together and I've created a proposal and corresponding PR that implements a test client for chalice. The only thing not implemented is support for testing websocket APIs but the proposal in #1468 has a proposed API on what this would look like. Let me know if you have any feedback.

Proposal: https://github.com/aws/chalice/issues/1468
PR: https://github.com/aws/chalice/pull/1469

[jamesls]

Closing out issue. The latest version of Chalice has its own test client, and I've added documentation on how to use it along with how to integrate it with pytest: https://aws.github.io/chalice/topics/testing.html

API reference: https://aws.github.io/chalice/api.html#testing

[mowliv]

@jamesls That page doesn't explain how to mock app.current_requests, which I'm trying to do and at least two people commented on above. Would you please provide an example?