Cookiecutter-django: The DATABASE_URL environment variable couldn't found in shell

[Neulana]

What happened?

/app # python manage.py shell_plus

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/environ/environ.py", line 273, in get_value
value = self.ENVIRON[var]
File "/usr/local/lib/python3.6/os.py", line 669, in __getitem__
raise KeyError(key) from None
KeyError: 'DATABASE_URL'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "manage.py", line 30, in
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 204, in fetch_command
settings.INSTALLED_APPS
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 56, in __getattr__
self._setup(name)
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 43, in _setup
self._wrapped = Settings(settings_module)
File "/usr/local/lib/python3.6/site-packages/django/conf/__init__.py", line 106, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/app/config/settings/local.py", line 1, in
from .base import * # noqa
File "/app/config/settings/base.py", line 41, in
'default': env.db('DATABASE_URL'),
File "/usr/local/lib/python3.6/site-packages/environ/environ.py", line 204, in db_url
return self.db_url_config(self.get_value(var, default=default), engine=engine)
File "/usr/local/lib/python3.6/site-packages/environ/environ.py", line 277, in get_value
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the DATABASE_URL environment variable

What should've happened instead?

I found the below command in ./compose/production/django/entrypoint that "export DATABASE_URL" is only valid for the current shell, when you start a new shell, there is no DATABASE_URL environment variable.
export DATABASE_URL="postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}"

why not place this variable in ./.env/.local/.django? It always seems to be valid in the container.

Steps to reproduce

I started a project powered by latest cookiecutter-django version using docker-compose, and when I executed "docker-compose -f local.yml up -d", the app had been successful launched, and the http://0.0.0.0:8000/ can be visited successfully.
But when I executed "docker exec -it bila_django_1(my container name) sh", and when I want to do anything like "python manage.py shell_plus", there is a error like above.

[webyneter]

In the entrypoint file you mentioned, there's an explicit explanation to as to why: # N.B. If only .env files supported variable expansion... Keep in mind, we make extensive use of .env files which, by nature, do not support shell variable expansion, therefore, entrypoint is the closest place to have such complex variables set.
You might wanna create a Makefile or something where the DATABASE_URL (and any other complex env var for that matter) would be provided in docker compose's -e argument while pre-emptively constructed from the underlying environment-specific envs. There's, by the way, an ongoing (proposed) Makefile effort already: https://github.com/pydanny/cookiecutter-django/pull/1879.
If you've got a better idea, feel free to send us a PR :)
Closing this one for now; suit yourself in re-opeining it should a PR from you to follow.

[ghost]

So how can we do if we want to run python manage.py in a new shell? What I have for now is use docker exec -it -e DATABASE_URL=database_url python manage.py，but that is not convenient and sometimes not secure. Any other better way?

[pySilver]

@zmrenwu see my comment at #1879

[simonbogarde]

@zmrenwu I just had the same problem and what works for me is to source the entrypoint file so that the env variables become available:
Within container shell: source compose/production/django/entrypoint

[Jtown0011]

You can echo the environment variables into ~/.bashrc file for the user in the container. See the below changes I've made to the django entrypoint file.

`

!/bin/sh

set -o errexit
set -o pipefail
set -o nounset

echo "export CELERY_BROKER_URL=${REDIS_URL}" >> ~/.bashrc

if [ -z "${POSTGRES_USER}" ]; then
base_postgres_image_default_user='postgres'
echo "export POSTGRES_USER=${base_postgres_image_default_user}" >> ~/.bashrc
fi
echo "export DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}" >> ~/.bashrc

postgres_ready() {
python << END
import sys

import psycopg2

try:
psycopg2.connect(
dbname="${POSTGRES_DB}",
user="${POSTGRES_USER}",
password="${POSTGRES_PASSWORD}",
host="${POSTGRES_HOST}",
port="${POSTGRES_PORT}",
)
except psycopg2.OperationalError:
sys.exit(-1)
sys.exit(0)

END
}
until postgres_ready; do

&2 echo 'Waiting for PostgreSQL to become available...'
sleep 1
done
&2 echo 'PostgreSQL is available'

exec "$@"
`

[luisroel91]

You can echo the environment variables into ~/.bashrc file for the user in the container. See the below changes I've made to the django entrypoint file.

`

!/bin/sh

set -o errexit
set -o pipefail
set -o nounset

echo "export CELERY_BROKER_URL=${REDIS_URL}" >> ~/.bashrc

if [ -z "${POSTGRES_USER}" ]; then
base_postgres_image_default_user='postgres'
echo "export POSTGRES_USER=${base_postgres_image_default_user}" >> ~/.bashrc
fi
echo "export DATABASE_URL=postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/${POSTGRES_DB}" >> ~/.bashrc

postgres_ready() {
python << END
import sys

import psycopg2

try:
psycopg2.connect(
dbname="${POSTGRES_DB}",
user="${POSTGRES_USER}",
password="${POSTGRES_PASSWORD}",
host="${POSTGRES_HOST}",
port="${POSTGRES_PORT}",
)
except psycopg2.OperationalError:
sys.exit(-1)
sys.exit(0)

END
}
until postgres_ready; do

&2 echo 'Waiting for PostgreSQL to become available...'
sleep 1
done
&2 echo 'PostgreSQL is available'

exec "$@"
`

Would there by any security risks by using this approach?

[elcolie]

After you are in django container run
source /entrypoint