Readthedocs.org: Python Interpreter for doc build is forced to Python 3.7 despite starting in virtualenv with python 3.6

[et304383]

Details

This happens trying to trigger a doc build, even for the demo project. Did this on a fresh install on both Ubuntu 18.04 and Amazon Linux 2.

My Ubuntu script:

apt update
apt upgrade -y
apt install -y latexmk texlive texlive-science texlive-formats-extra git redis python3-dev

git clone https://github.com/rtfd/readthedocs.org.git
cd readthedocs.org
echo "    SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')" >> readthedocs/settings/base.py

python3 -m venv --without-pip venv
source venv/bin/activate
curl -s https://bootstrap.pypa.io/get-pip.py | python

pip install -r requirements.txt
pip install django-allauth

python manage.py migrate
echo "from django.contrib.auth.models import User; User.objects.create_superuser('admin', 'admin@localhost', 'admin')" | python ./manage.py shell
python manage.py collectstatic
python manage.py loaddata test_data

python manage.py runserver 0.0.0.0:8000

Error:

[21/Feb/2019 19:22:26] readthedocs.doc_builder.environments:599[1832]: WARNING (Build) [admin-demo:latest] Command python3.7 -mvirtualenv --no-site-packages --no-download /home/ubuntu/readthedocs.org/user_builds/admin-demo/envs/latest failed:
Traceback (most recent call last):
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 178, in run
    env=environment,
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'python3.7': 'python3.7'
Traceback (most recent call last):
  File "/home/ubuntu/readthedocs.org/readthedocs/projects/tasks.py", line 558, in run_build
    self.setup_python_environment()
  File "/home/ubuntu/readthedocs.org/readthedocs/projects/tasks.py", line 784, in setup_python_environment
    self.python_env.setup_base()
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/python_environments.py", line 258, in setup_base
    cwd='$HOME',
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 612, in run
    return super().run(*cmd, **kwargs)
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 403, in run
    return self.run_command_class(cls=self.command_class, cmd=cmd, **kwargs)
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 618, in run_command_class
    return super().run_command_class(*cmd, **kwargs)
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 472, in run_command_class
    raise BuildEnvironmentWarning(msg)
readthedocs.doc_builder.exceptions.BuildEnvironmentWarning: Command python3.7 -mvirtualenv --no-site-packages --no-download /home/ubuntu/readthedocs.org/user_builds/admin-demo/envs/latest failed:
Traceback (most recent call last):
  File "/home/ubuntu/readthedocs.org/readthedocs/doc_builder/environments.py", line 178, in run
    env=environment,
  File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'python3.7': 'python3.7'

Expected Result

Docs build

Actual Result

Error on missing Python 3.7

This was not happening a few days ago when I tried this, using the same setup scripts.

[stsewd]

The docs building is separate from the application. We default to 3.7 in our newest releases. You can use a configuration file to use another python version or change the default settings from readthedocs/settings

[et304383]

Please explain how to do this. Is this documented?

[stsewd]

Using a config file https://docs.readthedocs.io/en/stable/config-file/v2.html

Or you can remove 3.7 from settings https://github.com/rtfd/readthedocs.org/blob/ed8dd29a68d009fc08929fabf7155883482619c1/readthedocs/settings/base.py#L292-L294 (because we take the latest version from the latest image)

And we use docker in production

[et304383]

This seems like a weird state to be in considering you don't support Python 3.7 for running the application (#4756).

Why would you require two versions of Python by default? That's not a good state to be in for default behaviour.