Trellis: Ubuntu 20.04 LTS support

[keybits]

• [x] I've read the guidelines for Contributing to Roots Projects
• [x] This request isn't a duplicate of an existing request
• [x] This is not a personal support request that should be posted on the Roots Discourse community

Summary

Support for Ubuntu 20.04 LTS

Motivation

Ability to use Ubuntu 20.04 with trellis

Additional context

Thanks for your work on trellis. Is Ubuntu 20.04 support planned?

[swalkinshaw]

Supporting the latest LTS is always the goal but no one has started work on it nor is a there a strict "plan". Usually the process is:

• try out the new version
• see what breaks
• fix it 😛

I haven't kept up with the release so I don't know if there's been large changes or not.

[hemangajmera]

I would also like to start with ubuntu 20.04 LTS for fresh setup. Please keep us updated once you plan to move forward with this.

Quick google reveals that python 3 and php 7.4 is default in 20.04 version vs python 2 and php 7.3.

[nextgenthemes]

Just tried, this came up.

TASK [mariadb : Add MariaDB PPA] *******************************************************************************
System info:
  Ansible 2.9.9; Linux
  Trellis version (per changelog): "Support Ansible 2.9"
---------------------------------------------------
apt cache update failed
fatal: [45.33.198.215]: FAILED! => {"changed": false}

On next try:

No package matching 'python-software-properties' is available
failed: [45.33.198.215] (item=python-software-properties) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "python-software-properties", "value": "present"}}                                                  
---------------------------------------------------
No package matching 'python-mysqldb' is available
failed: [45.33.198.215] (item=python-mysqldb) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "python-mysqldb", "value": "present"}}

I found this

apt_packages_python:
  '2':
    python-software-properties: "{{ apt_package_state }}"
    python-mysqldb: "{{ apt_package_state }}"
    python-pycurl: "{{ apt_package_state }}"
  '3':
    python3-software-properties: "{{ apt_package_state }}"
    python3-mysqldb: "{{ apt_package_state }}"
    python3-pycurl: "{{ apt_package_state }}"

So it seems the python3 version of those packages are already ready but the auto detection/switching to those seems to fail so I manually set the version to 3 on the cmd.

ansible-playbook server.yml -e "env=staging python_major_version=3"

Then I got the mariadb PPA error again. Doing manually apt upgrade and later error messages reveal that the PPA simple has no package for 20.04 and that causes errors.

I have to manually remove the PPA and disable the task that adds it. Then this:

TASK [mariadb : Set root user password] ************************************************************************
System info:
  Ansible 2.9.9; Linux
  Trellis version (per changelog): "Support Ansible 2.9"
---------------------------------------------------
failed: [45.33.198.215] (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}                                                           
---------------------------------------------------
failed: [45.33.198.215] (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}                                                           
---------------------------------------------------
failed: [45.33.198.215] (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}                                                           
---------------------------------------------------
failed: [45.33.198.215] (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}                                                           
---------------------------------------------------
fatal: [45.33.198.215]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 

So I enabled the log and it complaints about missing packages, installed python3-pymysql (that was NOT installed) but it did not help, I still get the same error. Found out I may need the python package and strangely sudo apt install python3-pip was not installed. But

sudo pip3 install pymysql
Requirement already satisfied: pymysql in /usr/lib/python3/dist-packages (0.9.3)

sudo pip3 install MySQL-python
Collecting MySQL-python
  Downloading MySQL-python-1.2.5.zip (108 kB)
     |████████████████████████████████| 108 kB 14.6 MB/s 
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-p9v45l0v/MySQL-python/setup.py'"'"'; __file__='"'"'/tmp/pip-install-p9v45l0v/MySQL-python/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-p9v45l0v/MySQL-python/pip-egg-info
         cwd: /tmp/pip-install-p9v45l0v/MySQL-python/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-p9v45l0v/MySQL-python/setup.py", line 13, in <module>
        from setup_posix import get_config
      File "/tmp/pip-install-p9v45l0v/MySQL-python/setup_posix.py", line 2, in <module>
        from ConfigParser import SafeConfigParser
    ModuleNotFoundError: No module named 'ConfigParser'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I think I got further with ansible-playbook server.yml -e "env=staging python_major_version=3 ansible_python_interpreter=/usr/bin/python3" specifying the python path as well.

TASK [mariadb : Set root user password] ************************************************************************
System info:
  Ansible 2.9.9; Linux
  Trellis version (per changelog): "Support Ansible 2.9"
---------------------------------------------------
unable to connect to database, check login_user and login_password are
correct or /root/.my.cnf has the credentials. Exception message: (1698,
"Access denied for user 'root'@'localhost'")
[WARNING]: Module did not set no_log for update_password

... I give up for now.

[nextgenthemes]

OK I got it working on 20.04. I think the apt cache errors were related to having PPAs added that do not have packages for 20.04. mariadb and nginx.

I do not know why the mysql tasks fail but all task that try to interact with the database fail. I have set the root pw manually and ran sudo mysql_secure_installation that does the same things the mysql tasks that failed tried.

I added this line when: ansible_distribution_release != 'focal' to the tasks that add the PPAs and the mysql security tasks to skip them.

trellis/roles/mariadb/tasks/main.yml

---
- block:
  - name: Add MariaDB APT key
    apt_key:
      keyserver: "{{ mariadb_keyserver }}"
      id: "{{ mariadb_keyserver_id }}"

  - name: Add MariaDB PPA
    apt_repository:
      repo: "{{ mariadb_ppa }}"
      update_cache: yes
    when: ansible_distribution_release != 'focal'

- name: Install MySQL client
  apt:
    name: "{{ mariadb_client_package }}"
    state: "{{ mariadb_client_package_state | default(apt_package_state) }}"
    cache_valid_time: "{{ apt_cache_valid_time }}"

- block:
  - name: Install MySQL server
    apt:
      name: "{{ mariadb_server_package }}"
      state: "{{ mariadb_server_package_state | default(apt_package_state) }}"
      cache_valid_time: "{{ apt_cache_valid_time }}"

  - name: Disable MariaDB binary logging
    template:
      src: disable-binary-logging.cnf
      dest: /etc/mysql/conf.d
      owner: root
      group: root
    when: mysql_binary_logging_disabled | bool
    notify: restart mysql server

  - name: Set root user password
    mysql_user:
      name: root
      host: "{{ item }}"
      password: "{{ mysql_root_password }}"
      check_implicit_admin: yes
      state: present
    no_log: true
    with_items:
      - "{{ inventory_hostname }}"
      - 127.0.0.1
      - ::1
      - localhost
    when: ansible_distribution_release != 'focal'

  - name: Copy .my.cnf file with root password credentials.
    template:
      src: my.cnf.j2
      dest: ~/.my.cnf
      owner: root
      group: root
      mode: 0600

  - name: Delete anonymous MySQL server users
    mysql_user:
      user: ""
      host: "{{ item }}"
      state: absent
    no_log: true
    with_items:
      - localhost
      - "{{ inventory_hostname }}"
      - "{{ ansible_hostname }}"
    when: ansible_distribution_release != 'focal'

  - name: Remove the test database
    mysql_db:
      name: test
      state: absent
    when: ansible_distribution_release != 'focal'

  when: not sites_using_remote_db | count

So seems to work fine.

[swalkinshaw]

@nextgenthemes thanks for trying things out 😄 You made me realize the check we had for python 3 wasn't great in hindsight. I've put up a draft PR working on 20.04: https://github.com/roots/trellis/pull/1197

I'll probably merge a few pieces independently since they can be done on their own:

• bumping to MariaDB 10.4 (10.2 isn't available on 20.04)
• switching Nginx PPAs (old one we used isn't on 20.04 either)
• the version checking

After that I think the only actual thing needed for 20.04 is to bump the default Vagrant box version

[nextgenthemes]

@swalkinshaw So the mariadb tasks fail because of the version that ships with 20.04? They should work anyway no?

Why are those PPAs even tied to a specific mariadb version, is there no PPA that just has the latest stable versions?

// OK I tried your PR and all tasks work for the server.yml work fine. However, deploy had an issue I filed.

[swalkinshaw]

We define the PPA like this:

mariadb_ppa: "deb [arch=amd64] http://nyc2.mirrors.digitalocean.com/mariadb/repo/10.2/ubuntu {{ ansible_distribution_release }} main"

So it ends up as deb .../10.2/ubuntu focal main but 10.2 doesn't even exist for the focal one 🤷‍♂️ Whereas 10.4 exists for both bionic and focal

That's just MariaDB's decision which we can't control using their official PPAs. I imagine 10.2 should work with 20.04 if there were a way to install it easily.

[nextgenthemes]

@swalkinshaw Yes I saw that, that is why I asked. I already understood that is why it broke apt. I was just curious if you know why it failed without the PPA. (I manually disabled the task that added it)

20.04 ships with 10.3 and the security tasks all failed on that version for some reason. But I forgot to run with -vvv b4 already moving on to the 10.04 PPA.

I did not even know you can define PPAs this way. I only saw ppa:xxx b4 and because of that DigitalOcean url I thought it's actually not the official PPA. Strange decision by them to do it that way. With a "normal" PPA this would never come up.

[nicegamer7]

MariaDB 10.5 was just released and the official PPA has packages for both bionic and focal. So that's also an option.

[swalkinshaw]

Kind of an oversight that we haven't keeping MariaDB up to date by bumping versions. So I think it makes sense to update to 10.5 anyway before the Ubuntu upgrade.

[kennethptchr]

Hi all, wanted to drop by and mention that the changes mentioned in PR are working nicely on my end, it saved me big time as our workflow broke because of a change in what ubuntu version Forge forces on creating a new droplet in Digital Ocean. Thanks!

[MikeiLL]

Noting here that roles/ngins/default/main.yml needs to look like this:

nginx_keyserver: "https://nginx.org/keys/nginx_signing.key"
nginx_keyserver_id: "ABF5BD827BD9BF62"
nginx_ppa: "deb http://nginx.org/packages/mainline/ubuntu {{ ansible_distribution_release }} nginx"
nginx_package: nginx
nginx_conf: nginx.conf.j2
nginx_path: /etc/nginx
nginx_worker_connections: 8000
nginx_logs_root: /var/log/nginx
nginx_user: www-data www-data
nginx_fastcgi_buffers: 8 8k
nginx_fastcgi_buffer_size: 8k
nginx_fastcgi_read_timeout: 120s

# Fastcgi cache params
nginx_cache_path: /var/cache/nginx
nginx_cache_key_storage_size: 10m
nginx_cache_size: 250m
nginx_cache_inactive: 1h

For some reason the one I created with Trellis-Cli (on Sept 8th) contained:

nginx_ppa: "ppa:nginx/mainline"
nginx_package: nginx
nginx_conf: nginx.conf.j2
nginx_path: /etc/nginx
nginx_worker_connections: 8000
nginx_logs_root: /var/log/nginx
nginx_user: www-data www-data
nginx_fastcgi_buffers: 8 8k
nginx_fastcgi_buffer_size: 8k
nginx_fastcgi_read_timeout: 120s

# Fastcgi cache params
nginx_cache_path: /var/cache/nginx
nginx_cache_key_storage_size: 10m
nginx_cache_size: 250m
nginx_cache_inactive: 1h

Added an issue (#131) to the Trellis-Cli Repo.

[MikeiLL]

Update. /srv/www hadn't been created. When I try to run provision task again it returns the following error:

task path: /Volumes/Oggun/Clients/etc/trellis/roles/nginx/tasks/main.yml:2
Using module file /Volumes/Oggun/Clients/etc/trellis/.trellis/virtualenv/lib/python3.8/site-packages/ansible/modules/packaging/os/apt_repository.py
Pipelining is enabled.
<206.189.189.62> ESTABLISH SSH CONNECTION FOR USER: admin
<206.189.189.62> SSH: EXEC ssh -vvv -o ForwardAgent=yes -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="admin"' -o ConnectTimeout=10 -o ControlPath=/Users/mikekilmer/.ansible/cp/963679c84f 206.189.189.62 '/bin/sh -c '"'"'sudo -H -S  -p "[sudo via ansible, key=xdgtbttbtbjnkxfjprglylrbtfihqkqr] password:" -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-xdgtbttbtbjnkxfjprglylrbtfihqkqr ; /usr/bin/python'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<206.189.189.62> (1, b'\n{"msg": "apt cache update failed", "failed": true, "exception": "WARNING: The below traceback may *not* be related to the actual failure.\\n  File \\"/tmp/ansible_apt_repository_payload_VrPEQh/ansible_apt_repository_payload.zip/ansible/modules/packaging/os/apt_repository.py\\", line 548, in main\\n  File \\"/usr/lib/python2.7/dist-packages/apt/cache.py\\", line 591, in update\\n    raise FetchFailedException(e)\\n", "invocation": {"module_args": {"install_python_apt": true, "filename": null, "repo": "deb http://nginx.org/packages/mainline/ubuntu focal nginx", "state": "present", "update_cache": true, "mode": null, "codename": null, "validate_certs": true}}}\n', b'OpenSSH_7.9p1, LibreSSL 2.7.3\r\ndebug1: Reading configuration data /Users/mikekilmer/.ssh/config\r\ndebug1: /Users/mikekilmer/.ssh/config line 1: Applying options for *\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 48: Applying options for *\r\ndebug2: resolve_canonicalize: hostname 206.189.189.62 is address\r\ndebug1: auto-mux: Trying existing master\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug2: mux_client_hello_exchange: master version 4\r\ndebug3: mux_client_forwards: request forwardings: 0 local, 0 remote\r\ndebug3: mux_client_request_session: entering\r\ndebug3: mux_client_request_alive: entering\r\ndebug3: mux_client_request_alive: done pid = 42030\r\ndebug3: mux_client_request_session: session request sent\r\ndebug3: mux_client_read_packet: read header failed: Broken pipe\r\ndebug2: Received exit status from master 1\r\n')
<206.189.189.62> Failed to connect to the host via ssh: OpenSSH_7.9p1, LibreSSL 2.7.3
debug1: Reading configuration data /Users/mikekilmer/.ssh/config
debug1: /Users/mikekilmer/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 48: Applying options for *
debug2: resolve_canonicalize: hostname 206.189.189.62 is address
debug1: auto-mux: Trying existing master
debug2: fd 3 setting O_NONBLOCK
debug2: mux_client_hello_exchange: master version 4
debug3: mux_client_forwards: request forwardings: 0 local, 0 remote
debug3: mux_client_request_session: entering
debug3: mux_client_request_alive: entering
debug3: mux_client_request_alive: done pid = 42030
debug3: mux_client_request_session: session request sent
debug3: mux_client_read_packet: read header failed: Broken pipe
debug2: Received exit status from master 1
System info:
  Ansible 2.9.13; Darwin
  Trellis 1.5.0: August 5th, 2020
---------------------------------------------------
apt cache update failed
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_apt_repository_payload_VrPEQh/ansible_apt_repository_payload.zip/ansible/modules/packaging/os/apt_repository.py", line 548, in main
  File "/usr/lib/python2.7/dist-packages/apt/cache.py", line 591, in update
    raise FetchFailedException(e)
fatal: [206.189.189.62]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "codename": null,
            "filename": null,
            "install_python_apt": true,
            "mode": null,
            "repo": "deb http://nginx.org/packages/mainline/ubuntu focal nginx",
            "state": "present",
            "update_cache": true,
            "validate_certs": true
        }
    }
}

When I add ansible_python_interpreter=/usr/bin/python3 to the provisioning command, it fails at connection : Warn about change in host keys. Since I'm running in the virtualenv generated by Trellis-cli, I tried using the path to the virtualenv python: .trellis/virtualenv/bin/python, which fails at Gathering Facts: The following modules failed to execute: setup.

[swalkinshaw]

@MikeiLL I'm confused about the changes to roles/nginx/default/main.yml. https://github.com/roots/trellis/pull/1208 was merged in separately to make this Ubuntu upgrade simpler. Is there an issue with the changes in that PR?

[MikeiLL]

Well, maybe. It seems like once I updated the nginx/default/main.yml file, the Add Nginx PPA task completed successfully, however there was no /srv/www directory created. And when I tried to re-provision, it gets stuck again at Add Nginx PPA.

[swalkinshaw]

Okay well I'll look into the /srv/www dir missing. I think I ran into that myself so I'll confirm

[MikeiLL]

Okay well I'll look into the /srv/www dir missing. I think I ran into that myself so I'll confirm

In the meantime, do you have a recommended workaround, or can you suggest one?

[swalkinshaw]

You'll need to add a task like this:

- name: Ensure webroot directory exists
  file:
    path: "{{ www_root }}"
    state: directory
    mode: '0755'

It could go in roles/nginx/tasks/main.yml

[MikeiLL]

I had to update (trellis/roles/common/tasks/main.yml):

- name: Update apt packages
  apt:
    update_cache: no (from yes)

Because of

Failed to update apt cache: W:Updating from such a repository can't be done
securely, and is therefore disabled by default., W:See apt-secure(8) manpage
for repository creation and user configuration details., W:GPG error:
http://nginx.org/packages/mainline/ubuntu focal InRelease: The following
signatures couldn't be verified because the public key is not available:
NO_PUBKEY ABF5BD827BD9BF62, E:The repository
'http://nginx.org/packages/mainline/ubuntu focal InRelease' is not signed.

Current snag is at Wordpress-setup: Create databases for sites:

The PyMySQL (Python 2.7 and Python 3.X) or MySQL-python (Python 2.X) module
is required.

Is this tied in with the PPA errors, because these apt packages weren't installed?

[MikeiLL]

These two tasks were missing from the Trellis-Init version

- name: Add Nginx APT key
  apt_key:
    keyserver: "{{ nginx_keyserver }}"
    id: "{{ nginx_keyserver_id }}"

and

- name: Ensure site directories exist
  file:
    path: "{{ nginx_path }}/{{ item }}"
    state: directory
  with_items:
    - sites-available
    - sites-enabled

I think this is what we want for roles/python_interpreter/tasks/main.yml:

- block:
  - name: Get Ubuntu version
    raw: lsb_release -rs
    register: ubuntu_version
    changed_when: false
  - name: Set ansible_python_interpreter for Ubuntu >= 20.04
    set_fact:
      ansible_python_interpreter: python3
    when: ubuntu_version.stdout | trim is version('20.04', '>=')
  when: ansible_python_interpreter is not defined
  tags: always

Also I had to add the when condition for:

- name: Update apt packages
  apt:
    update_cache: yes
  when: ansible_distribution_release != 'focal'

[maciekpaprocki]

I am not really sure if it adds anything good to the conversation, but I run multiple websites now with ubuntu 20 and I didn't have any issue with trellis except seeing a warning that ubuntu version is wrong.

I think the changes you were speaking of are already implemented or my setup is just too vanilla to catch issues.
Features I use that are not set up out of the box are:

• letsencrypt
• caching

[swalkinshaw]

I've finally got around to testing Ubuntu 20.04 more thoroughly and I haven't experienced any issues with Trellis on master either. This is using both Vagrant's 20.04 image and Digitalocean's default 20.04 image.

It is possible for images on providers to differ slightly, but generally a stock Ubuntu version is the same.

[strarsis]

Correct, I had issues in the past with installing Trellis on an Ubuntu base image provided by the hoster.
The solution was to install Ubuntu minimal onto an empty system, then optionally take that image for later reinstallations and run the ansible playbook on it.

[swalkinshaw]

Officially supported now as of 1.7 (https://github.com/roots/trellis/pull/1197)