Ddev: Support for Acquia's BLT

[rickmanelius]

Is your feature request related to a problem? Please describe.

We've had several requests for explicit support/tutorials for Acquia BLT. I would like to perform a PoC to determine the viability of this support. Here are two relevant docs:

• https://github.com/acquia/blt
• https://thinktandem.io/blog/2017/12/09/lando-blt-acquia/

Describe the solution you'd like
A working PoC and (ideally) a blog article similar to the article by the folks at Lando. cc @nearlythere.

[hmartens]

I've come across this requirement now as well :) Looking forward to this!

[rickmanelius]

I think this would be a solid Q4 goal along with an overall provider plugin. Assigning over.

[alex-moreno]

+1 for this feature

Is there anything I could help with to move this forward?

[rickmanelius]

Hi @alex-moreno! Yes, the first thing that could be of help is simply an audit of if it's possible to use ddev with Acquia BLT as is or if you get stopped at a certain point. Additionally, if there are any docs/tutorials beyond what I posted above, that would be helpful.

FWIW, I love when people provide step by step tests so we can get in their minds and see what they are trying to achieve. Of course, this may be overboard for a request given you are offering help! Here's an example of how I've done this in the past https://github.com/drud/ddev/pull/988#pullrequestreview-136685402. This helps us recreate, figure out a path forward if there is a blocker, etc.

[alex-moreno]

Awesome. I know a little bit of BLT, so I could try to configure it with ddev and see what I find and where I get. I'll document that in this ticket.

Not a problem, I'll try to understand your example and follow it for the tutorial, I may need some feedback during the process though :-)

[rickmanelius]

Awesome! And seriously, it's ok if you just give it a spin and let us know what your experience is/was. I haven't used BLT yet so I'm not sure if this is something that can already work largely out of the box or if it's a mountain to climb...

[alex-moreno]

quick one, there is a small trick in Lando, in which they hook blt so they can execute it inside the containers via lando itself:

```Hook up the Lando command we defined earlier in .lando.yml to the blt executable cli tool wyth a symlink:

lando ssh -u root -c "ln -s /app/lightning/vendor/acquia/blt/bin/blt /usr/bin/blt"

Then you can execute things like:

lando blt setup
lando blt doctor
```

how could you approach this? Not saying that it's necessarily needed, just exploring possibilities

[alex-moreno]

FORGET THIS COMMENT, I SOLVED IT USING --sites-subdir -> reading-error-messages++

---

ok, did some tests and hit a wall here:

$ ddev exec drush site-install --account-name=admin --account-pass=admin [error] Could not determine target sites directory for site to install. Use --sites-subdir to specify. Failed to execute command [drush site-install --account-name=admin --account-pass=admin]: Failed to run docker-compose [-f /Users/alejandromoreno/projects/personal/ddev/blt/ddevloveblt/.ddev/docker-compose.yaml exec -T web drush site-install --account-name=admin --account-pass=admin], err='exit status 1', stdout='', stderr=' [error] Could not determine target sites directory for site to install. Use --sites-subdir to specify. '

I'm using the blt build tool, which builds a D8 code base with blt and some other nicities:

$ ls README.md composer.json config/ drush/ phpcs.xml.dist tests/ blt/ composer.lock docroot/ patches/ salt.txt vendor/

[alex-moreno]

here is my feedback

Steps:

1. export SITENAME=ddevlovesblt
2. composer create-project --no-interaction acquia/blt-project ddevloveblt
3. ddev config --docroot docroot --projectname $SITENAME --projecttype drupal8
4. ddev start

First issue:

$ ddev start
Starting ddevlovesblt...
ddev needs to add an entry to your hostfile.
It will require administrative privileges via the sudo command, so you may be required
to enter your password for sudo. ddev is about to issue the command:
    sudo /usr/local/bin/ddev hostname ddevlovesblt.ddev.local 192.168.99.100
Please enter your password if prompted.
Running Command  Command=sudo /usr/local/bin/ddev hostname ddevlovesblt.ddev.local 192.168.99.100
Password:
Creating volume "ddevlovesblt-mariadb" with default driver
Creating ddev-ddevlovesblt-db ... done
Creating ddev-ddevlovesblt-web ... done
Creating ddev-ddevlovesblt-dba ... done

Stopping ddev-router ... done
Removing ddev-router ... done
Network ddev_default is external, skipping
Creating ddev-router ... done

Failed to start ddevlovesblt: db service health check timed out: labels map[com.ddev.site-name:ddevlovesblt com.docker.compose.service:db] timed out without becoming healthy, status=exited

Then, obviously this does not work:

 ddev exec drush site-install --account-name=admin --account-pass=admin
Project is stopped. Run 'ddev start' to start the environment.```


But firing ddev start a second time did the trick:

```$ ddev start
Starting ddevlovesblt...
Starting ddev-ddevlovesblt-db ... done
ddev-ddevlovesblt-web is up-to-date
ddev-ddevlovesblt-dba is up-to-date
Stopping ddev-router ... done
Removing ddev-router ... done
Network ddev_default is external, skipping
Creating ddev-router ... done

Ensuring write permissions for ddevlovesblt
Successfully started ddevlovesblt
Project can be reached at http://ddevlovesblt.ddev.local, https://ddevlovesblt.ddev.local, http://192.168.99.100:32791

Finally:

ddev exec drush site-install --account-name=admin --account-pass=admin

That complains about site folder

ddev exec drush site-install --account-name=admin --account-pass=admin --sites-subdir=docroot/

Fixed permissions:

chmod -R 0755 docroot/sites/

Latest blocker

$ ddev exec drush site-install --account-name=admin --account-pass=admin --sites-subdir=docroot/ [error] Failed to create database: ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111) Failed to execute command [drush site-install --account-name=admin --account-pass=admin --sites-subdir=docroot/]: Failed to run docker-compose [-f /Users/alejandromoreno/projects/personal/ddev/blt/ddevloveblt/.ddev/docker-compose.yaml exec -T web drush site-install --account-name=admin --account-pass=admin --sites-subdir=docroot/], err='exit status 1', stdout='', stderr=' [error] Failed to create database: ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111) '

[alex-moreno]

more data, following the UI from this point I found more folder permissions issues (mainly files-private, which is specific to BLT, and sites/default)

Files private does not exist

uid_501@71dd30260048:/var/www/html$ mkdir files-private
mkdir: cannot create directory 'files-private': Permission denied
uid_501@71dd30260048:/var/www/html$ sudo !!
sudo mkdir files-private
uid_501@71dd30260048:/var/www/html$

The directory sites/default/files does not exist

The directory sites/default/files is not writable

After solving that the UI was able to proceed, passing the db without problems. I wonder if the console was throwing a db error while it should be throwing permissions?

[alex-moreno]

blt from outside the container didn't work properly, i did:

$ docker exec -it ddev-ddevlovesblt-web /bin/bash

and, more issues, this time a missing bz2 lib

```uid_501@71dd30260048:/var/www/html/docroot$ ../vendor/acquia/blt/bin/blt setup
Setting up local environment for site default.
Using drush alias @self

source:build
[warning] The active configuration is not identical to the configuration in the export directory.
[warning] This means that you have not exported all of your active configuration.
[warning] Run drush cex to export the active config to the sync directory.
[warning] Continuing will overwrite the active configuration.
Continue? (y/n) y
tests:behat:init:config
source:build:composer
Gathering patches for root package.
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

Problem 1
- jakoch/phantomjs-installer 2.1.1-p07 requires ext-bz2 * -> the requested PHP extension bz2 is missing from your system.
- jakoch/phantomjs-installer 2.1.1-p07 requires ext-bz2 * -> the requested PHP extension bz2 is missing from your system.
- Installation request for jakoch/phantomjs-installer 2.1.1-p07 -> satisfiable by jakoch/phantomjs-installer[2.1.1-p07].

To enable extensions, verify that they are enabled in your .ini files:
- /etc/php/7.1/cli/php.ini
- /etc/php/7.1/cli/conf.d/10-mysqlnd.ini```

[alex-moreno]

I think they mysql issue from console happens because blt has its own settings file

[alex-moreno]

a bit more investigation. I don't get what's going on here, drush sqlc works fine (testing inside the web container), while drush site-install breaks can't find mysql???

uid_501@222629e0c5e9:/var/www/html/docroot$ drush site-install --account-name=admin --account-pass=admin --sites-subdir=docroot --db-url=mysql://[email protected]/db --yes

 // You are about to CREATE the 'db' database. Do you want to continue?: yes.

 [error]  Failed to create database: ERROR 2003 (HY000): Can't connect to MySQL server on '127.0.0.1' (111)
uid_501@222629e0c5e9:/var/www/html/docroot$ drush sqlc
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 155
Server version: 5.5.5-10.1.32-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

[alex-moreno]

I'm testing this again. It looks definitely more stable than a few months back.

The only thing it looks it's needed is php7.1-bz2, as it's a requirement for blt (see next).

How can we get this installed, as apt-get instal cannot be used?

ddev ssh
uid_501@828691836cce:/var/www/html$ blt setup
Setting up local environment for site default.
Using drush alias @self
> source:build
> tests:behat:init:config
> source:build:composer
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Component/Utility/Timer.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Component/Utility/Unicode.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Core/Database/Database.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Core/DrupalKernel.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Core/DrupalKernelInterface.php" which does not appear to be a file nor a folder
Could not scan for classes inside "/var/www/html/vendor/drupal/core/lib/Drupal/Core/Site/Settings.php" which does not appear to be a file nor a folder
Gathering patches for root package.
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - jakoch/phantomjs-installer 2.1.1-p07 requires ext-bz2 * -> the requested PHP extension bz2 is missing from your system.
    - jakoch/phantomjs-installer 2.1.1-p07 requires ext-bz2 * -> the requested PHP extension bz2 is missing from your system.
    - Installation request for jakoch/phantomjs-installer 2.1.1-p07 -> satisfiable by jakoch/phantomjs-installer[2.1.1-p07].

[alex-moreno]

updated steps:

1. export SITENAME=ddevlovesblt
2. composer create-project --no-interaction acquia/blt-project ddevloveblt
3. ddev config --docroot docroot --projectname $SITENAME --projecttype drupal8
4. ddev start
5. ddev ssh
6. blt 

config yaml needs this update too:

hooks:
  post-start:
  - exec: "ln -sf /var/www/html/vendor/acquia/blt/bin/blt /usr/bin/blt"
  - exec: bash -c "sudo apt-get update && sudo apt-get install -y php7.1-bz2"

[rfay]

In the container, sudo apt-get update && sudo apt-get install -y php7.1-bz2.

Or in a post-start step in config.yaml exec: bash -c "sudo apt-get update && sudo apt-get install -y php7.1-bz2"

It's annoying because it has to be done for each start when docker recreates.

It's not hard to get this into the standard web container, we've added many php packages people asked for. Is this really a requirement for BLT or just for your project?

[alex-moreno]

confirmed. BLT is actually working now, you just need to follow the steps and change the config. Easy :-)

I guess this issue can be closed now :-)

[rfay]

But you still needed php7.1-bz2 right?

[alex-moreno]

yes. I have updated my comment so it includes the bz2 install.

there's still this annoying issue that, if you re run ddev start, it fails because the ln -s fails as the link it's already there

[rfay]

Use ln -sf

[rfay]

It looks like we'll be able to get php-bz2 (https://github.com/drud/ddev/pull/1254) into this release, so this should be fairly simple, will hope you can write up the finished recipe on Stack Overflow or something. Thanks!

[alex-moreno]

I was thinking a post in dev.acquia, but it's that simple that not sure I can fill a post :-D.

Let me put it together tomorrow :-)

[alex-moreno]

done :-)

https://twitter.com/morenodotnet/status/1064528339967332352

[rfay]

Thanks to @alex-moreno this is documented in https://dev.acquia.com/blog/blt-and-ddev-together, yay!