Draft Release Notes for v1.8.0

Installation/Upgrade

See the installation instructions for details, but it's easy:

Please stop/remove all projects first, this does no harm: ddev remove --all --stop-ssh-agent
macOS Homebrew and Linux Linuxbrew: brew upgrade ddev
Linux or macOS via script:
curl https://raw.githubusercontent.com/drud/ddev/master/scripts/install_ddev.sh | bash
Windows: Download the ddev_windows_installer.v1.7.1.exe above or with Chocolatey choco install ddev or choco upgrade ddev (or choco install ddev --version=1.7.1 while it's in the approval process.)
And anywhere, you can just download the tarball or zipball, untar or unzip it, and place the executable in your path where it belongs.
For this release (all versions) you'll want to do mkcert -install; Watch what it says to you. You may have a little extra work to do on Linux
Please see the "Incompatibilities and Things to Upgrade" section below. You'll want to update config.yaml hooks that use bash -c, and if using custom nginx configuration will want to update your nginx config (but it's easier now!)

Key changes in v1.8.0:

Browsers and host OSs now trust ddev sites over https. You do have to take the one-time action to install the CA keys into your operating system and browsers: mkcert -install. If you use one of the package installation methods or the WIndows installer, you should already have mkcert. Otherwise, see the mkcert page for installation options. (Even curl and operating system tools generally trust the mkcert certs, and curl within the container also trusts them.)
Dynamic container updates: If you need extra Debian packages in your web or db container (or need to make more sophisticated adjustments) you no longer have to wait for them, or install them every time you start a project. You can add webimage_extra_packages to your config.yaml or build a free-form Dockerfile (Dockerfile.example provided in your .ddev folder)
The "stop" and "remove" commands have been changed a bit. ddev stop now does what ddev remove used to do. It removes the containers and saves resources on your host. ddev remove is still available as an alias of ddev stop. And ddev pause takes the place of the former ddev stop - it just does a "docker stop" on the project containers and leaves them in zombie state. This is the workflow we've promoted for some time, as we've been suggesting that people use "remove" in general, but now it's just named "stop".
nginx configuration in the web container has been reorganized and simplified. This mostly won't matter to you unless you use custom nginx configuration, in which case you'll want to upgrade your configuration (see below). docs
ddev exec and exec hooks now interpret commands using bash. This means you can have a hook like "sudo apt-get update && sudo apt-get install -y some-package" without putting "bash -c" in front of it. And you can ddev exec "sudo apt-get update && sudo apt-get upgrade -y some-package" as well, no bash -c required.
ddev exec can now work with interactive situations. So for example, you can ddev exec mysql and interact with the mysql program directly. Or ddev exec bash, which is the same as ddev ssh.

Smaller changes

There is now an exposed https port on localhost (not going through the router). It is reported on ddev start and via ddev describe. (As with the http port, the https localhost port can be locked down in the config.yaml, but most people won't need to do this. See the .ddev/config.yaml for instructions.)
Since https is preferred everywhere now, the reported URLs are now https-first, http second.
The composer cache has moved from a volume named "ddev-composer-cache" to a new volume named "ddev-global-cache". If you want to, you can docker volume rm ddev-composer-cache.
blackfire.io and php-pgsql packages was added to web container. If you were previously adding them with a post-start hook, you don't have to do that any more..
Windows users will be warned when they have more than 10 sites/hostnames represented in the hosts file, since WIndows stops resolving over 10 hostnames.
The webcache feature is officially deprecated and is no longer getting automated test coverage, but no date has been set to remove it. Maybe it will just live on until there's something to compete with it for automated testing speed. (docs)
New flag ddev --version gets just the ddev version without all the details in ddev version
Multiple hostname resolution in web container: There was a long-term bug (#1235) where you couldn't use curl against the hostname if additional_hostnames or additional_fqdns were enabled, that's fixed.
drush once again works on the host (macOS and Linux). It was broken on v1.7.1 because the ddev_drush_settings.php got dropped into the wrong place when generated.
nginx in the web container no longer intercepts and reports 40x errors, they'll be handled by the PHP application.
ddev config no longer searches for and uses a .ddev/config.yaml in a parent directory (although it does warn if it finds one). This led to people accidentally using unintentially created config files in non-project parent directories, including the home directory. Running ddev config in home directory is now disallowed.
ddev composer create --no-interaction passes the --no-interaction file to composer, but no longer will continue without asking for confirmation before destroying the project directory; confirmation is required.

Incompatibilities and things to fix during upgrade

The base nginx configuration has changed. If you are overriding configuration with an nginx-site.conf file, you'll want to re-do it (it's much easier now). In addition, small pieces of nginx configuration can now be added in the .ddev/nginx directory. See the docs for more information.
config.yaml "exec" hooks are now executed with bash context by default. If you have exec hooks that have "bash -c" in them, please remove that, as they can't be interpreted correctly by the new execution mechanism.
If you or your team used to use ddev stop instead of ddev remove, the new behavior of "stop" may seem different, but it hopefully won't affect most people. However, if you have an add-on docker-compose.*.yaml service that has volatile storage, you may want to change it to save its database on a volume, as ddev stop now completely brings down the containers, and if they had volatile content, it will be gone.
If you use drush in exec hooks, and the action you take requires confirmation, please change the exec hook to use drush --yes. Formerly, --yes was implied by exec hooks and ddev exec, but it isn't any more.

Thanks!

Many community members helped out with this release.

@damienmckenna , @wizonesolutions , and @mglaman among others worked to significantly improve the Apache Solr docs and example docker-compose.solr.yaml, and added a docker volume so the solr database would be nonvolatile after ddev stop.
@AronNovak created the PR to have the ddev-webserver nginx configuration not intercept 40x errors.
@yanniboi and @isholgueras were kind enough to catch and PR docs errors.
@dacostafilipe introduced a PR adding pgsql libraries
Thanks to all of you for the issues you created, bugs you pointed out, features you requested, support you gave in Slack and elsewhere, and for your support of ddev!

And the amazing mkcert from @FiloSottile made the seemingly impossible task of trusted local development certificates possible. Thanks! Is open source great or what?

Commits since v1.7.1

e85d60c2 Fix typo in description, change completion to 'has been stopped' (#1590)
4038b549 Improve mkcert docs for linux [skip ci][ci skip] (#1591)
f928dcb9 Improve wordpress quickstart with ddev config [skip ci][ci skip] (#1592)
689ea9ee Re-use the v1.8.0 tag even though small change was made in web container (#1589)
421efebb Remove --yes drush behavior when in ddev exec behavior, fixes #1584 (#1586)
16f109eb Revert parent config search, prevent directory destruction, hopefully fixes #1574 (#1576)
9b6fcadd Bump container versions to v1.8.0 (#1581)
96e78d8a Interpret ddev exec and hooks with bash, fixes #1023 (#1570)
a41256c2 Warn windows users when hosts file has too many entries on a line, fixes #948 (#1575)
24e1d8d2 Further improved Solr install docs. (#1568)
07ac3661 Add quickstart for PHP project [skip ci][ci skip] (#1573)
f3c9b0db ddev exec command should allow tty interactions (#1571)
2bf13b94 Use StartAndWaitForSync to accomodate docker toolbox slowness (#1572)
70917c6d Use docker-compose build for dynamic containers, allow easier container config (#1560)
81c4b2e0 ddev-webserver Nginx - do not intercept 400 errors (#1555)
7e9b172a Fix mark down format for hook examples. (#1565)
980cb5c9 Use sudo to copy mysql config files, fixes #1541 (#1563)
41e99543 apache-cgi should respect php overrides, fixes #1556 (#1557)
baef9b25 Test improvements for Win 10 Docker Toolbox (#1561)
dc340c9a Use restart: no on additional service examples (#1559)
49f3b508 Add blackfire.io to web container, fixes #653 (#1545)
e484c99e Improved Solr install docs, update to 6.6 (#1549)
761938f2 Use docker volume in sample apachesolr config (#1538)
7b2fbab6 SetSiteSettingsPaths() has to be set after we've read the config, fixes #1553 (#1554)
97c660af Trust SSL/TLS Certificates, fixes #1247, fixes #1501, fixes #849 (#1540)
be8505e3 Fix CaptureLogs to just capture stdout (#1550)
5940cdb3 Add separate lines for each external_links, fixes #1235 (#1546)
a2d191e8 Change PHP typo when it should be MySQL (#1532) [skip ci][ci skip]
beab35d9 Improve docs with DEBIAN_FRONTEND=noninteractive in apt-get examples [skip ci][ci skip] (#1543)
899386bc More robust handling of ddev stop -RO, remove from global list, fixes #1523 (#1531)
44f2c247 Rename stop-containers to pause, so ddev pause, follow up to #1526 (#1537)
93885347 Reorganize stop and remove commands (use stop), fixes #484 (#1526)
e64019e6 Improve "ddev version" and add --version flag, fixes #960 (#1527)
34c84c73 Deprecate webcache_enabled, fixes #1495 (#1530)
573fb002 Generate homebrew bottles for Linux and macOS for #1516 (#1524)
a9380962 Use CircleCI's new Ubuntu 16.04 machine image (#1513)
60e0d25d Minor release docs improvements (#1525)
0a70f468 Add wordpress composer recipe, simplify others, [skip ci][ci skip] (#1519)
21dbe0e1 Minor improvements to mariadb binlog management (#1515)
ce861669 Add missing php-pgsql packages to ddev-webserver container (#1514)
8b06e0fe Add fileinfo to required php extensions in docs [skip ci] [ci skip] (#1512)

rfay on 8 May 2019

🎉1

Test Plan for v1.8.0

Current testing is against v1.8.0-rc1.

Testing installation instructions:

Mac or Linux Homebrew: brew untap drud/ddev && brew tap rfay/ddev && brew upgrade ddev (Remember after testing completion to brew untap rfay/ddev && brew tap drud/ddev)
Windows: Just download the ddev_windows_installer from the release, v1.8.0-rc1
Or of course you can download any way you want and test from the release.

Problem Reporting

Please report your results in new comments below (you can edit your comment as you go along if you like), noting the test number and your reactions.
If you find things that really need to be described as issues, please open an issue.

Test Plan

New built-in Debian packages in web container: Use phpinfo.php or Drupal admin/reports/status/php to verify that the pgsql and blackfire.io packages are in there.
Wordpress composer recipe: Verify correctness of docs for wordpress composer install
PHP quickstart recipe: Verify correctness.
start/stop/pause: Test to make sure that ddev remove works as it always has, that ddev stop now behaves like ddev remove always did (with its flags). And that ddev pause can work with ddev start. Especially look for things that might trip people up as this is introduced to them.
Name resolution in container with additional_hostnames and additional_fqdns: Configure project with an additional_hostname and additional_fqdn and then make sure you can curl within the container against that URL (both http and https)
Drush should work if installed on the host (macOS and linux). v1.7.1 got the ddev_drush_settings.php moved to the wrong place, it should be back in sites/default now, so drush commands should work.
Trust SSL Certificates:
- make sure you've done the one-time mkcert -install
- Chrome, Safari, and Firefox should recognize the validity of https URL certificate on projects started.
- curl should work against https URLs (without having to use --insecure or -k). (macOS and Linux, doesn't work on Windows)
- curl should work inside the web container against the project https URL
- curl against the localhost https url (see ddev describe) should work fine on macOS and Linux.
Try webimage_extra_packages[] and dbimage_extra_packages[] and verify that the packages get installed on ddev start (ddev ssh in there and use dpkg or other technique)
Experiment with custom Dockerfile and make sure it does what's expected.
Use the interactive ddev exec in a few ways. Examples: ddev exec mysql, ddev exec vim /etc/my.cnf, ddev drush sqlc
Use ddev exec with commands that require bash interpretation. For example, ddev exec echo \$HOSTNAME and ddev exec "set | grep DDEV" andddev exec 'echo $HOSTNAME'andddev exec "ls >/tmp/junk.txt"`
Test exec hooks that require bash interpretation, like exec: set | grep DDEV and others like above.
ddev config in a subdirectory of a project should warn that a parent .ddev/config.yaml was found, but not automatically use it.
ddev composer create --no-interaction should still require confirmation before blowing away the project directory.
A warning should be issued on ddev start on Windows if the hosts file line has gone over 10 entries.

rfay on 9 May 2019

ok
did not check
ok
ok (also verified pause vs. stop which I have not used before) - I still think a new command remove-data would be favorable because stop --remove-data is not expected IMHO to non-docker-compose people.
ddev help stop has a little formatting error:
Stop and remove the containers of a project. You can run 'ddev stp[' from a project directory to stop/remove that project, or you can stop/remove projects in ...
Btw: For people not used to docker containers stop IS de facto a destructive operation as changed files outside of volumes WILL be gone.
ok (kudos for setting TYPO3s trustedHostPattern so that HTTPS proxy works out of the box!)
did not check
EDIT: ok after following the missing-package notice of mkcert
ok (great function! thank you!)
ok (kudos for automatically outputting all docker-compose build output on build error!)
ok (EDIT: stream redirection local -> container is a missing feature in docker-compose, works with plain docker, though)
ok
did not check
ok
ok
did not check

jonaseberle on 13 May 2019

👍1

Randy's testing:

Concerns

[x] Should blackfire.io be disabled by default? (@mglaman says no, I also tested it with and without and didn't see any particular issue)
[x] ddev help stop has a formatting error
[x] Message on ddev stop says "has been stopped and removed" and "has been stopped" would probably be better.
[x] Ubuntu Linux needs a little more explanation about SSL cert trust, as the mkcert -install warns about things (nss), but people may not see them. Also, on Ubuntu, /usr/sbin must be in the PATH to pick up /usr/sbin/update-ca-certificates.
> $ mkcert -install
> Created a new local CA at "/home/rfay/.local/share/mkcert" 
> Installing to the system store is not yet supported on this Linux  but Firefox and/or Chrome/Chromium will still work.
> You can also manually install the root certificate at "/home/rfay/.local/share/mkcert/rootCA.pem".
> Warning: "certutil" is not available, so the CA can't be automatically installed in Firefox and/or Chrome/Chromium! ⚠️
> Install "certutil" with "apt install libnss3-tools" or "yum install nss-tools" and re-run "mkcert -install" 
[x] Followups to Stack Overflow articles:
- [x] How to change the timezone in a container: Generally much easier now in Dockerfile.
- [x] Followup to SO article on bash constructs, easier now.

Execution (Mostly on Windows 10 Pro/Docker Desktop)

👍 admin/reports/status/php shows the pgsql and blackfire.io packages are in there.
👎 Wordpress composer recipe:
- This recipe is missing the initial ddev config to config initially.
- The recipe should use --prefer-dist to avoid stumbling on an amazing requirement for svn
- Running on windows I see > Installer::postPackageInstall Script Installer::postPackageInstall handling the post-package-install event terminated with an exception [ErrorException] symlink(): No such file or directory
  
  (However, this works with nfsmount_enabled)
👍 PHP quickstart recipe: (I just created an index.php with <?php phpinfo(); and everything was fine.
👍 start/stop/pause: Works fine. But I don't like the message on stop "Project d8composer has been stopped and removed.", it should probably be "has been stopped"
👍 Name resolution in container with additional_hostnames and additional_fqdns: Seems to work perfectly. I configured with "junker99" as additional hostname and "junk.example.com" as additional_fqdn, and curl against those, both http and https, inside container worked; also https://localhost.
👍 Drush should work if installed on the host (macOS and linux). Tested on Ubuntu 18.01, it worked, ddev_drush_settings.php was in the correct place.
👍 Trust SSL Certificates (works great, but see concerns about Linux above)
👍 Try [webimage_extra_packages[] and dbimage_extra_packages[]]
👍 Experiment with custom Dockerfile (Worked very nicely, even gave a warning that it was ignoring the webimage_extra_packages I configured in 8.)
👍 Use the interactive ddev exec in a few ways. Examples: ddev exec mysql, ddev exec vim /etc/my.cnf, ddev drush sqlc
👍 Use ddev exec with commands that require bash interpretation.
👍 Test exec hooks that require bash interpretation. (Tested a bunch of hooks, including post-start and post-import-db, with bash redirection and also traditional things like sudo apt-get; worked.)
👍 ddev config in a subdirectory of a project should warn that a parent .ddev/config.yaml was found, but not automatically use it. (This works; in my environment (cmder with bash:bash) on Windows it doesn't show the warning as brightly as I wish it would, it's white)
👍 ddev composer create --no-interaction should still require confirmation before blowing away the project directory. (This works as expected, confirmation is required)
👍 A warning should be issued on ddev start on Windows if the hosts file line has gone over 10 entries. I got:
> You have more than 9 entries in your (windows) hostsfile entry for 127.0.0.1
> Please use ddev hostname --remove-inactive or edit the hosts file manually
> Please see https://ddev.readthedocs.io/en/stable/users/troubleshooting/#windows-hosts-file-limited for more information

rfay on 13 May 2019

@rfay Blackfire shouldn't cause any performance problems idling in the background.

Blackfire profiles are generated only for requests triggered by a Blackfire user, or by Blackfire when profiling is automated. That is made possible thanks to our runtime instrumentation. As no other requests will be instrumented, Blackfire will add no overhead for your end users, which makes it safe to use in production.

I have the Blackfire extension installed in production on plenty of projects. The _only_ think I could think of is it someone installed Tideways or XHProf? But even then all of these runtimes only trigger when invoked. I've had XHProf and Blackfire on a site at the same time without errors.

mglaman on 13 May 2019

👍1

Test notes:

Did not test
WordPress Composer recipe: 👍
I was able to use the recipe documentation to start a fresh WordPress project using Composer, but I did hit the ddev config speed bump.
PHP Quickstart: 👍
An important addition to the documentation. I followed the steps to create a very simple project.
Reorganize start/pause/stop: ⚠️
I'm able to control the container lifecycle using start, pause, and stop instead of start, stop, and remove. I love the symmetry of start vs. stop, and pause is a natural addition to the command set. There is a typo in the Long: help text for ddev stop: You can run 'ddev stp['...
Additional hostnames/FQDNs resolution: :+1:
All additional hostnames and FQDNs defined in .ddev/config.yaml are recognized in-container, both for http and https.
Did not test
Trusting local certs: 👍
After configuring, the https local sites are trusted by the browser.
Additional packages: 👍
I'm able to define additional packages to be installed in a container. After starting or restarting containers, the packages are installed, as confirmed by ddev ssh.
Custom containers: 👍
Similarly, I'm able to define a custom container image for web and db containers. The expected packages are installed and other RUN steps are executed.
Interactive ddev exec: 👍
I'm able to interactively run commands in-container through ddev exec, e.g. ddev exec vim myfile.txt. This is very cool!
Bash interpretation in ddev exec: 👍
My go-to test command for this is ddev exec curl https://drud.com | tee drud.com.tmp. This works just as it would in a normal Bash context, and the contents of the resulting file is confirmed by ddev exec cat drud.com.tmp.
Bash interpretation in hooks: 👍
This was tested using the same command in 11. and was successful.
Rework config-in-subdirectory: 👍
When a .ddev/config.yaml exists in a parent directory, ddev config warns the user of the existing config, but continuing creates a new config in the current working directory.
ddev composer ... --no-interaction still requires confirmation: 👍
Despite the --no-interaction flag, the 'delete everything' step requires interactive confirmation.
Did not test

There are a ton of really major updates here — this is probably one of the biggest and best ddev updates yet!

andrewfrench on 14 May 2019

hi, i tested the rc on windows 10.

i ran mkcert --install, but chrome doesn't recognize the CA, also the CA is listed in the trust store.

hebbet on 14 May 2019

Hi @hebbet - I've tested this a number of times on WIndows 10 with Chrome. Could you recheck that you:

ddev remove --all before starting (so that there is no router running and the router gets recreated)
Restarted Chrome

Thanks for testing, and thanks for reporting. Hoping to understand this better.

rfay on 14 May 2019

This is all done, and I even went through Stack Overflow ddev articles looking for things that needed to be updated (bash -c, etc).

Elli and I are working on announcements and other communication.

rfay on 15 May 2019

close project board, milestone, update releases wiki page

dclear on 15 May 2019

Ddev: v1.8.0 Release Checklist Due 2019-05-14

All 10 comments