VVV 3 Roadmap

[tomjn]

Hello!

After some discussion with @LoreleiAurora, our consensus is that this is the approach to take for VVV 3 and the primary changes that need to happen.

The general gist being:

• Build our own Vagrant box and do the provisioning in advance, users won't have to provision the OS anymore
• Move mysql back to a shared/synced folder so that box updates are easier
• Replace Vagrant triggers with our own internal thing
• Have the vagrant file auto-install the hosts updater plugin
• Improve logging, with per site log files
• Move database creation into vvv-custom.yml on a per site basis so that backups can be turned on and off

This will likely involve a lot of rewriting of the vagrant file, and provisioner work.

Thoughts? Suggestions? Offers of assistance?

[tomjn]

Notes on infrastructure needed:

OVH

• 1 Jenkins BlueOcean/owncloud?/box storage - persistent
• 1 per build, ephemeral, Ubuntu, build runner, tester

Build pipeline

• git repos
  
  
  
  • 1 is a base box, ubuntu with all its sec updates, configured to work as a vagrantbox
    
    this process takes a long fucking time, runs once a week or something
  
  
  • 1 contains all our VVV specific provisioning things, but will run every build
  
  
  • 1 repo with the terraform and packer stuff, builds and destroys build servers
  
  
  • 1 repo with the jenkins stuff so that we can restore and keep our jenkins server versioned
  
  

install all php versions during the build process

• config turns them on and off, rather than installing and uninstalling

[aaronware]

I know it's relatively easy to remove an provisioned site; remove the host entry, kill the entry in the custom.yml kill the db but it would be cool to have some sort of uninstall process for a provisioned site.

[tomjn]

The DB change might help with that, but it would still need to be an additional process, and one that's explicitly triggered. There's too many things that could go wrong

I added a new doc in a PR here https://github.com/Varying-Vagrant-Vagrants/varyingvagrantvagrants.org/pull/83

But automatic removal would mean typos in vvv-custom.yml lead to data loss, or even commenting a site out. Nevermind the problem with sites that use a shared database. It would need to be an explicit command added in.

[Mte90]

Maybe plan also to improve the dashboard, like a button to do the provision of a specific site?
Or a command line way simplified, because maybe I du provision every day because I am contributing on core so I cannot wait for updates from the top for the box.

[tomjn]

There is a way to do it via CLI but it's not documented as it could open a can of worms :) Sadly without moving the dashboard to an electron app there's no way to provision from inside the VM

Fortunately, the dashboard was decoupled, so updates to the dashboard can be done without a VVV release

[tomjn]

Something like --provision-with="site-mysite" where mysite is the site to be provisioned

[Mte90]

Maybe we can write this command inside the dashboard so is easy a c&p and avoid interaction from dashboard.

[Mte90]

As I can see the new vagrant version integrate Vagrant triggers as core https://github.com/hashicorp/vagrant/blob/master/CHANGELOG.md so maybe is something not required anymore?

[tomjn]

That sounds like a good idea, can you raise it as a new issue?

[Mte90]

I am trying vagrant 2.1 with hosts-updater and see if works.

mte90:/var/www/VVV/www  ⑂ develop +  $  vagrant plugin uninstall vagrant-triggers
WARNING: Vagrant has detected the `vagrant-triggers` plugin. This plugin conflicts
with the internal triggers implementation. Please uninstall the `vagrant-triggers`
plugin and run the command again if you wish to use the core trigger feature. To
uninstall the plugin, run the command shown below:

  vagrant plugin uninstall vagrant-triggers

Note that the community plugin `vagrant-triggers` and the core trigger feature
in Vagrant do not have compatible syntax.

To disable this warning, set the environment variable `VAGRANT_USE_VAGRANT_TRIGGERS`.
Uninstalling the 'vagrant-triggers' plugin...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually                                                                                                                                               
caused by misconfigured plugin installations or transient network                                                                                                                                                   
issues. The error from Bundler is:                                                                                                                                                                                  

Unable to resolve dependency: user requested 'vagrant-hostsupdater (= 1.0.2)' 

[tomjn]

was that intended on #1475 ? ^^

[Mte90]

One of the common problems that we have on contributor days is the people on windows that is using the wrong version of Powershell that is not supported completely from Vagrant, but vagrant alert about that in a complicated way (write an error that require to search on internet).

I am wondering if we can speed up the provision process with parallel download and script running.

[tomjn]

Provisioning and downloading would only happen once on a build server, the
only things that would provision locally are the sites themselves
On Sat, 5 May 2018 at 15:38, Daniele Scasciafratte [e-mail]
wrote:

One of the common problems that we have on contributor days is the people
on windows that is using the wrong version of Powershell that is not
supported completely from Vagrant, but vagrant alert about that in a
complicated way (write an error that require to search on internet).

I am wondering if we can speed up the provision process with parallel
download and script running.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1469#issuecomment-386810057,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADl58zSDzuDx57-5wbH5wYgljLQK645ks5tvbkwgaJpZM4TmCBi
.

[Mte90]

Yes but I mean instead to process every single site as cascade maybe using python (instead of bash) can useful to provision the sites locally in parallel so the time for do that task can be more fast.

[tomjn]

That would speed things up, how might we achieve this technically?
On Sun, 6 May 2018 at 12:52, Daniele Scasciafratte [e-mail]
wrote:

Yes but I mean instead to process every single site as cascade maybe using
python (instead of bash) can useful to provision the sites locally in
parallel so the time for do that task can be more fast.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1469#issuecomment-386873778,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADl56OYIhmmy45hJOAPliinqINDKjmAks5tvuPogaJpZM4TmCBi
.

[Mte90]

There are solution built in linux like https://www.gnu.org/software/parallel/
I think that we have only to define the log, because with parallel execution of provisioning a log can be a mess.

I think that I can open a new ticket about the parallel execution with my ideas for this.

[tomjn]

See #1479 and #1478

We'll track feature requests and enhancements under the 3.x.x milestone as new issues

[tomjn]

As an aside, I've been speaking with @LoreleiAurora and she thinks it would be a good idea to ask for help going forward with regards to Jenkins so we don't have a single point of failure should things break and she's not around

[Mte90]

Cool!
I am not sure if I can help but I think also https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1417, https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1277, https://github.com/Varying-Vagrant-Vagrants/VVV/issues/710 seems interested for release 3.

[Mte90]

https://core.trac.wordpress.org/ticket/43711#ticket reading this about the new dev step https://make.wordpress.org/core/2018/05/16/preparing-wordpress-for-a-javascript-future-part-1-build-step-and-folder-reorganization/ about grunt maybe we can add a script for that.

I mean when you need to work on wordpress-develop you can run something like vagrant wp-dev and automatically is executed grunt-watch inside the machine avoding to enter with ssh like we are doing right now for xdebug on/off.

[tomjn]

@Mte90 the problem is that file watching inside the VM in vanilla vagrant isn't reliable, which can be fixed at the cost of disk performance. There's definitely things that can be done to improve it, but it will never be as great as a watch on the host. But I feel that should be another ticket #1447

For requests, lets keep things in separate issues, @LoreleiAurora is working off of a project board so it'd be easier for her to focus if everything appeared itemised

[tomjn]

A general update though, there's a CI server running a packer build that generates a box and runs a script to upload it to vagrant cloud. Currently there are going to be 2 boxes, vvv and vvv-base, with vvv-base being the Ubuntu box with the packages installed to speed up CI so that VVV provisioning doesn't have to wait for MariaDB etc to be installed.

The vvv-base box is what has been done so far. I'm told cleanup of boxes needs implementing, then there's the second box, which is what Vagrant would be grabbing on users machines, then finally the modified VVV itself with a rewritten provisioner/vagrant file

[benlumia007]

here is a thought, if we are going to build our own box, why not use .vhd instead of .vdi, that way .vhd can be use under hyper-v and virtualbox as well.

[Mte90]

I was thinking that can be helpful to have in the dashboard a button to execute scripts like xdebug_on so can be a real dashboard with actions and not only links.

[Mte90]

we have https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1792 so I think that we can close this one

[tomjn]

Agreed, lets close this out

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.