Vagrant: Install plugin dependencies from within Vagrantfile

[erran-r7]

The ability to install plugins programmatically would be beneficial in Vagrantfiles. My idea of how this would work is placing plugin dependencies outside of the Vagrant configure blocks like so:

plugin 'vagrant-berkshelf', '1.3.2'

Vagrant.configure('2') do |config|
  # Normal configuration here
end

Related issues:

• #1700
• #1736
• #1789
• I started this due to having a similar issue to: https://github.com/cityindex/wordpress-development-flow/issues/29

I started to create a work around at the top of my Vagrantfile, in accordance with my comment on #1736 , but I only gotten this far (by shelling out and grepping through the vagrant.d directory):

def plugin(name, version = nil, opts = {})
  @vagrant_home ||= opts[:home_path] || ENV['VAGRANT_HOME'] || "#{ENV['HOME']}/.vagrant.d"
  plugins = JSON.parse(File.read("#@vagrant_home/plugins.json"))

  if !plugins['installed'].include?(name) || (version && !version_matches(name, version))
    cmd = "vagrant plugin install"
    cmd << " --entry-point #{opts[:entry_point]}" if opts[:entry_point]
    cmd << " --plugin-source #{opts[:source]}" if opts[:source]
    cmd << " --plugin-version #{version}" if version
    cmd << " #{name}"

    result = %x(#{cmd})
  end
end

def version_matches(name, version)
  gems = Dir["#@vagrant_home/gems/specifications/*"].map { |spec| spec.split('/').last.sub(/\.gemspec$/,'').split(/-(?=[\d]+\.?){1,}/) }
  gem_hash = {}
  gems.each { |gem, v| gem_hash[gem] = v }
  gem_hash[name] == version
end

# With version/source: plugin 'vagrant-berkshelf', '1.3.2', source: 'https://rubygems.org'
plugin 'vagrant-berkshelf'
plugin 'vagrant-omnibus'

To me it looks like the solution would be to expose a similar method while Vagrantfile's are being parsed. This code in VagrantPlugins::CommandPlugin::Command::Install is probably how the plugin method would handle install. I'm not sure how to handle the versioning though.

[tknerr]

@ecarey-r7 I have started some work at vagrant-plugin-bundler. So far it only checks if the plugin requirements are met, but does not automatically install the missing plugin versions yet.

Versioning is a big deal though, since vagrant currently allows a plugin to be installed only in a single version (unlike Gems, which you can have in multiple versions).

As long as this is not fixed, installing the required plugins is a potentially dangerous operation as two parallel vagrant process might interfere each other due to different plugin requirements.

[erran-r7]

@tknerr I'm not sure about the syntax, but that looks great. One thing is the way you're listing plugins, you could download them from JSON (that's what seems to be going on in Vagrant):

  @vagrant_home ||= opts[:home_path] || ENV['VAGRANT_HOME'] || "#{ENV['HOME']}/.vagrant.d"
  plugins = JSON.parse(File.read("#@vagrant_home/plugins.json"))
plugins['installed']

Also, wouldn't the changes better be suited in the core Vagrant library? That is a user manually needs to install the vagrant-plugin-bundler plugin to manage other plugins through their Vagrant file. I could see this being a problem as a plugin that needs to be manually installed:

Vagrant.configure('2') do |config|
  config.plugin deps do
    depend 'vagrant-omnibus', '1.0.2'
    depend 'vagrant-cachier', '0.1.0'
  end
end

[ecarey @ dvwa]$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

Vagrant:
* Unknown configuration section 'berkshelf'.

[tknerr]

@ecarey-r7 yes, this should be fixed in vagrant core definitely. I see the vagrant-plugin-bunlder plugin as a temporary workaround until then only.

Hope that @mitchellh or @fgrehm will jump in here :-)

[fgrehm]

TBH I don't think this belongs on vagrant core itself. An analogy would be having this logic from inside Ruby on Rails instead of relying on Bundler, which IMHO does not make sense. Rails just provides the foundation for others to build plugins and relies on Bundler to get things in place and load plugins' code.
But that's just my 2 cents ;)

[erran-r7]

Vagrantfiles already manage boxes, which are a type of dependency. There are two solutions IMO:

1. The previously stated solution, adding some sort of plugin method to the Vagrantfiles either how @tknerr or I described.
2. Create a .vagrant_plugins or Vagrantplugins file which could be used like a Gemfile when invoking bundle install.

You manually need to install plugins regardless with vagrant at the moment:

[ecarey @ dvwa]$ vagrant up
Bringing machine 'standalone' up with 'virtualbox' provider...
Bringing machine 'xampp' up with 'virtualbox' provider...
There are errors in the configuration of this machine. Please fix
the following errors and try again:

Vagrant:
* Unknown configuration section 'berkshelf'.
* Unknown configuration section 'omnibus'.

[erran-r7]

The workflow could then be:

git clone https://github.com/user/repo
cd repo
bundle install # Install dependencies from Gemfile

# Either
vagrant plugin install # (with no args) could install from a new vagrant plugin file.
vagrant up

# Or
vagrant up # Installs boxes and plugins

[fgrehm]

Just to be clear, I really believe we need some sort of Bundler for Vagrant as there are plugins being created almost every month :P I just don't think it belongs on Vagrant's core :) let's wait for @mitchellh opinion on this

[erran-r7]

I can agree with that. I misunderstood you before. If that route's taken then I still think it'd be great to have a syntax in the Vagrantfile for installing plugins. (option two above)

[tknerr]

@fgrehm agree to the viewpoint that it does not belong to vagrant core, I would compare it with Ruby vs. Bundler.

But: in order to create something similar to bundler we would need the possibility to store multiple versions of the same plugin in parallel. Right now that is not possible as vagrant will automatically uninstall the previous version of a plugin once you install a newer one.

One option as suggested by @thedrow would be to replace the install_gem command with a custom implementation that bypasses the vagrant plugin mechanism and installs vagrant plugins to a separate sandbox.

While this would be the only possibility I see right now, I'm not sure whether this is a good idea or not.

@fgrehm, @mitchellh what do you think?

[thedrow]

Please do note that my suggestion also requires us to replace the uninstallation command so it will uninstall the plugin from the installation path of the sandbox. I know it's obvious but I'm just making sure everyone acknowledged that.
It can lead to nasty bugs otherwise.

[rjmunro]

Something like this is a great idea, but I want my non-expert users to be able to do:

git checkout [project url]
cd [project dir]
vagrant up

And the necessary plugins be installed or at least a message like "plugin foobar not found, type vagrant plugin install foobar to install it" be shown to the user.

Extra steps, particularly ones that vary depending on the project, hugely reduce the appeal of vagrant.

[fgrehm]

Hey guys, here's my take on the problem: https://github.com/fgrehm/vundler :)

[erran-r7]

@fgrehm Aside from the split second of name confusion I love it. I'm closing this issue as I feel that vundler's a good solution for it.

[fgrehm]

@ecarey-r7 No worries, I'm up for name suggestions, vagrant-bundler and vagrant-plugin-bundler were taken already and I ran out of ideas :)

[erran-r7]

It's not that I don't like it. I just thought of Vundle for a second. :laughing:

[patcon]

According to my most trusted source, "vundle" is not a thing:
http://www.urbandictionary.com/define.php?term=vundle

:P

EDIT: Ah...

EDIT2: AWESOME.

[erran-r7]

@patcon I realize you noticed now, but let me google that for you: http://lmgtfy.com/?q=vundle

[patcon]

:)

Also, throwing in my hat for the new name: https://github.com/mitchellh/vagrant/issues/1789#issuecomment-21018873

[rjmunro]

No no no no no no no!!!!!!

Something to do this must go into core.

The whole point of vagrant is that I can type vagrant up and have a working system. Having to install some other product is not an acceptable compromise. I may as well just install the plugins. It's bad enough having to install virtual box as well as vagrant.

[patcon]

Core could easily be _aware_ of an awesome tool, directing the user toward it, without pulling in the whole kit-and-caboodle. There are alternative ways to address user experience concerns aside from bringing in a potentially large chunk of codebase.

And vagrant uping to get plugins might solve a problem for you, but could just as easily cause the problem for others. (eg. one plugin doesn't work on their system). There are lots of reasons not to force one user's plugins on another.

[erran-r7]

I agree with @patcon. vundler is a good solution. If you disagree @rjmunro the you could always use my hack above.

It changes the workflow from:

• Install vagrant
• Install virtualbox
• In every project with different dependencies:
  
  
  
  • vagrant plugin install vagrant-some-plugin
  
  
  • vagrant plugin install vagrant-another-plugin
  
  

To:

• Install vagrant
• Install virtualbox
• vagrant plugin install vundler
• vagrant vundle setup - Only once
• vagrant plugin bundle - In any project where it's necessary, it even ties into core and warns tells the user that they can just plugin bundle

[gregelin]

I am looking at this issue. Typing vagrant up is magic. Manually installing plugins is like telling the audience to look the other way while the trick is done.

I like magic.

[thomaswelton]

I have the same opinion as @rjmunro
I want to be able to tell people. Hey dude just git clone and vagrant up & boom!
Having to explain to people, hey ok so Vagrant is a thing, it does virtual host management. And theres this bindle thing it does plugin management. Install both of these things then boom! isn't as sexy.

And in terms of the Ruby and Bundler comparison, Nodejs ships with NPM by default now. I'd love ti for Vagrant to ship with a tool to manage plugins.

[seclorum]

I want to be able to tell people. Hey dude just git clone and vagrant up & boom!

Me too, I share the same point of view - as a user.

The advantage of having something like “vagrant plugins manage” is that it gives us a central place to collect public plugins in a way that makes sense: through the project git repo. If vagrant has a master file of plugins available, all it takes is to put the plugin up somewhere on github, and add the link to the master file. This would mean - to use users - that we’d always pretty much be sure we know where to find plugins in the first place, and it’d also give the community a nice way to actually audit the plugins, in case something goes awry.

Just my .2c, plzignore.

j.

[fgrehm]

Guys, eventually this will come in to core

Regarding "where to find plugins", I've been doing my best to keep this Wiki page up to date and easy to find plugins ;) In fact I have even thought of creating a new category of deprecated / no longer maintained plugins to reduce the noise and save others some trouble, just haven't had the time to do it yet

[patcon]

The ability to "tag" github repos could potentially make the above concern much simpler. (ie. vagrant-plugin tag that we can use the github search api to filter for)

Feel free to :+1: this related issue: https://github.com/isaacs/github/issues/114

[suan]

Agreed with needing the ability to automatically install needed plugins upon vagrant up. Over here we use vagrant so that some not-too-technical people have an environment to work with. All the extra steps to enter commands to install plugins before they can vagrant up really gets in the way of things.

If there's concern about "pollution" of a user's vagrant install through this mechanism, might I suggest having Vagrant output a prompt like:

Vagrant will now install needed plugins X, Y, and Z. [Y/n]

so the user is aware? This seems to me like a sensible compromise.

[drpebcak]

You already can install plugins by just doing a vagrant up... Just shell out in the first lines of the vagrantfile and put in some lines to install plugins/ check if they are already installed.

[gregelin]

Taylor,

Can you please provide an example of "shell out?"

Thanks.

Greg Elin
P: 917-304-3488
E: [email protected]

Sent from my iPhone

On Jan 25, 2014, at 8:40 PM, Taylor Price [email protected] wrote:

You already can install plugins by just doing a vagrant up... Just shell out in the first lines of the vagrantfile and put in some lines to install plugins/ check if they are already installed.

—
Reply to this email directly or view it on GitHub.

[erran]

@gregelin If you read my actual issue here you'll see a way to do so. %x(COMMAND HERE) is a way to shell out in Ruby. It holds the exit status and output if you assign it to a variable. My example covers a lot though. It should get you what you need to wrap it up and make it pretty.

[gregelin]

@erran Just looked at example. I'm afraid my Ruby is not strong enough that I immediately follow the example.

The placement of the plugin reference makes sense:

plugin 'vagrant-berkshelf', '1.3.2'

Vagrant.configure('2') do |config|
  # Normal configuration here
end

Less clear to me is where your other code goes. I'm guessing you are recommending the other coded might be a solution for inside of Vagrant's ruby code?

I was hoping by "shell out" there was a way within the Vagrantfile itself to call a shell script that installed plugin. I understand there is a reluctance to do this as Vagrantfile targets the guest machine and having Vagrantfile also target changes on the host machine crosses a logical layer boundary.

Maybe another way to think of this Vagrantfilehost that is a distinct controller for controlling the host machine and the Vagrant environment?

[kevinmeredith]

Is there a known fix? Using vagrant 1.3.5, I'm running into this error when running vagrant up:

I added "require 'berkshelf/vagrant' to my Vagrantfile, yet I'm sttil getting this error:

unknown configuration section omnibus

[erran]

@kman007us vagrant plugin install vagrant-omnibus The berkshelf and omnibus dependencies are separate. You need to pull in the omnibus vagrant plugin to gain the ability to access the omnibus "configuration section".

[drpebcak]

@gregelin something like this:

%x(vagrant plugin install vagrant-omnibus) unless Vagrant.has_plugin?('vagrant-omnibus')

That will one line will check if vagrant-omnibus is installed, and if it is not, vagrant will install it for you.

[zoglmannk]

@gregelin I ended up doing something like this:

if !Vagrant.has_plugin?('vagrant-omnibus') &&  ENV['SKIP'] != 'true'
  if RUBY_PLATFORM =~ /win/ && RUBY_PLATFORM !~ /darwin/
    puts "The vagrant-omnibus plugin is required. Please install it with \"vagrant plugin install vagrant-omnibus\""
    exit
  end

  print "Installing vagrant plugin vagrant-omnibus..."
  %x(bash -c "export SKIP=true; vagrant plugin install vagrant-omnibus") unless Vagrant.has_plugin?('vagrant-omnibus') || ENV['SKIP'] == 'true'
  puts "Done!"
  puts "Please re-run your vagrant command..."
  exit
end

[gregelin]

This is great! Eager to give it a try.

Greg Elin
P: 917-304-3488
E: [email protected]

Sent from my iPhone

On Mar 28, 2014, at 12:33 PM, Kurt Zoglmann [email protected] wrote:

@gregelin I ended up doing something like this:

if !Vagrant.has_plugin?('vagrant-omnibus') && ENV['SKIP'] != 'true'
if RUBY_PLATFORM =~ /win/ && RUBY_PLATFORM !~ /darwin/
puts "The vagrant-omnibus plugin is required. Please install it with \"vagrant plugin install vagrant-omnibus\""
exit
end

print "Installing vagrant plugin vagrant-omnibus..."
%x(bash -c "export SKIP=true; vagrant plugin install vagrant-omnibus") unless Vagrant.has_plugin?('vagrant-omnibus') || ENV['SKIP'] == 'true'
puts "Done!"
puts "Please re-run your vagrant command..."
exit
end

—
Reply to this email directly or view it on GitHub.

[drpebcak]

You can also do stuff like this:

if Vagrant::Util::Platform.windows?

instead of using RUBY_PLATFORM

[kowal]

For the record, the issue described in here originally seems to be fixed in https://github.com/mitchellh/vagrant/pull/2769 by usinng Bundler and vagrant plugin install command. It seems natural because plugins are afterall Ruby gems so we should leverage Bundler for this task.

I'm also not happy about new vagrant plugin install step. Vagrant introduced higher level abstraction and now it got broken by some 'plugins' details :(

[mitchellh]

@kowal This issue was not resolved in any way by the switch to Bundler. It certainly makes it more possible but we're not quite there yet. It is a place I want to reach, but haven't yet.

[bdw429s]

After reading all these comments, I'm a little unclear on what the outcome was of this discussion. This ticket was closed with a reference to "vundle" which appears to be a vim bundler which is a little confusing. Another comment says "eventually this will come in to core" but links to a dead ticket of +1's and comments about how the suggested libraries are all unmaintained.

I using vagrant with teams of developers who don't really know or care what Vagrant is, they just need it to work with as little manual steps as possible. vagrant up is pretty straightforward, but requirement them to also install a collection of other plugins is a lot harder than you would think. I would LOVE vagrant to take care of installing in the necessary plugins the first time they "up". I need a solution that is as easy as possible without extra manual steps to get screwed up or missed.

[miroswan]

So far I got this to install the plugin, but vagrant isn't aware of the new config options introduced:

def ensure_plugin(plugin)
  pm = Vagrant::Plugin::Manager.new(
    Vagrant::Plugin::Manager.user_plugins_file
  )
  plugin_hash = pm.installed_plugins
  return if plugin_hash.has_key?(plugin)

  puts "Installing plugin #{plugin}"
  pm.install_plugin(plugin)
end

If there is a way to reload the config to recognize the new plugin, I think I might be on my way.

[miroswan]

I ended up just settling for this. Not single command, but better than nothing:

# Install vagrant plugin
#
# @param: plugin type: Array[String] desc: The desired plugin to install
def ensure_plugins(plugins)
  logger = Vagrant::UI::Colored.new
  result = false
  plugins.each do |p|
    pm = Vagrant::Plugin::Manager.new(
      Vagrant::Plugin::Manager.user_plugins_file
    )
    plugin_hash = pm.installed_plugins
    next if plugin_hash.has_key?(p)
    result = true
    logger.warn("Installing plugin #{p}")
    pm.install_plugin(p)
  end
  if result
    logger.warn('Re-run vagrant up now that plugins are installed')
    exit
  end
end

[ghost]

@miroswan how do I use this in my Vagrant file? I've been looking for this solution as well.

I need to ensure some plugins are installed before running vagrant up

[jaequery]

hi everyone, are there any update on this? honestly having another command to run defeats the whole purpose of Vagrant for me and i'm sure a lot of others. this is just begging for another "higher abstraction" tool that fully automates deployments to come into market, something like "magic up" maybe, someday...

in my typical project, I have, Dockerfile, Docker Compose file, Vagrantfile, (not to mention several others) and so far i've used Vagrant that encapsulates the entire entry point of deployment which was nice. but having another file like Vundlefile as a dependency would literally just push me over my edge of sanity.

i really liked the op's suggestion of putting this up top of your Vagrantfile:
plugin 'vagrant-berkshelf', '1.3.2'

i really think it's no harm no foul, because those are commands that the users would have to run manually in order for their app to work anyway and this would be just letting vagrant perform it for you.

[jbq]

FWIW I'm using @miroswan's solution and I'm happy with it. Native support in Vagrant would be nice though.

[devnix]

Hi guys, I've started a little plugin manager for my own. If you think it is useful for you, I will be glad to hear things that could be improved.

I'm maintaining this project as bugs appears at my daily job on different environments. If you have some issue or pull request, I will be constantly improving this.

Cheers!

http://devnix.github.io/Vagrant-dependency-manager/
https://github.com/DevNIX/Vagrant-dependency-manager

[jolohaga]

I find my users don't mind doing an init step before doing vagrant up. In developing init scripts, I wish Vagrant provided command line tools to query what plugins are installed.

[devnix]

@jolohaga Do you mean a command like vagrant plugin list?

[jolohaga]

@DevNIX Yes, that works for me. Obviously, I need to learn the command line features better. Thank you for pointing this out.

[nitrocode]

@DevNIX cool script (minus the tabs lol). Would you possibly turn that into a PR to merge it into vagrant core?

[devnix]

@nitrocode I don't know if my code would be the correct approach to do this job inside the core project. Maybe I could try to hack it some day, but I am not an experienced Ruby developer. Will fix the tabs ASAP!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.