Salt-bootstrap: Bootstrap fails on old Ubuntu images due to lack of `apt-get update`

[boboli]

Description of Issue/Question

When using an old Ubuntu base image and trying to provision using the salt bootstrap script (pointing to a git version), I run into the following error:

Fetched 3,276 kB in 3s (1,087 kB/s)
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/r/rsync/rsync_3.1.0-2ubuntu0.1_amd64.deb  404  Not Found [IP: 91.189.91.26 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/patch/patch_2.7.1-4ubuntu1_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
 * ERROR: Failed to run install_ubuntu_git_deps()!!!

I suspect this is because the bootstrap script doesn't call apt-get update before trying to install git. I hopped onto the Vagrant box and ran sudo apt-get update manually, and then was able to apt-get install git just fine.

What likely happened is that the old Vagrant box I'm using has an old apt index, and recently Ubuntu's servers have removed the old rsync and patch packages that the outdated index was referring to.

Setup

I'm using this Vagrant image: https://atlas.hashicorp.com/fgrehm/boxes/trusty64-lxc

Vagrantfile:

Vagrant.configure(2) do |config|
  config.vm.box = "fgrehm/trusty64-lxc"

  config.vm.provision :salt do |salt|
    salt.install_master = true
    salt.install_type = "git"
    salt.install_args = "v2015.8.7"
    salt.bootstrap_options = "-P"
  end
end

Steps to Reproduce Issue

Use the above Vagrantfile, and run vagrant up.

Versions and Systems

(salt --versions-report, bootstrap-salt.sh -v, system type and version,
cloud/VM provider as appropriate.)

The raw output of the bootstrap-salt.sh run:

vagrant@vagrant-ubuntu-trusty-64:~/saltbroke$ vagrant up
Bringing machine 'default' up with 'lxc' provider...
==> default: Importing base box 'fgrehm/trusty64-lxc'...
==> default: Checking if box 'fgrehm/trusty64-lxc' is up to date...
==> default: Setting up mount entries for shared folders...
    default: /vagrant => /home/vagrant/saltsuck
==> default: Starting container...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 172.20.7.30:22
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Running provisioner: salt...
Checking if salt-minion is installed
salt-minion was not found.
Checking if salt-call is installed
salt-call was not found.
Checking if salt-master is installed
salt-master was not found.
Bootstrapping Salt... (this may take a while)
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

/tmp/bootstrap_salt.sh  -P -M git v2015.8.7

Stdout from the command:

 *  INFO:  sh -- Version 2016.04.18

 *  INFO: System Information:
 *  INFO:   CPU:          GenuineIntel
 *  INFO:   CPU Arch:     x86_64
 *  INFO:   OS Name:      Linux
 *  INFO:   OS Version:   3.13.0-79-generic
 *  INFO:   Distribution: Ubuntu 14.04

 *  INFO: Installing minion
 *  INFO: Installing master
 *  INFO: Found function install_ubuntu_git_deps
 *  INFO: Found function install_ubuntu_git
 *  INFO: Found function install_ubuntu_git_post
 *  INFO: Found function install_ubuntu_restart_daemons
 *  INFO: Found function daemons_running
 *  INFO: Found function install_ubuntu_check_services
 *  INFO: Running install_ubuntu_git_deps()
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  git git-man liberror-perl patch rsync
Suggested packages:
  gettext-base git-daemon-run git-daemon-sysvinit git-doc git-el git-email
  git-gui gitk gitweb git-arch git-bzr git-cvs git-mediawiki git-svn ed
  diffutils-doc
The following NEW packages will be installed:
  git git-core git-man liberror-perl patch rsync
0 upgraded, 6 newly installed, 0 to remove and 0 not upgraded.
Need to get 3,643 kB of archives.
After this operation, 22.5 MB of additional disk space will be used.
Err http://archive.ubuntu.com/ubuntu/ trusty-updates/main rsync amd64 3.1.0-2ubuntu0.1
  404  Not Found [IP: 91.189.88.161 80]
Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main liberror-perl all 0.17-1.1 [21.1 kB]
Err http://security.ubuntu.com/ubuntu/ trusty-security/main rsync amd64 3.1.0-2ubuntu0.1
  404  Not Found [IP: 91.189.91.26 80]
Get:2 http://archive.ubuntu.com/ubuntu/ trusty/main git-man all 1:1.9.1-1 [698 kB]
Get:3 http://archive.ubuntu.com/ubuntu/ trusty/main git amd64 1:1.9.1-1 [2,555 kB]
Get:4 http://archive.ubuntu.com/ubuntu/ trusty/main git-core all 1:1.9.1-1 [1,452 B]
Err http://archive.ubuntu.com/ubuntu/ trusty-updates/main patch amd64 2.7.1-4ubuntu1
  404  Not Found [IP: 91.189.88.161 80]
Fetched 3,276 kB in 3s (1,087 kB/s)
E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/r/rsync/rsync_3.1.0-2ubuntu0.1_amd64.deb  404  Not Found [IP: 91.189.91.26 80]

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/p/patch/patch_2.7.1-4ubuntu1_amd64.deb  404  Not Found [IP: 91.189.88.161 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
 * ERROR: Failed to run install_ubuntu_git_deps()!!!


Stderr from the command:

stdin: is not a tty
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  237k  100  237k    0     0   340k      0 --:--:-- --:--:-- --:--:--  340k

[audleman]

I'm getting the same error trying to spin up EC2 images based on Ubuntu 14.04. Did you find a fix?

[JarenGlover]

Not ideal .. But force a "apt-get update" before your salt install via your VagrantFile 

On Sat, Apr 30, 2016 at 11:20 AM Kevin Audleman

<
mailto:Kevin Audleman [email protected]

wrote:

I'm getting the same error trying to spin up EC2 images based on Ubuntu 14.04. Did you find a fix?

—

You are receiving this because you are subscribed to this thread.

Reply to this email directly or
https://github.com/saltstack/salt-bootstrap/issues/821#issuecomment-215985045
https://github.com/saltstack/salt-bootstrap/issues/821#issuecomment-215985045

[audleman]

I found a fix for my situation too, though it's not very elegant. I created a new AMI where I had already run sudo apt-get update and used it as my base box.

[jfindlay]

Thanks for reporting. I was going to say that I wonder how many users would be unhappy if we ran apt-get update as part of bootstrap, but clearly bootstrapping a sufficiently old image is a problem. This is problematic in other ways, but that burden is more on the user than salt-bootstrap. Then I saw that this is done already in install_ubuntu_deps, and this is certainly not as bad as an apt-get -y upgrade. :-)

[boboli]

Yup, I think that if part of the bootstrap requires git to be installed, then the apt-get update should definitely happen, to make sure no apt-get installs can fail because of an outdated index.

[audleman]

@jfindlay I'm not sure what the issues are with running apt-get -y upgrade, but I am in favor of the salt bootstrap script running it.

[UtahDave]

doesn't the -U flag to the bootstrap script do a system upgrade first?

On Mon, May 16, 2016 at 3:21 PM, Kevin Audleman [email protected]
wrote:

@jfindlay https://github.com/jfindlay I'm not sure what the issues are
with running apt-get -y upgrade, but I am in favor of the salt bootstrap
script running it.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/saltstack/salt-bootstrap/issues/821#issuecomment-219551984

[audleman]

@UtahDave I can't quite remember right now, but the problem occurs when Salt Cloud initially copies /etc/salt/cloud.deploy.d/bootstrap-salt.sh to the remote server. Either it doesn't run or starts to but fails. I can check tonight if it's useful.

[marccardinal]

Any particular reason why https://github.com/saltstack/salt-bootstrap/blob/develop/bootstrap-salt.sh#L2400 has

if [ "$DISTRO_MAJOR_VERSION" -eq 12 ]; then
    apt-get update
fi

Running 14.04 here and an apt-get update would need to be run as part of bootstrap-salt.sh -P git

[rallytime]

This is fixed by #859 in the latest release.