Debian 10 is coming out on 2019-07-06 btw https://lists.debian.org/debian-devel-announce/2019/06/msg00003.html
root@hes:/home/debian# /tmp/.saltcloud-5d49e76b-7a75-4e68-bf7f-c087a112ee07/deploy.sh -c /tmp/.saltcloud-5d49e76b-7a75-4e68-bf7f-c087a112ee07 -x python3
* INFO: Running version: 2019.01.08
* INFO: Executed by: -
* INFO: Command line: '/tmp/.saltcloud-5d49e76b-7a75-4e68-bf7f-c087a112ee07/deploy.sh -c /tmp/.saltcloud-5d49e76b-7a75-4e68-bf7f-c087a112ee07 -x python3'
* INFO: Detected -x option. Using python3 to install Salt.
...
Hit:1 http://security.debian.org buster/updates InRelease
Get:2 http://deb.debian.org/debian buster InRelease [163 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [46.8 kB]
Get:4 http://deb.debian.org/debian buster-backports InRelease [44.5 kB]
Hit:5 https://repo.saltstack.com/py3/debian/9/amd64/latest stretch InRelease
Fetched 254 kB in 1s (467 kB/s)
Reading package lists...
Reading package lists...
...
Hit:1 http://deb.debian.org/debian buster InRelease
Hit:2 http://deb.debian.org/debian buster-updates InRelease
Hit:3 http://security.debian.org buster/updates InRelease
Hit:4 http://deb.debian.org/debian buster-backports InRelease
Hit:5 https://repo.saltstack.com/py3/debian/9/amd64/latest stretch InRelease
Reading package lists...
* INFO: Running config_salt()
* INFO: Running install_debian_stable()
Reading package lists...
Building dependency tree...
Reading state information...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
salt-minion : Depends: salt-common (= 2019.2.0+ds-1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
root@hes:/home/debian# apt-get install salt-common=2019.2.0+ds-1
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
salt-common : Depends: python3-tornado (< 5) but 5.1.1-4 is to be installed
Recommends: python3-croniter but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
Also see https://github.com/saltstack/salt-pack-py3/issues/116
The control file of the Salt 2018.3.4 package that is included in the Buster release build-depends on
python3-tornado4 (>= 4.2) | python3-tornado (<< 5),
python3-tornado4 (>= 4.2) | python3-tornado (>= 4.2),
since this patch https://salsa.debian.org/salt-team/salt/commit/555fd315e41e8746444f28bc0178d57acab7371e introduced in December 2018
The control file of the Salt 2018.3.4 package that is included in the Buster release build-depends on
python3-tornado4 (>= 4.2) | python3-tornado (<< 5), python3-tornado4 (>= 4.2) | python3-tornado (>= 4.2),since this patch https://salsa.debian.org/salt-team/salt/commit/555fd315e41e8746444f28bc0178d57acab7371e introduced in December 2018
This has already been mentioned in https://github.com/saltstack/salt-pack-py3/issues/116#issue-422035370
any update? 2019.2.1 is released
imo: __install_saltstack_debian_repository still detects debian 10 as stretch
The culprit seems to be this check. I changed the value 10 to something absurd (999) and ran with ./bootstrap-salt.sh -x python3 and it installed perfectly.
I believe that check is outdated and no longer needed.
@smitelli there is this issue to address this https://github.com/saltstack/salt-bootstrap/issues/1360
Thanks for the heads up on the 'this check'
This should be fixed by https://github.com/saltstack/salt-bootstrap/pull/1375
Can anyone verify as well?
Tried using a Vagrant VM I had lying around:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 10 (buster)
Release: 10
Codename: buster
$ wget https://raw.githubusercontent.com/saltstack/salt-bootstrap/develop/bootstrap-salt.sh
$ chmod +x bootstrap-salt.sh
$ sudo ./bootstrap-salt.sh -x python3
...snip...
* INFO: Salt installed!
$ salt-minion -V
Salt Version:
Salt: 2019.2.2
Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: 2.7.3
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 3.7.3 (default, Apr 3 2019, 05:39:12)
python-gnupg: Not Installed
PyYAML: 3.13
PyZMQ: 17.1.2
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.3.1
System Versions:
dist: debian 10.1
locale: UTF-8
machine: x86_64
release: 4.19.0-6-amd64
system: Linux
version: debian 10.1
Looks like it did the job!
Without -x python3, however, the install fails at install_debian_deps(). The underlying reason for that seems to be something wrong with the apt repo, since it breaks apt-get update too:
$ sudo apt-get update
Ign:1 https://repo.saltstack.com/apt/debian/10/amd64/latest buster InRelease
Hit:2 http://security.debian.org/debian-security buster/updates InRelease
Hit:3 http://deb.debian.org/debian buster InRelease
Err:4 https://repo.saltstack.com/apt/debian/10/amd64/latest buster Release
404 Not Found [IP: 99.84.127.46 443]
Reading package lists... Done
E: The repository 'https://repo.saltstack.com/apt/debian/10/amd64/latest buster Release' does not have a Release file.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
My guess is that's not something wrong with salt-bootstrap itself, but rather a packaging issue with Salt.
If it could be of any help:
https://repo.saltstack.com/apt/debian/10/amd64/latest doesn't exist but replace "apt" by "py3" and it will work:
http://repo.saltstack.com/py3/debian/10/amd64/latest/
I started running into the problem @smitelli mentioned either on Sunday or Monday, after I swapped my hypervisor to Debian 10. I had already been using Debian 10 instead of Ubuntu 18.04 on my salt master for at least a couple of days before this, and I stopped being able to build it. I thought it was my version of vagrant pulling in an old bootstrap script, but I downloaded the bootstrap script by hand and none of the options (git, develop, stable 2019.1.27, etc) I tried would get me past the "No release file" issue.
(Don't want to me too this thread, but I am hoping the timeline can help).
The problem for me is also that the bootscript now sucessfully detects Debian Buster but there is no repository anymore for it in path .../10/... so it fails!
Seems the internal communication between Saltstack teams need to be improved ?
$ vagrant provision virtualbox
==> virtualbox: Running provisioner: salt...
Copying salt minion config to vm.
Copying salt grains config to vm.
Checking if salt-minion is installed
salt-minion was not found.
Checking if salt-call is installed
salt-call was not found.
Using Bootstrap Options: -K -A -U -p curl -p patch -p virt-what -p libxml2-utils -p jq -i akeneo-vagrant -F -c /tmp stable
Bootstrapping Salt... (this may take a while)
* INFO: Running version: 2019.11.04
* INFO: Executed by:
* INFO: Command line: 'bootstrap-salt.sh -K -A -U -p curl -p patch -p virt-what -p libxml2-utils -p jq -i akeneo-vagrant -F -c /tmp stable'
* INFO: System Information:
* INFO: CPU: GenuineIntel
* INFO: CPU Arch: x86_64
* INFO: OS Name: Linux
* INFO: OS Version: 4.19.0-5-amd64
* INFO: Distribution: Debian 10
* INFO: Installing minion
* INFO: Found function install_debian_deps
* INFO: Found function config_salt
* INFO: Found function preseed_master
* INFO: Found function install_debian_stable
* INFO: Found function install_debian_restart_daemons
* INFO: Found function daemons_running
* INFO: Found function install_debian_check_services
* INFO: Running install_debian_deps()
Hit:1 http://security.debian.org/debian-security buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Ign:4 https://repo.saltstack.com/apt/debian/10/amd64/archive/2019.2.0 buster InRelease
Err:5 https://repo.saltstack.com/apt/debian/10/amd64/archive/2019.2.0 buster Release
404 Not Found [IP: 13.225.78.8 443]
Reading package lists...
EDIT: here the verification that Vagrant fetches the latest bootstrap script:
https://github.com/hashicorp/vagrant/blob/master/plugins/provisioners/salt/bootstrap-salt.sh
Same here, both repo paths for Debian 10 and CentOS 8 (RHEL 8) are absent:
http://repo.saltstack.com/yum/redhat/8/ - 404 not found
https://repo.saltstack.com/apt/debian/10 - 404 not found
Repos for rhel 7 and debian 9 are present.
P.S.
I've found that py3 versions are available: http://repo.saltstack.com/py3/debian/10/
Does it mean py2 is completely deprecated for RHEL8/Debian10? My understanding was that it is supported for now (for example, my master runs on python2.7)
I don't understand the issue anymore,
the salt-bootstrap script works for Debian 10 (verified like so)
@kiemlicz if you use default python (2.7 for example) - there are still no packages available for salt in repo.
see https://repo.saltstack.com/apt/debian/ - only 7, 8 and 9 are available.
Same issue with Centos8. You can add -x python3 to bootstrap options and that will work fine.
If I run with -x python3 - it works, installs py3 dependecies. If I run without - it does not. My understanding was that python 2.7 should still work on Debian 10/Centos 8. If it does not, should not bootstrap just go for python 3 ot throw an error "python versions prior to 3.x are not supported for Debian 10/Centos 8 etc"
@kiemlicz @mbochenk I would also consider the current behavior a bug. either in the script itself or its documentation. As an end-user, I feel one of the following options should occur:
./bootstrap-salt.sh without the -x option should use the default python2 and succeed../bootstrap-salt.sh without the -x option should default to python3 on these newer OS releases without requiring the user to specify this../bootstrap-salt.sh without the -x option should fail immediately on these newer OS releases, with a clear error message stating that -x python3 is required for the installation to succeed here.@smitelli On another hand I vaguely remember that currently python3 minions are not fully compatible with python2 master and vice-versa. I am actually running into issues with this, as all my masters are on python 2.7 (Centos 7)
So I would say option #1 or #3 in this case, number 2 may cause too many grief you are not ready to deal with.
@kiemlicz it would be much more nice if you add here the call you are using in line 10 of your linked script and explain why you solution works in your buster version against the default settings 馃樃 ...
I found this call in your reference:
sh /tmp/bootstrap-salt.sh -x python3 -X -n $salt_ver
And from checking documentation:
-x Changes the Python version used to install Salt.
For CentOS 6 git installations python2.7 is supported.
Fedora git installation, CentOS 7, Debian 9, Ubuntu 16.04 and 18.04 support python3.
-X Do not start daemons after installation
-n No colours
$salt_ver = ""
because then the default "stable" is used in Buster repository which isn't available as we use them also:
Installation types:
- stable Install latest stable release. This is the default
install type
- stable [branch] Install latest version on a branch. Only supported
for packages available at repo.saltstack.com
- stable [version] Install a specific version. Only supported for
packages available at repo.saltstack.com
- testing RHEL-family specific: configure EPEL testing repo
- git Install from the head of the develop branch
- git [ref] Install from any git ref (such as a branch, tag, or
commit)
Examples:
- ${__ScriptName}
- ${__ScriptName} stable
- ${__ScriptName} stable 2017.7
- ${__ScriptName} stable 2017.7.2
- ${__ScriptName} testing
- ${__ScriptName} git
- ${__ScriptName} git 2017.7
- ${__ScriptName} git v2017.7.2
- ${__ScriptName} git 06f249901a2e2f1ed310d58ea3921a129f214358
BTW the call
RUN apt-get update && apt-get install -y procps curl dumb-init &&
is inefficient because you can do it also within bootstrap options ;)
-U If set, fully upgrade the system prior to bootstrapping Salt
and
-p Extra-package to install while installing Salt dependencies. One package
per -p flag. You are responsible for providing the proper package name.
=>
-U -p procps -p curl -p dumb-init
This option was also not successful because it's need a mirror with /10/ version subfolder:
-R Specify a custom repository URL. Assumes the custom repository URL
points to a repository that mirrors Salt packages located at
repo.saltstack.com. The option passed with -R replaces the
"repo.saltstack.com". If -R is passed, -r is also set. Currently only
works on CentOS/RHEL and Debian based distributions.
but this option could help temporary:
-r Disable all repository configuration performed by this script. This
option assumes all necessary repository configuration is already present
on the system.
because Debian Buster has already saltstack packages which may be new enough:
salt-call 2018.3.4 (Oxygen)
@Reiner030 I am not sure i understand where this is going.
-x python3: if used with bootstrap deployment works. I am using stable 2019.2.2. I did not try develop, etc.
WIthout this option bootstrap tries to install python2 (python 2.7) and fails, as there are no public repositories available on https://repo.saltstack.com with python 2.7 parts - only py3. It does not matter which options you provide at this point (maybe pip install will do?) - it sill still fail as soon as tries to fetch packages from non-existing repo. You could try to install all packages manually via other repos or pip, but that does not resolve this issue, esp. if we use salt-cloud to create a VM and run bootstrap there.
@Reiner030 Thank you for pointing out the options!
I claimed that it works because this issue is about Debian 10 and Python3 - which works just fine
However you are right - it is impossible to install Salt on Debian 10 using Python2.7 ('my solution' won't work).
There simply is no repo available (https://repo.saltstack.com/#debian)
Maybe there are other ways to install (from git) however the 'proper' one (packages) is unavailable.
I think that the maintainers should make a statement about Debian 10 Python2 support
@kiemlicz In short I did not want to make duplicates and this one at the time seemed to fit the bill the best out of other issues I saw (for ex https://github.com/saltstack/salt-bootstrap/issues/1379 https://github.com/saltstack/salt-bootstrap/issues/1358 https://github.com/saltstack/salt-bootstrap/issues/1371)
Yes, I landed here after looking at a couple of other issues too.
For now, I'm using an older version since bootstrap_salt.sh's default behavior for Debian 10 doesn't work out of the box, which made automating the bootstrap a bit tougher than it needed to be, and I think I gave up in the end anyway. Maybe the newest latest does it.
you are accurate in your description that we only supply python3 packages for debian 10. You can install python2 with a git installation.
One thing that is making itself clear here though is we could probably improve the script with either these options:
a. always use python3 if installing with packages on debian 10
OR
b. print out an error stating it requires -x python3 so the user understands they will be getting python3 packages.
im leaning more towards option a. any thoughts?
In my opinion you should respect the system default version.
Since the default Python version for Debian 10 is Python 2.7 I'm in favour of option b
Because it is explicit and potentially avoids confusion
This is one of those situations where user experience may be more important than doing what is purely "correct." Yes, in an ideal world, Salt's Python version would match the distribution default. However, in this case, the packagers of Salt have made the decision that Python 2 and Debian 10 are not a combination they choose to package for. That limits our options here, simply because the ideal case can't work out of the box.
(An aside: I suppose it would be possible to force things to work with Python 2 using a scripted Git install instead of the apt packages and hiding that difference within bootstrap-salt, but that behavior would be unexpected and alarming for some. Speaking personally, I want Salt installed as a system package so I can get updates without having to re-run bootstrap-salt or think about Git.)
I'm leery of requiring users to explicitly state -x python3 since there is no other value for this argument that can work on their systems. It's going to lead to users blindly copying that into their install commands whether they need it or not, and keep it there for years to come -- long after Debian 10 is superseded and Python 3 becomes the default anyway.
As a user of Salt, I don't actually care what version of Python it's using. Maybe it matters to other people, but it's always been an opaque implementation detail to me. My main concerns are that it installs without hassle and works. That leads me more toward option A.
From my side: I do not care which version of python I am using as long as it works. In the past there were warnings about py3 minor incompatibility peppered through docs, that is why I was under impression that 2.7 is still somewhat supported.
A check and a clear message (python prior 3 is NOT supported on Debian10/CentOS8 /ETC.) would be nice too.
From my point of view: the app "should" respect the system default Python (but he Debian/Python 2 ship has sailed). But since I'm running the bootstrap script to install salt, and it doesn't support the default on my Debian 10 system, the bootstrap script should just do whatever it takes to install salt.
I vote for option A.
Not working for me, even with the '-x python3' option:
b@salt-master:~/Downloads$ sudo sh bootstrap-salt.sh -M -N -D
INFO: Distribution: Debian 10
DEBUG: Binaries will be searched using the following $PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
b@salt-master:~/Downloads$ sudo sh bootstrap-salt.sh -M -N -D -x python3
INFO: Distribution: Debian 10
DEBUG: Binaries will be searched using the following $PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
@b5n That second invocation might work if you removed /etc/apt/sources.list.d/saltstack.list. My guess is it contains
deb https://repo.saltstack.com/apt/debian/10/amd64/latest something something
while it needs
deb https://repo.saltstack.com/py3/debian/10/amd64/latest buster main
to succeed on Debian 10.
Looks like the first invocation of bootstrap-salt.sh left apt in a broken state for the second invocation.
Hello @Ch3LL,
tested today a new Saltstack setup based on my Vagrant VM because I have seen that above py3 packages are available again for Debian Buster since 2019.2.3is published.
2019.11.04 still can't automatically add the needed python3 reference.-x python3 set automatically in Debian Buster and similar distributions with python 3 ?I can reproduce this issue, after updating the bootstrap script to 2020.01.21 we got errors installing debian dependencies. The culprit was an erroneous entry in /etc/apt/sources.list.d/saltstack.list.
The URL added was:
https://repo.saltstack.com/apt/debian/10/amd64/latest
Adding script_args: -x python3 to /etc/salt/cloud sets the proper URL of:
https://repo.saltstack.com/py3/debian/10/amd64/latest
we are going to use this issue https://github.com/saltstack/salt-bootstrap/issues/1418 to track the work to not require -x python3 for OSs we no longer support python2. based on this are we okay to close here?
Thank You All. Closing.
Most helpful comment
we are going to use this issue https://github.com/saltstack/salt-bootstrap/issues/1418 to track the work to not require
-x python3for OSs we no longer support python2. based on this are we okay to close here?