Salt: host.present may break /etc/hosts file format
Dear all,
using host.present to add an host with alias names to /etc/hosts may break the file format as explained below.
I am not sure whether this is merely a case of unexpected behavior or wrong usage by me or might indeed qualify as a bug.
The format of a line in /etc/hosts on an unixoid system is fixed as
IP_address canonical_hostname [aliases...]
so to be more concrete usually 10.10.10.111 test.example.com test
please confirm with man (5) hosts. The FQDN has to be in front of the short-hand alias.
Naively using host.present to add the whole line 10.10.10.111 test.example.com test in one call generates instead:
test_input:
host.present:
- ip: 10.10.10.111
- names:
- fqdn.domain.com
- alias
Generated line in the minions /etc/hosts: 10.10.10.111 alias fqdn.domain.com
test_input:
host.present:
- ip: 10.10.10.111
- names:
- alias
- fqdn.domain.com
Generated line in the minions /etc/hosts: 10.10.10.111 alias fqdn.domain.com
Both break /etc/hosts format, the alias is in front of the FQDN.
Calling host.present twice can be used to generate the correct order of entries as shown below
input_fqdn:
host.present:
- ip: 10.10.10.111
- names:
- fqdn.domain.com
input_alias:
host.present:
- ip: 10.10.10.111
- names:
- alias
Generated line in the minions /etc/hosts: 10.10.10.111 fqdn.domain.com alias
but is awkward.
In any case I would suggest to mention this behavior in the salt.states.host documentation. If generating an FQDN and several aliases for one IP with one call of host.present is not the intended use case of the option to pass several names as a list (so in case I am using the features of host.present in a wrong way) it would also be nice to document the intended use.
Best Regards
Christof
Salt Version information:
Master
Salt Version:
Salt: 2015.8.3
Dependency Versions:
Jinja2: 2.7.3
M2Crypto: Not Installed
Mako: 1.0.0
PyYAML: 3.11
PyZMQ: 14.4.0
Python: 2.7.9 (default, Mar 1 2015, 12:57:24)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.5
cffi: 0.8.6
cherrypy: Not Installed
dateutil: 2.2
gitdb: 0.5.4
gitpython: 0.3.2 RC1
ioflo: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.2
mysql-python: 1.2.3
pycparser: 2.10
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: 0.8.2
timelib: Not Installed
System Versions:
dist: debian 8.2
machine: x86_64
release: 3.16.0-4-amd64
system: debian 8.2
Minion
Salt Version:
Salt: 2015.8.3
Dependency Versions:
Jinja2: 2.7.2
M2Crypto: Not Installed
Mako: 0.9.1
PyYAML: 3.10
PyZMQ: 14.0.1
Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
RAET: Not Installed
Tornado: 4.2.1
ZMQ: 4.0.4
cffi: Not Installed
cherrypy: Not Installed
dateutil: 1.5
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.3.0
mysql-python: 1.2.3
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: Not Installed
timelib: Not Installed
System Versions:
dist: Ubuntu 14.04 trusty
machine: x86_64
release: 3.13.0-57-generic
system: Ubuntu 14.04 trusty
ckoe-bccms
All 22 comments
@ckoe-bccms, I agree that the alias(es) should come after the FQDN. This is according to my own research and experience.
jfindlay
on 10 Dec 2015
Hi @jfindlay / @ckoe-bccms
I was trying to fix this, but looks like was already fixed.
Salt Version:
Salt: 2015.8.3-138-g654cab0
Dependency Versions:
Jinja2: 2.8
M2Crypto: 0.21.1
Mako: Not Installed
PyYAML: 3.10
PyZMQ: 15.1.0
Python: 2.7.6 (default, Jun 22 2015, 17:58:13)
RAET: Not Installed
Tornado: 4.3
ZMQ: 4.1.2
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
libnacl: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
python-gnupg: Not Installed
smmap: Not Installed
timelib: Not Installed
System Versions:
dist: Ubuntu 14.04 trusty
machine: x86_64
release: 3.13.0-68-generic
system: Ubuntu 14.04 trusty
Sample state
common_hostsfile:
host.present:
- ip: 10.10.10.123
- names:
- foo.local.domain
- foo
Hosts file content
cat /etc/hosts
127.0.0.1 localhost
Salt output
vagrant-ubuntu-trusty-64:
----------
ID: common_packages
Function: pkg.installed
Result: True
Comment: All specified packages are already installed
Started: 13:30:42.517345
Duration: 336.214 ms
Changes:
----------
ID: common_hostsfile
Function: host.present
Name: foo.local.domain
Result: True
Comment: Added host foo.local.domain (10.10.10.123)
Started: 13:30:42.854828
Duration: 1.119 ms
Changes:
----------
host:
foo.local.domain
----------
ID: common_hostsfile
Function: host.present
Name: foo
Result: True
Comment: Added host foo (10.10.10.123)
Started: 13:30:42.856035
Duration: 0.649 ms
Changes:
----------
host:
foo
Summary for vagrant-ubuntu-trusty-64
------------
Succeeded: 3 (changed=2)
Failed: 0
------------
Total states run: 3
Result looks good
cat /etc/hosts
127.0.0.1 localhost
10.10.10.123 foo.local.domain foo
Remving that line in hosts file and changing the order in names. Now I have alias first and fqdn next
common_hostsfile:
host.present:
- ip: 10.10.10.123
- names:
- foo
- foo.local.domain
Result looks good
vagrant-ubuntu-trusty-64:
----------
ID: common_packages
Function: pkg.installed
Result: True
Comment: All specified packages are already installed
Started: 13:34:13.998807
Duration: 300.357 ms
Changes:
----------
ID: common_hostsfile
Function: host.present
Name: foo.local.domain
Result: True
Comment: Added host foo.local.domain (10.10.10.123)
Started: 13:34:14.300372
Duration: 1.348 ms
Changes:
----------
host:
foo.local.domain
----------
ID: common_hostsfile
Function: host.present
Name: foo
Result: True
Comment: Added host foo (10.10.10.123)
Started: 13:34:14.301741
Duration: 0.726 ms
Changes:
----------
host:
foo
Summary for vagrant-ubuntu-trusty-64
------------
Succeeded: 3 (changed=2)
Failed: 0
------------
Total states run: 3
Host file
cat /etc/hosts
127.0.0.1 localhost
10.10.10.123 foo.local.domain foo
abednarik
on 30 Dec 2015
@ckoe-bccms, based on @abednarik's research, this may be fixed in 2015.8.4.
jfindlay
on 5 Jan 2016
Since we haven't heard anything back here, I will close this. However, if the issue remains, please leave a comment and we will gladly take another look.
rallytime
on 8 Apr 2016
For the record, this is still not working (tested with 2016.3.1).
This is very dependent on fqdn and alias name.
common_hostsfile:
host.present:
- ip: 10.10.10.123
- names:
- foo
- foo.local.domain
Will give:
10.10.10.123 foo.local.domain foo
But
test_input:
host.present:
- ip: 10.10.10.111
- names:
- fqdn.domain.com
- alias
Will give:
10.10.10.111 alias fqdn.domain.com
It will be great if this can be fixed in future release.
alssi
on 13 Jul 2016
It has a very strange behavior for me as well (on 2016.3.1), I have the exact similar output as @alssi, but even the 'working' one seems to have some issues.
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo
- foo.example.com
This works fine and render as: 127.0.1.1 foo.example.com foo
However if my fqdn have an additional subdomain like this:
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo
- foo.bar.example.com
it fails: 127.0.1.1 foo foo.bar.example.com
@rallytime could we re-open for investigation ? thanks
choopooly
on 27 Jul 2016
@choopooly Definitely. I'll removed the fixed pending label, too.
rallytime
on 27 Jul 2016
@abednarik You might be interested in taking another look at this one.
rallytime
on 27 Jul 2016
Sure, just give me a few days. I need spare time to work on this.
Cheers.
abednarik
on 27 Jul 2016
HI All
@rallytime I'm not a python expert, but could this issue be related to OrderedDict, see this
_Example 1_
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo
- foo.bar.example.com
Result
127.0.0.1 localhost
127.0.1.1 foo foo.bar.example.com
Debug
[ERROR ] OrderedDict([('127.0.0.1', ['localhost']), ('127.0.1.1', ['foo'])])
[ERROR ] OrderedDict([('127.0.0.1', ['localhost']), ('127.0.1.1', ['foo', 'foo.bar.example.com'])])
_Example 2_
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo.bar.example.com
- foo
Result
127.0.0.1 localhost
127.0.1.1 foo.bar.example.com foo
Debug
[ERROR ] OrderedDict([('127.0.0.1', ['localhost']), ('127.0.1.1', ['foo.bar.example.com'])])
[ERROR ] OrderedDict([('127.0.0.1', ['localhost']), ('127.0.1.1', ['foo.bar.example.com', 'foo'])])
Didn't found any code that take cares of values in that dict in order to guarantee fqdn is the first entry after IP that could be the cause we get random results.
abednarik
on 5 Aug 2016
@abednarik Nice find. I think you're right that the value order inside that dict is where this needs to be fixed.
rallytime
on 5 Aug 2016
Just a quick follow-up, with version 2016.3.* I still got the same issue but as a workaround I'm now using host.only on which the order seems to matter.
common_hostsfile:
host.only:
- name: 127.0.1.1
- hostnames:
- foo.bar.example.com
- foo
What exactly is the desired behaviour? That the entries should be listed in the precise order written in the YAML?
lordcirth
on 31 Aug 2017
Hi @lordcirth
The order should be
IP FQDN Alias1 Alias2
````
Where fqdn is the fully qualified domain name, basically the hostname + domain. Simple example:
ServerName: web1
Domain: company.com
Aliases: web1, web1-app, app1
IP: 192.168.23.44
192.168.23.44 web1.company.com web1 web1-app app1
````
I didn't review this is a long time, but looks like every entry in hostnames is a dictionary and entries there are added without proper order, just like a python dict.
abednarik
on 31 Aug 2017
@lordcirth I'm not sure what would be the best solution. The order would work fine if that's documented
Another option would be to add fqdn as another parameter of host.present so it's forced to add fqdn before any alias.
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo
- fqdn:
- foo.bar.example.com
So I understand that "IP FQDN Alias1 Alias2" is correct, the question is how to determine that based on user input. My personal inclination would be an 'fqdn' field as @choopooly mentions, but that requires a new argument for people to learn. I'm not a fan of using the ordering, though it would be an invisible change; IMHO the behaviour of such a simple state should be obvious by reading it.
lordcirth
on 31 Aug 2017
The extra option can be optional. I really like that or you can try to check for dots in hostnames, but is a bit hacky, you can also search in hostnames for the current hostname + domain in that list, still hacky.
Like I always say, there is no tool to fix human issues :) You can create some complex logic to try to determine which is the best entry to use as FQDN, which will work in simple cases, but nit in complex ones.
A simple option could be, the first entry is hostnames is used as FQDN, all other as alias, no extra option, simple to implement and if is well documented, problem fixed :)
abednarik
on 31 Aug 2017
While having an ordered dict could provide more deterministic results as expected/observed by a human, I am unclear what technical problem this is dealing with.
On a CentOS 7 system running man 5 hosts I have not seen any comment supporting the supposition that
The FQDN has to be in front of the short-hand alias.
A new install of CentOS 7.4 actually uses a non-FQDN first:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
Given that, I do not believe there is a bug here, as I am not sure how the order of the hosts entered should be expected to be preserved (if documented it will be and is not - documentation should be updated, but the order seems to have not functional basis/requirement/expectation). There have been some shifts in the discussion, so please let me know if I am not following something.
This seems to have been the comment which re-opened this issue and I am not sure what is believed to have failed:
common_hostsfile:
host.present:
- ip: 127.0.1.1
- names:
- foo
- foo.bar.example.com
it fails: 127.0.1.1 foo foo.bar.example.com
This appears to have done exactly what it was expected to do - the ip was set in the hosts file with the names provided.
leeclemens
on 23 Apr 2018
@leeclemens here is the man http://man7.org/linux/man-pages/man5/hosts.5.html, check the Description and also the Notes section.
I haven't tested this issue on Centos, but on Debian if the order is not respected the FQDN will not be set, which you can try using hostname -f and get an empty result or the alias set as FQDN.
choopooly
on 23 Apr 2018
@choopooly some confusion around FQDN and canonical_hostname. CentOS operates similarly, with the first value being returned by hostname -f - but it does not necessarily need to be an FQDN (and can be followed by any number of [other] FQDN's).
It seems host.present does not offer any guarantee as to order (the arguments used are alias, fwiw). One quick example: imagine I set the canonical_hostname to 'a'. Then you add the state host.present with 'b' and 'c'. You listed 'b' first, but should 'a' stay first? What if I need 'a' to stay first and host.present was meant to add aliases only? But since you added 'b' first, it pushes 'a' to the end (adding 'b' and 'c' first)? These issues could lead to unexpected behavior for existing and future setups.
To achieve what you are looking for, it may make more sense to use network.system, as that state/module is aware of the importance of the order and should handle it.
IMHO, host.present should not care about the order but merely its presence (as stated in its name). And as to not interfere with network.system, should continue to only append values to existing IP's.
leeclemens
on 23 Apr 2018
Looking at #49889, what would be appropriate for when lines need to be wrapped? Should that "canonical_hostname" be repeated on each line or only the first time?
I typically configure a system's hostname by editing /etc/hostname, so the need for specific order or canonical_hostname needing to be first is a bit foreign to me.
MTecknology
on 5 Oct 2018
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.
![stale[bot] picture](https://avatars.githubusercontent.com/in/1724?v=4&s=40)
stale[bot]
on 9 Jan 2020
Related issues
allyunion
路
3Comments
mooperd
路
3Comments
Arguros
路
3Comments
twangboy
路
3Comments
sfozz
路
3Comments
Most helpful comment
For the record, this is still not working (tested with 2016.3.1).
This is very dependent on fqdn and alias name.
Will give:
10.10.10.123 foo.local.domain fooBut
Will give:
10.10.10.111 alias fqdn.domain.comIt will be great if this can be fixed in future release.