Salt: file.managed Unable to manage file: 'hash_type' (2016.3.4)

[jinm]

Description of Issue/Question

I saw an issue with 2016.3.4 on CentOS 7.x.

The sls shows:

    /usr/lib/systemd/system/crond.service:
        ----------
        __env__:
            base
        __sls__:
            crond_service
        file:
            - managed
            |_
              ----------
              group:
                  root
              mode:
                  644
              skip_verify:
                  True
              source:
                  http://somehost/managed/crond.service
              user:
                  root
            |_
              ----------
              order:
                  10009

Minion log shows:

2016-12-22 16:18:16,265 [salt.utils.http  ][DEBUG   ][25065] Requesting URL http://somehost/managed/crond.service using GET method
2016-12-22 16:18:16,272 [salt.loaded.int.states.file][DEBUG   ][25065] Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/states/file.py", line 1794, in managed
    skip_verify)
  File "/usr/lib/python2.7/site-packages/salt/modules/file.py", line 4276, in manage_file
    name_sum = get_hash(real_name, source_sum['hash_type'])
KeyError: 'hash_type'

2016-12-22 16:18:16,272 [salt.state       ][ERROR   ][25065] Unable to manage file: 'hash_type'
2016-12-22 16:18:16,272 [salt.state       ][INFO    ][25065] Completed state [/usr/lib/systemd/system/crond.service] at time 16:18:16.271923 duration_in_ms=11.354

Setup

(Please provide relevant configs and/or SLS files (Be sure to remove sensitive info).)

Steps to Reproduce Issue

(Include debug logs if possible and relevant.)
Out of near 20 files deployed via salt, this issue always happens on three files every time. 2016.11.1 has this issue. 2016.3.1 does not have this issue.

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)
````

salt-call --versions-report

Salt Version:
Salt: 2016.3.4

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.8
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.4.8
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pygit2: Not Installed
Python: 2.7.5 (default, Jun 17 2014, 18:11:42)
python-gnupg: Not Installed
PyYAML: 3.11
PyZMQ: 14.3.1
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 3.2.5

System Versions:
dist: centos 7.1.1503 Core
machine: x86_64
release: 3.10.0-229.el7.x86_64
system: Linux
version: CentOS Linux 7.1.1503 Core
```

[cachedout]

Please give this fix a try: https://github.com/saltstack/salt/pull/38488

[githubcdr]

Sorry for re-opening this but ran into the same issue here on CentOS 7 with Salt 2016.11.3.

The following states gives an error;

{% set prefix = 'vim' %}
{{ prefix }}-check:
  pkg.installed:
    - name: vim-enhanced
  file.directory:
    - name: /usr/share/vim/vimfiles/colors/
{{ prefix }}-badwolf:
  file.managed:
    - name: /usr/share/vim/vimfiles/colors/badwolf.vim
    - source: https://raw.githubusercontent.com/sjl/badwolf/master/colors/badwolf.vim
    - source_hash: 2761baf1c74caceaad7ab5d67a032d2c
    - skip_verify: True
    - require:
      - pkg: {{ prefix }}-check

It seems skip_verify: True is the issue here, setting to False gives no error.

debug output

[DEBUG   ] Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/states/file.py", line 1836, in managed
    **kwargs
  File "/usr/lib/python2.7/site-packages/salt/modules/file.py", line 3748, in get_managed
    if cached_sum != source_sum['hsum']:
KeyError: 'hsum'

[ERROR   ] Unable to manage file: 'hsum'

[githubcdr]

This issue only appears when the minion has the file in cache!

[konstest]

+1 I have the same problem

[jinm]

Try this patch on /usr/lib/python2.7/site-packages/salt/modules/file.py?

--- file.py.old 2017-03-29 10:43:47.152598976 -0400
+++ file.py     2017-03-29 10:41:37.831364940 -0400
@@ -3745,7 +3745,7 @@
         if cached_dest and (source_hash or skip_verify):
             htype = source_sum.get('hash_type', 'sha256')
             cached_sum = get_hash(cached_dest, form=htype)
-            if cached_sum != source_sum['hsum']:
+            if cached_sum != source_sum.get('hsum', __opts__.get('hash_type', 'md5')):
                 cache_refetch = True
             elif skip_verify:
                 # prev: if skip_verify or cached_sum == source_sum['hsum']:

[vquiering]

Ran into this issue as well. The patch @jinm suggested worked here. I hope this will get merged soon.

[jdonofrio728]

I am also running into this issue.

[cachedout]

@jinm Would you like to submit that patch as a pull-request?

[jinm]

@cachedout , please review: https://github.com/saltstack/salt/pull/40546. Thank you. Jin.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.