Salt: [ERROR] Unable to manage file: 'utf8' codec can't decode byte

[valentin2105]

Description of Issue/Question

Hello,

I got an issue with latest Saltstack release on Debian 9, while trying to deploy some binary files with file.managed :

/opt/cni/bin/calico-ipam:
  file.managed:
    - source: salt://k8s/src/cni/{{ calicoCNIversion }}/calico-ipam
    - skip_verify: true
    - group: root
    - mode: 755

That result :

ID: /opt/cni/bin/calico-ipam
Function: file.managed
Result: False
Comment: Unable to manage file: 'utf8' codec can't decode byte 0x99 in position 25: invalid start byte
Started: 09:48:20.232602
Duration: 1472.851 ms
Changes:   

And here is the debug trace :

[DEBUG   ] Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/salt/states/file.py", line 2569, in managed
    **kwargs)
  File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 5158, in manage_file
    real_name, sfn, show_filenames=False)
  File "/usr/lib/python2.7/dist-packages/salt/modules/file.py", line 4921, in get_diff
    for x in fp_.readlines()])
  File "/usr/lib/python2.7/dist-packages/salt/utils/stringutils.py", line 114, in to_unicode
    return _normalize(s.decode('utf-8', errors))
  File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x99 in position 25: invalid start byte

[ERROR   ] Unable to manage file: 'utf8' codec can't decode byte 0x99 in position 25: invalid start byte
[INFO    ] Completed state [/opt/cni/bin/calico] at time 09:48:20.230734 (duration_in_ms=1670.37)

Here is the file :

$ file calico-ipam 
calico-ipam: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, stripped

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

Salt Version:
           Salt: 2018.3.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: 1.3.7
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.13 (default, Nov 24 2017, 17:33:09)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1

System Versions:
           dist: debian 9.4 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-6-amd64
         system: Linux
        version: debian 9.4 

[robkaper]

Specifying encoding and/or encoding_errors: ignore options does not help either, have the same problem where a vendor installs a file with the same cp1252 garbage. Workaround is to manually sed it out prior to managing the file, but that's not perfect.

[garethgreenaway]

@terminalmage You've been doing some UTF8 related cleanup this week. Thoughts on this one?

[terminalmage]

Fixed in https://github.com/saltstack/salt/pull/47061

[teran-mckinney]

I'm getting this with file.recurse, also with the latest release on Debian. Should the fix apply to both? Is there any workaround before a new release goes live in the apt repo?

Edit: show_changes: False does the trick.

[rnd0101]

Was same problem (Ubuntu 16.4) here in file.managed, workaround was effective, thanks.

[terminalmage]

@teran-mckinney file.recurse does a file.managed for each path under the source directory, so the fix should indeed apply to both.

[rallytime]

Closed via #47061

[neonknight]

I had this issue on a file name that definitely only contained ascii chars. wtf?

Edit: show_changes: False does the trick.
This worked for me, thank you for the hint.

• salt 2018.3.2 (Oxygen)
• Python 2.7
• CentOS 7.6
• Installed from: https://repo.saltstack.com/yum/redhat/7/x86_64/latest/