Salt: coercing to Unicode: need string or buffer, NoneType found - for mysql_user

Created on 27 Apr 2019  路  18Comments  路  Source: saltstack/salt

Description of Issue/Question

We are using mysql_user.present saltstate to maintain the db user via salstack
After the last release the we are not able to use the saltstate any more it is showing the error

Setup

 mysql_user.present:
    - name: {{ single_user['name'] }}
    - host: '%'
    - password_hash: '{{ single_user['password'] }}'
    - connection_user: '{{ mysql_user }}'
    - connection_pass: '{{ mysql_pass }}'
    - connection_host: '{{ mysql_host }}'`

Steps to Reproduce Issue

          ID: create admintest
    Function: mysql_user.present
        Name: admintest
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1475, in user_chpass
                  " IDENTIFIED BY '" + password + "';")
              TypeError: coercing to Unicode: need string or buffer, NoneType found
     Started: 03:24:56.335720
    Duration: 58.34 ms

Versions Report

Salt Version:
           Salt: 2019.2.0

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.4.2
      docker-py: 1.10.6
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.3
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Nov 12 2018, 14:36:49)
   python-gnupg: 0.3.8
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: UTF-8
        machine: x86_64
        release: 4.4.0-1079-aws
         system: Linux
        version: Ubuntu 16.04 xenial
Bug Confirmed fixed-pending-your-verification
Source
picture jblinuxclicks

All 18 comments

anyone facing the same issue

jblinuxclicks picture jblinuxclicks on 29 Apr 2019
👍1

@oliyavan Oh yeah, apparently that state is super broken in 2019.0 馃槥 . It was fixed with https://github.com/saltstack/salt/commit/ed62a2f87312ced2bc555cad2d756a2228f1987d which should be in the (coming soon) 2019.2.1 release

waynew picture waynew on 1 May 2019
😕1

I'm facing exactly the same error with Salt version 2019.2.0 (Fluorine)

----------
          ID: repl
    Function: mysql_user.present
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1475, in user_chpass
                  " IDENTIFIED BY '" + password + "';")
              TypeError: coercing to Unicode: need string or buffer, NoneType found
     Started: 09:56:22.456732
    Duration: 12.912 ms
     Changes:
tcristina picture tcristina on 30 Aug 2019
👍1

@tcristina if you can test it, does the aforementioned patch fix the problem for you?

waynew picture waynew on 6 Sep 2019

i'm having the same issue and i would like to test the patch. but i have no idea where the file is supposed to be. doing a locate only pointed me to python2.7 lib, and i remember salt now needing python3.
Is there a doc on how to test patch somewhere ?
(running in a lab on salt 2019.2.0)

poofyteddy picture poofyteddy on 11 Sep 2019

@poofyteddy you can do python -c 'import salt; print(salt.__file__)' and that should tell you where your salt is installed - from your description it's probably something like /usr/lib/python2.7/dist-packages/salt - you're going to want the salt/modules/mysql.py file. You should just be able to copy the relevant parts of the file to your local one (or possibly just copy the entire file). If you're not familiar with Python code, do make sure that your editor isn't set to indent with tabs or you might end out with some issues.

HTH!

waynew picture waynew on 11 Sep 2019

Thank's @waynew, I did this on the salt master last time, and it didn't work. i just swap the file on the minion, run a state.apply of my sls, and the issue is still there. Is there something i'm not doing right ?

The error changed for this

 ID: pkgs.mysql.install.server_root_access
    Function: mysql_user.present
        Name: root
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1933, in call
                  **cdata['kwargs'])
                File "/usr/lib/python2.7/dist-packages/salt/loader.py", line 1939, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python2.7/dist-packages/salt/states/mysql_user.py", line 174, in present
                  **connection_args):
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1556, in user_chpass
                  result = _execute(cur, qry, args)
                File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 609, in _execute
                  return cur.execute(qry, args)
                File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 210, in execute
                  query = query % args
              KeyError: 'password'
     Started: 18:05:18.567825
    Duration: 16.289 ms
     Changes:
poofyteddy picture poofyteddy on 12 Sep 2019

You would need to change this file on your salt minion, yeah. I believe that you can also drop it in as a custom module, which should send it to the minion on a highstate/saltutil.sync_all.

One way that you can test to make sure that the code you expect is being executed is by adding raise Exception('hello!') somewhere in the code. You should then get to see the message pop up somewhere, either in the state return, or the master or minion log (/var/log/salt/master or ../minion, probably)

waynew picture waynew on 12 Sep 2019

i can confirm i used the file from the "fix" and get the error i write about. I'm taking a look at it but i'm not a dev :(
It seam to be trigger by allow_passwordless

user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass frank localhost ''
hostname.domain
    False
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost toto
hostname.domain:
    True
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost fifi
hostname.domain:
    True
user@hostname:~$ sudo salt 'hostname*' mysql.user_chpass root localhost allow_passwordless=True
hostname.domain:
    The minion function caused an exception: Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1660, in _thread_return
        return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/executors/direct_call.py", line 12, in execute
        return func(*args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1556, in user_chpass
        result = _execute(cur, qry, args)
      File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 609, in _execute
        return cur.execute(qry, args)
      File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 210, in execute
        query = query % args
    KeyError: 'password'
ERROR: Minions returned with non-zero exit code
poofyteddy picture poofyteddy on 13 Sep 2019

so yea the fix break because of this

-                 qry = ("ALTER USER '" + user + "'@'" + host + "'"
-                        " IDENTIFIED BY '" + password + "';")
-                 args = {}
+                qry = "ALTER USER %(user)s@%(host)s IDENTIFIED BY %(password)s;"

Before when password wasn't defined i didn't force it, now it have to be there.
EDIT: if you take the actual file from developement it look like it work. i'm getting mysql error, but that might be on me

poofyteddy picture poofyteddy on 13 Sep 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

stale[bot] picture stale[bot] on 7 Jan 2020

Thank you for updating this issue. It is no longer marked as stale.

stale[bot] picture stale[bot] on 8 Jan 2020

@oliyavan / @tcristina / @poofyteddy

Can you verify whether or not this problem exists on Salt 3000?

waynew picture waynew on 26 Mar 2020

I'll need to test it, last time i try the mysql module i had issue with changing the root login from password to unix_socket, so i didn't check further
But i don't think i've seen this one since
I'll try this weekend. I happily do still work during the week

poofyteddy picture poofyteddy on 26 Mar 2020

yea, sadly i'm encountering other issue with the actual mysql module which stop me from testing this specific bug
i can't connect to the db from any salt state, but i can just sudo mysql without issue.

Sorry, but i can't help checking the validity of this issue while the mysql module is still broken on my side :(

ID: pkgs.mysql.install.server_root_access
    Function: mysql_user.present
        Name: root
      Result: False
     Comment: An exception occurred in this state: Traceback (most recent call last):
                File "/usr/lib/python3/dist-packages/salt/state.py", line 1981, in call
                  **cdata['kwargs'])
                File "/usr/lib/python3/dist-packages/salt/loader.py", line 1977, in wrapper
                  return f(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/salt/states/mysql_user.py", line 131, in present
                  **connection_args):
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 1266, in user_exists
                  server_version = salt.utils.data.decode(version(**connection_args))
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 858, in version
                  dbc = _connect(**connection_args)
                File "/usr/lib/python3/dist-packages/salt/modules/mysql.py", line 392, in _connect
                  dbc = MySQLdb.connect(**connargs)
                File "/usr/lib/python3/dist-packages/MySQLdb/__init__.py", line 86, in Connect
                  return Connection(*args, **kwargs)
                File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 172, in __init__
                  for k, v in conv.items():
              AttributeError: 'str' object has no attribute 'items'
     Started: 16:48:56.392379
    Duration: 17.315 ms
     Changes:   

{# Ensure the root user is present and can only connect to  mysql from the unix socket #}
{{ sls }}_root_access:
  mysql_user.present:
    - name: root
    - host: localhost
    - allow_passwordless: True
    - unix_socket: True



md5-149d9e5555f9276d504a1b96d24f12b0



local:
    Salt Version:
               Salt: 3000

    Dependency Versions:
               cffi: Not Installed
           cherrypy: Not Installed
           dateutil: 2.7.3
          docker-py: Not Installed
              gitdb: Not Installed
          gitpython: Not Installed
             Jinja2: 2.10
            libgit2: Not Installed
           M2Crypto: Not Installed
               Mako: Not Installed
       msgpack-pure: Not Installed
     msgpack-python: 0.5.6
       mysql-python: 1.3.10
          pycparser: Not Installed
           pycrypto: 2.6.1
       pycryptodome: Not Installed
             pygit2: Not Installed
             Python: 3.7.3 (default, Dec 20 2019, 18:57:59)
       python-gnupg: Not Installed
             PyYAML: 3.13
              PyZMQ: 17.1.2
              smmap: Not Installed
            timelib: Not Installed
            Tornado: 4.5.3
                ZMQ: 4.3.1

    System Versions:
               dist: debian 10.3 
             locale: UTF-8
            machine: x86_64
            release: 4.19.0-8-amd64
             system: Linux
            version: debian 10.3

56124 should have fixed it, but i can't tell if salt 3000.1 is out yet or not ... all i know is that i'm up to date with the latest salt-server and minion on salt-stack official debian repository

poofyteddy picture poofyteddy on 29 Mar 2020

the packages will be available soon on repo.saltstack.com

sagetherage picture sagetherage on 2 Apr 2020

i'll try again this weekend with 3000.1

poofyteddy picture poofyteddy on 2 Apr 2020

fixes were released in Neon, please open a new ticket if issues persist

sagetherage picture sagetherage on 29 May 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

icycle77 picture icycle77  路  3Comments
Inveracity picture Inveracity  路  3Comments
commutecat picture commutecat  路  3Comments
sagetherage picture sagetherage  路  3Comments
layer3switch picture layer3switch  路  3Comments