Salt: State.event can't JSON serialize datetime in data

[jtraub91]

Description of Issue

When trying to view the salt minion local event bus with salt-call state.event, data returned after issuing salt-call event.fire '{"func": "list"}' 'manage_schedule' causes the state.event command to exit with the following error:

# salt-call state.event
manage_schedule {"func": "list", "where": null, "_stamp": "2020-03-16T17:10:47.596871"}

Passed invalid arguments: Object of type datetime is not JSON serializable.

The posted event displays, but I suspect the following data isn't properly serialized by salt.utils.json.dumps here: https://github.com/saltstack/salt/blob/master/salt/modules/state.py#L2420

Setup

Normal salt installation

Steps to Reproduce Issue

Issue the following commands on a salt minion

# salt-call --local state.event pretty=True

# salt-call --local event.fire '{"func": "list"}' 'manage_schedule'



md5-42900399719d7475c8d4ae56d7575d99



# salt --versions-report
Salt Version:
           Salt: 3000

Dependency Versions:
           cffi: 1.12.3
       cherrypy: unknown
       dateutil: 2.7.2
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.9.6
        libgit2: 0.28.2
       M2Crypto: 0.33.0
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: 0.28.2
         Python: 3.7.5 (default, Oct 17 2019, 12:25:15)
   python-gnupg: 0.4.2
         PyYAML: 5.1.1
          PyZMQ: 18.0.1
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-862.3.2.el7.x86_64
         system: Linux
        version: Not Installed

[frogunder]

@jtraub91 Thank you for reporting this issue.

I am seeing the same thing.

Thanks.

[omegarus]

Same problem with 2019.2.4 version but with salt-run state.event

[omegarus]

This issue is not allowing us to use engines on 2019.2.4 salt, also it's very hard to debug any master related issues without salt-run state.event.
Everything worked fine on 2018.3.4.
Any suggestions how we can workaround this issue ?

[adzafar]

Has there been any progress on this issue?

[garethgreenaway]

Apologies for the delay on this one, while working on another issue and finding the cause, I believe the fix will also resolve this one, the root of the issue is that the data element contains bytes which is incompatible with salt.utils.json.dumps, the fix I've been looking at is this:

diff --git a/salt/modules/state.py b/salt/modules/state.py
index 09e03c9981..61ba8c6ea2 100644
--- a/salt/modules/state.py
+++ b/salt/modules/state.py
@@ -2477,7 +2477,7 @@ def event(
                         "{}\t{}".format(  # future lint: blacklisted-function
                             salt.utils.stringutils.to_str(ret["tag"]),
                             salt.utils.json.dumps(
-                                ret["data"],
+                                salt.utils.data.decode(ret["data"]),
                                 sort_keys=pretty,
                                 indent=None if not pretty else 4,
                             ),

Using the salt.utils.data.decode we can ensure that data is in the right format regardless of if that element is a string, a list, or a dictionary. If you're able to, can you test this fix out and see if it resolves the issue for you?

[garethgreenaway]

@jtraub91 I am unable to reproduce this one with the latest version, can you update and test again?

[jtraub91]

@garethgreenaway It is still happening for me on 3002.2 but I haven't applied the fix you suggested in https://github.com/saltstack/salt/issues/56389#issuecomment-756287821

[garethgreenaway]

@jtraub91 and the steps you're using to reproduce is having the state.event runner running and then fire an event like this:
salt-call --local event.fire '{"func": "list"}' 'manage_schedule'?

[jtraub91]

@garethgreenaway Actually an important distinction is I have salt-call state.event running, _not_ salt-run state.event

[garethgreenaway]

@jtraub91 Thanks. That is different. I'll test with that.

[garethgreenaway]

@jtraub91 Updated diff:

diff --git a/salt/utils/data.py b/salt/utils/data.py
index 4cda10dba1..19e3476c1f 100644
--- a/salt/utils/data.py
+++ b/salt/utils/data.py
@@ -5,6 +5,7 @@ and data structures.


 import copy
+import datetime
 import fnmatch
 import functools
 import logging
@@ -286,6 +287,8 @@ def decode(
                 to_str,
             )
         )
+    if isinstance(data, datetime.datetime):
+        return str(data)
     try:
         data = _decode_func(data, encoding, errors, normalize)
     except TypeError:
@@ -316,11 +319,6 @@ def decode_dict(
     # Clean data object before decoding to avoid circular references
     data = _remove_circular_refs(data)

-    _decode_func = (
-        salt.utils.stringutils.to_unicode
-        if not to_str
-        else salt.utils.stringutils.to_str
-    )
     # Make sure we preserve OrderedDicts
     ret = data.__class__() if preserve_dict_class else {}
     for key, value in data.items():
@@ -343,7 +341,14 @@ def decode_dict(
             )
         else:
             try:
-                key = _decode_func(key, encoding, errors, normalize)
+                key = decode(key,
+                             encoding,
+                             errors,
+                             normalize,
+                             preserve_dict_class,
+                             preserve_tuples,
+                             to_str)
+
             except TypeError:
                 # to_unicode raises a TypeError when input is not a
                 # string/bytestring/bytearray. This is expected and simply
@@ -400,8 +405,14 @@ def decode_dict(
             )
         else:
             try:
-                value = _decode_func(value, encoding, errors, normalize)
-            except TypeError:
+                value = decode(value,
+                               encoding,
+                               errors,
+                               normalize,
+                               preserve_dict_class,
+                               preserve_tuples,
+                               to_str)
+            except TypeError as e:
                 # to_unicode raises a TypeError when input is not a
                 # string/bytestring/bytearray. This is expected and simply
                 # means we are going to leave the value as-is.
@@ -431,11 +442,6 @@ def decode_list(
     # Clean data object before decoding to avoid circular references
     data = _remove_circular_refs(data)

-    _decode_func = (
-        salt.utils.stringutils.to_unicode
-        if not to_str
-        else salt.utils.stringutils.to_str
-    )
     ret = []
     for item in data:
         if isinstance(item, list):
@@ -479,7 +485,14 @@ def decode_list(
             )
         else:
             try:
-                item = _decode_func(item, encoding, errors, normalize)
+                item = decode(item,
+                              encoding,
+                              errors,
+                              normalize,
+                              preserve_dict_class,
+                              preserve_tuples,
+                              to_str)
+
             except TypeError:
                 # to_unicode raises a TypeError when input is not a
                 # string/bytestring/bytearray. This is expected and simply