Salt: /jobs call is broken in 2016.11.4

[e-senthilkumar]

Description of Issue/Question

/jobs call is broken in 2016.11.4.

Exception:
TypeError: list_job() got an unexpected keyword argument '__current_eauth_groups'n'

Setup

2016.11.4 installation.

Steps to Reproduce Issue

1. Install latest 2016.11.4 and hit /jobs/

API log

Traceback (most recent call last):
2017-04-24 05:31:32,848 [salt.loaded.int.netapi.rest_cherrypy.app][DEBUG   ][4301] Error while processing request for: /jobs/20170418110002213864/
  File "/root/.pex/install/salt-2016.11.3.1-py2-none-any.whl.878d23086c75679e3d8a3a02c0bff23b52c6b529/salt-2016.11.3.1-py2-none-any.whl/salt/netapi/rest_cherrypy/app.py", line 719, in hypermedia_handler
    ret = cherrypy.serving.request._hypermedia_inner_handler(*args, **kwargs)
  File "/root/.pex/install/CherryPy-3.2.2-py2-none-any.whl.2a7b0546eca0957845d935d65af710afe659f3b8/CherryPy-3.2.2-py2-none-any.whl/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/root/.pex/install/salt-2016.11.3.1-py2-none-any.whl.878d23086c75679e3d8a3a02c0bff23b52c6b529/salt-2016.11.3.1-py2-none-any.whl/salt/netapi/rest_cherrypy/app.py", line 1334, in GET
    returns = job_ret_info[0].get('Result')
AttributeError: 'str' object has no attribute 'get'

eventlisten.py event

Event fired at Mon Apr 24 08:03:58 2017
*************************
Tag: salt/run/20170424080355186610/ret
Data:
{'_stamp': '2017-04-24T08:03:58.566547',
 'fun': 'runner.jobs.list_job',
 'fun_args': [{'__current_eauth_groups': ['xxxxxx',
                                          'xxxxxx'],
               '__current_eauth_user': 'xxxxxx',
               'jid': '20170418110002213864'}],
 'jid': '20170424080355186610',
 'return': 'Exception occurred in runner jobs.list_job: Traceback (most recent call last):\n  File "/root/.pex/install/salt-2016.11.3.1-py2-none-any.whl.878d23086c75679e3d8a3a02c0bff23b52c6b529/salt-2016.11.3.1-py2-none-any.whl/salt/client/mixins.py", line 395, in _low\n    data[\'return\'] = self.functions[fun](*args, **kwargs)\nTypeError: list_job() got an unexpected keyword argument \'__current_eauth_groups\'\n',
 'success': False,
 'user': 'xxxxxx'}

Versions Report

Salt Version:
           Salt: 2016.11.4

Dependency Versions:
           cffi: 1.5.2
       cherrypy: 3.2.2
       dateutil: 2.6.0
      docker-py: Not Installed
          gitdb: 2.0.0
      gitpython: git
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: 1.0.1
   msgpack-pure: Not Installed
 msgpack-python: 0.4.5
   mysql-python: Not Installed
      pycparser: 2.14
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.11 (default, Dec  9 2015, 09:37:09)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.4.1
            ZMQ: 4.1.6

System Versions:
           dist: redhat 6.6 Santiago
        machine: x86_64
        release: 2.6.32-504.el6.x86_64
         system: Linux
        version: Red Hat Enterprise Linux Workstation 6.6 Santiago

[djsly]

hey @e-senthilkumar , unrelated question, how did you install 2016.11.4 ? I want to update our stack to this version, but I cannot see it being released. The tag seems to have been created, but I cannot see the release. thanks

[gtmanfred]

The release is on pypi, we are still working on packages.

[e-senthilkumar]

@djsly We built from 2016.11.4 tag.

[martinhoefling]

I can confirm that issue with the new packages.

pool/main/s/salt/salt-master_2016.11.3+ds-1_all.deb

was still ok but

pool/main/s/salt/salt-master_2016.11.4+ds-1_all.deb

is broken.

[Ch3LL]

Looks like i'm able to replicate this. git bisect shows commit 27f7fd9ad4e7198c02ffb889d41e69d8db042a97 which is in PR https://github.com/saltstack/salt/pull/39472

[Ch3LL]

Sorry I hit enter too soon. Looks like we will need to get this fixed up.

Here is a docker container for anyone to quickly replicate the issue:

1. docker run -it -v /home/ch3ll/git/salt/:/testing/ ch3ll/issues:40845 (where /home/ch3ll/git/salt is a local cloned git repo of salt)
2. salt-master -d; salt-minion -d; salt-api -d
3. curl -k -c /tmp/cookies.txt -sS localhost:8000/login -H "Accept:application/x-yaml" -d username='saltdev' -d password='saltdev' -d eauth='pam'
4. curl -sSk http://localhost:8000/jobs -H 'Accept: application/x-yaml' -H 'X-Auth-Token: <session token from earlier command'

ping @whiteinge any ideas on this one?

[whiteinge]

I have a very incomplete understanding of the auto-injected __current_eauth_groups (and related) kwargs in the client interfaces -- unsure of when/why are they added, who uses them, when/why they're removed. My PR above doesn't touch them directly so perhaps I cut out a call that munges those. I'll take a closer look.

[stamak]

Have a chance to face this problem as well after update (
Folks, it is not serious - minor release and such kind of problems!