Salt: AttributeError: 'Minion' object has no attribute 'schedule'

[ch-acctg]

Hi, I am trying to upgrade our salt from v2014.7.0 to v2015.8.0 and I am getting some early errors. We are testing this running bootstrap-salt.sh executed from Vagrant masterless and we are getting the following error:

2015-10-05 17:39:01,126 [tornado.application                      ][ERROR   ][11202] Exception in callback <functools.partial object at 0x2225cb0>
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 592, in _run_callback
    ret = callback()
  File "/usr/local/lib/python2.7/dist-packages/tornado/stack_context.py", line 275, in null_wrapper
    return fn(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 598, in <lambda>
    self.add_future(ret, lambda f: f.result())
  File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 215, in result
    raise_exc_info(self._exc_info)
  File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 230, in wrapper
    yielded = next(result)
  File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1462, in handle_event
    self.module_refresh(notify=data.get('notify', False))
  File "/usr/lib/python2.7/dist-packages/salt/minion.py", line 1296, in module_refresh
    self.schedule.functions = self.functions
AttributeError: 'Minion' object has no attribute 'schedule'

[jfindlay]

@ch-acctg, thanks for the report. We have tested upgrading from 2014.7.5 to 2015.8.0 and have not encountered this problem. Are you able to workaround this, for example, by restarting the minion or does it persist?

[ch-acctg]

Well, it is only in bootstrapping. When I run a salt-call -l debug state.highstate inside the vm after, the error doesn't seem to occur but other errors in how our grains are now interpreted might be blocking. I'll try upgrading through some intermediate releases.

[cachedout]

This sounds a lot like stale .pyc files. Can you try cleaning out any stale files and trying this again?

[ch-acctg]

We rebuild the systems from scratch from a fresh debian jessie build VM with package updates before running the salt. I'll try and remove the pyc's before saving the next base.

[gravitypaint]

Hi @cachedout I am having a similar issue (#29835). What do you mean by stale .pyc's?

[jfindlay]

@gravitypaint, the python interpreter can dynamically compile python code, changing a file.py into a file.pyc. If those are left in place after an upgrade/downgrade, they can cause mysterious problems. You can either start with a new, clean system, or you will have to know where and how to remove the .pyc files.

[CGenie]

Isn't this because minion's schedule is created only on _post_master_init so after minion connects to master? This way in a standalone minion the schedule attribute will not be present?

https://github.com/saltstack/salt/blob/develop/salt/minion.py#L868

[jessecollier]

Bump.

Is there any status on this issue and when we might expect this to be resolved? I'm completely at a standstill for masterless setup.

[developerinlondon]

+1 to that. i have the same issue.

[developerinlondon]

The error I am getting is as below:

2016-04-08 13:48:43,449 [tornado.application][ERROR   ][4301] Exception in callback <functools.partial object at 0x1ffab50>
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 591, in _run_callback
    ret = callback()
  File "/usr/lib64/python2.7/site-packages/tornado/stack_context.py", line 274, in null_wrapper
    return fn(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/tornado/ioloop.py", line 597, in <lambda>
    self.add_future(ret, lambda f: f.result())
  File "/usr/lib64/python2.7/site-packages/tornado/concurrent.py", line 214, in result
    raise_exc_info(self._exc_info)
  File "/usr/lib64/python2.7/site-packages/tornado/gen.py", line 230, in wrapper
    yielded = next(result)
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 1554, in handle_event
    self.module_refresh(notify=data.get('notify', False))
  File "/usr/lib/python2.7/site-packages/salt/minion.py", line 1388, in module_refresh
    self.schedule.functions = self.functions
AttributeError: 'Minion' object has no attribute 'schedule'

version:

[root@2usvretzer vagrant]# salt --versions-report
Salt Version:
           Salt: 2015.8.8.2

Dependency Versions:
         Jinja2: 2.7.2
       M2Crypto: 0.21.1
           Mako: Not Installed
         PyYAML: 3.11
          PyZMQ: 14.7.0
         Python: 2.7.5 (default, Nov 20 2015, 02:00:19)
           RAET: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.0.5
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 1.5
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
        libgit2: Not Installed
        libnacl: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.7
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
         pygit2: Not Installed
   python-gnupg: Not Installed
          smmap: Not Installed
        timelib: Not Installed

System Versions:
           dist: centos 7.2.1511 Core
        machine: x86_64
        release: 3.10.0-327.el7.x86_64
         system: CentOS Linux 7.2.1511 Core

I can reproduce this by running vagrant up from my stack here - https://github.com/developerinlondon/vagrant-salt-lxc

[obimod]

+1 (vagrant salt provisioning)

[jaceq]

+1 I get that on 2016.03.1 too

[svrist]

+1 on 2016.03.03

[attiasr]

+1 using vagrant on:
```Salt Version:
Salt: 2016.11.1

Dependency Versions:
cffi: 1.5.2
cherrypy: Not Installed
dateutil: 2.6.0
gitdb: 0.6.4
gitpython: 1.0.1
ioflo: Not Installed
Jinja2: 2.8
libgit2: 0.24.0
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.3
msgpack-pure: Not Installed
msgpack-python: 0.4.6
mysql-python: Not Installed
pycparser: 2.14
pycrypto: 2.6.1
pygit2: 0.24.0
Python: 2.7.12 (default, Nov 19 2016, 06:48:10)
python-gnupg: 0.3.8
PyYAML: 3.11
PyZMQ: 15.2.0
RAET: Not Installed
smmap: 0.9.0
timelib: Not Installed
Tornado: 4.2.1
ZMQ: 4.1.4

System Versions:
dist: Ubuntu 16.04 xenial
machine: x86_64
release: 4.4.0-57-generic
system: Linux
version: Ubuntu 16.04 xenial
```

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.