Luigi: Install issues caused by `python-daemon`

[EdMan1022]

Just a heads up, the python-daemon package was updated from 2.1.2 to 2.2.0, and for some reason, in some instances, setuptools doesn't properly handle a docutils requirement. Since the python-daemon requirement for luigi is only python-daemon<3.0, I'm having builds that previously passed fail. If anyone else has this issue, you can easily fix it by either manually specifying docutils in the requirements, or by explicitly setting python-daemon==2.1.2

python-daemon repository
Issue for problem on python-daemon repo

[swardlincoln]

Having the same issue. Thanks for the work around.

[dlstadther]

Thanks for raising @EdMan1022 !

Feel free to submit a PR to resolve this - either link a specific 2.1.2 version or add docutils. I'll leave the details of the solution up to the submitter :)

[EdMan1022]

I'm not sure if there should be a change to the luigi requirements for this. I couldn't replicate the issue in my local environment, although admittedly I didn't try too hard to do so. So it's either some strange OS specific thing causing it, or it could have something to do with the version of setuptools or pip being used. Either way, I'd be worried about locking the package to a minor version, since that makes the install process less flexible, and there might be useful changes introduced later in the python-daemon package. I'd also hesitate to add docutils to the requirements, since it's really only needed during the install process for python-daemon and could bloat production images. If a lot of other users seem to hit this issue, and the person responsible for python-daemon doesn't figure out some sort of hotfix on their side, then I'd open a PR.

[chaspy]

I have the same issue. thanks.

[dlstadther]

@EdMan1022 Since you believe this might be some other cause, could you denote your OS/setuptools/pip versions where having issues?

[pybokeh]

I have similar problem, but getting different error message related to 'pwd' module not found. I'm on Windows 7 machine and I think luigi is trying to start a daemon in a manner that is only valid for UNIX/POSIX machines, but not valid for Windows.

Collecting python-daemon<3.0 (from luigi==2.7.5)
  Using cached https://files.pythonhosted.org/packages/99/2a/75fe6aa7086e838570f29899f674e7896a42be26d9fff33f90d990e599d2/python-daemon-2.2.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\ma17151\AppData\Local\Temp\pip-install-2ae_dkbk\python-daemon\setup.py", line 34, in <module>
        level=0, fromlist=main_module_fromlist)
      File "C:\Users\ma17151\AppData\Local\Temp\pip-install-2ae_dkbk\python-daemon\daemon\__init__.py", line 38, in <module>
        from .daemon import DaemonContext
      File "C:\Users\ma17151\AppData\Local\Temp\pip-install-2ae_dkbk\python-daemon\daemon\daemon.py", line 19, in <module>
        import pwd
    ModuleNotFoundError: No module named 'pwd'

Downgrading to or installing python-daemon==2.1.2 fixes the problem.

[pybokeh]

I hope with Luigi's dependence on python-daemon, that doesn't mean Luigi may no longer work on Windows. Please keep Luigi cross platform! I have been using Luigi because Apache Airflow does not work on Windows.

[EdMan1022]

@dlstadther Sure thing:

• OS: RHEL 7
• pip: 7.1.2
• setuptools 18.2

[swardlincoln]

Is there any update on this issue? I'm still running into this problem and using the specific python-daemon lib (2.1.2) in my requirements.txt to get around the problem.

[honnix]

Will do a release very soon to push out #2536. If possible please help verify and close the issue. Thanks! @EdMan1022

[honnix]

Released 2.7.9.