Fpm: FPM virtualenv packaging not working with Python 3.6

Created on 2 Sep 2017 · 12Comments · Source: jordansissel/fpm

Hi guys, hope you're doing great! 😄

The FPM virtualenv extension first performs an update and install of both pip and distribute. However, installation of distribute fails on Python 3.6 (due to a problem in distribute). To my knowledge, distribute is not actually all that useful anymore (as it is also listed as a legacy package on PyPI).

Here's sample output on CentOS 7 using Python 3.6 from the IUS YUM repository and the latest FPM release:

fots > ~ $ fpm --verbose --input-type virtualenv --output-type rpm Flask
Setting workdir {:workdir=>"/tmp", :level=>:info}
Requirement already up-to-date: pip in /tmp/package-virtualenv-build-dead82952af5ad484714f9f4cfb239635b116711834e708f5cedff8eb4d7/usr/share/python/Flask/lib/python3.6/site-packages {:level=>:info}
Collecting distribute {:level=>:info}
  Using cached distribute-0.7.3.zip {:level=>:info}
    Complete output from command python setup.py egg_info: {:level=>:info}
    Traceback (most recent call last): {:level=>:info}
      File "<string>", line 1, in <module> {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/setuptools/__init__.py", line 2, in <module> {:level=>:info}
        from setuptools.extension import Extension, Library {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/setuptools/extension.py", line 5, in <module> {:level=>:info}
        from setuptools.dist import _get_unpatched {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/setuptools/dist.py", line 7, in <module> {:level=>:info}
        from setuptools.command.install import install {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/setuptools/command/__init__.py", line 8, in <module> {:level=>:info}
        from setuptools.command import install_scripts {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/setuptools/command/install_scripts.py", line 3, in <module> {:level=>:info}
        from pkg_resources import Distribution, PathMetadata, ensure_directory {:level=>:info}
      File "/tmp/pip-build-fx3jcul7/distribute/pkg_resources.py", line 1518, in <module> {:level=>:info}
        register_loader_type(importlib_bootstrap.SourceFileLoader, DefaultProvider) {:level=>:info}
    AttributeError: module 'importlib._bootstrap' has no attribute 'SourceFileLoader' {:level=>:info}
     {:level=>:info}
    ---------------------------------------- {:level=>:info}
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-fx3jcul7/distribute/ {:level=>:info}
Process failed: /tmp/package-virtualenv-build-dead82952af5ad484714f9f4cfb239635b116711834e708f5cedff8eb4d7/usr/share/python/Flask/bin/python failed (exit code 1). Full command was:["/tmp/package-virtualenv-build-dead82952af5ad484714f9f4cfb239635b116711834e708f5cedff8eb4d7/usr/share/python/Flask/bin/python", "/tmp/package-virtualenv-build-dead82952af5ad484714f9f4cfb239635b116711834e708f5cedff8eb4d7/usr/share/python/Flask/bin/pip", "install", "-U", "-i", "https://pypi.python.org/simple", "pip", "distribute"] {:level=>:error}
fots > ~ $

Any help would be greatly appreciated! 😄
Fotis

Source

fgimian

👍6

Most helpful comment

Any movement on this?

JackLeo on 8 Jan 2018

👍6

All 12 comments

was this looked at?

JackLeo on 17 Oct 2017

Hi guys, is there a work around for this? Getting exactly the same issue. Is there a way of stopping distribute being installed?

wileycoyote on 21 Nov 2017

I'm currently patching it after installation. Not ideal, but it's all I can do for now.

Save the following into virtualenv.rb.patch:

--- virtualenv.rb       2017-09-02 06:30:06.000000000 +0000
+++ virtualenv.patched.rb       2017-09-02 06:32:18.313428934 +0000
@@ -111,8 +111,7 @@
     # Why is this hack here? It looks important, so I'll keep it in.
     safesystem(python_exe, pip_exe, "install", "-U", "-i",
                attributes[:virtualenv_pypi],
-               "pip", "distribute")
-    safesystem(python_exe, pip_exe, "uninstall", "-y", "distribute")
+               "pip")

     extra_index_url_args = []
     if attributes[:virtualenv_pypi_extra_index_urls]

Then apply as follows:

patch /usr/local/share/gems/gems/fpm-1.9.3/lib/fpm/package/virtualenv.rb < virtualenv.rb.patch

I put this in my Dockerfile which builds an image that uses FPM to build me some Python-based RPMs 😄

Hope this helps a little
Fotis

fgimian on 1 Dec 2017

👍4 🎉1

Interesting. Can anyone else test @fgimian's patch? I'm open to including this patch in fpm. I can test it sometime this or next week maybe.

jordansissel on 1 Dec 2017

Hey @jordansissel, I should clarify that the patch is purely intended to work with Python 3.6. I'm not sure of the implications with Python 2.7 mate.

Perhaps trying to understand why distribute was needed in the first instance could be handy. I personally never install distribute in my own virtualenvs as it largely appears to be a legacy package: https://pypi.python.org/pypi/distribute/0.7.3

Perhaps this is required for much older versions of Python.

Kindest Regards
Fotis

fgimian on 1 Dec 2017

@fgimian that is helpful to know, thank you!

jordansissel on 1 Dec 2017

Any movement on this?

JackLeo on 8 Jan 2018

👍6

The distribute package is (for a long time now) deprecated; it should be okay to replace distribute with setuptools, and it will work on all current Python versions.

samh on 23 Jul 2018

I can confirm the patch logic (remove "distribute", and remove the "uninstall", "-y", "distribute" line) works with Python 3.6.6 and fpm 1.10.2.

I can also confirm that distribute is a legacy package that has been superseded by setuptools across all Python versions, and should be removed not replaced. If fpm users are building against very old versions of Python, and require distribute, they can add it to their requirements.txt file. See https://pypi.org/project/distribute/ for a summary.

That said, it sounds like a test is needed. Does this sound right?