Fpm: python package rpm

[raymatos]

When trying to install an RPM in centos, dependencies are looked up in the yum repos vs pypi. How can we force pip/pypi?

[anantshri]

either we have a way of looking dependencies on pypi or we have a way by which dependencies are also auto generated at the package generation time. any one of the following would defeintely help.

[monokal]

I'm hitting this problem too. I'm using fpm with the -t rpm -s python arguments and pointing it at my setup.py which contains the following:

...
    install_requires=[
        'daemonize',
        'influxdb',
        'requests',
        'edgegrid-python'
    ]
)

But on trying to install the RPM via Yum I get a bunch of errors:

...
Error: Package: sbg-akagraph-1.0.0-1.noarch (/sbg-akagraph)
           Requires: python-edgegrid-python
Error: Package: sbg-akagraph-1.0.0-1.noarch (/sbg-akagraph)
           Requires: python-influxdb
Error: Package: sbg-akagraph-1.0.0-1.noarch (/sbg-akagraph)
           Requires: python-daemonize
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest

It seems that dependancies are been looked for in the RPM repos whereas it'd be much more reliable (at least in my case) to tell FPM to get them via easy_install or pip as running the setup.py would do. Many modules are only available via PyPI.

How about a --[no-]python-pypi-dependencies argument?

[hatt]

To my knowledge no packages work this way. Since yum does the dependency resolution of an RPM (or dpkg for a deb), it doesn't know about other systems. When we package up a pip module, it gets installed into the directories pip would do itself but rpm tracks the files and their permissions etc. If pip looks at what it has, the modules show up properly.

To get all you dependencies into the same management system you need recursive building, so for each dependency building it as an rpm. For upstream modules which EPEL or other repos already have, there is no need to do that. FPM doesn't get the dependencies for you at all, it just sets them in the package metadata. Since the install itself is done with yum, it falls to yum to handle dependencies for you.

[monokal]

I guess a workaround would be to put the pip/easy_install dependancy logic in a --pre-install script, but it's obviously not ideal when packaging for multiple platforms.

How about a feature where FPM will generate a pre-install script on-the-fly for the relevant platform based on the -t <PACKAGE_FORMAT> arg if we pass something like --python-pypi-dependencies.

Anyway, here's my current Bash-based --pre-install workaround for those interested.

#!/usr/bin/env bash

modules=( \
    daemonize \
    influxdb \
    requests \
    edgegrid-python \
)

package_managers=( \
    pip \
    easy_install \
)

echo "Installing PyPI module dependencies..."

for p in "${package_managers[@]}"; do
    echo "Checking system for \"${p}\"..."

    if [ -x "$(command -v ${p})" ]; then
        echo "Found \"${p}\"." >&2

        p_args=''
        if [ "${p}" == 'pip' ]; then
            p_args='install'
        fi

        for m in "${modules[@]}"; do
            echo "Installing ${m} module..."
            ${p} ${p_args} ${m}

            if [ $? -eq 0 ]; then
                echo 'OK.'
            else
                echo "Failed to install \"${m}\" module via ${p}."
                exit 1
            fi
        done
        echo "Completed."
        exit 0
    else
        echo "\"${p}\" is not installed."
    fi
done

echo "Failed to find a PyPI package manager."
exit 1

[Alan-R]

I suppose that one could imagine declaring a dependency on a pip package, then having fpm generate the native (RPM, DEB) package for it automagically.

[westurner]

If vendoring the dependencies (committing to upgrading them all yourself) is an option, you can create a PEP441 zipapp (.pyz, .pyzw)

https://www.python.org/dev/peps/pep-0441/

https://www.python.org/dev/peps/pep-0441/#minimal-tooling-the-zipapp-module

PEXes are zipapps:
https://github.com/pantsbuild/pex/blob/master/docs/whatispex.rst

https://www.pantsbuild.org/python_readme.html

https://docs.bazel.build/versions/master/be/python.html

https://buckbuild.com/rule/python_binary.html

[westurner]

1059

[maximumG]

I'm having the same issue here as my dependencies are not in official CentOS repository.

Currently I used the same hack as @monokal. Would still be a nice feature being able to have a pre-install script to generate the requirement pip command.

[samh]

Another option is to package a virtualenv. Supposedly fpm can do this, but I've never gotten it to work. I use something based on this article: https://hynek.me/articles/python-app-deployment-with-native-packages/

The advantage of this is that you don't rely on pip when installing (internet could be down, pypi could be down, etc.).

[morph027]

https://github.com/jordansissel/fpm/issues/264#issuecomment-437856623

[mathieu-aubin]

Wouldn't it be a wise thing to have a rpm pre-install script (which i know fpm can do) run which in turns notices the user about a few more dependencies, then asks a quick yes or no so not to be intrusive and give the user a choice, then upon yes, have that script install (either via easy_install or pip) the required dependencies?

The dependencies of your rpm would only need to be python and python-pip

I have not tested this but to me, logically, it seems like this would flow and work just perfect to achieve what you want.

Packaging a virtualenv is one way to circumvent the issue

* after writing this, i realized that @monokal suggestion is pretty much what i had in mind.