Dietpi: bazarr | Python requirements fail to build on ARM

[Dr0bac]

raspberry pi 3, v6.21.1

Installation order:

1. apt-get install git-core python-pip
2. apt-get install python-dev
3. cd /opt/
4. git clone https://github.com/morpheus65535/bazarr.git
5. cd bazarr and pip install -r requirements.txt

and at this step i got problems, first it said something about setuptools

so i installed it sudo apt-get install -y python-setuptools again error

Then i tried pip --no-cache-dir install greenlet and again same error

then i saw something about missing libxml2 libxslt

i installed these also sudo apt-get install libxml2-dev libxslt-dev again nothing fixed

this is what i get:

root@DietPi:/opt/bazarr# pip install -r requirements.txt
Collecting gevent==1.4.0 (from -r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/ed/27/6c49b70808f569b66ec7fac2e78f076e9b204db9cf5768740cff3d5a07ae/gevent-1.4.0.tar.gz
  Ignoring cffi: markers 'sys_platform == "win32" and platform_python_implementation == "CPython"' don't match your environment
Collecting lxml==4.3.0 (from -r requirements.txt (line 2))
  Using cached https://files.pythonhosted.org/packages/16/4a/b085a04d6dad79aa5c00c65c9b2bbcb2c6c22e5ac341e7968e0ad2c57e2f/lxml-4.3.0.tar.gz
Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" (from gevent==1.4.0->-r requirements.txt (line 1))
  Using cached https://files.pythonhosted.org/packages/f8/e8/b30ae23b45f69aa3f024b46064c0ac8e5fcb4f22ace0dca8d6f9c8bbe5e7/greenlet-0.4.15.tar.gz
Building wheels for collected packages: gevent, lxml, greenlet
  Running setup.py bdist_wheel for gevent ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-heyxQG/gevent/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpM1M69jpip-wheel- --python-tag cp27:
  /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'cffi_modules'
    warnings.warn(msg)
  /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'project_urls'
    warnings.warn(msg)
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for gevent
  Running setup.py clean for gevent
  Running setup.py bdist_wheel for lxml ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-heyxQG/lxml/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpAXlR81pip-wheel- --python-tag cp27:
  Building lxml version 4.3.0.
  Building without Cython.
  Using build configuration of libxslt 1.1.29
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for lxml
  Running setup.py clean for lxml
  Running setup.py bdist_wheel for greenlet ... error
  Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-heyxQG/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpIgM3Lxpip-wheel- --python-tag cp27:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help

  error: invalid command 'bdist_wheel'

  ----------------------------------------
  Failed building wheel for greenlet
  Running setup.py clean for greenlet
Failed to build gevent lxml greenlet
Installing collected packages: greenlet, gevent, lxml
  Running setup.py install for greenlet ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-heyxQG/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-RretMp-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_ext
    building 'greenlet' extension
    creating build
    creating build/temp.linux-armv7l-2.7
    arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-Ub3vap/python2.7-2.7.13=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c greenlet.c -o build/temp.linux-armv7l-2.7/greenlet.o
    unable to execute 'arm-linux-gnueabihf-gcc': No such file or directory
    error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-heyxQG/greenlet/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-RretMp-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-heyxQG/greenlet/

python version is 2.7.13, this version is listed on bazarr wiki

Installed pip install wheel code went further this time, but again Failed building wheel

[MichaIng]

@Dr0bac
Thanks for your report.

I never tested to install bazarr, so need to consult official install instructions. Software request is up: https://github.com/MichaIng/DietPi/issues/2045

---

Following the official install instructions works fine here on x86_64, however python-dev can be skipped there: https://github.com/morpheus65535/bazarr/wiki/Installation-Running-from-Source

I guess it is due to some python module mismatch. Could you try: pip install -U setuptools

And to get rid of the wheel build warning/failure, do: pip install wheel

• This is not required but cleans the output from the mentioned warnings so we can identify the real module build error better.
• Do ask me why pip always tries to build wheels even that the module is not installed 🤔.

---

Ah okay I found the actual issue, I guess:

error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

apt-get install libc6-dev gcc should resolve it.

[Dr0bac]

@MichaIng
installed apt-get install libc6-dev gcc
gevent, greenlet went trough, failed on lxml, cffi

again same error

compilation terminated.
error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

why is this happening, on bazarr wiki there are only few lines of commands that i need to enter, and yet, after 5+ additional packages that i installed after, problems are not solved..
What is the best way to install bazarr on fresh dietpi, what steps i need to follow?

[MichaIng]

@Dr0bac

why is this happening, on bazarr wiki there are only few lines of commands that i need to enter, and yet, after 5+ additional packages that i installed after, problems are not solved..

Ask this the bazarr devs 😉.

What is the best way to install bazarr on fresh dietpi, what steps i need to follow?

There is no "best way" to install bazarr on DietPi, since we did no implement this into DietPi-Software yet. So you need to follow guides you find about how to install on ARM devices with Debian OS for now.
DietPi is not more than a tidied Debian, so any guide that applies to Debian systems, does apply for DietPi as well. If it doesn't, then the guide is not complete, respectively expects some packages to be installed already instead of naming all dependencies explicitly.

In case of the bazarr wiki, it seems to be a bid outdated as well. libc6-dev is a package that is marked as "recommended" for python-dev. On default Debian, recommended packages are installed as well automatically, however on DietPi only "dependencies" are installed automatically. It seems like for bazarr on ARM libc6-dev is indeed a dependency, which is why this should be added to the guide. gcc was just a guess since the error is related to gcc.

Note that the install worked very well on my x86_64 test system, where also according to the guide python-dev is not required. It seems that on ARM (SBCs) certain python modules need to be built/compiled that are already available as compiled binaries on x86 systems. Such build processes always have many dependencies that are sometimes hard to estimate and change often, especially since in this case there are several different python modules included.

However last resort is to install the whole build essentials:

apt install build-essential

If it still does not work, please try to update setuptools to the more current version available via pip:

pip install -U setuptools

• Ah I already mentioned that above, did you do it already? As well:

pip install wheel

[arkhub]

I have similar problem on my raspberry pi 3 b+ after upgrading to 6.22.3 , when trying to install build-essential shows

The following packages have unmet dependencies:
 build-essential : Depends: libc6-dev but it is not going to be installed or
                            libc-dev
                   Depends: g++ (>= 4:6.3) but it is not going to be installed

also trying to install libpython-dev also met with "unmet dependencies"

[MichaIng]

@arkhub
Thanks for your report.

However this is a different issue. Could you please open a new issue to not mix things up here?

[arkhub]

oh ok sorry about that

[morpheus65535]

sudo apt-get install python-libxml2 python-lxml?

Don't worry about pip not being able to install lxml because you've already installed it through apt-get.

[Dr0bac]

@MichaIng i tried like bazarr developer said to run bazarr with these errors and it's working..so i can close this issue now..

[MichaIng]

@morpheus65535
Thanks for jumping in. Ah jep great idea to install the APT package for Python lxml. Generally good to try APT packages if pip fails.

As of the other error messages, might be another required module: python-cffi?

So either apt install python-cffi or pip install cffi?
Looks like somehow pip fails to fetch the correct architecture of this, as above:

Ignoring cffi: markers 'sys_platform == "win32" and platform_python_implementation == "CPython"' don't match your environment

I saw you already updated to Wiki:

(Raspbian and maybe other ARM based distro) Install `python-dev` with `apt-get install python-libxml2 python-lxml`

• Should it be apt-get install libc6-dev python-dev python-libxml2 python-lxml?
• As libc6-dev is only a recommendation of python-dev (no requirement) and allowed to install gevent and greenlet via pip. Not sure about gcc itself.

@Dr0bac
Great it works! Would be another test if apt install python-cffi allows to run pip install -r requirements.txt without any error (besides the wheel building in case).

[morpheus65535]

@MichaIng I haven't installed neither python-cffi, python-dev or libc6-dev on my Raspberry Pi and it's working with Raspbian out of the box. I don't think either of them are required to be manually installed. Do you have different result with DietPi?

[MichaIng]

@morpheus65535
Ah okay, nope assumption was just as of OP reports. Not sure which packages are pulled automatically as recommendations on default Raspbian or which are installed by default there (Raspbian Lite stock). DietPi has vastly reduced amount of APT packages installed by default and recommendations for packages are not pulled with them automatically.

We aim to add bazarr natively as install option to DietPi soon (have required install steps automated based on device/architecture), then I will test it thoroughly and in case report back.
I think so far it's fine, since OP has it working now 😃.

[morpheus65535]

@MichaIng Do not hesitate to contact me if you need further assistance!

[Dr0bac]

@MichaIng One more thing..i want to create bazarr service but something is off..
https://github.com/morpheus65535/bazarr/wiki/systemd-service-file-for-Debian-Ubuntu

i changed user to bazarr group to dietpi and bazarr location is at /opt/bazarr
i guess python folder is not in /usr/bin/python

[MichaIng]

@Dr0bac
You can find the python binary via which python. If you installed python via APT (apt install python(-pip)) then the above should be right. If you installed/updated it via pip, then AFAIK another binary is created at /usr/local/bin/python, which should then be used.

About the bazarr user:

• You did create it, e.g. via useradd -r bazarr -G dietpi -d /opt/bazarr -s /usr/sbin/nologin?
• And you chowned the bazarr install dir accordingly, via: chown -R bazarr:bazarr /opt/bazarr?

[Dr0bac]

@MichaIng Thanks for tips on creating new user..
service still doesn't start with boot

● bazarr.service - bazarr Daemon
   Loaded: loaded (/etc/systemd/system/bazarr.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

when i call it with service bazarr start or systemctl start bazarr.service than i get it to load..

● bazarr.service - bazarr Daemon
   Loaded: loaded (/etc/systemd/system/bazarr.service; disabled; vendor preset: enabled)
   Active: active (running) since Wed 2019-03-27 23:20:00 CET; 51s ago
 Main PID: 1939 (python)
   CGroup: /system.slice/bazarr.service
           ├─1939 /usr/bin/python /opt/bazarr/bazarr.py
           └─1941 /usr/bin/python -u /opt/bazarr/bazarr/main.py

do i need to set some permissions on bazarr.service file?

[MichaIng]

@Dr0bac
You need to enable it: systemctl enable bazarr

Note that DietPi-known services are controlled by DietPi-Services thus started on boot even if disabled. But manually created services need to be enabled to have them started on boot by systemd.