Supervisor: Custom Components Dependencies Error

[mouth4war]

Hi!

It looks like HASSIO has trouble with installing dependencies for custom components. This is a pretty major issue for anyone using custom components.

The error seems to come from this command: Command “/usr/bin/python3 -u -c “import setuptools

I have noted two examples of this issue below with the most details.

1. https://community.home-assistant.io/t/custom-component-dependency-package-install-failure-bluepy-setup-py-error-code-1/23423
2. https://community.home-assistant.io/t/hass-io-custom-component-dependencies/20394

These are not related to custom components but seem to have the same error message (bluetooth tracker issues).

1. https://community.home-assistant.io/t/after-adding-platform-bluetooth-tracker-error-message/18391
2. https://community.home-assistant.io/t/bluetooth-miflora/22769

Thanks for looking at this. HASSIO looks really promising especially the addons.

[pvizeli]

Hi @mouth4war

We support only official components. I refactory the hole image for next release and we support a lot more bluetooth devices. If a custom component depends on a requerement of a official component it will work, it work also if the module is a pure python modul.

Most of that bluetooth stuff need a complex build system local, that build system use 200-300mb only for compile a 60kb library. We ship the same image to a lot of users and most of it don't need a build system.

If this component will a part of official home-assistant component or use librarys that is used by other core components we can make that it will work.

A very big problem is, that the bluetooth module of raspberry pi3 work over the CPU and slow done the hole system. That make that on search the cpu go to slow for home-assistant. So our official statement is, don't use bluetooth on same device as home-assistant run.

It is also possible to run hass.io with a custom home-assistant image (look into API.md).

[mouth4war]

Hi @pvizeli, I checked and it only uses bluepy 1.1.1 and webcolors. Would it be possible to include these in the next image please?

[pvizeli]

bluepy is included in next image and webcolors is pure python. So I see no problem for next image release (with hass 0.51).

[tomh05]

@pvizeli where can I find which packages are included? Can't find a reference to bluepy in this repo.
I want to find which version is included so I can test against that.
Thanks!

[Betree]

@tomh05 I think dependencies are defined in https://github.com/home-assistant/home-assistant/ which is linked to this project using https://github.com/home-assistant/hassio-build (correct me if I'm wrong).

Thus the requirements file currently states: bluepy==1.1.1

[tomh05]

Thanks @Betree! Bluepy seems to be commented out there though.

Interestingly bluepy is included for decora, the docs of which also seem to mention issues with getting it running with bluepy.

[Betree]

@tomh05 You're right I didn't noticed that, and it's seems to be the same for 0.51 and 0.54 tags.

[gollo]

homeassistant determines dependencies for components by finding references in the component's REQUIREMENTS constant. Eg you can see the decora component has REQUIREMENTS = ['decora==0.6', 'bluepy==1.1.1']. homeassistant will try to install this on startup if it attempts to load the decora component because decora is configured in configuration.yaml. This happens whether or not homeassistant is running in a container. Ignore requirements_*.txt for now (see below).

The short answer is that if your component needs bluepy and webcolors and they are pip installable it should be enough to put this in your component REQUIREMENTS = ['webcolors==x.x.x', 'bluepy==1.1.1']. This is sufficient for homeassistant to know to try to install this dependency at startup, container or not. There should be no need to modify any container images.

However, if the installation of the dependency is failing, that means you have successfully told homeassistant of your dependency requirement, so that's not a problem, the actual installation is. It needs to be determined if the installation failure is because of lower lever (not just pure python, but perhaps OS package) dependencies. If that is the case, then image modification may be required.

You can see what Alpine packages are installed in base homeassistant here

and then arch specific additions (eg RPi) here

I hope this helps clarify which type of dependencies need different treatment between _standalone homeassistant_ vs _container (hassio) homeassistant_ dependency/requirements and how/where they are specified.

requirements_*.txt
As an aside, requirements_all.txt is really used for running tests and setting up a dev environment, so should not impact homeassistant runtime (container or otherwise). Some things are commented out in requirements_all.txt because they will only work the target platforms at runtime, not on common dev/test envs (a pip install for an armhf component will only work at runtime on a pi and not when setting up a development env on x64 for example).

[Betree]

@gollo Thanks for these great explanations!

Seems like one of bluepy's dependencies, glib2-dev, is not available on alpine (only glib) but I'll do more testing to try to get it running.

If missing packages are identified, will there be a way for us to specify these dependencies at component's level ?

[gollo]

If there are new os level packages required (either apk add or compile from scratch), the relevant Dockerfiles (referred to above) would need to include them. Once you find out exactly what is needed update this issue and we can help get it in.

[pvizeli]

https://github.com/home-assistant/hassio-build/blob/master/homeassistant/generic/Dockerfile#L12

We install bluepy on base image in the version they will suggest by component. Webcolor is pure python and will be installed into dept folder if you have the correct REQUREMENTS

[tomh05]

bluepy-1.1.1 is indeed installed now, but frustratingly magicblue skips that version from bluepy-1.1.0 or bluepy-1.1.2. I've forked magicblue to tomh05/magicblue, changing it to use 1.1.1, but HASSIO doesn't seem to like using a URL or git repository in the REQUIREMENTS array.

I then tried seeing if I could manually install it in the docker container:

ssh [email protected] -p 22222
docker exec -ti homeassistant bash
pip3 install https://github.com/tomh05/magicblue/archive/master.zip

click to see command output

Collecting https://github.com/tomh05/magicblue/archive/master.zip
  Downloading https://github.com/tomh05/magicblue/archive/master.zip
Requirement already satisfied: bluepy==1.1.1 in /usr/lib/python3.6/site-packages (from magicblue==0.4.3)
Collecting webcolors (from magicblue==0.4.3)
  Using cached webcolors-1.7.tar.gz
Installing collected packages: webcolors, magicblue
  Running setup.py install for webcolors ... done
  Running setup.py install for magicblue ... done
Successfully installed magicblue-0.4.3 webcolors-1.7

This installs okay and notices that bluepy=1.1.1 exists, but sadly HASSIO's watchdog freaks out when the container restarts, so I can't see if it works.

@Betree could you create a package of magicblue that uses bluepy=1.1.1 so we can point pip to that?

[Betree]

@tomh05 I just pushed a special 0.5.0.dev1 version on pypi using bluepy=1.1.1. Tell me if it helps

[pvizeli]

@tomh05 only http URL should work. You need strict follow the Home-Assistant rules for this.

You can also use a own homeassistant docker image, but modify the exists is not a good idea. But that is Beta.

[tomh05]

@Betree Thanks for doing that! Yes, using 0.5.0.dev1 works a treat 👍