Conan: [bug] system_requirements is ignored when using python_requires

Created on 17 Sep 2020 · 3Comments · Source: conan-io/conan

Hello!

We have a base recipe with a system_requirements method which parses a file and generates a list of dependencies which are then installed with SystemPackageTool. This works quite well when executing the conan commands one by one.
But when using conan create . directly, without previously running conan install, system_requirements is not called and building fails due to missing libraries. This mostly is a problem in a new docker container, which doesn't have the dependencies yet.

From debugging it a bit my best guess is that it fails here. It only checks the attributes and methods of the current conanfile, but doesn't resolve or check the base class.
In manager.py it also is building the code waybefore it can even reach call_system_requirements.

A simple workaround is of course defining system_requirements and just calling super().system_requirements() in the recipe. But it would be nice if it could be resolved correctly.
Or if this is expected behavior, it would be nice to update the documentation, because it's not mentioned there.

Sorry if this is already mentioned somewhere, I haven't found anything which addressed this issue!

Environment Details (include every applicable attribute)

Operating System+version: Ubuntu 20.04
Compiler+version: gcc version 9.3.0 (Ubuntu 9.3.0-10ubuntu2)
Conan version: 1.29.0
Python version: Python 3.8.2

Steps to reproduce (Include if Applicable)

Create base recipe with a system_requirements method with some output and some requirement
Create a simple conan package which inherits from the base recipe
Call conan create . on the conan package

I've also pushed the sample code here.

Logs (Executed commands with output) (Include/Attach if Applicable)

I've added two snippets, one where it doesn't work and one where I applied the workaround and it works.

Relying on base_recipe's system_requirements

Installing (downloading, building) binaries...
system_reqs_test/0.1.0: Running build_id.
system_reqs_test/0.1.0: Configuring sources in /home/developer/.conan/data/system_reqs_test/0.1.0/_/_/source
system_reqs_test/0.1.0: Running source.
system_reqs_test/0.1.0: Building your package in /home/developer/.conan/data/system_reqs_test/0.1.0/_/_/build/e05a9ef4fc5005a542b3c1b2d495c890d9ce4531
INFO  :installer.py   [124]: GENERATORS: Writing generators [2020-09-17 14:46:57,442]
system_reqs_test/0.1.0: Generator cmake created conanbuildinfo.cmake
INFO  :installer.py   [127]: TOOLCHAIN: Writing toolchain [2020-09-17 14:46:57,442]
DEBUG :build.py       [11]: Call conanfile.build() with files in build folder: ['conanbuildinfo.cmake'] [2020-09-17 14:46:57,442]
system_reqs_test/0.1.0: Calling build()

Inheriting system_requirements

Installing (downloading, building) binaries...                                                                                                                                                                    
system_reqs_test/0.1.0: Running system_requirements.
INFO  :runners.py     [85]: Calling command: /usr/bin/apt-get moo > "/tmp/tmpj8g28ywq" [2020-09-17 12:50:51,739]
INFO  :runners.py     [88]: Return code: 0 [2020-09-17 12:50:51,746]
INFO  :runners.py     [96]: Output: in file:                 (__) 
                                             (oo) 
                                       /------\/ 
                                      / |    ||   
                                     *  /\---/\ 
                                        ~~   ~~   
                            ..."Have you mooed today?"...

                            stdout: None
                            stderr:b'' [2020-09-17 12:50:51,746]
dpkg-query: no packages found matching qt5-default
Running: sudo apt-get update
Hit:2 http://de.archive.ubuntu.com/ubuntu focal InRelease   
Hit:3 http://de.archive.ubuntu.com/ubuntu focal-updates InRelease
Hit:4 http://de.archive.ubuntu.com/ubuntu focal-backports InRelease
Hit:7 http://de.archive.ubuntu.com/ubuntu focal-security InRelease
Hit:1 https://apt.llvm.org/focal llvm-toolchain-focal-10 InRelease
Reading package lists... 
Running: sudo apt-get install -y --no-install-recommends qt5-default
Reading package lists...                                                                                                                                                                                          
Building dependency tree...
Reading state information...
The following NEW packages will be installed:                                                            
  qt5-default                                                                                                                                                                                                     
0 upgraded, 1 newly installed, 0 to remove and 152 not upgraded.
Need to get 24.4 kB of archives.                    
After this operation, 170 kB of additional disk space will be used.
Get:1 http://de.archive.ubuntu.com/ubuntu focal/universe amd64 qt5-default amd64 5.12.8+dfsg-0ubuntu1 [24.4 kB]                               
debconf: delaying package configuration, since apt-utils is not installed
Fetched 24.4 kB in 0s (95.8 kB/s)                                                                                                                                                                                 
                                 Selecting previously unselected package qt5-default:amd64.
(Reading database ... 81546 files and directories currently installed.)
Preparing to unpack .../qt5-default_5.12.8+dfsg-0ubuntu1_amd64.deb ...            
Unpacking qt5-default:amd64 (5.12.8+dfsg-0ubuntu1) ...                           
Setting up qt5-default:amd64 (5.12.8+dfsg-0ubuntu1) ...
system_reqs_test/0.1.0: Running build_id.
system_reqs_test/0.1.0: Configuring sources in /home/developer/.conan/data/system_reqs_test/0.1.0/_/_/source
system_reqs_test/0.1.0: Running source.

Thanks a lot!

bug

Source

wadimklincov

All 3 comments

Hi @wadimklincov

You are hitting some old legacy inside the system_requirements() code. I am proposing https://github.com/conan-io/conan/pull/7721, but we need to check and make sure that this will not be breaking anything else.