Platformio-core: Build fails because of missing dependency of part of library not included in project

[simonvanderveldt]

Configuration

Operating system: Linux
PlatformIO Version: 3.1.0

Description of problem

When using a part of a library without other dependencies whilst there are parts of the library that do have dependencies building will fail if those dependencies aren't installed.

This happens for example with the DHT sensor library when you're only including DHT.h, yet when building DHT_U.cpp/h is getting built.

Steps to Reproduce

# Install DHT lib without installing it's dependency (it doesn't get installed automatically, which I guess is a bug as well?)
$ platformio lib install 19
# Use `DHTTester.ino` sample from http://platformio.org/lib/show/19/DHT%20sensor%20library
# build
$ platformio run
[Sun Nov  6 22:44:01 2016] Processing esp01 (platform: espressif8266, board: esp01, framework: arduino)
---------------------------------------------------------------------------------------------------------
Verbose mode can be enabled via `-v, --verbose` option
Collected 25 compatible libraries
Looking for dependencies...
Library Dependency Graph
|-- <DHT sensor library> v1.3.0
Compiling .pioenvs/esp01/src/main.o
Compiling .pioenvs/esp01/lib/DHT sensor library_ID19/DHT_U.o
In file included from .piolibdeps/DHT sensor library_ID19/DHT_U.cpp:22:0:
.piolibdeps/DHT sensor library_ID19/DHT_U.h:25:29: fatal error: Adafruit_Sensor.h: No such file or directory
#include <Adafruit_Sensor.h>
^
compilation terminated.
*** [.pioenvs/esp01/lib/DHT sensor library_ID19/DHT_U.o] Error 1

Expected Results

I expected that only parts of the library I actively include would be included in the build.

[ivankravets]

That is not our issue. We made a few PRs with library.json to Adafruit's repo. However, @ladyada rejected our efforts and said that plain TXT with arduino.library.properties without dependencies is our future. See official Adafruit's answer https://github.com/platformio/platformio/issues/499#issuecomment-189571349

Just compare 2 manifests:

• http://platformio.org/lib/show/19/DHT%20sensor%20library/manifest
• http://platformio.org/lib/show/18/Adafruit%20DHT%20Unified/manifest

P.S: To avoid this issue I don't recommend installing library manually using CLI. Please take a look at lib_deps option. In your case, the final project config will look as:

[env:esp01]
platform = espressif8266
board = esp01
framework = arduino
lib_deps = 
    DHT sensor library
    Adafruit Unified Sensor

[simonvanderveldt]

@ivankravets thanks for the explanation of why dependencies don't work.
I don't really understand why there's 2 versions of that lib in the platformio lib list though, wouldn't the one from platformio-libmirror be enough?
I couldn't use that one because it says it's only compatible with arduino and I'm building for esp at the moment.

Anyway :) My actual question is about the compilation of files that I haven't included using include statements. I expected platformio/gcc/make to inspect my own sources and create the relevant list of dependencies (pretty much the same as gcc -M does) and then use that list to automatically get the relevant libraries from the installed libraries.
Is this not how it should work?

[ivankravets]

I don't really understand why there's 2 versions of that lib in the platformio lib list though, wouldn't the one from platformio-libmirror be enough?

These are 2 different libraries.

Is this not how it should work?

Please take a look at http://docs.platformio.org/en/stable/librarymanager/ldf.html

[simonvanderveldt]

These are 2 different libraries.

Thanks for the hint, I totally missed that because the non-unified library also includes the unified version's files :|
Created an issue for that https://github.com/adafruit/DHT-sensor-library/issues/64

Please take a look at http://docs.platformio.org/en/stable/librarymanager/ldf.html

The way I understand the docs platformio doesn't work correctly.

• lib_ldf_mode is on it's default setting, which should be 1
• lib_ldf_mode=1 means: platformio parses ALL C/C++ source code of the project and follows only by nested includes (#include ..., chain...) from the libraries. (emphasis mine)

I have the following files (removed the irrelevant dirs):

├── .piolibdeps
│   ├── DHT sensor library_ID19
│       ├── DHT.cpp
│       ├── DHT.h
│       ├── DHT_U.cpp
│       ├── DHT_U.h
│       ├── examples
│       ├── .github
│       ├── keywords.txt
│       ├── .library.json
│       ├── library.properties
│       └── README.md
└── src
    └── main.cpp

Where src/main.cpp has an #include <DHT.h>. Both DHT.cpp and DHT.h don't include any other libs than their own and some default/Arduino stdlib ones.

Yet the build fails with the error message from my first message when the AdaFruit unified lib isn't installed because that's included in DHT_U.h.
The way I understood the docs this shouldn't happen because nowhere is DHT_U.h actually included in the chain from my project's sources (i.e. main.cpp).

Or does this chain start from the libraries? If so, wouldn't it make more sense to start from the project's own source files?

[jeffeb3]

w.r.t. compiling DHT_U.cpp, platformio follows this logic (I think, I'm not the expert here):
main.cpp includes DHT.h
DHT sensor library_ID19 has a DHT.h
All the .cpp files in DHT sensor library_ID19 are compiled*
DHT_U.cpp includes Adafruit_Sensor.h

*** This step is what you're surprised by, but even though 99% of .h/.cpp files follow a 1:1, it's not required. You could have a library like this:
Library/lib.h
Library/lib_part_a.cpp
Library/lib_part_b.cpp

or like this:
Library/lib.h
Library/foo.cpp

or even:
Library/a.h
Library/a.cpp
Library/b.h
Library/b.cpp
with some crazy interdependencies that didn't end up including b.h from a.h or a.cpp...

The trouble is, you can't assume that even though DHT.h and DHT.cpp don't include DHT_U.h that it shouldn't be included in the build. You have to specifically tell the build system that DHT.h/cpp is a separate unit from DHT_U.h/cpp. That probably is a feature of this build system, and it's probably just not being configured properly. Unfortunately, I don't know this build system well enough to tell you what's wrong.

On a completely different, but related topic, why doesn't the library.json or libraries.properties suggest that "Adafruit Unified Sensor" is a dependency? Seems like a bug to me. If not that, then it should be configured to not build the DHT_U.cpp as Simon was suggesting.

[ivankravets]

Guys, I close this issue because you correctly described LDF.

If you need DEEP Search, please use lib_ldf_mode=2 and PIO will parse all files including library source code. Default behaviour is good for 99%. The other option, you can help PlatformIO to build a project if you specify what do you need. You can put mised library to lib_deps or included it directly in main project. For example, please to one of src file this #include <Adafruit_Sensor.h>

[ivankravets]

Manifest has been updated with dependencies http://platformio.org/lib/show/19/DHT%20sensor%20library/manifest