Pybind11: Wrong python system path import when add PCL library in CMakelists

[ltphy]

Issue description

I use Python C++ Interface to access the python module by Pybind11 and import the python module in C++ code.
There is a wrong python path imported when including PCL library together with Pybind11 in CMakelists.txt file, but it imports the python environment correctly when there is only pybind11.

Installation

conda install pybind11

Expected Behavior
Should import the python from the current environment path.

Reproducible example code

When CMakelists only includes pybind11, the python path environment is used correctly.

• Python code:

import sys
print(sys.path)

['/home/phy/.conda/envs/env/lib/python37.zip', '/home/phy/.conda/envs/env/lib/python3.7', '/home/phy/.conda/envs/env/lib/python3.7/lib-dynload', '/home/phy/.local/lib/python3.7/site-packages', '/home/phy/.conda/envs/env/lib/python3.7/site-packages', '/home/phy/.conda/envs/env/lib/python3.7/site-packages/PyYAML-5.3.1-py3.7-linux-x86_64.egg', '/home/phy/.conda/envs/env/lib/python3.7/site-packages/numba-0.50.1-py3.7-linux-x86_64.egg', '/home/phy/.conda/envs/env/lib/python3.7/site-packages/llvmlite-0.33.0-py3.7-linux-x86_64.egg', '/home/phy/env']



md5-237f0dc1013ecd83e332f042ad44412d



import sys
print(sys.path)



md5-abea0460962e276c0d6b120d6be4d7fa



['/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/home/phy/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', u'.',]



md5-e07804f4776a932b066e554c550d7a18



cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
project(cli_cplusplus)
find_package(Python3 COMPONENTS Interpreter)
find_package( pybind11 REQUIRED)
find_package( PCL 1.8 REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories( ${Boost_INCLUDE_DIR} ${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions("-Wall -DCPU_ONLY -std=c++11 -O3 -g")
target_link_libraries (main PRIVATE pybind11::embed ${PCL_LIBRARIES} -lstdc++fs)

[YannickJadoul]

What's the version of Python that's used? If you print sys.version and/or sys.version_info, does it still show Python 3?

If it does, we might have to blame PCL, because I don't know how pybind11 would change that, then.
If it doesn't, @henryiii's recent work (partically in progress) on the CMake build system might fix this?

[henryiii]

There are several issues.

1. Never put 2.8 as the minimum version. Since you use find_package(Python3..., your minimum version is 3.12. FATAL_ERROR hasn't been needed since CMake 2.6.
2. You should always list LANGUAGES CXX in project unless you are building both C++ and C code.
3. You are using find_package(Python..., but that's not supported until #2370 is merged.
4. You are using find_package(Python3, which is really only intended to be used if you _also_ call find_package(Python2 in the same project. If you don't intend to target both, you should use find_package(Python ..., possibly with VERSION 3 if you don't want Python 2.
5. You are only finding Interpreter, not Interpreter Development. You will not be filling in the variables and targets needed for writing a Python extension, only for calling Python.
6. Pybind11 is finding Python with a modified "classic" FindPythonLibs/FindPythonInterp search, which is often wrong. You should set -DPYTHON_EXECUTABLE=/path/to/python to ensure the right one is being found.
7. You should probably not use include_directories, but use target_include_directories instead. Also you don't need to include python if you also link to the pybind11::embed target.
8. Don't use link directories. Link paths are absolute in CMake.
9. Don't add C++ standard flags. CMake knows you need at least C++11 if you use our targets. set(CMAKE_CXX_STANDARD 11) if you want to force a specific standard.

cmake_minimum_required(VERSION 3.12...3.18)
project(cli_cplusplus LANGUAGES CXX)

find_package(pybind11 CONFIG REQUIRED)
find_package(PCL 1.8 REQUIRED)

target_compile_options(main PRIVATE -Wall -DCPU_ONLY -O3 -g)
target_link_libraries (main PRIVATE pybind11::embed ${PCL_LIBRARIES} -lstdc++fs)
target_include_directories(main SYSTEM PRIVATE ${Boost_INCLUDE_DIR} ${PCL_INCLUDE_DIRS})

Build with <cmake commands> -DPYTHON_EXECUTABLE=$(which python)

Does the problem persist?

[ltphy]

I cannot build by make. It produces the following error:

<command-line>:0:1: error: macro names must be identifiers

[henryiii]

Replace target_compile_definitions with target_compile_options. Only one of them was actually a definition; options doesn't assume that you have definitions. (Edited above)

[ltphy]

Thank you very much. It works now. You save my day.

[henryiii]

Glad I could help! Maybe Modern CMake and/or the workshop linked on the front page (may move soon, so not adding a link here) would be helpful in the future. :)