Pybind11: CMake 3.12+ Python Find Routines, IPO/LTO

[ax3l]

CMake 3.12.0+ deprecates the old find modules for Python and unifies them into a new call find module:

• https://cmake.org/cmake/help/v3.17/module/FindPythonInterp.html
• https://cmake.org/cmake/help/v3.12/module/FindPython.html

We should consider mid to long-term to update our CMake scripts accordingly, as soon as we can require CMake 3.12+ (released July 2018).

Furthermore, CMake 3.9.0+ introduces an abstraction for IPO/LTO, which can simplify our code base.

• https://cmake.org/cmake/help/v3.9/module/CheckIPOSupported.html

[RUrlus]

This is also helpful for those using Pyenv, the old find module doesn't detect the active python version but rather the newest one. The new version does correctly identify the active version .

[henryiii]

I highly recommend supporting this (and will try to help tackle this in the near future), but even if we jump to supporting 3.12+ only tomorrow, I think we will need to support both methods for a long time. If a package using PyBind11 does anything else with CMake + Python, either another package or even just using the variables they expect to be defined, they will probably have to update to the new module as well (unless they mix better than I think they do).

Quickly thinking about it, I would have a PYBIND_FIND_PYTHON variable that defaults to "OFF" for now, and later (3.0?) will default to "ON", that selects between the two implementations. I think the new implementation is probably quite simple, as quite a bit of what we do is already fixed or added to CMake now. Also, we might need to make the minimum version for PYBIND_FIND_PYTHON=ON a little higher than 3.12, since there were lots of fixes and additions in the subsequent versions as they polished off the new modules.

Another option would be to simply respect an existing find package run - so if you do:

find_package(Python COMPONENTS Development)
find_package(pybind11)

then it picks up the discovered new-style Python instead of self discovering with the old mechanism. That might be the best way, actually.

I haven't used the new module much yet, since basically whenever I need Python + CMake, it's probably for a PyBind11 module. :)

[dvj]

To expand upon @henryiii 's point: The new FindPython module allows the users to supply many hints, which are very useful in complex environments (think: in a virtualenv, but building for a set of different interpreters). Setting these hints as a user should pass seamlessly through pybind11 in a 3.12+ system using FindPython, and if changes are made in the near-term to support both systems (which sounds like a winning idea!), I would definitely appreciate a method (as Henry says) "to respect an existing find package run" for similar complex-environment reasons, as the current system doesn't allow enough flexibility in supplying appropriate interpreters, libs and headers in CMake.

[henryiii]

Dup of #2139, by the way.