Pybind11: CMake configuration files missing for PyPi install

[cklb]

As it has already surfaced in #1379, when installing pybind11 via pip form the python package index, only the header files of pybind are installed. After browsing a little in the project history, the functionality to add pybind11 as an external cmake package via find_package() has been added in #506.
In #769 the installation of the config files was added / adjusted for conda based builds with this comment stating what can also be seen when looking at the pypi tarball of pybind11 and could maybe be stated somewhere in the docs.

Now, as the conda recipe shows, an installation of the cmake configuration files when performing a pip install seems to be possible. My question now is if nobody did have the time to do it yet or whether there are more serious problems for this case.
Alternatively, many linux distros install the config files in their packaged versions of pybind11. However, as my project targets a multi platform user base I would really like to stick with pip.

Finally, I'm well aware of https://github.com/sdhnshu/pybind_demo which uses a manual build for the pip install and thus requires no cmake dependencies at all.

[Xeverous]

Apart from the issue that only header files are installed, there is another issue even if you download the entire library repo:

cmake 3.12.1

CMake Error at CMakeLists.txt:8 (find_package):
  Could not find a package configuration file provided by "pybind11" with any
  of the following names:

    pybind11Config.cmake
    pybind11-config.cmake

The file that pybind offers is named pybind11Config.cmake.in

[cklb]

Well this is actually not really an error. If you check out the the source of pybind11 you have to build the cmake package first. As stated here (assuming linux) you should run something along the lines of

make
make all mock_install

to build and copy them into some subfolder. The files you got to make available to cmake are

pybind11/mock_install/share/cmake/pybind11/pybind11Config.cmake
pybind11/mock_install/share/cmake/pybind11/pybind11ConfigVersion.cmake
pybind11/mock_install/share/cmake/pybind11/FindPythonLibsNew.cmake
pybind11/mock_install/share/cmake/pybind11/pybind11Tools.cmake
pybind11/mock_install/share/cmake/pybind11/pybind11Targets.cmake

If cmake is able to find those, you can use pybind via find_packe().

[Xeverous]

Thanks, I was mislead by the claim that this is a header-only library (and so does not require any building)

How/where exactly should I copy these files? I want to workaround the problem that right now pip installation is insufficient to build any project (I want to store pybind out-of-tree)

[loriab]

@Xeverous, I believe that the lines @cklb supplied will get you the out-of-tree pb11 you want (cmake -DCMAKE_INSTALL_PREFIX=/path/to/outoftree/destination). It's true that pb11 is header-only in that nothing gets compiled at build/install time. But it's also true that that make install step is necessary to get the headers copied to the right place and to generate the .cmake files for find_package(pybind11 CONFIG) to consume.

I wholly agree that it'd be nice if the pypi package included the cmake helpers. Unfortunately, they can only be generated when the ~build/install runs through cmake (there's analogous problems with other (compiled) projects where cmake files can't be generated by libtool), not when files are just copied which is I think what's going on in setup.py now. Altering this would require an administrative decision to build the pypi pkg through cmake or to hook cmake up more deeply as a backend to setup.py.

[atilag]

It's been a year and seems like the pypi version of pybind11 still has the problem of not including the cmake required files to use find_package(). Is there a technical reason to not include these files? Will make our life easier :)
Thanks for the project btw!

[NikolausDemmel]

This prevents sophuspy to be installable via pip on Ubuntu 18.04, since the system packaged version of pybind11, that comes with cmake modules, is too old, and the up-to-date pip version doesn't come with cmake modules, see https://github.com/pybind/pybind11/issues/890#issuecomment-570630304

[fstuck123]

So cmake is not able to find your pybind11 installation.
For the following I am assuming pybind11 is installed here:
/opt/pybind11/v2.5/pybind11
I assume, you have already executed the following instructions:
mkdir build
cd build
cmake ..
make check -j 4

You can either solve the problem by

1. Install with sudo rights. I think this should be sufficient. (Is a long time ago since I last did this)
   sudo make install

2. If you do not have sudo rights on your machine, you can install pybind11 locally and add install location to your CMAKE_PREFIX_PATH.
   First create a custom install directory
   mkdir install
make DESTDIR=/opt/pybind11/v2.5/pybind11/build/install/ install
   Now modify the CMAKE_PREFIX_PATH to point to the install directory
   export CMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH:/opt/pybind11/v2.5/pybind11/build/install/usr/local/share/cmake/pybind11

Now cmake will find pybind11.

[NikolausDemmel]

@henryiii, nice, thanks for your effort!

Now I can do

python3 -m pip install https://github.com/pybind/pybind11/archive/master.zip
pybind11_DIR="`python3 -c 'import pybind11; print(pybind11.get_cmake_dir())'`" python3 -m pip install sophuspy

instead of previously

git clone https://github.com/pybind/pybind11.git
cd pybind11
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../install/ -DPYBIND11_TEST=Off
make install
pybind11_DIR=$PWD/../install/share/cmake/pybind11/ python3 -m pip install sophuspy

However, you need to still point cmake to the pybind11 modules manually. Is it conceivable that pip would install / symlink the cmake files into

<PREFIX>/lib/share/cmake/pybind11

or similar, not just

<PREFIX>/lib/python3.6/dist-packages/pybind11/share/cmake/pybind11

?

See also https://cmake.org/cmake/help/latest/command/find_package.html#search-procedure.

[YannickJadoul]

@NikolausDemmel, @henryiii provided a global installation, as well. To install from PyPI (after the 2.6.0 release), pip install pybind11[global] (or pip install pybind11-global) should work.

Building from source, I thínk you can set the PYBIND11_GLOBAL_SDIST environment variable to do a global installation, but I might be wrong.

[NikolausDemmel]

Thanks for the reply. Indeed this works, I didn't realize:

PYBIND11_GLOBAL_SDIST=1 python3 -m pip install https://github.com/pybind/pybind11/archive/master.zip
python3 -m pip install sophuspy

(the second step still installs the released pybind11 package from pypi since sophuspy specifies the dependency. But cmake then finds the files from pybind11-global)

Looking forward to the 2.6.0 release. Thanks again!