Nixpkgs: Broken cmake file with Qt5.10 full

[dwarfmaster]

When invoking CMake on a project that depends on Qt5, cmake complain that the /nix/store/...-qt-full-5.10.1/lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake file refer to the /nix/store/...-qtbase-5.10.1-bin/lib/qt-5.10/plugins/imageformats/libqicns.so file, and that this last one doesn't exists, thus failing.

EDIT After some more investigations, it turns out that the Qt5Gui_QICNSPlugin.cmake file only refer to imageformats/libqicns.so, which is not (and as never been) part of qtbase but is part of qtimageformats. This is because Qt5GuiConfig.cmake (in qt-full, but generated from the qtbase repository), when calling a *Plugin.cmake, prefix the path they give by the absolute path to qtbase.

Steps to reproduce

Try to invoque cmake on this repository https://github.com/paceholder/nodeeditor for example.

Technical details

• system: "x86_64-linux"
• host os: Linux 4.9.30, NixOS, 17.09.git.bdf11d9f973 (Hummingbird)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 1.11.9
• channels(root): "nixos-16.09.1824.ae92cb4"
• nixpkgs: /home/luc/.nixpkgs/nixpkgs

[nioncode]

Did you find out how to fix this? I'm currently facing the same problem compiling a CMake project.

[dwarfmaster]

I did not find how to fix it due to insufficient Nix knowledge. It seems the errors comes from the fact that when the cmake files are created, they think the plugins are in the qtbase installation, while they are in the qt-full installation. All my tries to change it failed, it has something to do with the CMAKE_PLUGIN_DIR variable (but I don't know where it comes from).

My (ugly) workaround for now is to develop Qt in a nix-shell using an older version of the nixpkgs. For that I am using the commit 168fbde17a2 (which comes with qt 5.9 and not qt 5.10).

[vandenoever]

I'm seeing the same problem with qt59 on NixOS 18.03.

The problem could perhaps be fixed by patching lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake after install. I do not know what QICNSPlugin is for. Perhaps a workaround is to disable it.

[dwarfmaster]

I believe the problem is not specific to the Qt5Gui_QICNSPlugin.cmake file. It is the first one to fail but others also fail. As I said, the problem is that when generating the files, the CMAKE_PLUGIN_DIR variable contains the installation of qtbase, while it should be that of qt-full.

[arximboldi]

I am facing the same problem with qt56.full

[cryptix]

A friend of mine tried to package another QT application (his nixpkgs brandch to add beam-mw) and stumbled over this.

I got interested in nodeeditor and made this branch, using a copy of sqlitebrowsers approach (which is also qt). It works but just installs the examples.

[viric]

Maybe @ttuegel knows how to fix this quickly. I hit the same problem.

It is related to qtbase.patch touching features/data/cmake/Qt5BasicConfig.cmake.in :
https://github.com/NixOS/nixpkgs/blob/release-18.03/pkgs/development/libraries/qt-5/5.10/qtbase.patch#L252-L262

That macro is called by Qt:Gui plugins (which may be from different nix store paths) but the import location is set always to:

        set(imported_location "/nix/store/hgkyq8h7hs5whn3kih7ilbhn1vz1k366-qtbase-5.10.1-bin/lib/qt-5.10/plugins/${PLUGIN_LOCATION}")

The original post tells about an imageformats plugin, but it happens with others too, like
/nix/store/k2pnhw6b40ly9lvcpzlghcyc8cvbn90i-qt-full-5.10.1/lib/cmake/Qt5Gui/Qt5Gui_QVirtualKeyboardPlugin.cmake

[ttuegel]

The _populate_Gui_plugin_properties macro is supposed to be internal to Qt5Gui. qtimageformats should be generating its own CMake modules, not "borrowing" macros from another submodule. Qt probably gets away with this in most distributions because having a "split" Qt build is uncommon, and installing into different prefixes is even more uncommon. The possible solutions for this, on the Nixpkgs side, are:

1. Provide a correct CMake module for qtimageformats, or
2. Bundle qtimageformats with qtbase.

[viric]

Notice that QVirtualKeyboardPlugin suffers of the same problem as qtimageformats. I didn't explore what else.
Are you sure the macro is supposed to be internal to Qt5Gui? The schema seems to show the opposite.

[ttuegel]

Are you sure the macro is supposed to be internal to Qt5Gui?

Uh, well, yes and no. :grinning: CMake has a single global namespace so nothing is ever "private" except by convention. I think it is usually the case that macros intended for internal use begin with _, as this troublesome one does. I have observed that KDE packages, for example, refrain from calling these macros and generate their own instead. If this macro is supposed to be public, I don't see how we could support that in Nixpkgs; at minimum, we would have to return to providing only a monolithic Qt build.

The schema seems to show the opposite.

I'm not sure what you mean. What schema are you referring to?

[viric]

By schema I only meant that the pattern they follow does not take that macro as internal to Qt5Gui. So maybe that's how it should work. :)
What about patching that cmake file at qt env time so it points to the env directory?

[viric]

I have an idea how to fix this. I'm trying it. I will try to open a PR if it works.

[viric]

Check my proposal here over release-18.03: https://github.com/NixOS/nixpkgs/compare/release-18.03...viric:qt5-fix-full-cmake?expand=1

It's simple, it works and keeps the separated store paths.

Maybe people would like a single patch file for qtbase, or a different patch file name, ... no idea. That's the easiest I could do.

[ttuegel]

@viric Please make a separate patch for now. I'm in the process of breaking out the separate patches because the one-patch-to-rule-them-all approach is completely unmaintainable.

[viric]

It is a separate patch now. Please pick it in any rework you do because it makes all work.

[nioncode]

Any chances of getting this merged soon?

[viric]

Did you try it?

[nioncode]

No, I'm new to nixos and don't really know how I can try your patch.

[viric]

Whatever nix command you use to get your cmake thing, use it with a checkout of my branch of nixpkgs, and tell it "-I nixpkgs=path-to-the-checkout".

[nioncode]

After a few hours of building Qt, I can confirm that your patch makes my cmake based project successfully compile and launch again. Thanks, I can now develop without a VM again.

We should also add this patch to the Qt 5.6 and 5.9 LTS versions and possibly backport this to fix Qt development on older channels.

[viric]

I pushed the change to master for qt 5.11. @ttuegel do you want to rearrange the patches before any backport?

[viric]

I reopen the issue for the backports. The title says 5.10.

[nioncode]

@viric This must be patched also for 5.6 + 5.9 LTS versions on master, since they are still available for install using Qt56.full and Qt59.full. I verified that both are broken, but did not have the time to build Qt with your patch for 5.6 and 5.9 to check if it fixes them.

In backports, this should be patched for 5.6 + 5.9 LTS and whatever the latest Qt version is at the time in the channel.

[nschoe]

Thanks @viric you're a life saver. I've been struggling with this issue for months!

I have tried testing your patch, but the full qt rebuild that it triggered took 10 hours on my laptop and it ended up failing due to lack of space...

A good friend lended me his server which I'm using as a build farm (I'm hoping to be able to copy the clojure after or use it as a binary cache), it's been 4 hours, still building at the moment.
I'll report when the build completes.

[nschoe]

Wow! After about 20hours of build on my own machine (which I restarted several times, etc.) qt finally built with your patch.
Thanks a lot. Now I have to examine it and understand why/how it works.

[nioncode]

Can you guys still run qtcreator from a nix-shell -p qtcreator -p qt5.full environment? With @viric 's patch, running qtcreator prints:

libGL error: unable to load driver: i965_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: i965
libGL error: unable to load driver: i965_dri.so
libGL error: driver pointer missing
libGL error: failed to load driver: i965
libGL error: unable to load driver: swrast_dri.so
libGL error: failed to load driver: swrast
Unrecognized OpenGL version
Unrecognized OpenGL version

Afterwards, it just hangs.

[viric]

It may be that you are mixing QTs in your build environment and your OS environment. Qt doens't want to be mixed with other Qts; it loads plenty of plugins per process and the path to many of them come through OS common vars (QTPLUGINPATH or similar).

[nioncode]

Isn't this what environments built by nix-shell or nix packages in general try to solve? If I create an environment and list all the packages I want, why does it then still depend on some system packages?

[nioncode]

I just pulled nixpkgs again and built my environment from master. Building my project with CMake still works and running QtCreator now also works. I don't really know what was broken before, but at least with the current master everything works for me.

I filed a PR for getting this fix on Qt 5.6 and 5.9.

[nioncode]

Ok, so with my PR being merged (thanks for the fast response!), the only thing left to do is to merge those 2 commits into the appropriate channels and possibly fix the directory names if the latest Qt version is not 5.11.

[husnoo]

It's back...

Linux heisenbug 4.19.23 #1-NixOS SMP Fri Feb 15 08:09:54 UTC 2019 x86_64 GNU/Linux
$ nix --version
nix (Nix) 2.1.3
$ nixos-version 
18.09.2203.9bd45dddf81 (Jellyfish)

CMake Error at /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:13 (message):
  The imported target "Qt5::Gui" references the file

     "/nix/store/mlsgfqvixginx8wjzs6k64bz930ascl0-qtbase-5.11.1-bin/lib/qt-5.11/plugins/imageformats/libqicns.so"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:155 (_qt5_Gui_check_file_exists)
  /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake:4 (_populate_Gui_plugin_properties)
  /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:163 (include)
  /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:87 (find_package)
  /nix/store/1shkzgz3vpymq4igismi28wl7gsyi3zy-qt-full-5.11.1/lib/cmake/Qt5/Qt5Config.cmake:28 (find_package)
  cmake/SearchForStuff.cmake:601 (find_package)
  CMakeLists.txt:147 (include)


-- Configuring incomplete, errors occurred!

[nschoe]

Any news on this?
I've just updated my channels and I still have this issue. Hasn't this been fixed again?
Am I missing something?

[nioncode]

It's really unfortunate that this seems to break every now and then, which makes my nixos machine totally unreliable for development. I'd have to create a nix-shell that points to a specific nixpkgs commit and try to update it every now and then (and make sure to test compiling a qt gui project).

Can't we add a test case that compiles a qt gui project so that breakages are detected immediately and automatically? (I'm not too familiar with nixos' automatic tests)

[nioncode]

@viric This is still broken for me. Does it work for you? I can't build a GUI application for months and have to go through a VM all the time, which is a pita:

CMake Error at /nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:13 (message):
  The imported target "Qt5::Gui" references the file

     "/nix/store/bnjabqcfd05dwynjcw8ic8rjwfjiqwa5-qtbase-5.11.3-bin/lib/qt-5.11/plugins/imageformats/libqicns.so"

  but this file does not exist.  Possible reasons include:

  * The file was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and contained

     "/nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake"

  but not all the files it references.

Call Stack (most recent call first):
  /nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:155 (_qt5_Gui_check_file_exists)
  /nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Gui/Qt5Gui_QICNSPlugin.cmake:4 (_populate_Gui_plugin_properties)
  /nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Gui/Qt5GuiConfig.cmake:163 (include)
  /nix/store/phh2xpfr93d6gf136f8f6m76qwsvckrv-qt-full-5.11.3/lib/cmake/Qt5Widgets/Qt5WidgetsConfig.cmake:87 (find_package)
  CardManager/lib/CMakeLists.txt:6 (find_package)

EDIT: Switching from 18.09 to 19.03 on my dev machine made the compilation work again, so I guess this is fixed in 19.03, but broken in 18.09.

[stale[bot]]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.