Nixpkgs: nix-shell -p qt5.qtbase drops files into my current directory

[nh2]

Running nix-shell -p qt5.qtbase the first time works.

Running it again produces mkdir: cannot create directory ‘/path/to/pwd/__nix_qt5__/bin’: File exists.

And indeed the first invocation created:

$ find __nix_qt5__/
__nix_qt5__/
__nix_qt5__/lib
__nix_qt5__/bin
__nix_qt5__/bin/qt.conf
__nix_qt5__/bin/qmake
__nix_qt5__/share
__nix_qt5__/mkspecs
__nix_qt5__/nix-support
__nix_qt5__/nix-support/qt-inputs
__nix_qt5__/include

Is this expected? I thought nix-shell by itself should never put stuff into my current dir. Am I wrong?

[FRidh]

The files are created by pkgs/development/libraries/qt-5/qtbase-setup-hook.sh which is a setupHook of qtbase.

I know a shellHook can create files outside the store when used with nix-shell but I am a bit surprised as well to see this happening by other parts.

[avnik]

Is a ugly workaround for qmake.
I have idea, to wrap it into intermediate derivation.
so instead

    buildInputs = [ qtbase qtgui otherQtBasedLib nonQtBasedLib

will be

     let qmakeEnv = makeQmakeBuildEnv [ qtbase qtgui otherQtBaseLib ];
     in
     mkDerivation {
          ...
          buildInputs = [qmakeEnv nonQtLib ];
     }

[bjornfor]

One could start by changing the qmake hook from doing "mkdir x" (fails on 2nd invokation) to "rm -rf x; mkdir x".

[nh2]

One could start by changing the qmake hook from doing "mkdir x" (fails on 2nd invokation) to "rm -rf x; mkdir x".

Wait wait, can we first clarify _why_ this should put _anything_ into my directory?

Ideally I'd really not have it put anything into my current directory (also, there is no guarentee that I can even write into the current directory; I could be in /), and I certainly wouldn't want it to delete anything from it.

[bjornfor]

CC @ttuegel, who knows the qmake hook much better than I.

[ttuegel]

It's not the qmake hook specifically, it actually belongs to qtbase.

Qt absolutely cannot tolerate having different modules installed in different prefixes. At the start of each build, we link all the active modules for the current build into a temporary location. Then we lie to Qt and pretend it was installed to that temporary prefix. The only alternative is going back to building all Qt modules as one giant monster package.

The temporary prefix is supposed be in $TMP created using mktemp -d, which should always be writable. I don't know why it is created in your current directory.

[avnik]

@ttuegel hack with __nix_qt5__ was introduced to avoid impurity from mktemp -d, because that paths somehow hit executables. What you think about make these temporary location a derivation (one per packages set)?

[nh2]

The temporary prefix is supposed be in $TMP created using mktemp -d, which should always be writable. I don't know why it is created in your current directory.

@ttuegel So for you, nix-shell -p qt5.qtbase doesn't create the mentioned directory in $PWD?

[ttuegel]

So for you, nix-shell -p qt5.qtbase doesn't create the mentioned directory in $PWD?

@nh2 No, I mean: when I wrote the code, I wanted the directory to be in $TMP and I consider anything else a bug. I was not aware or did not recall the changes @avnix mentions.

hack with nix_qt5 was introduced to avoid impurity from mktemp -d, because that paths somehow hit executables.

@avnik The builder is ultimately still impure because it would contain the value of $TMP, which can change from build to build.

I don't know which paths are included, but an educated guess: paths to header files, pulled in by assert()?

What you think about make these temporary location a derivation (one per packages set)?

@avnik That would be great, but I do not know a way to create a derivation inside a builder.

[avnik]

@ttuegel you was approved this (current) approach in #20856 and specifically, because $PWD/__nix__qt5__ replace random paths with more predictabe (/tmp/$derivation path is always same now)

What you think about make these temporary location a derivation (one per packages set)?

@avnik That would be great, but I do not know a way to create a derivation inside a builder.

Not in builder, but in expression (I put example in one of comments above)

[ttuegel]

Not in builder, but in expression (I put example in one of comments above)

The problem with this approach is:
The package being built will refer to the intermediate expression, which cannot be garbage collected. The .dev outputs and symlink farms are very big, so we do not want to retain them at runtime. (I know, symlink farms seem lightweight, but until I removed them, KDE had ~100MB of symlinks.)

In the immediate term:
I believe we can check for IN_NIX_SHELL and fall back to using mktemp -d in the qtbase setup hook if it is being run from nix-shell. In this case, I think the principle of "don't litter in $PWD" overrides the principle "be totally predictable".

Long term, I think the only solution which solves all our problems is to fix Qt so it does not reference the temporary paths.

you was approved this (current) approach in #20856 and specifically, because $PWD/__nix__qt5__ replace random paths with more predictabe (/tmp/$derivation path is always same now)

It is... probable... that I did not understand what I was agreeing to. 😺
Sorry for the churn. ☹️

[das-g]

Are there any plans to get these fixes/workarounds into a release?

[avnik]

@das-g was fixed in #28237
And should hit 18.03 release.
Does you hit problem again?

[das-g]

Does you hit problem again?

I thought so, but I guess it was just an old __nix_qt5__ directory I encountered. (Can't tell anymore, because I've already deleted it).

I tried

nix-shell -p kdiff3 --run kdiff3

as well as

nix-shell -p kdiff3-qt5 --run kdiff3

and neither did produce a new __nix_qt5__ directory in the current working dir. So I guess this issue can be closed?