Nixpkgs: fetchGit does not find private git repo

[ghost]

Issue description

All permutations to fetch a private git:

let test = (import (builtins.fetchGit git://github.com/private/private-repo)); in "nixops deploy ${test}";
let test = (import (builtins.fetchGit "git://[email protected]/private/private-repo.git")); in "nixops deploy ${test}";
let test = (import (builtins.fetchGit [email protected]/private/private-repo)); in "nixops deploy ${test}";
let test = (import (builtins.fetchGit [email protected]/private/private-repo.git)); in "nixops deploy ${test}";
fail with:

building Nix...
building the system configuration...
fatal: remote error: 
  Repository not found.

Steps to reproduce

Write function to fetch private git repo using Nix 2.0's fetchGit, then run nixos-rebuild switch.

Technical details

• system: "x86_64-linux"
  
  
  
  • host os: Linux 4.14.20, NixOS, 18.03pre129076.831ef4756e3 (Impala)
  
  
  • multi-user?: no
  
  
  • sandbox: yes
  
  
  • version: nix-env (Nix) 2.0pre5968_a6c0b773
  
  
  • channels(root): "nixos-18.03pre129076.831ef4756e3"
  
  
  • nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs
  
  

[dhess]

I had a similar problem. Assuming the user as whom you are running the Nix command has an SSH key that works with the private repo on GitHub, try this URL, which works for me:

ssh://[email protected]/private/private-repo.git

[CMCDragonkai]

@dhess Did you achieve this by using $SSH_AUTH_SOCK or via some special path to an SSH key available to the nixbld group users.

[dhess]

It's the Hydra user that needs the key, since Hydra fetches the repo.

You'll want to do this securely, since the private key could be used for nefarious purposes. That means keeping it out of the Nix store, which is world-readable (or worse, in a binary cache somewhere). To accomplish this, I wrote a bit of NixOS+NixOps config that ensures the key is uploaded to the /run/keys filesystem and then copied with secure permissions to the Hydra host's local (persistent) filesystem. You can see that here:

https://gist.github.com/dhess/6bbb00100b0fe9b8e17472c0c62bfb10

Note that the config shown there assumes your Hydra username is hydra.

Also note that, in addition to installing the private key for your repo, you'll also want to pre-seed the GitHub public host key so that ssh can connect without trying to prompt whether you want to accept the host public key. The config shown above does that as well, assuming you have the key plaintext stored in a file named github.com.pub.

(I would also suggest creating a private key explicitly for Hydra's use, so that if it is compromised or retired, it's easy to revoke and doesn't affect other users of your repo.)

[tbenst]

@dhess do you know how this works on local NixOS? I have the proper key in /home/$USER/.ssh/id_rsa, but when I run nix-build as that user, fetchgit fails with:

Could not create directory '/var/empty/.ssh'.
Host key verification failed.
fatal: Could not read from remote repository.

more info in: https://github.com/NixOS/nixpkgs/issues/43583#issuecomment-510665973

[dhess]

@tbenst I think that means that nix-daemon doesn't know the GitHub SSH public host key. You can do something like this in your NixOS config:

   programs.ssh.knownHosts = [
      {
        hostNames = [ "github.com" ];
        publicKey = "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==";
      }
    ];

[tbenst]

@dhess, thanks getting warmer! Now I get,

Could not create directory '/home/tbenst/.ssh'.
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

This directory already exists. nixbld1 doesn't have access to it, as it's not part of the users group. Do I really need to add nixbld1 to users? it could then access my home directory.

[paluh]

@tbenst Did you manage to overcome this issue? I'm getting the same error message...

[tbenst]

@paluh I gave up and used git submodules