terraform source not able to pull github enterprise repository.

[nallanisai]

$terraform init
Initializing modules...
- module.terraform_test_vm
  Getting source "git::[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"
Error downloading modules: Error loading modules: error downloading 'file:///Users/user/terraform/git/terraform-live-va/[email protected]:user/terraform-module-va.git?ref=v0.0.2': /usr/local/bin/git exited with 128: Cloning into '.terraform/modules/7f6caca0c76cc98bed17baabce74edf7'...
fatal: '/Users/user/terraform/git/terraform-live-va/[email protected]:user/terraform-module-va.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$

If i do the git clone iam able to do clone repo without any issues.

$git clone [email protected]:user/terraform-module-va.git
Cloning into 'terraform-module-va'...
remote: Counting objects: 16, done.
remote: Compressing objects: 100% (13/13), done.
remote: Total 16 (delta 0), reused 11 (delta 0), pack-reused 0
Receiving objects: 100% (16/16), 5.47 KiB | 5.47 MiB/s, done.
$

My main.tf looks like this.

module "terraform_test_vm" {
  source = "git::[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

  servers = "1"
  name = "${var.system_name}"
}

[mildwonkey]

Hi @nallanisai, sorry to hear you're having an issue with this.

Can we please double check the source syntax in your config and try again? According to the documentation you do not need the preceding git:: so I think you can try the below in your main.tf:

module "terraform_test_vm" {
  source = "[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

  servers = "1"
  name = "${var.system_name}"
}

[flickerfly]

I'm currently troubleshooting an issue with git source where it is trying to resolved the DNS name with a colon on the end. If I remove the colon between the dns name and the rest of the request it works on CentOS, but breaks on MacOS. Could this be related to what you are experiencing?

[nallanisai]

@mildwonkey I tried removing with git::. Now iam getting source path error.

module "terraform_test_vm" {
  source = "[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

  servers = "1"
  name = "${var.system_name}"
}

Initializing modules...
- module.terraform_test_vm
  Getting source "[email protected]:snall5/terraform-module-va.git//instances?ref=v0.0.2"
Error downloading modules: Error loading modules: error downloading 'file:///Users/user/terraform/git/terraform-live-va/[email protected]:user/terraform-module-va.git?ref=v0.0.2': source path error: stat /Users/user/terraform/git/terraform-live-va/[email protected]:snall5/terraform-module-va.git: no such file or directory.

The above code with git:: works for bitbucket. With below format. But not working with github.

module "terraform_test_vm" {
  source = "git::ssh://[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

  servers = "1"
  name = "${var.system_name}"
}

[mildwonkey]

Alright, I'm sorry if I gave you bad advice, but thanks for trying. It looks like you're using GitHub enterprise, is that correct? That's why you needed the git:: syntax. My mistake!

Can you remove ?ref=v0.0.2 and see if you get the same error, or a new one? I don't think it's related but it'd be good to check.

[nallanisai]

@mildwonkey Iam using github enterprise version, get the same error when removed ?ref=v0.0.2.

$terraform init
Initializing modules...
- module.terraform_test_vm
  Getting source "git::git@[email protected]:user/terraform-module-va.git//instances"
Error downloading modules: Error loading modules: error downloading 'file:///Users/user/terraform/git/terraform-live-va/git@[email protected]/user/terraform-module-va.git': /usr/local/bin/git exited with 128: Cloning into '.terraform/modules/f5aa4a3e98735ddc4ca2538d03a1c973'...
fatal: '/Users/user/terraform/git/terraform-live-va/git@[email protected]/user/terraform-module-va.git' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
$

[nallanisai]

@flickerfly I tried in rhel7 also by removing replacing : with /, but got the same error.

Initializing modules...
- module.terraform_test_vm
  Getting source "git::[email protected]/user/terraform-module-va.git//instances?ref=v0.0.2"
Error downloading modules: Error loading modules: error downloading 'file:///var/lib/jenkins/terraform-live-va/[email protected]/snall5/terraform-module-va.git?ref=v0.0.2': /bin/git exited with 128: Cloning into '.terraform/modules/9b3f61bc33503b9b1da728fc37a052a7'...

[mildwonkey]

What version of terraform are you using, @nallanisai?

[nallanisai]

@mildwonkey 11.7

$terraform --version
Terraform v0.11.7

$

[mildwonkey]

Thanks! Hm ..... what else can we try?

You mentioned that this format works for bitbucket:
"git::ssh://[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

Have you tried this?
"git::ssh://[email protected]:user/terraform-module-va.git//instances?ref=v0.0.2"

Just to give you a heads up, I'm going to go back and edit some of your past comments so they are a bit easier to read - I will only edit the formatting, not the content.

[nallanisai]

@mildwonkey I tried with "ssh://" and replacing : with /, now i get permission denied.
Not sure, i was able to clone with any issue with.
$git clone [email protected]:user/terraform-module-va.git

$terraform init
Initializing modules...

• module.terraform_test_vm
  Getting source "git::ssh://git@[email protected]/user/terraform-module-va.git//instances?ref=v0.0.2"
  Error downloading modules: Error loading modules: error downloading 'ssh://git@[email protected]/user/terraform-module-va.git?ref=v0.0.2': /usr/local/bin/git exited with 128: Cloning into '.terraform/modules/fc80d1a7db4af97713da568f78a24924'...
  git@[email protected]: Permission denied (publickey).
  fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

$

[apparentlymart]

I'm not sure if there's any overlap between them, but I just wanted to note that we've been looking at another Git-SSH-source-related issue over in #18546 where I looked into how the source string gets gradually processed by various parts of Terraform and then by git. Perhaps the info there gives a clue as to how the source addresses in _this_ issue might be being misinterpreted.

[nallanisai]

Issue resolved with below command in centos, but its not working on mac.

source = "git::ssh://[email protected]/user/terraform-module-va.git//instances?ref=v0.0.2"

Thanks everyone for the support.

[agmaverick]

Is there any solution to this issue yet? I'm experiencing the same problem described above on Ubuntu VM's (16.04 and 18.04 running with Vagrant):

ssh: Could not resolve hostname github.org.company_name.com:project: Name or service not known
fatal: Could not read from remote repository.

Please make sure you have the correct access rights

My modules are hosted in my company's GHE. I'm able to pull the module directly using git clone but Terraform fails.

I've tried the solutions from previous comments with no joy.

[apparentlymart]

Hi all,

For some reason there was a sudden spike in this sort of git+ssh source string syntax issue even though nothing has changed in Terraform in this area for some time. We (the Terraform team at HashiCorp) are unsure as to why so many of you ran into similar issues in the space of one week after no such reports prior to that.

I want to try to sum up what we learned and discussed across some of the other issues and forums where people ran into problems like this.

There are two different ways to write a Git SSH "URL" for Terraform:

# "scp-style":
git::username@hostname:path

# "URL-style":
git::ssh://username@hostname/path

In both of these cases, Terraform is just taking the portion after the git:: prefix (after also removing any //subdir and ?rev=... portions) and passing it to git clone:

git clone username@hostname:path
git clone ssh://username@hostname/path

How the rest of this is interpreted is entirely up to git. Notice that the scp-style string uses a colon to separate the path from the hostname, while the URL style uses a slash, as described in the official git documentation.

What isn't clear yet is why the ssh:// "URL-style" address seems to be working on some platforms and not others. Across the various discussions the common theme seems to be that git on a Mac does not accept the URL-style address. Given how Terraform processes these strings, I have to assume that this is something strange about the version of git installed on those Macs, which may be configured differently or may be a different version than is being used on other systems.

Although this particular issue wasn't related to the colon vs. slash mix-up exactly, many of the other issues were rooted in that. If anyone following this thread can given a reference to any example they found that shows using the URL-style form with a colon incorrectly separating the path, please let us know and if it's documentation _we_ control then we'll get it repaired ASAP.

[sunggun-yu]

Hi all and @apparentlymart

I was successfully init the module from enterprise github repo with following configs...

Environment

• OS: macOS High Sierra, v10.13.6
• Terraform: Terraform v0.11.7

Config

1. Config ~/.ssh/config for your enterprise github account

   Host github.your-company.com
       HostName github.your-company.com
       User git
       IdentityFile ~/.ssh/your-key
   

2. Terraform Module source:

   module "terraform_test_vm" {
     source = "git::github.your-company.com:user/terraform-module-va.git//instances?ref=v0.0.2"
   
     servers = "1"
     name = "${var.system_name}"
   }
   

The point is

• git::github.your-company.com:org/repo.git not git::[email protected]:org/repo.git: It resolves does not appear to be a git repository error.
• HostName github.your-company.com: It resolves ssh: Could not resolve hostname error on Mac OS.
• User git: Terraform will access to github repo with your computers account name without this config.

Please test. Thank you

[sunggun-yu]

Hi all,
I did same testing on Windows 10 power shell.
and it works fine with same configuration as above https://github.com/hashicorp/terraform/issues/18541#issuecomment-416369712

• Git with OpenSSL Library
• C:\Users\your-name\.ssh\config

[suvhm]

Actually i tried to add my bitbucket repository to my launchpad account for automatic snap creation.
I have added my bitbucket repository with launchpad but when launchpad tried to clone the source code from repository it through warning: You appear to have cloned an empty repository but it has all the source code in that repository.
I tried it manually by cloning the bitbucket into my local dir. it was cloned successfully.
I appreciate any support here.

[lhagemann]

As a note regarding @apparentlymart comment above about the sudden increase in errors with GH:E.

I just came across this issue today myself. My situation was an existing TF module, to which I added a new remote submodule. One that had only been added to GH:E that very day.

Our GH:E was updated from v 2.13.3 to 2.15.2 on Nov 15, 2018. The first reported issue with the git source url format was reported on July 25, 2018. Note that the 2.13 branch of GH:E had a release on July 24, 2018: https://enterprise.github.com/releases/

I haven't dug too deeply into the changes to git clone urls between those versions of the GH:E --- but it seems reasonable that is related.

[cmcconnell1]

TL;DR 'Hoping to save others some time regarding github enterprise accounts and TF issue:
The following two options worked for me adding to the code within the main.tf calling the module.
YMMV.

enterprise github ssh auth:

module "my-mod" {
  source = "[email protected]:my-company/my-tf-modules-repo.git//modules/mod-foo"
...

Notes:

• I needed to have the double slash // at the base of the repo above since the modules are in a subdir in the above example.
• I didn't need any special configurations in the ~/.ssh/config file.

S3 modules keeping control locally within your org (requires manually zipping and versioning)

module "my-mod" {
  source = "s3::https://s3-us-west-2.amazonaws.com/mycompany-tf-modules/mod-foo/mod-foo-1.0.1.tgz"
...

It should be noted that you need to ensure that your structure for the archive file must be as terraform expects else you will get errors.

Hope this helps save others some time and frustration.

[enrichman]

This worked for me, thanks @sunggun-yu.
The problem is that I would like to set the username programmatically and not in the config file.
Is there a way to do this?

I'm running the terraform init from a docker container where the user is root so I'm getting some issues with that.

EDIT:

I've tried with this and it worked also without the config file!

module "my_mod" {
  source = "git::ssh://[email protected]/user/terraform_modules.git//my_mod"
}

[teamterraform]

Hi all!

We eventually learned, after a lot of trial and error, that the root problem here was coming from an ambiguity between the URL-style syntax and the scp-style syntax as @apparentlymart described above. The source string parser in Terraform was being a little too liberal in how it was handling these, causing confusing/incorrect results in some cases.

The URL parsing behavior now (in the current Terraform release) supports only two permutations, each of which are equally valid, but no mixtures of the two are allowed:

# "scp-style":
git::username@hostname:path

# "URL-style":
git::ssh://username@hostname/path

We recommend using the "URL-style" because it's consistent with the other URL forms accepted in module source addresses and thus probably more familiar/intuitive to readers.

If your SSH server is running on a non-standard TCP port (not port 22) then you can include a port number _only with a URL-style address_ by introducing a colon after the hostname:

# URL-style with port number
git::ssh://username@hostname:port/path

Using the ssh:// URL-like prefix but also using a colon to delimit the path is now _never_ valid, whereas before it would be interpreted differently in different situations based on an incorrect heuristic.

Since this change was already included in a Terraform release, we're going to close this out now. If you're still seeing weird behavior for certain source addresses or if you have any other follow-up questions, please feel free to start a new topic in the community forum. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.