Dep: the newest dep version cant't get golang package

[godcong]

the newest dep version cant't get golang package
What is the difference between the two versions?

What version of dep are you using (dep version)?

dep 0.3.0
dep 0.3.1/0.3.2

What dep command did you run?

dep ensure

Gopkg.toml
[[constraint]]
name = "golang.org/x/text"
branch = "master"
source = "github.com/golang/text"

What did you expect to see?

i want get the text package from github.com
when i used dep 0.3.0 that's ok.

What did you see instead?

when i use dep 0.3.1/0.3.2：
error:
✗ unable to deduce repository and source type for "golang.org/x/text": unable to read metadata: unable to
fetch raw metadata: failed HTTP request to URL "http://golang.org/x/text?go-get=1": Get http://golang.org/x
/text?go-get=1: dial tcp 216.239.37.1:80: connectex: A connection attempt failed because the connected party
did not properly respond after a period of time, or established connection failed because connected host ha
s failed to respond.

[darkowlzz]

@godcong hi, this looks like a network error. Seems to work as expected for me. Can you try again?

[ibrasho]

Are you behind the GFW? I remember people having issues with golang.org behind the GFW.

[5dao]

Yes ! GFW problem !

Are you behind the GFW? I remember people having issues with golang.org behind the GFW.

to do :
export http_porxy = your GFW proxy

`
[[constraint]]

name = "golang.org/x/text"

branch = "master"

source = "github.com/golang/text"

`

But there is another problem. @ibrasho @darkowlzz
why not directly download from [github.com/golang/text]
send ( http://golang.org/x/text?go-get=1 ) to get meta data ,should on the back!

[localvar]

I'm behind the GFW and met same problem.

However, in my understanding, source is a solution for this: when dep cannot access golang.org, use github instead.

[localvar]

this issue is really annoying to me, could it be made higher priority?

[sdboyer]

to do :
export http_porxy = your GFW proxy

assuming that it's spelled correctly (http_proxy, or HTTP_PROXY), these should be picked up and used automatically by dep, including by the bit where we look for go-get metadata. that should work on tip, as well as pretty much every released version of dep.

[mspiegel]

Confirmed. On 0.3.0 I can use source as a solution when I'm having issues with my corporate proxy. On 0.3.2 this no longer works.

[ixre]

use ignored label:

 # required = ["github.com/user/thing/cmd/thing"]
 ignored = [
    "git.apache.org/thrift.git/lib/go/thrift",
    "golang.org/x/sys/unix",
 ]

[meilihao]

Me too, source does't work in a92678159ec741a4d28f4b2ea4727de1156ab823.

i use:

[[constraint]]
  branch = "master"
  name = "golang.org/x/crypto"
  source = "https://github.com/golang/crypto.git"

run dep ensure -update golang.org/x/crypto, get:

  ✗ unable to deduce repository and source type for "golang.org/x/net": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://golang.org/x/net?go-get=1": Get http://golang.org/x/net?go-get=1: dial tcp 216.239.37.1:80: i/o timeout

[darkowlzz]

Since you guys can reproduce it, can you git bisect and find out the exact commit where this broke? That would help in finding out what changed that's could be causing this.

[diesang]

I met the same issue with 0.3.2

[godcong]

an example toml
Gopkg.toml
[[constraint]]
name = "golang.org/x/crypto"
branch = "dev"
source = "github.com/golang/crypto"

when use

dep ensure

it will first deduce the golang.org/x/crypto

i logged out on version 0.3.2
golangorgtest>dep ensure:

its run case like this:

/src/github.com/golang/dep/cmd/dep/main.go
->if err := dep.ValidateProjectRoots(ctx, p.Manifest, sm); err != nil

/src/github.com/golang/dep/manifest.go:
->for pr := range m.Constraints {
wg.Add(1)
go validate(pr)
}
->origPR, err := sm.DeduceProjectRoot(string(pr))

/src/github.com/golang/dep/internal/gps/source_manager.go
->pd, err := sm.deduceCoord.deduceRootPath(context.TODO(), ip)

/src/github.com/golang/dep/internal/gps/deduce.go
->return hmd.deduce(ctx, path)

after that deduce will be error:

Signal received: waiting for 1 ops to complete...
D:/Godcong/Workspace/g7n3/src/github.com/golang/dep/manifest.go:174: orig
The following issues were found in Gopkg.toml:

✗ unable to deduce repository and source type for "golang.org/x/crypto": unable to read metadata: unable to fetch raw metadata: failed HTTP request to URL "http://golang.org/x/crypto?go-get=1": Get http://golang.org/x/crypto?go-
get=1: context canceled

but behind the GFW, it has some problems.
i think so

[godcong]

i removed two function calls:
ValidateParams
ValidateProjectRoots
to build the dep file

after that ,when use:

dep ensure

the mirror was download correct...

[n3integration]

FWIW - I am also experiencing the same results where the 0.3.0 release is working as expected with the source override. Whereas, the 0.3.1+ releases fail with:

unable to deduce repository and source type for "golang.org/x/sys": unable to read metadata: go-import metadata not found

As an aside, is there any disadvantage to making this the default behavior (i.e. pointing all golang.org repos to github.com/golang in Gopkg.toml), at least in the case of golang.org/*? It seems to be tripping up a fair amount of users.

@darkowlzz

7a32e011e28c77ff0abed52513b536a39352dd01 is the first bad commit
commit 7a32e011e28c77ff0abed52513b536a39352dd01
Author: Sunny <[email protected]>
Date:   Sun Sep 3 20:49:25 2017 +0530

    fix(manifest): validate project roots in manifest

    Validating ProjectRoot(s) require source manager, which is created after
    loading the project. Hence, ProjectRoot validation can't be done in
    existing validateManifest.

    This change adds `ValidateProjectRoots()` which validates only the Constraint
    and Override names to be valid ProjectRoot. Warnings are issued at stderr when
    invalid ProjectRoot is found in manifest.

    `ensure` and `status` call `ValidateProjectRoots()` expecitly.

:040000 040000 b145004b62e6928ee39441b608d1635f7050f77a 3b4852ec46de875a65a759dac9b22b5f73c51354 M  cmd
:100644 100644 11472a64a51d3dbc2b27020bbfa9d0819f2f090f 74a64da273d4cb83b62d7f55cdc103c67f7c931e M  manifest.go
:100644 100644 c79f5e02b1bef6f905c81c82ff422a44e9532c90 f206db8b822752088005fb7d063650ddaa4b9d27 M  manifest_test.go

[godcong]

@n3integration
you can do some like me
remove two check function
(ValidateParams ValidateProjectRoots) and rebuild dep.
then it will run well like 0.3.0

the ValidateProjectRoots will check golang.org/x/sys first
even if you defined it direct to github.com/golang/sys in gopkg.toml

[darkowlzz]

Can you guys, who can reproduce it, try https://github.com/golang/dep/pull/1527 and see if that fixes the issue?

[n3integration]

@godcong @darkowlzz - confirmed that #1527 resolves the issue on my machine. 🎉

[sdboyer]

The original issue, as posted, is resolved by using HTTP proxy variables. If those aren't working, please file a separate bug. Because that issue is, AFAIK, addressed, i'm closing this issue.

There are some other issues being brought up that are ultimately more about #860, and should be considered there; #1527 could be a helpful step towards that, i'm pondering that independently.

[mspiegel]

I'm not sure why this issue was closed. I'm behind a proxy that allows "github.com" access but not "golang.org" access. Using the source option is supposed to solve this issue but it is not working. I have my HTTP proxy variables configured correctly.

[sdboyer]

The issue is closed because the source option is not _supposed_ to help you work around your local machine's network issues. It's a global setting that potentially pollutes the ecosystem with conflicts when it's used in that way.

The expectation is that If your proxy doesn't allow access to golang.org, then you need a different proxy.

The more proper solution for this is registries. Progress towards that is also happening.

[mspiegel]

Ok that is helpful information. Thank you.

On Mar 1, 2018 11:25 AM, "sam boyer" [e-mail] wrote:

The issue is closed because the source option is not supposed to help you
work around your local machine's network issues. It's a global setting that
potentially pollutes the ecosystem with conflicts when it's used in that
way.

The expectation is that If your proxy doesn't allow access to golang.org,
then you need a different proxy.

The more proper solution for this is registries. Progress towards that is
also happening.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/golang/dep/issues/1322#issuecomment-369647091, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAtnbvOGCegzhCVwiM-xXNpFNL2qcx2ks5taCD4gaJpZM4QIu1K
.

[godcong]

if you have a proxy client
put your proxy on in you cmd:
run set http_proxy=ip:port before use dep ensure.

example(windows):
set http_proxy=127.0.0.1:1235
set https_proxy=127.0.0.1:1235

it can visit golang.org to fix this issus

[bensonfx]

I add code like this.
the mirror rule can use getProjectConstraint to parse,like dep ensure -add

[cdz620]

will it be special env variable like DEP_HTTP_PROXY in future?
now I did below to solve this problem
alias dep='http_proxy=http://me:1087 https_proxy=http://me:1087 dep'
but this is not gorgeous.

[waliguder]

@godcong how to remove two check function(ValidateParams ValidateProjectRoots) and rebuild dep.

I can't find their exact location！

[godcong]

@waliguder fix the function other error will happened.
the best is to set the proxy

[crwgregory]

I had this issue when using dep inside docker containers on windows. Restarting docker fixed the issue. Which leads me to believe it was a network issue.