Velero: Cannot install velero with go get + go install

[georgettica]

What steps did you take and what happened:

$ GO111MODULE=on go get -v -u github.com/vmware-tanzu/velero
go get: github.com/vmware-tanzu/[email protected] requires
        github.com/kubernetes-csi/external-snapshotter/[email protected] requires
        k8s.io/[email protected] requires
        k8s.io/[email protected]: reading k8s.io/api/go.mod at revision v0.0.0: unknown revision v0.0.0

$ cd $(go env GOPATH)/src/github.com/vmware-tanzu/velero
$ go install .
go: github.com/kubernetes-csi/external-snapshotter/[email protected] requires
        k8s.io/[email protected] requires
        k8s.io/[email protected] requires
        vbom.ml/[email protected]: unrecognized import path "vbom.ml/util": https fetch: Get "https://vbom.ml/util?go-get=1": dial tcp: lookup vbom.ml on XXXXXXXX:53: no such host

What did you expect to happen:
that the velero cli will be installed successfully

Environment:
running inside a fedora container with go installed

• OS (e.g. from /etc/os-release):
  > $ cat /etc/os-release
  > NAME=Fedora
  > VERSION="32 (Container Image)"
  > ID=fedora
  > VERSION_ID=32
  > VERSION_CODENAME=""
  > PLATFORM_ID="platform:f32"
  > PRETTY_NAME="Fedora 32 (Container Image)"
  > ANSI_COLOR="0;34"
  > LOGO=fedora-logo-icon
  > CPE_NAME="cpe:/o:fedoraproject:fedora:32"
  > HOME_URL="https://fedoraproject.org/"
  > DOCUMENTATION_URL="https://docs.fedoraproject.org/en-US/fedora/f32/system-administrators-guide/"
  > SUPPORT_URL="https://fedoraproject.org/wiki/Communicating_and_getting_help"
  > BUG_REPORT_URL="https://bugzilla.redhat.com/"
  > REDHAT_BUGZILLA_PRODUCT="Fedora"
  > REDHAT_BUGZILLA_PRODUCT_VERSION=32
  > REDHAT_SUPPORT_PRODUCT="Fedora"
  > REDHAT_SUPPORT_PRODUCT_VERSION=32
  > PRIVACY_POLICY_URL="https://fedoraproject.org/wiki/Legal:PrivacyPolicy"
  > VARIANT="Container Image"
  > VARIANT_ID=container

Vote on this issue!

This is an invitation to the Velero community to vote on issues, you can see the project's top voted issues listed here.
Use the "reaction smiley face" up to the right of this comment to vote.

• :+1: for "I would like to see this bug fixed as soon as possible"
• :-1: for "There are more important bugs to focus on right now"

[nrb]

Thanks for this - I was able to replicate the issue.

In general, our supported delivery methods are via the GitHub releases, and there is a Linux client available.

I'll file this as a bug though, as our dependency tree should resolve correctly via go get.

[nrb]

It looks like we may have to update to external-snapshotter v3 for better Go modules support, but I'm not 100% sure.

[zubron]

It looks like we may have to update to external-snapshotter v3 for better Go modules support, but I'm not 100% sure.

I think this will still be an issue in the new version 😞 v3.0.0 still relies on the Kubernetes repo which doesn't support being used as a library. It looks like the folks working/relying on external-snapshotter have been trying to get the parts that the project relies on to be usable outside of that repo (kubernetes/kubernetes#85692 , kubernetes/kubernetes#85693)

It looks like there are some workarounds which we could propose to that project.

[zubron]

I looked at this a bit more, and the external-snapshotter project actually created a separate go module for the client and APIs which _doesn't_ have the dependency on k/k and using it fixes the problem here. This new go module is available in v2.2.0-rc3, so we wouldn't have to upgrade the API types to v3 _but_ updating to that newer RC means that we pull in v0.19.0-rc2 versions of some of the Kubernetes modules. v3 of the module uses the GA release of the Kubernetes modules: v0.19.0.

Either way, to fix this we're going to have to do some upgrading of the Kubernetes modules we're using.

[georgettica]

Thatks all ( that includes you @zubron ) for pushing this issue forward

I agree @nrb that this should be done via the normal releases and still it's a good self check to see all the modules are in order.

If you need 🙌 for this, would gladly see what I can do to make this a reality

[nrb]

@georgettica Thanks!

I think we'll need to move this to v1.5.3 or v1.6.0 because v1.5.2 is going to get cut this week.

[pires]

Well, this doesn't affect installing Velero CLI alone but also any plug-ins that import Velero code. Maybe I'll just wait before polluting my go.mod w/ something like this.

[nrb]

@pires Unfortunately, those requires directives are for Kubernetes, not Velero. Every project importing Kubernetes needs it :(

[carlisia]

Hey @georgettica, do you have bandwidth to tackle this for our v1.6 release?

[georgettica]

I will do my best and will consult when necessary.
Sign me up!

[lmdkfs]

i have the same problem， when i execute go get github.com/vmware-tanzu/velero/pkg/plugin/framework, the error is reported as follows:

go: found github.com/vmware-tanzu/velero/pkg/plugin/framework in github.com/vmware-tanzu/velero v1.5.2
go get: github.com/vmware-tanzu/[email protected] requires
  github.com/kubernetes-csi/external-snapshotter/[email protected] requires
  k8s.io/[email protected] requires
  k8s.io/[email protected]: reading k8s.io/api/go.mod at revision v0.0.0: unknown revision v0.0.0

How to solve this problem before 1.6 is released

[georgettica]

I am having trouble fixing the issue, tried to tackle from multiple angles and to no avail :( I will upload a PR with a failing test and I hope you could guide me to a suceeding build

[brito-rafa]

+1 while trying to compile a velero plugin:

$ go mod tidy
go: finding module for package k8s.io/apimachinery/pkg/runtime/schema
go: finding module for package github.com/openshift/api/route/v1
go: finding module for package k8s.io/client-go/tools/clientcmd
go: finding module for package k8s.io/api/core/v1
go: finding module for package k8s.io/client-go/kubernetes
go: finding module for package github.com/brito-rafa/k8s-mutators/pkg/route2httpproxy
go: finding module for package k8s.io/client-go/dynamic
go: finding module for package k8s.io/apimachinery/pkg/apis/meta/v1
go: finding module for package github.com/projectcontour/contour/apis/projectcontour/v1
go: finding module for package github.com/vmware-tanzu/velero/pkg/plugin/velero
go: finding module for package github.com/sirupsen/logrus
go: finding module for package github.com/vmware-tanzu/velero/pkg/plugin/framework
go: finding module for package k8s.io/apimachinery/pkg/apis/meta/v1/unstructured
go: found github.com/sirupsen/logrus in github.com/sirupsen/logrus v1.7.0
go: found github.com/vmware-tanzu/velero/pkg/plugin/framework in github.com/vmware-tanzu/velero v1.5.3
go: found github.com/brito-rafa/k8s-mutators/pkg/route2httpproxy in github.com/brito-rafa/k8s-mutators v0.2.0
go: found github.com/openshift/api/route/v1 in github.com/openshift/api v3.9.0+incompatible
go: found github.com/projectcontour/contour/apis/projectcontour/v1 in github.com/projectcontour/contour v1.11.0
go: found k8s.io/api/core/v1 in k8s.io/api v0.20.2
go: found k8s.io/apimachinery/pkg/apis/meta/v1 in k8s.io/apimachinery v0.20.2
go: found k8s.io/client-go/dynamic in k8s.io/client-go v11.0.0+incompatible
go: github.com/vmware-tanzu/[email protected] requires
        github.com/kubernetes-csi/external-snapshotter/[email protected] requires
        k8s.io/[email protected] requires
        k8s.io/[email protected]: reading k8s.io/api/go.mod at revision v0.0.0: unknown revision v0.0.0

[georgettica]

@brito-rafa it's still being worked on in #3202
you can track progress there :)

[arianitu]

I'm running into this, is there any workarounds to make it compile with v1.5.x? @georgettica

[georgettica]

Once my PR is merged yes, but so far I am not done on it so it will take a while longer.

If someone wiser than me has a quick fix, feel free to.speak up

[brito-rafa]

I used the following workaround on my go.mod (as pointed in a previous comment by @pires ):
https://github.com/vmware-tanzu/velero-plugin-for-vsphere/blob/master/go.mod#L28-L68

I added all these "replace" directives as specified on this file.

[arianitu]

@brito-rafa thank you, that worked perfectly! I'll remove those when it's fixed upstream. Cheers.