Operator-sdk: Makefile should not need to be modified to get tests to run

Created on 7 Aug 2020  路  12Comments  路  Source: operator-framework/operator-sdk

Bug Report

A user should not have to update their Makefile after scaffolding an operator to get make test to work.

What did you do?

mkdir zeus-operator
cd zeus-operator
operator-sdk init --repo=github.com/jmrodri/zeus-operator
operator-sdk create api --group test --version v1 --kind Zeus --controller --resource
make test

What did you expect to see?
I expected make test to run the tests and if it needs other binaries to install them.

What did you see instead? Under which circumstances?
After running make test it spewed tons of errors:
```/home/devel/jesusr/dev/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
/home/devel/jesusr/dev/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
go test ./... -coverprofile cover.out
? github.com/jmrodri/zeus-operator [no test files]
? github.com/jmrodri/zeus-operator/api/v1 [no test files]

Running Suite: Controller Suite

Random Seed: 1596830400
Will run 0 of 0 specs

STEP: bootstrapping test environment
2020-08-07T16:00:00.616-0400 DEBUG controller-runtime.test-env starting control plane {"api server flags": []}
2020-08-07T16:00:00.619-0400 ERROR controller-runtime.test-env unable to start the controlplane {"tries": 0, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file
or directory"}
github.com/go-logr/zapr.(zapLogger).Error
/home/devel/jesusr/dev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).startControlPlane
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:279
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).Start
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:241
github.com/jmrodri/zeus-operator/controllers.glob..func1
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:61
reflect.Value.call
/usr/lib/golang/src/reflect/value.go:460
reflect.Value.Call
/usr/lib/golang/src/reflect/value.go:321
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(runner).runAsync.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-08-07T16:00:00.620-0400 ERROR controller-runtime.test-env unable to start the controlplane {"tries": 1, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file
or directory"}
github.com/go-logr/zapr.(zapLogger).Error
/home/devel/jesusr/dev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).startControlPlane
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:279
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).Start
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:241
github.com/jmrodri/zeus-operator/controllers.glob..func1
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:61
reflect.Value.call
/usr/lib/golang/src/reflect/value.go:460
reflect.Value.Call
/usr/lib/golang/src/reflect/value.go:321
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(runner).runAsync.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-08-07T16:00:00.620-0400 ERROR controller-runtime.test-env unable to start the controlplane {"tries": 2, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file
or directory"}
github.com/go-logr/zapr.(zapLogger).Error
/home/devel/jesusr/dev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).startControlPlane
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:279
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).Start
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:241
github.com/jmrodri/zeus-operator/controllers.glob..func1
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:61
reflect.Value.call
/usr/lib/golang/src/reflect/value.go:460
reflect.Value.Call
/usr/lib/golang/src/reflect/value.go:321
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(runner).runAsync.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-08-07T16:00:00.621-0400 ERROR controller-runtime.test-env unable to start the controlplane {"tries": 3, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file
or directory"}
github.com/go-logr/zapr.(zapLogger).Error
/home/devel/jesusr/dev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).startControlPlane
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:279
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).Start
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:241
github.com/jmrodri/zeus-operator/controllers.glob..func1
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:61
reflect.Value.call
/usr/lib/golang/src/reflect/value.go:460
reflect.Value.Call
/usr/lib/golang/src/reflect/value.go:321
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(runner).runAsync.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
2020-08-07T16:00:00.621-0400 ERROR controller-runtime.test-env unable to start the controlplane {"tries": 4, "error": "fork/exec /usr/local/kubebuilder/bin/etcd: no such file
or directory"}
github.com/go-logr/zapr.(zapLogger).Error
/home/devel/jesusr/dev/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).startControlPlane
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:279
sigs.k8s.io/controller-runtime/pkg/envtest.(Environment).Start
/home/devel/jesusr/dev/pkg/mod/sigs.k8s.io/[email protected]/pkg/envtest/server.go:241
github.com/jmrodri/zeus-operator/controllers.glob..func1
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:61
reflect.Value.call
/usr/lib/golang/src/reflect/value.go:460
reflect.Value.Call
/usr/lib/golang/src/reflect/value.go:321
github.com/onsi/ginkgo/internal/leafnodes.newRunner.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:49
github.com/onsi/ginkgo/internal/leafnodes.(runner).runAsync.func1
/home/devel/jesusr/dev/pkg/mod/github.com/onsi/[email protected]/internal/leafnodes/runner.go:86
Failure [0.005 seconds]
[BeforeSuite] BeforeSuite
/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:52

Unexpected error:
<*fmt.wrapError | 0xc00000f4a0>: {
msg: "failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory",
err: {
Op: "fork/exec",
Path: "/usr/local/kubebuilder/bin/etcd",
Err: 0x2,
},
}
failed to start the controlplane. retried 5 times: fork/exec /usr/local/kubebuilder/bin/etcd: no such file or directory
occurred

/home/devel/jesusr/dev/zeus-operator/controllers/suite_test.go:62

Ran 0 of 0 Specs in 0.006 seconds
FAIL! -- 0 Passed | 0 Failed | 0 Pending | 0 Skipped
--- FAIL: TestAPIs (0.01s)
FAIL
coverage: 0.0% of statements
FAIL github.com/jmrodri/zeus-operator/controllers 0.027s
FAIL
make: * [Makefile:32: test] Error 1

**Environment**
* operator-sdk version:

$ operator-sdk version
operator-sdk version: "v1.0.0-alpha.2-16-g8bca0209", commit: "8bca020931136d196ae5f4135c1d70738a501135", kubernetes version: "v1.18.2", go version: "go1.13.11 linux/amd64", GOOS: "linux", GOARCH: "amd64"

* go version:

go version go1.13.11 linux/amd64

* Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.0", GitCommit:"91e7b4fd31fcd3d5f436da26c980becec37ceefe", GitTreeState:"clean", BuildDate:"2018-06-27T20:17:28Z", GoVersion:"go1.10.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-30T20:19:45Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

* Kubernetes cluster kind:

$ kind version
kind v0.8.1 go1.13.11 linux/amd64

* Are you writing your operator in ansible, helm, or go?
Go

**Possible Solution**
If the `test` target requires the user to modify it to work, why can't the scaffolder generate a `Makefile` with the proper `test` target as outlined below.

**Additional context**
This issue apparently was brought up in Issue #3461 because kubebuilder executables are required for the tests. They are still not in the Prerequisites: [Golang Based Operator SDK Installation](https://master.sdk.operatorframework.io/docs/building-operators/golang/installation/).  The solution suggested in the [documentation](https://master.sdk.operatorframework.io/docs/building-operators/golang/references/envtest-setup/), is for the user to modify the newly scaffolded `Makefile` with the following:

Run tests

ENVTEST_ASSETS_DIR=$(shell pwd)/testbin
test: generate fmt vet manifests
mkdir -p ${ENVTEST_ASSETS_DIR}
test -f ${ENVTEST_ASSETS_DIR}/setup-envtest.sh || curl -sSLo ${ENVTEST_ASSETS_DIR}/setup-envtest.sh https://raw.githubusercontent.com/kubernetes-sigs/controller-runtime/master/hack/setup-envtest.sh
source ${ENVTEST_ASSETS_DIR}/setup-envtest.sh; fetch_envtest_tools $(ENVTEST_ASSETS_DIR); setup_envtest_env $(ENVTEST_ASSETS_DIR); go test ./... -coverprofile cover.out
```

languaggo prioritimportant-soon
Source
picture jmrodri

Most helpful comment

It will be solved when we are able to merge; https://github.com/kubernetes-sigs/kubebuilder/pull/1626 and support the go/v3-alpha plugin from kubebuilder. Needs to be post 1.0.

picture camilamacedo86 on 7 Aug 2020
👍2

All 12 comments

haha, I just hit this same error! The docs are misleading to me, the part about 'setting up your test environment' is something I skipped because I really just wanted to build my operator not test it, however, that test step actually provides the missing dependencies required to do a build.

jmccormick2001 picture jmccormick2001 on 7 Aug 2020
👍2

It will be solved when we are able to merge; https://github.com/kubernetes-sigs/kubebuilder/pull/1626 and support the go/v3-alpha plugin from kubebuilder. Needs to be post 1.0.

camilamacedo86 picture camilamacedo86 on 7 Aug 2020
👍2

Yes, I don't understand the situation here.
I have just created a new project with operator-sdk v1.0.0, then all I did was copy the bit of Makefile from the documentation into my makefile and the result worked. I did nothing else. I didn't understand the linked documentation section "Setup the envtest binaries and environment". The Makefile section it tells you to add downloads the envtest stuff if it's not present.

tomqwpl picture tomqwpl on 7 Sep 2020

There are 3 possible solutions:

  1. Update the quickstart documentation to ensure folks always add it by hand: https://sdk.operatorframework.io/docs/building-operators/golang/quickstart/
  2. Update our scaffolding code to add it automatically to the generated Makefile
  3. Update the Makefile so that the docker-build target does not depend on the test target.
jmrodri picture jmrodri on 30 Sep 2020

Consensus on the grooming call was to do option 2.

jmrodri picture jmrodri on 30 Sep 2020

It is great news @jmrodri. Also, I think the best option is 2.

camilamacedo86 picture camilamacedo86 on 1 Oct 2020

@camilamacedo86 Doesn't your comment indicate we get this for free because it's been fixed upstream? We just need to bump the version?

jberkhahn picture jberkhahn on 1 Oct 2020

waiting for 1711 in kubebuilder to be merged before we bump our dependency on it

jberkhahn picture jberkhahn on 7 Oct 2020
👍1

We're going to merge #3983 and wait on the upstream changes to fix it for good.

jberkhahn picture jberkhahn on 12 Oct 2020

I'd like to keep this issue open to track the issue upstream. Will move it into the next release and take off the blocker tag.

jberkhahn picture jberkhahn on 15 Oct 2020

It was

camilamacedo86 picture camilamacedo86 on 15 Oct 2020

In SDK, this problem is solved with the PR #3983 and it was cherry-picked to v1.0.x with https://github.com/operator-framework/operator-sdk/pull/4038.

Also, the same solution is merged already for the v3+ plugins in upstream and a better approach has been discussed and might be pushed to v2+ as well. More info: https://github.com/kubernetes-sigs/kubebuilder/pull/1711.

In this way, we can consider it solved.

camilamacedo86 picture camilamacedo86 on 15 Oct 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

kristiandrucker picture kristiandrucker  路  5Comments
hasbro17 picture hasbro17  路  3Comments
surajssd picture surajssd  路  3Comments
linuxbsdfreak picture linuxbsdfreak  路  4Comments
ricardozanini picture ricardozanini  路  4Comments