Community: Update e2e testing guide

Created on 6 Oct 2018  ·  21Comments  ·  Source: kubernetes/community

The doc https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests.md is very stale and lots of portions need a re-write.

Ref: https://github.com/kubernetes/community/pull/2701#discussion_r219757973

/cc @BenTheElder @spiffxp @cjwagner

/sig testing
/area contributor-guide
/kind bug

arecontributor-guide aredeveloper-guide help wanted kinbug kindocumentation lifecyclfrozen prioritimportant-longterm sicontributor-experience sitesting
Source
picture nikhita
👍3

All 21 comments

/milestone 2019
/cc

also contributors/devel/e2e-node-tests.md here
probably will do after a kubetest overhaul

krzyzacy picture krzyzacy on 15 Oct 2018
👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 13 Jan 2019

/remove-lifecycle stale

On Mon 14 Jan, 2019, 12:28 AM fejta-bot, notifications@github.com wrote:

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually
close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta
https://github.com/fejta.
/lifecycle stale

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/community/issues/2759#issuecomment-453856010,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APXA0OVhu8AMmfcXMlvQg0D1B7t9EXW2ks5vC4HsgaJpZM4XLd3X
.

nikhita picture nikhita on 13 Jan 2019

/kind documentation

nikhita picture nikhita on 18 Mar 2019

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 26 Nov 2019

/assign

vishakhanihore picture vishakhanihore on 18 Dec 2019

/remove-lifecycle stale

vishakhanihore picture vishakhanihore on 18 Dec 2019

/unassign @markyjackson-taulia now that @vishakhanihore is working on this

markyjackson-taulia picture markyjackson-taulia on 4 Jan 2020

I was trying to follow the instructions to install kubetest but it failed, being unable to find some of its dependencies : 

➜ go get -u k8s.io/test-infra/kubetest
package github.com/aws/aws-k8s-tester/ekstester: cannot find package "github.com/aws/aws-k8s-tester/ekstester" in any of:
    /usr/local/go/src/github.com/aws/aws-k8s-tester/ekstester (from $GOROOT)
    /Users/ale/code/go/src/github.com/aws/aws-k8s-tester/ekstester (from $GOPATH)
package k8s.io/client-go/vendor/github.com/golang/protobuf/proto: cannot find package "k8s.io/client-go/vendor/github.com/golang/protobuf/proto" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/github.com/golang/protobuf/proto (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/github.com/golang/protobuf/proto (from $GOPATH)
package k8s.io/client-go/vendor/github.com/googleapis/gnostic/OpenAPIv2: cannot find package "k8s.io/client-go/vendor/github.com/googleapis/gnostic/OpenAPIv2" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/github.com/googleapis/gnostic/OpenAPIv2 (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/github.com/googleapis/gnostic/OpenAPIv2 (from $GOPATH)
package k8s.io/client-go/vendor/github.com/davecgh/go-spew/spew: cannot find package "k8s.io/client-go/vendor/github.com/davecgh/go-spew/spew" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/github.com/davecgh/go-spew/spew (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/github.com/davecgh/go-spew/spew (from $GOPATH)
package k8s.io/client-go/vendor/golang.org/x/crypto/ssh/terminal: cannot find package "k8s.io/client-go/vendor/golang.org/x/crypto/ssh/terminal" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/golang.org/x/crypto/ssh/terminal (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/golang.org/x/crypto/ssh/terminal (from $GOPATH)
package k8s.io/client-go/vendor/golang.org/x/oauth2: cannot find package "k8s.io/client-go/vendor/golang.org/x/oauth2" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/golang.org/x/oauth2 (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/golang.org/x/oauth2 (from $GOPATH)
package k8s.io/client-go/vendor/golang.org/x/time/rate: cannot find package "k8s.io/client-go/vendor/golang.org/x/time/rate" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/golang.org/x/time/rate (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/golang.org/x/time/rate (from $GOPATH)
package k8s.io/client-go/vendor/github.com/imdario/mergo: cannot find package "k8s.io/client-go/vendor/github.com/imdario/mergo" in any of:
    /usr/local/go/src/k8s.io/client-go/vendor/github.com/imdario/mergo (from $GOROOT)
    /Users/ale/code/go/src/k8s.io/client-go/vendor/github.com/imdario/mergo (from $GOPATH)
alex-leonhardt picture alex-leonhardt on 26 Jan 2020

It looks like there are broken dependencies:

$ go get -u k8s.io/test-infra/kubetest
package github.com/aws/aws-k8s-tester/ekstester: cannot find package "github.com/aws/aws-k8s-tester/ekstester" in any of:
/usr/local/go/src/github.com/aws/aws-k8s-tester/ekstester (from $GOROOT)
/home/me/go/src/github.com/aws/aws-k8s-tester/ekstester (from $GOPATH)

Taking a look at the aws-k8s-tester repo, there is no file or directory called ekstester

Any advice?

go: extracting github.com/mattn/go-ieproxy v0.0.0-20200121124216-6d733ce01d10
build k8s.io/test-infra/kubetest: cannot load github.com/aws/aws-k8s-tester/ekstester: git fetch --unshallow -f origin in /home/me/go/pkg/mod/cache/vcs/b1ddf91d095fd86fbe3ddfb9b2f554ce75ef6fe70ad2e3fedeaa08b431c2ac40: exit status 128:
fatal: git fetch-pack: expected shallow list

casibbald picture casibbald on 28 Jan 2020

@vishakhanihore vishakhanihore Please may you kindly advise if this is something you have started looking at?

casibbald picture casibbald on 30 Jan 2020

/assign @hh

mrbobbytables picture mrbobbytables on 22 Apr 2020

hello/

i ran into the issues with the installation instructions today and created a pr to fix that part. not sure what else is stale, but this should help.

elmiko picture elmiko on 5 Jun 2020

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 3 Sep 2020

/remove-lifecycle stale

I believe this is being worked on 👍

mrbobbytables picture mrbobbytables on 3 Sep 2020

@BenTheElder @spiffxp @stevekuznetsov
This has been open for awhile, what do you think? Is the e2e testing docs up to date enough to call this good?

mrbobbytables picture mrbobbytables on 7 Oct 2020

They are not up to date enough IMO, I would happily review any changes to them. My bandwidth for updating them myself is low

spiffxp picture spiffxp on 13 Oct 2020

At a glance, a few things I would recommend:

  • drop references to ci-test.k8s.io and pr-test.k8s.io, instead refer to:

    • prow.k8s.io

    • testgrid.k8s.io

    • prow.k8s.io/an/example/spyglass/path/for/a/specific/ci-job

    • prow.k8s.io/an/example/spyglass/path/for/a/specific/pr-job

  • instructions on how to get kubetest (IIRC it can't be installed via go get)
  • rewrite kinds of tests to instead be test tags, include test tags that aren't yet documented ([DisabledForLargeClusters] is one I'm aware of off the top of my head)
  • I might recommend using kind to run e2e tests (and examples to do so), that may already exist elsewhere @BenTheElder ?
  • the "performance evaluation" content may be stale, I feel like that should move to or reference scalability docs with a link
spiffxp picture spiffxp on 13 Oct 2020

@pymander is doing great work on https://github.com/kubernetes/community/blob/master/contributors/devel/sig-testing/e2e-tests-kubetest2.md
I share information with him on the points @spiffxp mentioned above to add to the document.

Riaankl picture Riaankl on 23 Oct 2020
👍1

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot picture fejta-bot on 20 Jan 2021

/remove-lifecycle stale
/lifecycle frozen
/priority important-longterm

spiffxp picture spiffxp on 9 Feb 2021
Was this page helpful?
0 / 5 - 0 ratings

Related issues

embano1 picture embano1  ·  4Comments
dddd45 picture dddd45  ·  4Comments
dims picture dims  ·  4Comments
bboreham picture bboreham  ·  4Comments
spiffxp picture spiffxp  ·  5Comments