Test-infra: prow: reply to PR with pass/fail status of merged postsubmit

[cblecker]

Post-submit jobs that are tied to a PR should be able to post the results of the job back to the PR.

We do something similar for the updateconfig plugin, but job status (pass/fail) and a link to the job log should be something all postsubmits can do.

/kind feature

[krzyzacy]

/assign

[aszecowka]

Do you plan to provide retest functionality for postsubmit jobs from PR view?

[BenTheElder]

/cc
do want this feature!

[alvaroaleman]

@krzyzacy Are you currently working on this?

[krzyzacy]

yes, I'm trying to migrate reporter to crier first. https://github.com/kubernetes/test-infra/pull/10675 and https://github.com/kubernetes/test-infra/pull/10818 are moving towards that. Once that's done technically we only need to enable Postsubmit to https://github.com/kubernetes/test-infra/blob/master/prow/github/reporter/reporter.go#L55 - will make a follow up PR in the week. (Friday afternoon? 😏 )

[krzyzacy]

Current status:

I added the report field for postsubmit jobs (not reporting by default), and found out the report lib need to be updated in order to update status on a BaseSHA instead of PullSHA.

Once it works I'll post the announcement and we have plan to switch on report for all postsubmit jobs and unify the report field in config with presubmit jobs.

[krzyzacy]

After https://github.com/kubernetes/test-infra/pull/11168, we will have postsubmit setting status context by default for all postsubmit jobs.

Currently the knative-postsubmit worked for test-infra repo.

Will implement comment on a commit & respond to trigger from a commit in the near future.

(It's tied to each merge commit, instead back to PR itself, since postsubmit is triggered upon pushevents, and not necessarily all push will have a corresponding PR)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[cblecker]

/remove-lifecycle stale

Would still love to see a comment at some point.

[krzyzacy]

/unassign
/help

not actively working on this, posting a comment is trivial but we probably also want to handle retrigger a postsubmit job.

[k8s-ci-robot]

@krzyzacy:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/unassign
/help

not actively working on this, posting a comment is trivial but we probably also want to handle retrigger a postsubmit job.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[cblecker]

/remove-lifecycle stale

[stevekuznetsov]

We only need the comment on a failure, right?

[krzyzacy]

I think @fejta also wanted a trigger, or else we can just let it make the same comment as presubmit

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[cblecker]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[cblecker]

/remove-lifecycle rotten

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[cblecker]

/reopen
/remove-lifecycle rotten

[k8s-ci-robot]

@cblecker: Reopened this issue.

In response to this:

/reopen
/remove-lifecycle rotten

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[spiffxp]

Reporting status contexts to merge commits is a good start, but I would still rather see comments on the PR(s) in question for success or failure. Specific use cases in mind:

• tell me when the PR that merged caused something to deploy
• tell me when the PR that merged caused something to get pushed

This would require at least some changes to crier's github reporter, which currently:

• always adds a status context (we might want this to be configurable)
• always uses a hardcoded commenting scheme (we might want report-on-success on a per-job basis, or we might want a per-job report template)