Community: Add a link checker for this repo

[pmorie]

While looking at the API conventions docs this morning, I found that many of the links in that doc were broken. We should add a link checker for this repo (and eventually other repos in the community).

[pmorie]

@duglin any chance you might be interested in doing this?

[chupman]

I'd like to work on this. I've looked at verify-links.sh script that @duglin wrote, but I'm not totally clear on how you want the link checking implemented. Are you wanting something like a scheduled task to regularly scan for dead links? Or just something to manually run from time to time?

[duglin]

@chupman it should run as part of the verification checks that are part of the normal kube build process. So basically it should run on every PR. Similar to what is done for lint and gofmt checks.

[chupman]

I'm most likely overthinking it, but my concern was that if you run on every PR it could cause unrelated errors if an existing link goes dead. Would it make sense to expect an extra arg with the files changed in addition to the repo base dir? Or is it better to just leave it simple and just scan the whole repo and let the reviewer read the error log?

[duglin]

The purpose behind running the test is exact what you said... to find dead links. So we want things to fail if there area dead links. I suspect the first time we turn this on we'll find a TON and it'll result in the PR to add this to also include a ton of md file fixes. :-)

Now if you're concerned about having to check too many files and it being slow, we should wrapper the call to the checker with code that only calls it for files changed in the current PR/commit.

[duglin]

For reference, here the repo with the link checker: https://github.com/duglin/vlinker

[jamiehannaford]

If nobody else is already working on this, I'd like to help out.

@duglin are there any docs about how to set up CI on a new repo? I'm not sure what the conventions are - e.g. is it okay to use Travis or does everything need to go through the existing https://github.com/kubernetes/test-infra?

[jamiehannaford]

Also, instead of writing a new script, perhaps we could use or refactor https://github.com/kubernetes/kubernetes/blob/master/cmd/mungedocs/links.go and run all the mung scripts on a PR?

[chupman]

I'm currently waiting on code review from @duglin for some enhancements on his link checker. Despite the fact that I've already invested some time in this I agree that if there's something out there that's already maintained it's probably a better option. In case we do go with vlinker I'd also like to know about the CI integration. I was using travis on my own forks for testing while I was whittling down false positives.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

[cblecker]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[jamiehannaford]

/remove-lifecycle stale

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[castrojo]

/remove-lifecycle rotten

[cblecker]

I came across this in my OSS travels: https://github.com/hashicorp/terraform-website/blob/master/content/scripts/check-links.sh

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[idealhack]

/remove-lifecycle stale

[nikhita]

xref the "404 Hunter for kubernetes community" role on the role board: https://discuss.kubernetes.io/t/404-hunter-for-kubernetes-community/2629

[spiffxp]

https://github.com/kubernetes/kubernetes/pull/70052#discussion_r239686687
https://github.com/kubernetes/kubernetes/pull/70052#issuecomment-445112134

Turns out we _have_ a linkchecker in kubernetes/kubernetes: https://github.com/kubernetes/kubernetes/tree/master/cmd/linkcheck

It's just been broken and unused for two years. It was in use before we heavily leaned on kubernetes/website and kubernetes/community the way we do today. I would suggest we break that code out and see what can be done with it.

k/test-infra can be a home for it if there's no more appropriate place for it

[spiffxp]

/help
/kind cleanup

[chupman]

Hey @spiffxp. If next week I were to dump linkcheck in test-infra and get it running for external links and possibly also get it to check relative links within github (if wanted) would that be enough to close out this issue?

I don't have any exposure to bazel or how the CI is setup so I'll need some help there, but I've already researched how the link checker should work. Plus if it's apache licensed I can use it in JanusGraph, and it'll save my inbox an email every 90 days.

[chupman]

PR is in for the link check, but a separate PR will be needed in this repo for the wrapper script. Changes will need to be made to specify which directories should be scanned or skipped.

[chupman]

Hi @spiffxp I got the PR in (#3324) to actually add in the link checker to CI. I'm hoping someone can let me know where to save the output from the command. Once I update that it should be ready to test. We'll probably need a combination of dead link fixes and whitelisting before it can actually be merged though.

[nikhita]

/priority important-longterm
/lifecycle active

[nikhita]

/remove-help

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[nikhita]

/remove-lifecycle stale

On Tue, Jun 18, 2019 at 6:45 PM fejta-bot [e-mail] wrote:

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually
close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta
https://github.com/fejta.
/lifecycle stale

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/kubernetes/community/issues/359?email_source=notifications&email_token=AD24BUH2EL352FRQLC5UQQ3P3DNXVA5CNFSM4C7Y5L4KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODX6SQLY#issuecomment-503130159,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AD24BUAQUMEKT2F6QL3FH5LP3DNXVANCNFSM4C7Y5L4A
.

[spiffxp]

/assign
As I'm helping review PR from @chupman

[spiffxp]

/unassign
/help
@chupman wasn't able to complete https://github.com/kubernetes/community/pull/3324

[k8s-ci-robot]

@spiffxp:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/unassign
/help
@chupman wasn't able to complete https://github.com/kubernetes/community/pull/3324

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.