Go: x/pkgsite: automatically fetch a requested package if it does not exist on pkg.go.dev

[natefinch]

Currently, if you request package docs from pkg.go.dev for a package that hasn't been sent to the go proxy yet, then it just returns 404.

Requesting a url from pkg.go.dev should do whatever "add your package" does, if that's required to view the docs, because clearly I want to view the docs and the server knows exactly what step is missing to allow me to view the docs, so just do it, please.

By returning a 404 page, it looks like there's a bug in the website. The suggested "how to add your package" requires some arcane steps (I presume? it's not actually spelled out) that simply shouldn't be necessary.

See https://github.com/golang/go/issues/36986 for an example of someone reporting a "bug" because a package didn't show up.

This is actually super common when using godoc.org in my experience, as I will often want to view the docs on godoc.org as I'm writing the code, to see how they look there. I'm sure others will want to do the same on pkg.go.dev.

This can also easily happen if most people in your company use a corporate proxy, so the code isn't in go's proxy yet, but then you decide you want to view the docs and you're just used to doing it on pkg.go.dev (which I do all the time for random go packages on godoc.org... rather than download them and view the docs locally, I view 99% of docs directly on godoc.org).

[hunterloftis]

I remember the "magic" of having my open-source projects auto-doc'd for me. Someone had to point out to me that I no longer needed to build my own markdown-generating toolchain for docs - instead, they would "just exist" and even be searchable, alongside major, important packages.

I believe this suggestion aligns with that beginner-friendly "it just works" philosophy that has long been a part of the language and community.

[ghost]

If immediate fetch is not possible for some reason, then add a "request documentation" button to the 404 page. Clicking the button kicks of whatever needs to be done and shows UI explaining when the documentation will be available.

[deeper-x]

I'm joining this thread because I consider this issue a detrimental change in the language.
The way godoc.org works is how things should be, as Andrew Gerrand says

[...] it should be coupled to the code itself so the documentation evolves along with the code. The easier it is for programmers to produce good documentation, the better for everyone

Plain and simple: in Go, code & documentation are tightly coupled by default. No surprises expected, because publishing your code you're publishing the documentation on godoc.org as well.

What you quietly introduced with pkg.go.dev and mention there breaks this natural flow introducing a weird exception, silently assuming that now could exist public code-without-documentation.
Furthermore, suggesting a workaround like forcing a fake download in order to trigger the documentation generation, cannot be considered a stable solution.

[gopherbot]

Change https://golang.org/cl/240457 mentions this issue: internal/frontend: add fetch endpoint to robots.txt

[gopherbot]

Change https://golang.org/cl/240459 mentions this issue: content: update frontend fetch UI

[gopherbot]

Change https://golang.org/cl/242357 mentions this issue: internal/fetch: populate go_mod_path for standard library

[gopherbot]

Change https://golang.org/cl/242358 mentions this issue: internal/frontend: fix typo in logic for when to fetch path@master

[gopherbot]

Change https://golang.org/cl/242437 mentions this issue: migrations: set not-null constraint on version_map.go_mod_path

[gopherbot]

Change https://golang.org/cl/242458 mentions this issue: internal/frontend: return 303 for alternative modules

[gopherbot]

Change https://golang.org/cl/242921 mentions this issue: internal/frontend: update response text for fetch endpoint

[gopherbot]

Change https://golang.org/cl/242923 mentions this issue: internal/frontend: increase pollEvery to 1 second

[gopherbot]

Change https://golang.org/cl/242922 mentions this issue: content: update search page for no results

[gopherbot]

Change https://golang.org/cl/244543 mentions this issue: internal/frontend: update response text for fetch timeout

[gopherbot]

Change https://golang.org/cl/244601 mentions this issue: internal/frontend: refetch paths that previously 404ed

[gopherbot]

Change https://golang.org/cl/244698 mentions this issue: internal/frontend: display path result if previously fetched

[gopherbot]

Change https://golang.org/cl/244600 mentions this issue: internal/postgres: fetch version_map.updated_at

[gopherbot]

Change https://golang.org/cl/245121 mentions this issue: internal/frontend: fix timeout error from checkForPath

[gopherbot]

Change https://golang.org/cl/245021 mentions this issue: internal/frontend: delete s.fetchModule

[gopherbot]

Change https://golang.org/cl/245020 mentions this issue: internal/frontend: use StatusNotFoundInVersionMap for frontend fetch

[gopherbot]

Change https://golang.org/cl/245437 mentions this issue: content: alphabetize classes with prefix Fetch

[gopherbot]

Change https://golang.org/cl/245402 mentions this issue: internal/worker: don't delete 40x results from version_map

[gopherbot]

Change https://golang.org/cl/245401 mentions this issue: content: center text in h3.FetchMessage

[gopherbot]

Change https://golang.org/cl/245640 mentions this issue: internal/frontend: tweak response text for fetch requests

[gopherbot]

Change https://golang.org/cl/245646 mentions this issue: content: update text in fetch.tmpl for 404s

[gopherbot]

Change https://golang.org/cl/245880 mentions this issue: internal/fronted: update text in fetch.tmpl

[gopherbot]

Change https://golang.org/cl/245878 mentions this issue: internal/frontend: return 404 on invalid stdlib paths

[gopherbot]

Change https://golang.org/cl/245897 mentions this issue: internal/frontend: improve validation of candidate module paths for fetch

[julieqiu]

Users can now request a package by clicking a button on the 404 page.

Here's an example: