go.dev: open source pkg.go.dev

[marwan-at-work]

Hi there,
I know this topic must have come up a lot but I haven't found an open issue for it.

Will pkg.go.dev ever be open sourced?

It would be incredibly useful for companies running a documentation server internally especially if pkg.go.dev spoke the GOPROXY download protocol and a company had an internal GOPROXY as well.

Thanks!

[tv42]

I'm amazed I can't quickly find the source. I want to know what the exact criteria passed to the license checker is, to get my project above whatever detection threshold you use.

Frankly, this reflects poorly on the Go project.

[mvdan]

I also think it would be depressing if godoc.org starts redirecting to pkg.go.dev, a site that's entirely closed source. The Go project has been built on open source since the very early days, so this seems to go directly against its history. The forced switch would be a significant downgrade.

It's also discouraging that https://go.dev talks so much about the power of open source, but the first major product that comes from it is a black box.

[qaisjp]

I'm amazed I can't quickly find the source. I want to know what the exact criteria passed to the license checker is, to get my project above whatever detection threshold you use.

Frankly, this reflects poorly on the Go project.

I want go.dev and pkg.go.dev to be open source too. But the bit that you're asking for is already open source: https://github.com/google/licensecheck

[tv42]

@qaisjp licensecheck lists probabilities, the yes/no decision criteria is hidden in pkg.go.dev.

[rsc]

This general topic also came up on golang-dev over the weekend, and I wrote a long reply there:
https://groups.google.com/d/msg/golang-dev/mfiPCtJ1BGU/qtCrqlrEEwAJ

It would be incredibly useful for companies running a documentation server internally especially if pkg.go.dev spoke the GOPROXY download protocol and a company had an internal GOPROXY as well.

Agreed. I definitely want to open source a server like that. I am not convinced that pkg.go.dev in its current form is the right server for "companies running a documentation server internally" - I think we can and should aim a little higher and differently for that. See my mailing list reply for more about that.

But even if we release a different server as the best thing to run in non-global-scale settings, we should still look more carefully at open-sourcing pkg.go.dev's code, as an example if nothing else. As I said in the mail, I am going to look into that.

[qaisjp]

This general topic also came up on golang-dev over the weekend, and I wrote a long reply there:
groups.google.com/g/golang-dev/c/mfiPCtJ1BGU/m/qtCrqlrEEwAJ

That link didn't work for me. This one does, though: https://groups.google.com/d/msg/golang-dev/mfiPCtJ1BGU/qtCrqlrEEwAJ

p.s. i'm quite satisfied with that reply. thank you!

[julieqiu]

In case anyone following this issue has not seen @rsc's post on golang-dev, we are working towards open sourcing pkg.go.dev!

Updates will be posted on this issue.

[l0nax]

@julieqiu can you give an approximate date when this will be the case?

[julieqiu]

Not at the moment - we're still figuring out the details for what is involved with open sourcing the site. We'll post on this issue once we have an update!

[carnott-snap]

I get that you are not able to commit to anything, but can we get an order of magnitude? (e.g. days, weeks, months, or years)

[mvdan]

@carnott-snap given that their initial plan was to phase out godoc.org towards the end of 2020, and that the codebase will probably require some work before being open sourced, one can imagine it's in the order of months.

[julieqiu]

+1 to @mvdan's comment - on the order of months. We are planning to open source before traffic from godoc.org is redirected to pkg.go.dev.

[gopherbot]

Change https://golang.org/cl/222737 mentions this issue: repos: add golang.org/x/discovery repository

[gopherbot]

Change https://golang.org/cl/234677 mentions this issue: repos: add API for opting-in to use pkg.go.dev for documentation

[gopherbot]

Change https://golang.org/cl/234678 mentions this issue: cmd/golangorg: add golang.org/x/pkgsite repository

[julieqiu]

Pkg.go.dev is now open source! You can find the codebase at go.googlesource.com/pkgsite. The repository is also mirrored to github.com/golang/pkgsite. For details, see blog.golang.org/pkgsite.

[marwan-at-work]

That's very exciting, thank you all for your work open sourcing this to the community!

[mvdan]

@julieqiu perhaps you could confirm if the project will use this issue tracker as the only source of truth going forward, like gopls? I know we started using issues a while back, but as far as I understand it was still partly a mirror of sorts, where the actual fixes and deployments were tracked internally. It would be nice if all of that now happened here too.

I'm also interested to know if the existing issues are up to date with what's been open sourced, or if someone should update the issue tracker

[julieqiu]

We do plan to continue using only the Go issue tracker for x/pkgsite. You can continue filing issues using this template.

The existing issues related to pkg.go.dev should be up to date. We will continue using the go.dev label to track issues. I plan to add the help wanted label to relevant issues later today.

Additional details are in the blog post and contributing guidelines :)