Go: proposal: go build flag to disable internal path check

[JavierZunzunegui]

This is a proposal regarding go internal packages, probably know to many by the compilation error imports ***/internal/***: use of internal package not allowed.

Can we have some means to disable it, perhaps via a build flag?

While /internal is very useful, I have come across multiple times since it was added when it became a problem, specially when writing scripts that ran on otherwise internal-compliant codebase. For example:

• go library internal code: there are some very useful packages in go itself behind internal packages. You can (and I sometimes have had to) copy large amounts of code and modify some internal paths, but that takes time, is ugly and may even be in breach of copyright rules.
• when doing code generation I have at times needed to add many packages to access the operations in func init() {...} or top level flags - see this post. Whenever some packages where under internal that was a problem.
• when hacking some third party libraries. The solution is often to create a branch and remove internal (or add some means to bypass it), again a tedious task.
• when doing a large code refactoring. Being able to build half way through it without internal might help check you are in the right path.

Basically in my use case (there may be others, would like to hear from other gophers) these are all scripts doing some form of hack over the codebase, be it generate code, inspect some code, etc. or as part of a quick experimentation before making proper (internal-compliant) changes. While disabling internal to build production applications seems like a bad idea, I think we could have a way of relaxing it when hacking something together. It can certainly be a productivity burden at times.

[mvdan]

I think point 1 is not a good argument - these libraries are internal for a reason, so you should not be able to use them directly in any way. You could say that a compiler flag is going out of one's way to use them, but so is copying the code or using a tool to do that work for you.

I'm not sure about point 2. It seems like a niche use-case.

I don't understand point 3. When you hack on a third party library, you don't need to change any of the library's import paths, so you should be unaffected by the presence of internal packages.

As for point 4, I think vanilla Go is already somewhat against half-baked refactors. For example, the compiler doesn't allow unused variables or imports, and there's no way to disable either of those checks.

So I think the answer here should be tooling. In a similar way, goimports helps get rid of unused imports, instead of having a flag to skip that check.

[JavierZunzunegui]

these libraries are internal for a reason, so you should not be able to use them directly in any way

My point is when I am hacking something together I want my compiler to be as 'relaxed' as possible. I know it is not the way the libraries are meant to be used, but as long as it does not make it to production I see no reason I shouldn't be allowed so tell the compiler to not get in my way (so long as it can build it).

I'm not sure about point 2. It seems like a niche use-case.

It is a very specific use-case. Would like to hear of other equally specific use-cases others may have come across though.

When you hack on a third party library, you don't need to change any of the library's import paths

The issue I was thinking of is when a third party package, say X, has some parameter (or uses some function, ...) from X/internal/Y. X uses Y in some sensible way, but I am doing something unexpected to package X and want it changed. Changing the path of X/internal/Y or adding some aliases in X is still more tedious that adding a flag on the build.

As for point 4, I think vanilla Go is already somewhat against half-baked refactors.

That wasn't what I meant, and in fact what I want is to make big refactors easier. I don't want to wait until I finish refactoring (probably adding or removing internal somewhere) to find out I have a bigger issue, say a circular dependency. If I can check for that earlier and plan accordingly, that's a productivity gain.

Overall, my point is one of productivity when doing exploratory work, and nothing more. There have come multiple times when I was hacking stuff when internal got in the way (and disabling internal was the easiest solution in such cases). It is never going to be a full blocker (worst case scenario you copy and modify whatever library you need), but why would go's developer productivity stop at production code? Making experimenting easier will also, indirectly, result in better production code.

And to emphasis, the flag should never used in production. No proposed change to that, internal is very useful and I recognize that.

[rsc]

We've had this since Go 1.5 without significant pushback. The moment where you start breaking down these barriers is where they stop being real barriers. Internal needs to mean _really_ internal.