Go: proposal: Go 2: find a way to export uncased identifiers

[rsc]

https://github.com/golang/go/issues/5763#issue-51284151 observes “It is very strange to use, say Z成本 or Jぶつける as identifiers.” In that issue we discussed potentially changing the default export rule, but as of https://github.com/golang/go/issues/5763#issuecomment-333669811, which seemed to have general agreement, we decided against that.

Even so we do want to find a way to export uncased identifiers, or at least consider ways, in order to address the original observation.

This issue is for discussion of non-breaking ways to export uncased identifiers.

[ianlancetaylor]

This has been discussed before, but to note it on this issue: one possible approach would be to designate a specific Unicode character, not otherwise available for use in identifiers, to designate the identifier as exported.

For example, if we use the character $, then people would write $成本 to indicate that this identifier is exported. $ is just an example, it could instead be ↑ or ⍟ or pretty much anything.

This then leads to another decision point. We could treat the $ as a required part of the identifier, which then means that people will write p.$成本 to refer to the symbol from a different package. The effect is that we will see .$ everywhere; it almost becomes a token in itself. Or, we could say that the $ is only required for references within the package where the symbol is defined, and that for references from a different package the $ is implied. After all, if the $ were not there, the symbol could not be referenced from a different package anyhow. We would then have to consider the interaction with method names and interface satisfaction; there is an obvious set of rules but is it clear enough for the reader?

[griesemer]

In addition to what @ianlancetaylor said: There's also a third design choice (besides requiring the special Unicode character always for exported identifiers, or only inside the package that exports the identifier). The third choice is to only require the special Unicode character at the declaration; basically a marker indicating that the following (or perhaps preceding) identifier is exported. There's also an obvious drawback* with this choice which is that one won't be able to tell (at a use site) whether an identifier is exported or not by simply looking at it. ( * In the past we have eschewed this idea).

Regarding the choice of the special Unicode character: One could chose . (period). Inside a package p that declares the exported identifier, say 成本, one would have to write .成本. Inside a package that imports package p one would continue to write p.成本. This would be pretty natural in imported packages, and fairly unobtrusive (but still visible) in the declaring package.

[bcmills]

If we are contemplating a change to export rules anyway, we should evaluate whether the same mechanism could be used for fields of cgo-imported structs, as described in #13467.

As far as I can tell, the constraints to address that use-case are:

1. When used to export all of the fields of a struct type, aliases of that struct type defined in different packages must be identical.
2. Within a package, the use of a struct field exported by this mechanism must not require a distinguished prefix.

A distinguished Unicode character only satisfies those constraints if it is not required for field references within the same package.

[lijhello]

maybe this can be solved in some other project's own coding rules

// if you don't understand the requirement and abstract the concepts very well and cant' come up with good names

// E for Export
var E_成本1 double
var E_成本2 double
var E_成本3 double

// or use a Getter
func Get_成本() double

some time some coder can't come up with a good name in English, this has nothing to do with the programming language.

variable or function names in Chinese are not used so much in source code. other part of source code are still not Chinese: if, for, func, return. you don't use variable or function names in Chinese in C, C++ too.

[Davidc2525]

if they come to place $ to declare a variable, I will hate to go

[kstenerud]

Why not just make go 2.x be explicit, using keywords like public and private to specify visibility? "Clever" hacks upon existing systems not designed for that purpose (e.g. letter case) are what cause these kinds of messes in the first place. Adding further hacks on top of the broken ones just makes it worse.

To support legacy code, you could make fields of unspecified visibility use the old, broken behavior so that code still compiles and runs as expected.

[ianlancetaylor]

@kstenerud We believe that the fact that one can immediately tell by looking at an identifier whether it is exported or private is a feature. We don't consider it to be broken behavior.

In any case this is now so fundamental to Go code that it would not be feasible to change it at this point.

[beoran]

In Oberon * or - were used as export marks, so using, for example . as the export mark is not unprecedented. (https://cseweb.ucsd.edu/~wgg/CSE131B/oberon2.htm) I feel that the go package system as well as the imports and exports are inspired by Oberon, but with the advantage that upper case identifiers are exported "automatically". Still, I think it is an omission that there can't be an explicit export mark as well. Furthermore, for interoperation between Go and other languages, it might be desirable to export a lower case identifier as well.

The fact that one can immediately tell by looking at an identifier whether it is exported or private is only of limited use, since that is only so at the local package level. If the identifier comes from another package, normally, if not using dot imports, it is immediately clear that an identifier is imported, thanks to the package prefix. Therefore I think that the '.' prefix to signify an exported identifier like @griesemer proposes is the best solution for this problem.

[dreamgonfly]

Why not keyword "export"? It's readable, explicit, and understandable even for new comers. Please do not give more special characters special meanings, which is incomprehensible at first glance.

I also support @kstenerud 's opinion.

[beoran]

Go is partially inspired on oberon-2, but it has a more C like syntax. Otherwise we would write begin in stead of { and end in stead of }, and pointer in stead of *. So special characters with special meaning is somewhat normal for Go.
.

[mdempsky]

With the Go language version in go.mod, I think one option here is we could just make uncased identifiers exported iff they're in a package that uses Go 1.17 (or whatever -- I'm going to use 1.17 for concreteness). That is, 成本 would continue being non-exported for packages using Go 1.16 or older (including being inaccessible from newer packages compiled using Go 1.17+), but would be exported from packages compiled using Go 1.17+ (but would similarly continue being inaccessible from older packages using Go 1.16 or older).

This would mean users upgrading from Go 1.16 to Go 1.17 might need to rename their identifiers to prevent them from being exported (e.g., rename 成本 to x成本). We'd probably want to give a substantial heads up to developers about this planned change.

I'm pretty sure the compilers and reflect API can handle this fine. I'm a little worried about go/token.IsExported; e.g., I see go/doc and net/rpc use it, but maybe they can be handled some other way.

[dsnet]

With the Go language version in go.mod

As someone who writes lots of tooling that parses Go source files, there's a strong benefit to being able to understand what the source code for a single Go file means without needing to consult some other piece of information (i.e., the go.mod file).

For example, ast.Ident.IsExported would be broken since it has no concept of whether it is operating under pre-Go1.17 semantics or not.

[crvv]

That was proposed in 2013 but rejected. I think it is a good idea. But if that won't happen, I think it's better to keep the rule.
https://github.com/golang/go/issues/5763#issuecomment-66081539

95% of Chinese programmers won't to use Chinese variable name.
For the rest of the 5%, I am very sure they won't use Chinese variable name in 95% of their code.
It is not worth adding a marker in the variable declaration.
https://github.com/golang/go/issues/5763#issuecomment-316421809

[beoran]

@crvv, I can see that not many Chinese Go programmers want to use Chinese function names or variable names _up to now now_, but perhaps this like the story of the fox that tries to eat grapes from a vine but cannot reach them? If it becomes possible, then I think we are likely to see more people who will want to use Go because now they are able to use it, as is suggested by @lych77 in #5763.

Furthermore, there is also a problem of interoperating Go code with other programming languages. In some of those programming languages, the convention is that function or method names should be all lowercase. Therefore if would be convenient for that use case to allow certain non-exported identifiers to become exported, preferably by a . marker as was suggested by @griesemer.

[crvv]

Yes, there are some use cases where Chinese variable name are very helpful.
https://github.com/golang/go/issues/5763#issuecomment-245828791
And I think it can be solved by just making uncased identifiers exported.

https://github.com/golang/go/issues/30572#issuecomment-469381004
I agree that using case to distinguish export status is a great feature of Go.
If I can write var .lower_case_variable int to make it public, the great feature will be broken.

var Upper int // public
var lower int // private
var .lower int // public
var 成本 int // private
var .成本 int //public

If the great feature is broken, why not just accept #30572 ?

[jimmyfrasche]

So:

Export all uncased identifiers

• swaps having to write X成本 to export for having to write x成本 to stop it from being exported
• sharp transition where a lot of unexported names are suddenly exported. Automatable but still a lot of changes at once, even with the gate on language version
• doesn't change cased languages (:+1:)
• breaks IsExported (could have two versions for pre- and post-this change though that's a bit awkward and hard to use)

Export sigil only used at declaration, like .成本

• lightweight notation (:+1:)
• breaks IsExported

Export keyword

• same as sigil used at declaration
• introduces a new keyword
• breaks IsExported

Export sigil used only in defining package

• I don't see a benefit for doing this vs. only using it at declaration or always using it. I could be missing something here.
• breaks IsExported

Export sigil always used, like $成本

• not much different from X成本, just uses a different rune
• does not break IsExported

Would anything other than lexical checks for exported-ness break in a user-facing manner?

IsExported would only break in token/ast: the versions in go/types and reflect (soon to be added #41563) would continue to be precise. Would many tools be broken irreparably by this? Would it help to deprecate token/ast IsExported well before making a change?

Could an explicit notation be limited to uncased identifiers and cased identifiers continue to use the current rules? (:+1:)