Go: all: support gradual code repair while moving a type between packages

[rsc]

Original Title: proposal: support gradual code repair while moving a type between packages

Go should add the ability to create alternate equivalent names for types, in order to enable gradual code repair during codebase refactoring. This was the target of the Go 1.8 alias feature, proposed in #16339 but held back from Go 1.8. Because we did not solve the problem for Go 1.8, it remains a problem, and I hope we can solve it for Go 1.9.

In the discussion of the alias proposal, there were many questions about why this ability to create alternate names for types in particular is important. As a fresh attempt to answer those questions, I wrote and posted an article, “Codebase Refactoring (with help from Go).” Please read that article if you have questions about the motivation. (For an alternate, shorter presentation, see Robert's Gophercon lightning talk. Unfortunately, that video wasn't available online until October 9. Update, Dec 16: here's my GothamGo talk, which was essentially the first draft of the article.)

This issue is _not_ proposing a specific solution. Instead, I want to gather feedback from the Go community about the space of possible solutions. One possible avenue is to limit aliases to types, as mentioned at the end of the article. There may be others we should consider as well.

Please post thoughts about type aliases or other solutions as comments here.

Thank you.

Update, Dec 16: Design doc for type aliases posted.
Update, Jan 9: Proposal accepted, dev.typealias repository created, implementation due at the start of the Go 1.9 cycle for experimentation.

---

Discussion summary (last updated 2017-02-02)

Do we expect to need a general solution that works for all declarations?

If type aliases are 100% necessary, then var aliases are maybe 10% necessary, func aliases are 1% necessary, and const aliases are 0% necessary. Because const already has = and func could plausibly use = too, the key question is whether var aliases are important enough to plan for or implement.

As argued by @rogpeppe (https://github.com/golang/go/issues/16339#issuecomment-258771806) and @ianlancetaylor (https://github.com/golang/go/issues/16339#issuecomment-233644777) in the original alias proposal and as mentioned in the article, a mutating global var is usually a mistake. It probably doesn't make sense to complicate the solution to accommodate what is usually a bug. (In fact, if we can figure out how, it would not surprise me if in the long term Go moves toward requiring global vars to be immutable.)

Because richer var aliases are likely not important enough to plan for, it seems like the right choice here is to focus only on type aliases. Most of the comments here seem to agree. I won't list everyone.

Do we need a new syntax (= vs => vs export)?

The strongest argument for new syntax is the need to support var aliases, either now or in the future (https://github.com/golang/go/issues/18130#issuecomment-264232763 by @Merovius). It seems okay to plan not to have var aliases (see previous section).

Without var aliases, reusing = is simpler than introducing new syntax, whether => like in the alias proposal, ~ (https://github.com/golang/go/issues/18130#issuecomment-264185142 by @joegrasse), or export (https://github.com/golang/go/issues/18130#issuecomment-264152427 by @cznic).

Using = in would also exactly match the syntax of type aliases in Pascal and Rust. To the extent that other languages have the same concepts, it's nice to use the same syntax.

Looking ahead, there could be a future Go in which func aliases exist too (see https://github.com/golang/go/issues/18130#issuecomment-264324306 by @nigeltao), and then all declarations would permit the same form:

const C2 = C1
func F2 = F1
type T2 = T1
var V2 = V1

The only one of these that wouldn't establish a true alias would be the var declaration, because V2 and V1 can be redefined independently as the program executes (unlike the const, func, and type declarations which are immutable). Since one main reason for variables is to allow them to vary, that exception would at least be easy to explain. If Go moves toward immutable global vars, then even that exception would disappear.

To be clear, I am not suggesting func aliases or immutable global vars here, just working through the implications of such future additions.

@jimmyfrasche suggested (https://github.com/golang/go/issues/18130#issuecomment-264278398) aliases for everything except consts, so that const would be the exception instead of var:

const C2 = C1 // no => form
func F2 => F1
type T2 => T1
var V2 => V1
var V2 = V1 // different from => form

Having inconsistencies with both const and var seems more difficult to explain than having just an inconsistency for var.

Can this be a tooling- or compiler-only change instead of a language change?

It's certainly worth asking whether gradual code repair can be enabled purely by side information supplied to the compiler (for example, https://github.com/golang/go/issues/18130#issuecomment-264205929 by @btracey).

Or maybe if the compiler can apply some kind of rule-based preprocessing to transform input files before compilation (for example, https://github.com/golang/go/issues/18130#issuecomment-264329924 by @tux21b).

Unfortunately, no, the change really can't be confined that way. There are at least two compilers (gc and gccgo) that would need to coordinate, but so would any other tools that analyze programs, like go vet, guru, goimports, gocode (code completion), and others.

As @bcmills said (https://github.com/golang/go/issues/18130#issuecomment-264275574), “a ‘non-language-change’ mechanism which must be supported by all implementations is a de facto language change — it’s just one with poorer documentation.”

What other uses might aliases have?

We know of the following. Given that type aliases in particular were deemed important enough for inclusion in Pascal and Rust, there are likely others.

1. Aliases (or just type aliases) would enable creating drop-in replacements that expand other packages. For example see https://go-review.googlesource.com/#/c/32145/, especially the explanation in the commit message.

2. Aliases (or just type aliases) would enable structuring a package with a small API surface but a large implementation as a collection of packages for better internal structure but still present just one package to be imported and used by clients. There's a somewhat abstract example described at https://github.com/golang/go/issues/16339#issuecomment-232813695.

3. Protocol buffers have an "import public" feature whose semantics is trivial to implement in generated C++ code but impossible to implement in generated Go code. This causes frustration for authors of protocol buffer definitions shared between C++ and Go clients. Type aliases would provide a way for Go to implement this feature. In fact, the original use case for import public was gradual code repair. Similar issues may arise in other kinds of code generators.

4. Abbreviating long names. Local (unexported or not-package-scoped) aliases might be handy to abbreviate a long type name without introducing the overhead of a whole new type. As with all these uses, the clarity of the final code would strongly influence whether this is a suggested use.

What other issues does a proposal for type aliases need to address?

Listing these for reference. Not attempting to solve or discuss them in this section, although a few were discussed later and are summarized in separate sections below.

1. Handling in godoc. (https://github.com/golang/go/issues/18130#issuecomment-264323137 by @nigeltao and https://github.com/golang/go/issues/18130#issuecomment-264326437 by @jimmyfrasche)

2. Can methods be defined on types named by alias? (https://github.com/golang/go/issues/18130#issuecomment-265077877 by @ulikunitz)

3. If aliases to aliases are allowed, how do we handle alias cycles? (https://github.com/golang/go/issues/18130#issuecomment-264494658 by @thwd)

4. Should aliases be able to export unexported identifiers? (https://github.com/golang/go/issues/18130#issuecomment-264494658 by @thwd)

5. What happens when you embed an alias (how do you access the embedded field)? (https://github.com/golang/go/issues/18130#issuecomment-264494658 by @thwd, also #17746)

6. Are aliases available as symbols in the built program? (https://github.com/golang/go/issues/18130#issuecomment-264494658 by @thwd)

7. Ldflags string injection: what if we refer to an alias? (https://github.com/golang/go/issues/18130#issuecomment-264494658 by @thwd; this only arises if there are var aliases.)

Is versioning a solution by itself?

"In that case maybe versioning is the whole answer, not type aliases."
(https://github.com/golang/go/issues/18130#issuecomment-264573088 by @iainmerrick)

As noted in the article, I think versioning is an complementary concern. Support for gradual code repair, such as with type aliases, gives a versioning system more flexibility in how it builds a large program, which can be difference between being able to build the program and not.

Can the larger refactoring problem be solved instead?

In https://github.com/golang/go/issues/18130#issuecomment-265052639, @niemeyer points out that there were actually two changes for moving os.Error to error: the name changed but so did the definition (the current Error method used to be a String method).

@niemeyer suggests that perhaps we can find a solution to the broader refactoring problem that fixes types moving between packages as a special case but also handles things like method names changing, and he proposes a solution built around "adapters".

There is a fair amount of discussion in the comments that I can't easily summarize here. The discussion isn't over, but so far it is unclear whether "adapters" can fit into the language or be implemented in practice. It does seem clear that adapters are at least one order of magnitude more complex than type aliases.

Adapters need a coherent solution to the subtyping problems noted below as well.

Can methods be declared on alias types?

Certainly aliases do not allow bypassing the usual method definition restrictions: if a package defines type T1 = otherpkg.T2, it cannot define methods on T1, just as it cannot define methods directly on otherpkg.T2. That is, if type T1 = otherpkg.T2, then func (T1) M() is equivalent to func (otherpkg.T2) M(), which is invalid today and remains invalid. However, if a package defines type T1 = T2 (both in the same package), then the answer is less clear. In this case, func (T1) M() would be equivalent to func (T2) M(); since the latter is allowed, there is an argument to allow the former. The current design doc does not impose a restriction here (in keeping with the general avoidance of restrictions), so that func (T1) M() is valid in this situation.

In https://github.com/golang/go/issues/18130#issuecomment-267694112, @jimmyfrasche suggests that instead defining "no use of aliases in method definitions" would be a clear rule and avoid needing to know what T is defined as to know if func (T) M() is valid. In https://github.com/golang/go/issues/18130#issuecomment-267997124, @rsc points out that even today there are certain T for which func (T) M() is not valid: https://play.golang.org/p/bci2qnldej. In practice this doesn't come up because people write reasonable code.

We will keep this possible restriction in mind but wait until there is strong evidence that it is needed before introducing it.

Is there a cleaner way to handle embedding and, more generally, field renames?

In https://github.com/golang/go/issues/18130#issuecomment-267691816, @Merovius points out that an embedded type that changes its name during a package move will cause problems when that new name must eventually be adopted at the use sites. For example if user type U has an embedded io.ByteBuffer that moves to bytes.Buffer, then while U embeds io.ByteBuffer the field name is U.ByteBuffer, but when U is updated to refer to bytes.Buffer, the field name necessarily changes to U.Buffer.

In https://github.com/golang/go/issues/18130#issuecomment-267710478, @neild points out that there is at least a workaround if references to io.ByteBuffer must be excised: the package P that defines U can also define 'type ByteBuffer = bytes.Buffer' and embed that type into U. Then U still has a U.ByteBuffer, even after io.ByteBuffer is gone entirely.

In https://github.com/golang/go/issues/18130#issuecomment-267703067, @bcmills suggests the idea of field aliases, to allow a field to have multiple names during a gradual repair. Field aliases would allow defining something like type U struct { bytes.Buffer; ByteBuffer = Buffer } instead of having to create the top-level type alias.

In https://github.com/golang/go/issues/18130#issuecomment-268001111, @rsc raises yet another possibility: some syntax for 'embed this type with this name', so that it is possible to embed a bytes.Buffer as the field name ByteBuffer, without needing a top-level type or an alternate name. If that existed, then the type name could be updated from io.ByteBuffer to bytes.Buffer while preserving the original name (and not introducing a second, nor a clumsy exported type).

These all seem worth exploring once we have more evidence of large-scale refactorings blocked by problems with fields changing names. As @rsc wrote, "If type aliases help us get to the point where lack of field aliases is the next big roadblock for large-scale refactorings, that will be progress!"

There was a suggestion of restricting the use of aliases in embedded fields or changing the embedded name to use the target type's name, but those make the alias introduction break existing definitions that must then be fixed atomically, essentially preventing any gradual repair. @rsc: "We discussed this at some length in #17746. I was originally on the side of the name of an embedded io.ByteBuffer alias being Buffer, but the above argument convinced me I was wrong. @jimmyfrasche in particular made some good arguments about the code not changing depending on the definition of the embedded thing. I don't think it's tenable to disallow embedded aliases completely."

What is the effect on programs using reflection?

Programs using reflection see through aliases. In https://github.com/golang/go/issues/18130#issuecomment-267903649, @atdiar points out that if a program is using reflection to, for example, find the package in which a type is defined or even the name of a type, it will observe the change when the type is moved, even if a forwarding alias is left behind. In https://github.com/golang/go/issues/18130#issuecomment-268001410, @rsc confirmed this and wrote "Like the situation with embedding, it's not perfect. Unlike the situation with embedding, I don't have any answers except maybe code shouldn't be written using reflect to be quite that sensitive to those details."

The use of vendored packages today also changes package import paths seen by reflect, and we have not been made aware of significant problems caused by that ambiguity. This suggests that programs are not commonly inspecting reflect.Type.PkgPath in ways that would be broken by use of aliases. Even so, it's a potential gap, just like embedding.

What is the effect on separate compilation of programs and plugins?

In https://github.com/golang/go/issues/18130#issuecomment-268524504, @atdiar raises the question of the effect on object files and separate compilation. In https://github.com/golang/go/issues/18130#issuecomment-268560180, @rsc replies that there should be no need to make changes here: if X imports Y and Y changes and is recompiled, then X needs to be recompiled too. That's true today without aliases, and it will remain true with aliases. Separate compilation means being able to compile X and Y in distinct steps (the compiler does not have to process them in the same invocation), not that it is possible to change Y without recompiling X.

Would sum types or some kind of subtyping be an alternative solution?

In https://github.com/golang/go/issues/18130#issuecomment-264413439, @iand suggests "substitutable types", "a list of types that may be substituted for the named type in function arguments, return values etc.". In https://github.com/golang/go/issues/18130#issuecomment-268072274, @j7b suggests using algebraic types "so we also get an empty interface equivalent with compile time type checking as a bonus". Other names for this concept are sum types and variant types.

In general this does not suffice to allow moving types with gradual code repair. There are two ways to think about this.

In https://github.com/golang/go/issues/18130#issuecomment-268075680, @bcmills takes the concrete way, pointing out that algebraic types have a different representation than the original, which makes it not possible to treat the sum and the original as interchangeable: the latter has type tags.

In https://github.com/golang/go/issues/18130#issuecomment-268585497, @rsc takes the theoretical way, expanding on https://github.com/golang/go/issues/18130#issuecomment-265211655 by @gri pointing out that in a gradual code repair, sometimes you need T1 to be a subtype of T2 and sometimes vice versa. The only way for both to be subtypes of each other is for them to be the same type, which not concidentally is what type aliases do.

As a side tangent, in addition to not solving the gradual code repair problem, algebraic types / sum types / union types / variant types are by themselves hard to add to Go. See
the FAQ answer and the Go 1.6 AMA discussion for more.

In https://github.com/golang/go/issues/18130#issuecomment-265206780, @thwd suggests that since Go has a subtyping relationship between concrete types and interfaces (bytes.Buffer can be seen as a subtype of io.Reader) and between interfaces (io.ReadWriter is a subtype of io.Reader in the same way), making interfaces "recursively covariant (according to the current variance rules) down to their method arguments" would solve the problem provided that all future packages only use interfaces, never concrete types like structs ("encourages good design, too").

There are three problems with that as a solution. First, it has the subtyping issues above, so it doesn't solve gradual code repair. Second, it doesn't apply to existing code, as @thwd noted in this suggestion. Third, forcing the use of interfaces everywhere may not actually be good design and introduces performance overheads (see for example https://github.com/golang/go/issues/18130#issuecomment-265211726 by @Merovius and https://github.com/golang/go/issues/18130#issuecomment-265224652 by @zombiezen).

Restrictions

This section collects proposed restrictions for reference, but keep in mind that restrictions add complexity. As I wrote in https://github.com/golang/go/issues/18130#issuecomment-264195616, "we should probably only implement those restrictions after actual experience with the unrestricted, simpler design helps us understand whether the restriction would bring enough benefits to pay for its cost."

Put another way, any restriction would need to be justified by evidence that it would prevent some serious misuse or confusion. Since we haven't implemented a solution yet, there is no such evidence. If experience did provide that evidence, these will be worth returning to.

Restriction? Aliases of standard library types can only be declared in standard library.

(https://github.com/golang/go/issues/18130#issuecomment-264165833 and https://github.com/golang/go/issues/18130#issuecomment-264171370 by @iand)

The concern is "code that has renamed standard library concepts to fit a custom naming convention", or "long spaghetti chains of aliases across multiple packages that end up back at the standard library", or "aliasing things like interface{} and error".

As stated, the restriction would disallow the "extension package" case described above involving x/image/draw.

It's unclear why the standard library should be special: the problems would exist with any code. Also, neither interface{} nor error is a type from the standard library. Rephrasing the restriction as "aliasing predefined types" would disallow aliasing error, but the need to alias error was one of the motivating examples in the article.

Restriction? Alias target must be package-qualified identifier.

(https://github.com/golang/go/issues/18130#issuecomment-264188282 by @jba)

This would make it impossible to make an alias when renaming a type within a package, which may be used widely enough to necessitate a gradual repair (https://github.com/golang/go/issues/18130#issuecomment-264274714 by @bcmills).

It would also disallow aliasing error as in the article.

Restriction? Alias target must be package-qualified identifier with same name as alias.

(proposed during alias discussion in Go 1.8)

In addition to the problems of the previous section with limiting to package-qualified identifiers, forcing the name to stay the same would disallow the conversion from io.ByteBuffer to bytes.Buffer in the article.

Restriction? Aliases should be discouraged in some way.

"How about hiding aliases behind an import, just like for "C" and “unsafe”, to further discourage it's usage? In the same vein, I would like the aliases syntax to be verbose and stand out as a scaffold for on going refactoring." - https://github.com/golang/go/issues/18130#issuecomment-264289940 by @xiegeo

"Should we also automatically infer that an aliased type is legacy and should be replaced by the new type? If we enforce golint, godoc and similar tools to visualize the old type as deprecated, it would limit the abuse of type aliasing very significantly. And the final concern of aliasing feature being abused would be resolved." - https://github.com/golang/go/issues/18130#issuecomment-265062154 by @rakyll

Until we know that they will be used wrong, it seems premature to discourage usage. There may be good, non-temporary uses (see above).

Even in the case of code repair, either the old or new type may be the alias during the transition, depending on the constraints imposed by the import graph. Being an alias does not mean the name is deprecated.

There is already a mechanism for marking certain declarations as deprecated (see https://github.com/golang/go/issues/18130#issuecomment-265294564 by @jimmyfrasche).

Restriction? Aliases must target named types.

"Aliases shouldn't not apply to unnamed type. Their is no "code repair" story in moving from one unnamed type to another. Allowing aliases on unnamed types means I can no longer teach Go as simply named and unnamed types." - https://github.com/golang/go/issues/18130#issuecomment-276864903 by @davecheney

Until we know that they will be used wrong, it seems premature to discourage usage. There may be good uses with unnamed targets (see above).

As noted in the design doc, we do expect to change the terminology to make the situation clearer.

[variadico]

I like how visually uniform this looks.

const OldAPI => NewPackage.API
func  OldAPI => NewPackage.API
var   OldAPI => NewPackage.API
type  OldAPI => NewPackage.API

But since we can almost gradually move most elements, maybe the simplest
solution _is_ just to allow an = for types.

const OldAPI = NewPackage.API
func  OldAPI() { NewPackage.API() }
var   OldAPI = NewPackage.API
type  OldAPI = NewPackage.API

[zquestz]

So first, I just wanted to thank you for that excellent write-up. I think the best solution is to introduce type aliases with an assignment operator. This requires no new keywords/operators, uses a familiar syntax, and should solve the refactoring problem for large code bases.

[iand]

As Russ's article points out, any alias-like solution needs to gracefully solve https://github.com/golang/go/issues/17746 and https://github.com/golang/go/issues/17784

[travisjeffery]

Thank you for the write up of that article.

I find the type-only aliases using the assignment operator to be best:

type OldAPI = NewPackage.API

My reasons:

• It's simpler.
  The alternative solution => having subtly different meaning based on its operand feels out of place for Go.
• It's focused and conservative.
  The issue at hand with types is solved and you don't need to worry about imagining the complications of the generalized solution.
• It's aesthetic.
  I think it looks more pleasing.

All of these above: the result being simple, focused, conservative, and aesthetic make it easy for me to picture of it being a part of Go.

[cznic]

If the solution would be limited to types only then the syntax

type NewFoo = old.Foo

already considered before, as discussed in the @rsc's article, looks very good to me.

If we would like to be able to do the same for constants, variables and functions, my preferred syntax would be (as proposed before)

package newfmt

import (
    "fmt"
)

// No renaming.
export fmt.Printf // Note: Same as `export Printf fmt.Printf`.

export (
        fmt.Sprintf
        fmt.Formatter
)

// Renaming.
export Foo fmt.Errorf // Foo must be exported, ie. `export foo fmt.Errorf` would be invalid.

export (
    Bar fmt.Fprintf
    Qux fmt.State
)

As discussed before, the disadvantage is that a new, top-level only keyword is introduced, which is admittedly akward, even though technically feasible and fully backwards compatible. I like this syntax because it reflects the pattern of imports. It would seem natural to me that exports would be permitted only in the same section where imports are allowed, ie. between the package clause and any var, type, constant or function TLD.

The renaming identifiers would be declared in the package scope, however, the new names are not visible in the package declaring them (newfmt in the example above) above with respect to redeclaration, which is disallowed as usual. Given the previous example, TLDs

var v = Printf // undefined: Printf.
var Printf int // Printf redeclared, previous declaration at newfmt.go:8.

In the importing package the renaming identifiers are visible normally, as any other exported identifier of the (newftm's) package block.

package foo

import "newfmt"

type bar interface {
    baz(qux newfmt.Qux) // qux type is identical to fmt.State.
}

In conclusion, this approach does not introduce any new local name binding in newfmt, which I believe avoids at least some of the problems discussed in #17746 and solves #17784 completely.

[4ad]

My first preference is for a type-only type NewFoo = old.Foo.

If a more general solution is desired, I agree with @cznic that a dedicated keyword is better than a new operator (especially an asymetric operator with confusing directionality[1]). That being said, I don't think the export keyword conveys the right meaning. Neither the syntax, nor semantics mirrors import. What about alias?

I understand why @cznic doesn't want the new names to be accesible in the package declaring them, but, to me at least, that restriction feels unexpected and artificial (although I perfectly well understand the reason behind it).

[1] I have been using Unix for almost 20 years, and I still can't create a symlink on the first try. And I usually fail even on the second try, after I have read the manual.

[iand]

I would like to propose an additional constraint: type aliases to standard library types may only be declared in the standard library.

My reasoning is that I don't want to work with code that has renamed standard library concepts to fit a custom naming convention. I also don't want to deal with long spaghetti chains of aliases across multiple packages that end up back at the standard library.

[quentinmit]

@iand: That constraint would block the use of this feature to migrate anything into the standard library. Case in point, the current migration of Context into the standard library. The old home of Context should become an alias for the Context in the standard library.

[iand]

@quentinmit that is unfortunately true. It also limits the use case for golang.org/x/image/draw in this CL https://go-review.googlesource.com/#/c/32145/

My real concern is with people aliasing things like interface{} and error

[joegrasse]

If it is decided to introduce a new operator, I would like to propose ~. In the English language, it is generally understood to mean "similar to", "approximately", "about", or "around". As @4ad above stated, the => is an asymetric operator with confusing directionality.

For example:

const OldAPI ~ NewPackage.API
func  OldAPI ~ NewPackage.API
var   OldAPI ~ NewPackage.API
type  OldAPI ~ NewPackage.API

[jba]

@iand if we limit the right-hand side to a package-qualified identifier, then that would eliminate your specific concern.

It would also mean you couldn't have aliases to any types in the current package, or to long type expressions like map[string]map[int]interface{}. But those uses have nothing to do with the main goal of gradual code repair, so maybe they are no great loss.

[rsc]

@cznic, @iand, others: Please note that _restrictions add complexity_. They complicate the explanation of the feature, and they add cognitive load for any user of the feature: if you forget about a restriction, you have to puzzle through why something you thought should work doesn't.

It's often a mistake to implement restrictions on a trial of a design solely due to hypothetical misuse. That happened in the alias proposal discussions, and it made the aliases in the trial unable to handle the io.ByteBuffer => bytes.Buffer conversion from the article. Part of the goal of writing the article is to define some cases we know we want to be able to handle, so that we don't inadvertently restrict them away.

As another example, it would be easy to make a misuse argument to disallow non-pointer receivers, or to disallow methods on non-struct types. If we'd done either of those, you couldn't create enums with String() methods for printing themselves, and you couldn't have http.Headers both be a plain map and provide helper methods. It's often easy to imagine misuses; compelling positive uses can take longer to appear, and it's important to create space for experimentation.

As yet another example, the original design and implementation for pointer vs value methods did not distinguish between the method sets on T and *T: if you had a *T, you could call the value methods (receiver T), and if you had a T, you could call the pointer methods (receiver *T). This was simple, with no restrictions to explain. But then actual experience showed us that allowing pointer method calls on values led to a specific class of confusing, surprising bugs. For example, you could write:

var buf bytes.Buffer
io.Copy(buf, reader)

and io.Copy would succeed, but buf would have nothing in it. We had to choose between explaining why that program ran incorrectly or explaining why that program didn't compile. Either way there were going to be questions, but we came down on the side of avoiding incorrect execution. Even so, we still had to write a FAQ entry about why the design has a hole cut out of it.

Again, please remember that restrictions add complexity. Like all complexity, restrictions need significant justification. At this stage in the design process it is good to think about restrictions that might be appropriate for a particular design, but we should probably only implement those restrictions after actual experience with the unrestricted, simpler design helps us understand whether the restriction would bring enough benefits to pay for its cost.

[rsc]

Also, my hope is that we can reach a tentative decision about what to try and then have something ready for experimentation at the beginning of the Go 1.9 cycle (ideally the day the cycle opens). Having more time to experiment will have many benefits, among them an opportunity to learn whether a particular restriction is compelling. One mistake with alias was not committing a complete implementation until near the end of the Go 1.8 cycle.

[btracey]

One thing about the original alias proposal is that in the intended use case (enabling refactoring) the actual use of the alias type should only be temporary. In the protobuffer example, the io.BytesBuffer stub was deleted once the gradual repair had been concluded.

If the alias mechanism should only be seen temporarily, does it actually require a language change? Perhaps instead there could be a mechanism to supply gc with a list of "aliases". gc could temporarily make the substitutions, and the author of the downstream codebase could gradually remove items in this file as fixes are merged. I realize this suggestion also has tricky consequences, but it at least encourages a temporary mechanism.

[Merovius]

I will not participate in the bikeshedding about syntax (I basically don't care), with one exception: If adding aliases is decided and if it's decided to restrict them to types, please use a syntax that is consistently extensible to at least var, if not also func and const (all proposed syntactical constructs allow for all, except type Foo = pkg.Bar). The reason is that, while I agree that cases where aliases for var make the difference might be rare, I don't think they are non-existent and as such believe that we might well at some point decide to add them too. At that point we definitely will want to have all alias declarations be consistent, it would be bad if it's type Foo = pkg.Bar and var Foo => pkg.Bar.

I'd also slightly argue for having all four. The reasons are

1) there is a distinction for var and I do sometimes use it. For example I often expose a global var Debug *log.Logger, or reassign global singletons like http.DefaultServeMux to intercept/remove registrations of packages that add handlers to it.

2) I also think that, while func Foo() { pkg.Bar() } does the same thing as func Foo => pkg.Bar, the intention of the latter is much clearer (especially if you already know about aliases). It clearly states "this isn't really meant to be here". So while technically identical, the alias syntax might serve as documentation.

It's not the hill I'd die on, though; type-aliases alone for now would be fine with me, as long as there is the option to extend them later.

I'm also super glad that this was written up like it was. It summarizes a bunch of opinions I had about API design and stability for a while and will, in the future, serve as a simple reference to link people too :)

However, I also want to emphasize that there where additional use cases covered by aliases that are different from the doc (and AIUI the more general intention of this issue, which is to find some solution to solve gradual repair). I am very glad if the community can agree on the concept of enabling gradual repair, but if a different decision from aliases is decided to reach it, I'd also think that in that case there should be simultaneously talk about if and how to support things like the protobuf public imports or the x/image/draw use case of drop-in replacement packages (both somewhat near to my heart too) with a different solution. @btracey's proposal of a go-tool/gc flag for aliases is an example where I believe that, while it covers gradual repair relatively well, it is not really acceptable for those other usecases. You can't really expect everyone who wants to compile something that uses x/image/draw to pass those flags, they should just be able to go get.

[bcmills]

@jba

@iand if we limit the right-hand side to a package-qualified identifier, then that would eliminate your specific concern.

It would also mean you couldn't have aliases to any types in the current package, […]. But those uses have nothing to do with the main goal of gradual code repair, so maybe they are no great loss.

Renaming within a package (e.g. to a more idiomatic or consistent name) is certainly a type of refactoring one might reasonably want to do, and if the package is used widely then that necessitates gradual repair.

I think a restriction to only package-qualified names would be a mistake. (A restriction to only exported names might be more tolerable.)

[bcmills]

@btracey

Perhaps instead there could be a mechanism to supply gc with a list of "aliases". gc could temporarily make the substitutions, and the author of the downstream codebase could gradually remove items in this file as fixes are merged.

A mechanism for gc would either mean that the code is only buildable with gc during the repair process, or that the mechanism would have to be supported by the other compilers (e.g. gccgo and llgo) too. A "non-language-change" mechanism which must be supported by all implementations is a de facto language change — it's just one with poorer documentation.

[rsc]

@btracey and @bcmills, and not just the compilers: any tool that analyzes source code, like guru or anything else people have built. It's certainly a language change no matter how you slice it.

[btracey]

Okay, thanks.

[jimmyfrasche]

Another possibility is aliases for everything except consts (and @rsc please forgive me for proposing a restriction!)

For consts, => is really just a longer way to write =. There's no new semantics, as with types and vars. There's no saved keystrokes as with funcs.

That would resolve #17784 at least.

The counterargument would be that tooling could treat the cases differently and that it could be an indicator of intent. That's a good counterargument, but I don't think it outweighs the fact that it's basically two ways to do exactly the same thing.

That said, I'm fine with just type aliases for now, they are certainly the most important. I definitely agree with @Merovius that we should strongly consider retaining the option for adding var and func aliases in the future, even if those doesn't happen for some time.

[xiegeo]

How about hiding aliases behind an import, just like for "C" and “unsafe”, to further discourage it's usage? In the same vein, I would like the aliases syntax to be verbose and stand out as a scaffold for on going refactoring.

[josharian]

As an attempt to open up the design space a little, here are some ideas. They're not fleshed out. They're probably bad and/or impossible; the hope is mainly to trigger new/better ideas in others. And if there's any interest, we can explore further.

The motivating idea for (1) and (2) is to somehow use conversion instead of aliases. In #17746, aliases ran into issues around having multiple names for the same type (or multiple ways to spell the same name, depending on whether you think of aliases as like #define or as like hard links). Using conversion sidesteps that by keeping the types distinct.

1. Add more automatic conversion.

When you call fmt.Println("abc") or write var e interface{} = "abc", "abc" is automatically converted to an interface{}. We could change the language so that when you have declared type T struct { S }, and T has no non-promoted methods, the compiler will automatically convert between S and T as necessary, including recursively inside other structs. T could then serve as a de-facto alias of S (or vice versa) for gradual refactoring purposes.

1. Add a new "looks like" kind of type.

Let type T ~S declare a new type T that is a type that "looks like S". More precisely, T is "any type convertible to and from type S". (As always, syntax could be discussed later.) Like interface types, T cannot have methods; to do basically anything at all with T, you need to convert it to S (or a type convertible to/from S). Unlike interface types, there is no "concrete type", conversion between S to T and T to S involves no representation changes. For gradual refactoring, these "looks like" types would allow authors to write APIs accepting both old and new types. ("Looks like" types are basically a highly restricted, simplified union type.)

1. Type tags

Bonus super-hideous idea. (Please don't bother telling me this is awful--I know it. I'm only trying to spur new ideas in others.) What if we introduced type tags (like struct tags), and used special type tags to set up and control aliases, like say type T S "alias:\"T\"". Type tags will have other uses as well and it provides scope for more specification of aliases by the package author than merely "this type is an alias"; for example, the author of the code could specify embedding behavior.

[nigeltao]

If we do try aliases again, it might be worth thinking about "what does godoc do", similar to the "what does iota do" and "what does embedding do" issues.

Specifically, if we have

type  OldAPI => NewPackage.API

and NewPackage.API has a doc comment, are we expected to copy/paste that comment next to "type OldAPI", are we expected to leave it un-commented (with godoc automatically providing a link or automatically copy/pasting), or will there be some other convention?

[nigeltao]

Somewhat tangential, while the primary motivation is and should be supporting gradual code repair, a minor use case (going back to the alias proposal, since that is a concrete proposal) could be to avoid a double function-call overhead when presenting a single function backed by multiple, build-tag-dependent implementations. I'm only hand-waving right now, but I feel like aliases could have been useful in the recent https://groups.google.com/d/topic/golang-nuts/wb5I2tjrwoc/discussion "Avoiding function call overhead in packages with go+asm implementations" discussion.

[jimmyfrasche]

@nigeltao re godoc, I think:

It should always link to the original, regardless.

If there's docs on the alias, those should be displayed, regardless.

If there are not docs on the alias, it's tempting to have godoc display the original docs, but the name of the type would be wrong if the alias also changed the name, the docs could refer to items not in the current package, and, if it's being used for gradual refactoring, there could be a message that says "Deprecated: use X" when you're looking at X.

However, maybe that wouldn't matter for the majority of use cases. Those are things that could go wrong, not things that will go wrong. And some of them could be detected by linting, like renamed aliases and accidentally copying deprecation warnings.

[tux21b]

I am not sure if the following idea had been posted before, but what's about a mostly tool-based "gofix" / "gorename" like approach? To elaborate:

• any package can contain a set of rewriting rules (e.g. mapping pkg.Ident => otherpkg.Ident)
• those rewriting rules can be specified with //+rewrite ... tags inside arbitrary go files
• those rewriting rules are not limited to ABI compatible changes, it's also possible to do other things (e.g. pkg.MyFunc(a) => pkg.MyFunc(context.Contex(), a))
• a gofix like tool can be used to apply all transformations to the current repository. This makes it easy for users of a package to update their code.
• it's not necessary to call the gofix tool in order to compile successfully. A library that still wants to use the old API of a dependency X (to stay compatible with old and new versions of X) can still do so. The go build command should apply the transformations (specified in the rewrite tags of package X) on-the-fly without changing the files on disk.

The last steps might complicate / slow-down the compiler a bit, but it's basically just a pre-processor and the amount of rewrite rules should be kept small anyway. So, enough brainstorming for today :)

[uluyol]

Using aliases to avoid function call overhead seems like a hack to work around the compiler's inability to inline non-leaf functions. I don't think implementation deficiencies should influence the language spec.

[Merovius]

@josharian While you didn't intend them as full proposals, let me response (even if only, so that whoever is inspired by you can take the immediate criticism into account):

1. Doesn't really solve the problem, because conversions aren't really the issue. x/net/context.Context is assignable/convertable/whateverable to context.Context. The problem are higher-order types; namely the types func (ctx x/net/context.Context) and func (ctx context.Context) are not the same, even though the arguments are assignable. So, for 1 to solve the problem, type T struct { S } would need to mean, that T and S are identical types. Which means, that you are simply using a different syntax for aliases after all (just that this syntax already has a different meaning).

2. Again has a problem with higher-order types, because assignable/convertible types do not necessarily have the same memory representation (and if they do, the interpretation might change significantly). For example, an uint8 is convertible to an uint64 and vice-versa. But that would mean, that, e.g. with type T ~uint8, the compiler can't know how to call a func(T); does it need to push 1, 2,4 or 8 bytes on the stack? There might be ways around this issue, but it sounds pretty complicated to me (and harder to understand than aliases).

[josharian]

Thanks, @Merovius.

1. Yes, I missed interface satisfaction here. You're right, this doesn't do the job.

2. I had in mind "have the same memory representation". Convertible back-and-forth is clearly not the right elucidation of that--thanks.

[nigeltao]

@uluyol yes, it's largely about the compiler's inability to inline non-leaf functions, but explicit aliasing might be less surprising with respect to whether or not inlined calls to non-leafs should show up in stack traces, runtime.Callers, etc.

In any case, as I said, it's a minor tangent.

[Merovius]

@josharian Similar problem: [2]uintptr and interface{} have the same memory representation; so only relying on memory representation will allow circumventing type safety. uint64 and float64 have both the same memory representation and are convertible back-and-forth, but would still lead to really weird results at the least, if you don't know which is which.

You might get away with "same underlying type", though. Not sure what the implications would be for that. Off the top of my hat, that might lead to wrongness if a type is used in fields, for example. If you have type S1 struct { T1 } and type S2 struct { T2 } (with T1 and T2 the same underlying type), then under type L1 ~T1 both might be work as type S struct { L1 }, but as T1 and T2 still have a different (though looking alike) underlying type, with type L2 ~S1 you won't have S2 looking alike S1 and not be usable as an L2.

So you'd have to, in a bunch of places in the spec, replace or amend "identical types" with "same underlying type" to make this work, which seems unwieldy and will probably have unforeseen consequences for type safety. "look-alike" types also seem to have an even greater abuse and confusion potential than aliases, IMHO, which seem to be the main arguments against aliases.

If anyone can come up with a simple rule for it, though, that doesn't have these problems, it should definitely be considered as an alternative :)

[iand]

Following on from @josharian's ideation, here's a variation of his number 2:

Allow the specification of "substitutable types". This is a list of types that may be substituted for the named type in function arguments, return values etc. The compiler would allow calling of a function with an argument of the named type or any of its substitutes. The substitute types must have a compatible definition with the named type. Compatible here means identical memory representations and identical declarations after allowing for other substitute types in the declaration.

One immediate problem is that the directionality of this relationship is opposite to the alias proposal which inverts the dependency graph. This alone might make it unworkable but I propose it here because others might think of a way around this. One way might be to declare substitutes as //go comments rather than via the import graph. In this way they perhaps become more like macros.

Conversely there are some advantages to this reversal of directionality:

• the set of substitutable types is controlled by the author of the new package who is better placed to guarantee semantics
• no code changes are required in the original package so clients don't have to update until they start using the new package

Applying this to the Context refactoring: the standard library context package would declare that context.Context may be substituted by golang.org/x/net/context.Context. This means any usage that accepts context.Context may also accept a golang.org/x/net/context.Context in its place. However functions in the context package that return a Context would always return a context.Context.

This proposal circumvents the embedding issue (#17746) because the name of the embedded type never changes. However, an embedded type could be initialised using a value of a substitute type.

[4ad]

@iand @josharian you are asking for a certain variant of covariant types.

[rsc]

@josharian, thanks for the suggestions.

Re type T struct { S }, that looks like a different syntax for alias, and not necessarily a clearer one.

Re type T ~S, I am either not sure how it differs from alias or not sure how it helps refactoring. I guess in a refactoring (say, io.ByteBuffer -> bytes.Buffer), you would write:

package io
type ByteBuffer ~bytes.Buffer

but then if, as you say, "to do basically anything at all with T, you need to convert it to S", then all the code doing anything with io.ByteBuffer still breaks.

Re type T S "alias": A key point @bcmills made above is that having multiple equivalent names for types is a language change, no matter how it is spelled. All compilers need to know that, say, io.ByteBuffer and bytes.Buffer are the same, as do any tools that analyze or even type-check code. The key part of your suggestion seems to me something like "maybe we should plan ahead for other additions". Maybe, but it's unclear that a string would be the best way to describe those, and it's also unclear we want to design syntax (like Java generalized annotations) without a clear need. Even if we did have a general form, we'd still need to consider carefully all the implications of any new semantics we introduced, and most would still be language changes that would require updating all tools (except gofmt, admittedly). On balance it seems simpler to continue to find the clearest way to write the forms we need one by one instead of creating a meta-language of one kind or another.

@Merovius FWIW, I would say that [2]uintptr and interface{} do not have the same memory representation. An interface{} is a [2]unsafe.Pointer not a [2]uintptr. A uintptr and a pointer are different representations. But I think your general point is right, that we do not want to necessarily allow direct conversion of that kind of thing. I mean, can you convert from interface{} to [2]*byte too? It's a lot more than is needed here.

[rsc]

@jimmyfrasche and @nigeltao, re godoc: I agree that we need that working early too. I agree that we should not hard-code the assumption "the new feature - whatever it is - will only be used for codebase refactoring". It may have other important uses, like Nigel found for helping to write a draw extension package with aliases. I expect that deprecated things will be marked deprecated in their doc comments explicitly, as Jimmy said. I did think about generating a doc comment automatically if one is not there, but there's nothing obvious to say that shouldn't already be clear from the syntax (speaking generally). To make a specific example, consider the old Go 1.8 aliases. Given

type ByteBuffer => bytes.Buffer

we could synthesize a doc comment saying "ByteBuffer is an alias for bytes.Buffer", but that seems redundant with displaying the definition. If someone writes "type X struct{}" today, we don't synthesize "X is a named type for a struct{}".

[rsc]

@iand, thanks. It sounds like your proposal requires the author of the new package to write the exact definition from the old package and then also a declaration linking the two, like (making up syntax):

package old
type T { x int }

package new
import "old"
type T1 { x int }
substitutable T1 <- old.T

I agree that the import reversal is problematic and may be a show-stopper by itself, but let's skip that. At this point the codebase seems like it is in a fragile state: now package new can be broken by a change to add a struct field in package old. Given the substitutable line, there is only one possible definition for T1: exactly the same as old.T. If the two types still have distinct definitions then you also have to worry about the methods: do the method implementations need to match too? If not, what happens when you put a T in an interface{} and then pull it out using a type assertion as a T1 and call M()? Do you get T1.M? What if you pull it out as an interface { M() }, without naming T1 directly, and call M()? Do you get T.M? There's a lot of complexity caused by the ambiguity of having both definitions in the source tree.

Of course, you could say that the substitutable line makes the rest redundant and not require a definition for type T1 or any methods. But then that's basically the same as writing (in the old alias syntax) type T1 => old.T.

Getting back to the import graph issue, although the examples in the article all made the old code defined in terms of the new code, if the package graph were such that new had to import old instead, it's equally effective to put the redirect in the new package during the transition.

I think this shows that in any transition like this, there's probably not a useful distinction between the author of the new package and the author of the old package. By the end, the goal is that code has been added to new and deleted from old, so both authors (if they're different) need to be involved then. And the two need some kind of coordinated compatibility during the middle too, whether explicit (some kind of redirect) or implicit (type definitions must match exactly, as in the substitutability requirement).

[iand]

@rsc that breakage scenario suggests that any type aliasing needs to be bidirectional. Even under the previous alias proposal any change in new package could potentially break any number of packages that happen to have aliased the type.

[rsc]

@iand If there's only one definition (because the other says "same as _that_ one") then there's no worry about them not being in sync.

[ianlancetaylor]

Over in #13467, @joegrasse points out that it would be nice if this proposal provided a mechanism for permitting identical C types to become identical Go types when using cgo in multiple packages. That is not at all the same problem as the one that this issue is for, but both problems are related to type aliasing.

[thwd]

Is there any summary of proposed/accepted/rejected restrictions/limitations on aliases? Some questions that pop to mind are:

• Is the RHS always fully-qualified?
• If aliases to aliases are allowed, how do we handle alias cycles?
• Should aliases be able to export unexported identifiers?
• What happens when you embed an alias? (how do you access the embedded field)
• Are aliases available as symbols in the built program?
• ldflags string injection: what if we refer to an alias?

[iand]

@rsc I don't want to divert the conversation too much but under the alias proposal if "new" removes a field that "old" relied on it means clients of "old" now can't compile.

However, under the substitute proposal I think it could be arranged that only clients that use both old and new together would break. For that to be possible then the substitution directive would have to validated only when the compiler detects a use of "old" types in "new" package.

[Merovius]

@thwd I don't think there is a good writeup yet. My notes:

• Alias cycles are a non-issue. In case of package-crossing aliases, a cycle is already disallowed because of an import-cycle. In case of non package-crossing aliases, they obviously need to be disallowed, which is very similar to cycles in initialization order. Personally, I'd like to have aliases to aliases, because I don't think they should be restricted to gradual repair use cases (see my comment above) and it would be sad, if package A could break by someone moving a type in package B with an alias (imagine x/image/draw.Image aliasing draw.Image and then someone deciding to move draw.Image into image.Draw via an alias, assuming it safe. Suddenly x/image/draw breaks, because aliases to aliases are not allowed).
• I think previously proponents of aliases have agreed that alias-exporting unexported identifiers is probably a bad idea due to the weirdness it may cause. Effectively that means that aliases to unexported identifiers are useless and may be disallowed completely.
• The embedding question, AFAIK, is as of yet unsolved. There is a whole discussion in #17746, I expect that this discussion will be continued if/when/before it is decided to go forward with aliases (but there's still the possibility of an alternative solution or the decision to not make gradual repairs a goal at all)

[rsc]

@iand, re "only clients that use both old and new together would break", that's the only interesting case. It's the mixed clients that make it a gradual code repair. Clients that use only the new code or only the old code will work today.

[iainmerrick]

There's something else to consider, that I haven't seen mentioned elsewhere yet:

Since an explicit goal here is to allow large, gradual refactoring in large decentralized codebases, there'll be situations where a library owner wants to do some kind of cleanup that will require an unknown number of clients to change their code (in the final "retire the old API" step). A common way to do that is to add a deprecation warning, but the Go compiler doesn't have any warnings.

Without any kind of compiler warning, how can a library owner be confident that it's safe to complete the refactoring?

One answer might be some kind of versioning scheme — it's a new release of the library with a new incompatible API. In that case maybe versioning is the whole answer, not type aliases.

Alternatively, how about allowing the library author to add a "deprecation warning" that actually causes a compile _error_ for clients, but with an explicit algorithm for the refactoring that they need to perform? I'm imagining something like:

Error: os.time is obsolete, use time.time instead. Run "go upgrade" to fix this.

For type aliases, I guess the refactoring algorithm would just be "replace all instances of OldType with NewType", but there might be subtleties, I'm not sure.

Anyway, that would allow the library author to make a best-effort attempt to warn all clients that their code is about to break, and give them an easy way to fix it, before deleting the old API completely.

[zombiezen]

@iainmerrick There are bugs open for these: golang/lint#238 and golang/gddo#456

[jimmyfrasche]

Solving the gradual code repair problem, as outlined in @rsc's article, reduces to requiring a way for two types to be interchangeable (as workarounds exist for vars, funcs, and consts).

This needs either a tool or a change to the language.

Since making two types interchangeable is, by definition, changing how the language works, any tool would be a mechanism for simulating the equivalence outside of the compiler, likely by rewriting all the instances of the old type to the new type. But this means such a tool would have to rewrite code you don't own, like a vendored package that uses golang.org/x/net/context instead of the stdlib context package. The specification for the change would either have to be in a separate manifest file or a machine-readable comment. If you don't run the tool you get build errors. That all gets messy to deal with. It seems like a tool would create as many problems as it solves. It'd still be a problem everyone using these packages has to deal with, albeit somewhat nicer since a portion is automated.

If the language is changed, code only needs to be modified by its maintainers, and, for most people, things just work. Tooling to aid the maintainers is still an option, but it'd be much simpler since the source is the specification, and only the maintainers of a package would need to invoke it.

As @griesemer pointed out (I don't recall where, there have been so many threads about this) Go already has aliasing, for stuff like byte ↔ uint8, and when you import a package twice, with different local names, into the same source file.

Adding a way to explicitly alias types in the language is just allowing us to use semantics that already exist. Doing so solves a real problem in a manageable way.

A language change is still a big deal and a lot of things need to be worked out, but I think that it's ultimately the right thing to do here.

[akavel]

As far as I'm aware, one "elephant in the room" is the fact, that for type aliases, introducing them will allow for non-temporary (i.e. "non-refactoring") usages. I've seen those mentioned in passing (for example, "reexporting type identifiers in different package to simplify API"). Keeping up with good tradition of previous proposals, please list all the known alternative usages of type aliases under "impact" subsection. This should also bring the benefit of fueling people's imagination for inventing additional possible alternative usages and bringing them into light in the current discussion. As is now, the proposal appears to pretend that authors are completely unaware of other possible uses of type aliases. Also, as to reexporting, Rust/OCaml may have some experience with how those work for them.

Additional question: please clarify if type aliases would allow adding methods to the type in the new package (arguably breaking encapsulation) or not? also, would the new package get access to private fields of old structs, or not?

[jba]

Additional question: please clarify if type aliases would allow adding methods to the type in the new package (arguably breaking encapsulation) or not? also, would the new package get access to private fields of old structs, or not?

An alias is just another name for a type. It doesn't change the type's package. So no to both of your questions (unless new package == old package).

[rsc]

@akavel As of now, there is no proposal at all. But we do know of two interesting possibilities that came up during the Go 1.8 alias trials.

1. Aliases (or just type aliases) would enable creating drop-in replacements that expand other packages. For example see https://go-review.googlesource.com/#/c/32145/, especially the explanation in the commit message.

2. Aliases (or just type aliases) would enable structuring a package with a small API surface but a large implementation as a collection of packages for better internal structure but still present just one package to be imported and used by clients. There's a somewhat abstract example described at https://github.com/golang/go/issues/16339#issuecomment-232813695.

[niemeyer]

The underlying goal of aliases is great, but it still sounds like we're not being quite honest to the goal of refactoring code, despite it being the number one motivator for the feature. Some of the proposals suggest locking down the name, and I haven't seen it mentioned yet that types usually change their surface with such refactorings too. Even the example of os.Error => error often mentioned around aliases ignores the fact that os.Error had a String method and not Error. If we just moved the type and renamed it, all error handling code would be broken regardless. That's common place during refactorings.. old methods get renamed, moved, dropped, and we don't want them in the new type as it would preserve the incompatibility with new code.

In the interest of helping out, here is a seed idea: what if we looked at the problem in terms of adapters, instead of aliases? An adapter would give an existing type an alternative name _and interface_, and it can be used unadorned in places where the original type was seen before. The adapter would need to explicitly define the methods it supports, rather than assuming the same interface of the underlying adapted type are present. This would be much like the existing type foo bar behavior, but with some additional semantics.

io.ByteBuffer

For instance, here is an example skeleton addressing the io.ByteBuffer case, using the temporary "adapts" keyword for the time being:

type ByteBuffer adapts bytes.Buffer

func (old *ByteBuffer) Write(b []byte) (n int, err error) {
        buf := (*bytes.Buffer)(old)
        return buf.Write(b)
}

(... etc ...)

So, with that adapter in place, this code would all be valid:

func newfunc(b *bytes.Buffer) { ... }
func oldfunc(b *io.ByteBuffer) { ... }

func main() {
        var newvar bytes.Buffer
        var oldvar io.BytesBuffer

        // New code using the new type obviously just works.
        newfunc(&newvar)

        // New code using the old type receive the underlying value that was adapted.
        newfunc(&oldvar)

        // Old code using the old type receive the adapted value unchanged.
        oldfunc(&oldvar)

        // Old code gets new variable adapted on the way in. 
        oldfunc(&newvar)
}

The interfaces of newfunc and oldfunc are compatible. Both actually accept *bytes.Buffer, with oldfunc adapting it to *io.BytesBuffer on the way in. The same concept works for assignments, results, etc.

os.Error

The same logic probably be made to work on interface too, although the compiler implementation of it is a bit trickier. Here is an example for os.Error => error, which handles the fact the method was renamed:

package os

type Error adapts error

func (e Error) String() string { return error(e).Error() }

This case needs further thinking, though, because methods such as:

func (v *T) Read(b []byte) (int, os.Error) { ... }`

Will be returning a type that has a String method, so we'd usually want to adapt in the opposite direction so code can be gradually fixed.

_UPDATED: Needs further thinking._

Embedding problem

In terms of the embedding bug that dragged the feature out of 1.8, the outcome is a bit more clear with adapters, since they're not just new names for the same thing: if the adapter is embedded, the field name used is that of the adapter so old logic remains working, and accessing the field will use the adapter interface unless explicitly handed into a context that takes the underlying type. If the unadapted type is embedded, the usual happens.

kubernetes, docker

The problems stated in the post seem like variations of the above issues, and solved by the proposal.

vars, consts

It wouldn't make much sense to adapt variables or constants under that scenario, since we can't really associate methods with them directly. It's their types that would be adapted or not.

godoc

We'd be explicit about the fact the thing is an adapter, and show the documentation for it as usual, since it contains an independent interface from the adapted thing.

syntax

Please pick something nice. ;)

[rakyll]

@iainmerrick @zombiezen

Should we also automatically infer that an aliased type is legacy and should be replaced by the new type? If we enforce golint, godoc and similar tools to visualize the old type as deprecated, it would limit the abuse of type aliasing very significantly. And the final concern of aliasing feature being abused would be resolved.

[ulikunitz]

Two observations:

1. Semantics of type references depend on supported refactoring use case

Gustavo's proposal shows that there needs to more work on the use case for type references and the resulting semantics.

Ross' new proposal includes a new syntax type OldAPI = newpkg.newAPI. But what are the semantics? Is it impossible to extend OldAPI with legacy public methods or fields? Assuming yes as an answer that requires the newAPI to support all public methods and fields of OldAPI to maintain compatibility. Please note that any code in the package with OldAPI that relies on private fields and methods must be rewritten to use only the public newAPI assuming that modifying the visibility constraints of packages is off the table.

The alternate path would be to allow additional methods to be defined for OldAPI. That could ease the burden on NewAPI to provide all public old methods. But that would make OldAPI a different type than NewAPI. Some form of assignability between values of the two types has to be maintained, but rules would become complex. Allowing the addition of fields would result in more complexity.

2. Package with NewAPI cannot import package with OldAPI

Redefinition of the OldAPI requires that package O containing the definition of OldAPI imports package N with the NewAPI. That implies that package N cannot import O. Maybe it is so obvious that it hasn't been mentioned, but it seems to me an important constraint for the refactoring use case.

Update: Package N cannot have any dependency on package O. For example it cannot import a package that imports O.

[Merovius]

@niemeyer Changes like renaming a method are already gradually possible: a) Add the new method, call the old one under the hood (or vice versa), b) gradually change all users to the new method, c) delete the old method. You can combine that with a type alias. The reason this focuses on type-moving is, that this is the only thing identified, that is not yet possible. All other identified changes are possible, even if they may use several steps (for example changing the set of arguments of a method without renaming it). I believe choosing a fix with less surface area (less things to understand) is preferable.

@rakyll Personally, if I'd consider aliases useful for something non-refactoring (like wrapper-packages, which I find an excellent use case) I'd just use them, deprecation-warnings be damned. I'd be pissed at whoever artificially crippled them and made them confusing for my users, but I wouldn't be discouraged.

I think at some point it needs to be debated whether we actually consider wrapper-packages, protobuf public imports or exposing internal package-APIs such a bad thing (and I don't know how to best debate something this subjective without one side just repeating over and over that they're unreadable and the other saying "no they're not". There isn't a lot of objective argument to be had here, it seems to me).

I at least (obviously) think they're a good thing and I'm also of the opinion that adding a language feature and artificially restricting it to only one use case is a bad thing; an orthogonal, well-designed language allows you to do as much as possible with as little features as possible. You want your features to expand the "spanned vector space of possible programs" as much as possible, so adding a feature which only adds a single point to the space seems weird to me.

[glyn]

I'd like another, slightly different use case to be born in mind as any type alias proposal is developed.

Although the main use case we are discussing in this issue is type _replacement_, type aliases would also be very useful for weaning a body of code off a dependency on a type.

For example, suppose a type turned out to be "unstable" (i.e. it keeps being changed, perhaps in incompatible ways). Then some its users might want to migrate to a "stable" replacement type. I'm thinking of development on github etc. where the owners of a type and its users do not necessarily work closely together or agree on the objective of stability.

Other examples would be where a single type is the only thing stopping a dependency on a large or problematic package from being deleted, e.g. where a license incompatibility has been discovered.

So the process here would be:

1. Define the type alias
2. Change the relevant body of code to use the type alias
3. Replace the type alias with a type definition.

At the end of this process, there would be two independent types which would be free to evolve in their own directions.

Note that in this use case:

• there is no option of changing the package containing the original type definition to add a type alias there (since the owners are unlikely to agree to this)
• the original type is not deprecated (although it might be considered as such in the body of code in the process of being "weaned off" the type).

[niemeyer]

@Merovius The moment you delete or rename the old method, you kill every client that was using it, at once. If you're willing to do that, the whole non-trivial exercise of adding a language feature to prevent all-at-once breakage is moot. We might as well say exactly the same thing for moving the code: just rename the type on every call site at once. Done. Both actions are simply atomic renames, which have in common the fact they assume complete access to every line of code in the call sites. This might be the case for Google, but as a maintainer of large open source applications and libraries, it's not the world I live in.

In most cases I find that criticism unfair, since the Go team goes through great lengths to make the project inclusive to external parties, but the moment you assume you have access to every single line of code that is calling a given package, that's a walled garden which doesn't match the context of an open source community. Adding a language-level refactoring feature that only works inside walled gardens would be atypical, to say the least.

[Merovius]

@niemeyer I apparently didn't make myself clear. I wasn't advocating for deleting the old API in any case, I was just pointing out that any workflow that we want to enable with type aliases is already possible with renaming methods (whether at the same time or not). So, no matter what you want to do, for

1. Add new API, interchangeable with old API
2. Gradually change consumers over to new API
   3a. Once everything is migrated or the deprecation period runs out, delete the old API
   3b. Provide indefinite stability by keeping both APIs around forever (see e.g. this part of the article)

You seem to be arguing about doing 3a vs. 3b. But what I was pointing out, that 1. is already possible for method-names but not possible for types, which is what this is about.

Though, I now realize that I think I misunderstood you :) You might have been pointing out, that os.Error are different interface-definitions, so the move doesn't really pan out. I think that is true; if you forbid removing APIs, type aliases would not make it possible to rename methods of interface types.

Maybe you can clarify something about your adapter idea for me, though: Wouldn't that also allow to use (for example, in the os.Error case) any fmt.Stringer as an os.Error?

In any case, the adapter idea seems worthwile to develop further, even if I'm slightly sceptical about it. But having a way to gradually refactor interfaces without breaking possible implementers and/or consumers is a good goal.

[rsc]

@niemeyer Yes, you bring up a good point about the method name changing in error too. That introduces many complications, and it's not something I'm trying to tackle here. Because only a fraction of code mentioning error/os.Error actually calls the method, the move was the more painful part than the method change. I think we can treat method renamings as an independent problem from changing code location. If the move were happening today and we could do the package reorg seamlessly but be stuck with the old method name, that would still be significant progress. Focusing this issue on code location is meant to try to simplify.

I agree that if there were some general fix that handled both kinds of changes, that would be great. I don't see what that fix is. In particular I don't understand how type switches work with the adapters you described: does the value somehow automatically convert during the type switch? What about reflection? Having only one type with two names avoids many problems that arise with having two types that auto-convert back and forth.

[niemeyer]

@rsc Yes, the adapter would consistently automatically convert in every situation, so type switches would be no different. We'd forbid type switches containing both the adapter and its underlying type, as that would be ambiguous. I might be missing something, but can't yet see a problem with reflection, since every code context needs to necessarily either be using the adapted type, or its underlying type, explicitly. Just like today, we can't go into an interface{} without knowing how we got there, if that makes sense.

@Merovius My two comments above address precisely the points you're still making. If you move a type today, you break code that needs fixing. If you rename a method, you break code that needs fixing. If you delete a method, change their arguments, you break code that needs fixing. When refactoring code in any of these cases, the fixes need to be done atomically with the breakage in every call site for things to continue working. Allowing the type to be moved but completely untouched is a very limited case of refactoring, which IMO doesn't justify a language feature.

[rsc]

@niemeyer That would handle the concrete types. What about a type assertion for .(interface{String() string}) vs .(interface{Error() string}) or whatever specific pieces of interface changed? Does the check have to consider both possible underlying types somehow?

[Merovius]

@niemeyer No. Renaming a method is possible non-atomically. e.g. to move a method from A.Foo to A.Bar, do

1. Add method A.Bar as a wrapper around A.Foo
2. Migrate users to only call A.Bar via arbitrarily many commits
3. Either delete A.Foo, or don't, depending on if you are willing to enforce a deprecation.

Change functions arguments is possible non-atomically. e.g. to add a parameter x int to a func Foo(), do

1. Add func FooWithInt(x int) { Foo(); // use x somehow; }
2. Migrate users to add the parameter via arbitrarily many commits
3. If you are not willing to enforce a deprecation (or you are not bothered by having the WithInt), you are done. Otherwise modify Foo to func Foo(x int) { FooWithInt(x) }.
4. Migrate users with s/FooWithInt/Foo/g via arbitrarily many commits.
5. Delete FooWithInt.

The same works for pretty much all cases except moving types (and, strictly speaking, vars). atomicity isn't required. You either break compatibility when enforcing deprecation, or you don't, but that's completely orthogonal to atomicity. The ability to use two different names to refer to the same thing is, what allows you to sidestep atomicity when doing basically arbitrary changes and you have that ability for all cases except types. Yes, to do an actual move, instead of an amendment, you need to be willing to enforce deprecation (so breaking the build of potentially unknown code, meaning this needs wide and timely announcement). But even if you are not, the ability to augment APIs with a more convenient name or other useful wrapping (see x/image/draw) also depends on the ability to refer to the old thing by the new name and vice-versa.

The difference between moving types today and renaming a function today is, that in the former case, you actually need an atomic change, whereas for the latter, you can make the change gradually, over independent repos and commits. Not as a "I'll make a commit that does s/Foo/Bar/", but there is a process to do it.

Anyway. I don't know where we are, apparently, talking past each other. I find @rsc's document pretty clear to convey my POV and don't really get yours :)

[niemeyer]

@rsc I can see two reasonable answers. The simple one that the interface carries the type that went in, adapter or otherwise, and usual semantics apply when interface-asserting. The other one is that the value may be unadapted if it doesn't satisfy the interface but the underlying value does. The former one is simpler and perhaps enough for the refactoring use cases we have in mind, while the latter is perhaps more consistent with the idea that we can type-assert it to the underlying type as well.

@Merovius Sure, renaming a method is possible as long as _you don't actually rename it_ and force call sites to use a new API instead. Likewise, moving a type is possible as long as _you don't actually move it,_ and force call sites to use a new API instead. We've all been doing both of these things for years to preserve old code working.

[Merovius]

@niemeyer But again: For types, you can't even add things in a decent manner. See x/image/draw. And not everyone might have such an absolute view of stability; I, myself, am fine with saying "in 6,12,… months $function,$type,… is going away, be sure that you are migrated away from it at that point" and then just break unmaintained code that doesn't manage to follow that deprecation notice (if someone thinks they need long-term-support for APIs, they surely can find someone to pay to provide that). I'd even claim that most people don't have that absolute view on stability; see the recent push for semantic versions, which only really makes sense if you do want to have the option to break compatibility. And the doc argues very well, how, even in that case, you would still profit from the ability of having gradual repairs and how it can alleviate, if not essentially solve the diamond dependency problem.

You might dismiss most of the use cases of aliases for gradual repairs because your stance on stability is absolute. But I'd claim that for most of the go community, that is different, that there is a want for breakages and a use in making them as smoothly as possible when they do happen.

[thwd]

@niemeyer @rsc @Merovius I have been following your discussion (and the entire discussion) and I'd like to blatantly smack this post right in the middle of it.

The more we iterate over the issue, the closer we get to some form of extended covariance semantics. So, here goes a thought: we already have subtype semantics ("is-a") defined from concrete types to interfaces and among interfaces. My proposal is to make interfaces recursively covariant (according to the current variance rules) down to their methods arguments.

This does not solve the problem for all current packages. But it can solve the problem for all future packages, yet to be written, in that the "moveable parts" of the API can be interfaces (encourages good design, too).

I think we can solve all requirements by (ab)using interfaces in this way. Are we breaking Go 1.0? I dont know but I think we're not.

[griesemer]

@thwd I think you need to define more precisely what you mean by "making interfaces recursively covariant". Usually, in subtyping, method arguments need to change in contravariant ways, and the results in covariant ways. Also, from what you're saying this wouldn't solve any existing problem with concrete (non-interface) types.

[Merovius]

@thwd I disagree, that interfaces (even covariant ones) are a good solution to any of these problems (only to very specific instances of it). To make them one, you'd need to make everything in your API an interface (because you never know what you might want to move/change at some point), including vars/consts/funcs/… and I don't think at all, that that is good design (I've seen that in java. It aggravates me). If something is a struct, just make it a struct. Everything else just adds weird syntactic overhead in your package and every reverse dependency for virtually no benefit. It's also the only way to stay sane when you begin; start simple and move to something more general later. A lot of complications in API I've seen so far come from people overthinking API design and planning for way more generality than will ever be needed. And then, in 80% (that number is an obvious lie) of the cases, nothing happens at all, because there is no "clean API design".

(to be clear: I'm not saying that covariant interfaces aren't a good idea. I'm just saying that they are not a good solution to these problems)

[zombiezen]

To add to @Merovius's point, many gradual code repairs that I've seen have taken the form of moving a generally useful non-interface type out of a much larger package. Consider the following:

package foo

type Authority struct {
  Host string
  Port int
}

Over time, the foo package grows and it ends up gaining more responsibility (and code size) than someone who just needs the Authority type really wants. So having a way of creating a fooauthority package that just contains Authority and having existing users of foo.Authority still work is a desirable use-case. Note that any solution that only considers interface types would not help here.

[niemeyer]

@Merovius Your last comment has been entirely subjective and is addressing me personally instead of my proposal. This won't end well, so I'll stop that line of discussion here.

[thwd]

@griesemer @Merovius I agree with both of you. To close the loop, then, we can agree that the discussion so far has lead us to some notion of subtypes/covariance. Also, that any implementation of it should incur no runtime indirection. That's kind of what @niemeyer was proposing (if I understood him right). But I'd love to read more ideas. I'll be thinking about the problem, too.

[jba]

@niemeyer There was nothing _ad hominem_ in @Merovius's comments. His claim that "your stance on stability is absolute" is an observation about your position, not you, and is a reasonable inference from some of your statements, like

The moment you delete or rename the old method, you kill every client that was using it, at once.

and

Sure, renaming a method is possible as long as you don't actually rename it and force call sites to use a new API instead. Likewise, moving a type is possible as long as you don't actually move it, and force call sites to use a new API instead. We've all been doing both of these things for years to preserve old code working.

I got the same impression as Merovius from those statements—that you aren't sympathetic to deprecating something for a while, then eventually removing it; that you are committed to keeping code in the wild working indefinitely; that "your stance on stability is absolute". (And to forestall further misunderstanding, I'm using "you" to refer to your ideas, not your personality.)

[bcmills]

@niemeyer The adapts declaration you're suggesting seems closely related to instance from Haskell typeclasses. Loosely translating that to Go, it might look something like:

package os

type Error interface {
  String() string
}

instance error Error (
  func (e error) String() string { return e.Error() }
)

Unfortunately (as @zombiezen notes), it's not clear how this would help for non-interface types.

It's also not obvious to me how it would interact with function types (arguments and return-values); for example, how would the semantics of adapts help with migrating Context to the standard library?

[niemeyer]

I got the same impression as Merovius from those statements—that you aren't sympathetic to deprecating something for a while

@jba These are absolute facts, not an absolute opinions. If you delete a method or a type, Go code using it breaks, so these changes need to be done atomically. My proposal, though, is about gradual refactoring of code, which is the subject here and implies deprecation. That process of deprecation, though, isn't a matter of sympathy. I've got multiple public Go packages with thousands of in-the-wild dependencies each, and multiple independent APIs due to that gradual evolution. When we break an API, it's good to do such breakages in batches, instead of streaming them, if we expect to not get people insane. Unless, of course, you live in a walled garden and can reach out to every call site to fix it. But I'm repeating myself.. all of that can be read in the original proposal above in a more articulated way.

[rakyll]

@Merovius

Personally, if I'd consider aliases useful for something non-refactoring (like wrapper-packages, which I find an excellent use case) I'd just use them, deprecation-warnings be damned.

We maintain packages with wildly large number of new and deprecated APIs and having aliases without clear explanation of the state of the old (aliased) type is not going to help the gradual code repair and will only contribute to the overwhelmingness of the increased API surface. I agree with @niemeyer that our solution needs to address the requirements of a distributed developer community which currently doesn't have any other signals than free-form godoc text saying that an API is "deprecated". Adding a language feature to help deprecating old types is the topic of this thread, hence it naturally leads to question of what's the state of the old (aliased) type.

I would love to discuss type aliasing under a different theme such as providing extension to a type or partial packages but not on this thread. That topic itself has various encapsulation-specific problems to be addressed before any consideration.

A specific operator or implying that the aliased typed is somewhat replaced could be healthy to communicate to the users that they need to switch. Such differentiability would enable tools to automatically report the replaced APIs.

To be clear, deprecation policy is not technically possible for types outside of the standard library. A type is only old from the perspective of an aliasing package. Given we can never enforce this in the ecosystem, I would still like to see standard library aliases implies deprecation strictly (hinted by proper deprecation notices).

I am also suggesting we standardize the notion of deprecation in a parallel discussion and land support for them in our core tools (golint, godoc, etc). Lacking deprecation notices is the biggest problem in the Go ecosystem and is more wide-spread than the problem of gradual code repair.

[Merovius]

@rakyll I'm sympathetic to the use case of having computer-readable deprecation notices; I just object to the notion of a) aliases being that and b) emitting them as compiler warnings.

For a), apart from the fact that I'd like to use aliases productively for other things than moves, it would also only apply for a very small set of deprecations. For example, say I'd want to remove some parameters from a function in a couple of releases; I can't use aliases, really, because the signature of the new API will differ, but I'd still want to announce that. For b), IMHO compiler warnings are universally bad. I think this is mostly in line with what go is already doing, so I don't think it requires justification.

I agree with all of what you are saying about deprecation notices. There already is a syntax for this, apparently: #10909, so the next step to make it more useful would be to enhance tool-support by highlighting them in godoc and have a check that warns about their usage (say go vet, golint or a separate tool alltogether).

[jimmyfrasche]

@rakyll I agree that the stdlib should start with a conservative use of type aliases, should they be introduced.

---

Sidebar:

Background for those unaware of the status of deprecation comments in Go and related tooling, as it is rather spread out:

As @Merovius mentions above, there is a standard convention for marking items as deprecated, #10909, see https://blog.golang.org/godoc-documenting-go-code

TL;DR: make a paragraph in the docs of the deprecated item that begins with "Deprecated: " and explains what the replacement is.

There's an accepted proposal for godoc to display deprecated items in a more useful manner: #17056.

@rakyll proposed that golint warn when deprecated items are used: golang/lint#238.

---

Even if the stdlib takes a conservative stance on use of aliases within the stdlib, I don't think that the existence of a type alias should imply (in any way that is detected mechanically or denoted visually) that the old type is deprecated, even if it always means that in practice.

Doing so would mean one of:

• scanning other stdlib pakages to see if any type, not explicitly marked as deprecated, is aliased elsewhere
• hardcoding all stdlib aliases into automated tools
• only reporting that the old type is deprecated when you are already looking at its replacement, which does not help discovery

When a type alias is introduced because the old type has been deprecated, it needs to be handled by marking the old type deprecated, with a reference to the new type, regardless.

This enables better tooling to exist by allowing it to be simpler and more general: it doesn't need to special case anything or even know about type aliases: it just needs to match "Deprecated: " in doc comments.

An official, if perhaps temporary, policy that an alias in the stdlib is only for deprecation is good, but it should only be enforced with the standard deprecation comments and by disallowing other uses to make it past code review.

[Merovius]

@niemeyer My previous reply got lost due to loss of power :( out of order:

But I'm repeating myself..

FWIW, I found your last reply pretty helpful. It convinced me, that we are more in agreement, than it previously seemed (and than it may still seem to you). There still seems to be miscommunication somewhere, though.

My proposal, though, is about gradual refactoring of code

This is non-contentious, I think. :) I agreed, from the beginning, that your proposal is an interesting alternative to be considered to address the problem. What confuses me are statements like this:

If you delete a method or a type, Go code using it breaks, so these changes need to be done atomically.

I still wonder what your reasoning here is. I understand the unit of atomicity to be a single commit. With that assumption, I simply do not understand why you are convinced that the deletion of a method or type can not first happen in separate, arbitrarily numerous commits in the depending repositories and then, once there is no apparent user anymore (and an ample deprecation interval has passed) the method or type is deleted in a commit upstream (without breaking anything, as no one depends anymore). I agree that there is a certain fuzziness factor around reverse dependencies that don't adhere to the deprecation or that you can not find (or reasonably fix), but that, to me, seems largely independent of the matter at hand; you'll have that problem whenever you apply a breaking change and no matter how you try to orchestrate it.

And, to be fair: The confusion isn't really helped by sentences like

Unless, of course, you live in a walled garden and can reach out to every call site to fix it.

If anything I said gave you the impression that this is the point I'm arguing from, I'd hope you can take a step back and maybe re-read it under the assumption that I am arguing completely from the position of the open source community (if you don't believe me, feel free to look up my previous contributions to this topic; I'm always the first to point out that this by fare more a community problem, than a monorepo problem. Monorepos have ways around this, as you pointed out).

Anyway. I find this just as draining as you. I hope I'll understand your position at some point, though.

[tombergan]

simultaneously talk about if and how to support things like the protobuf public imports ...
I think at some point it needs to be debated whether we actually consider wrapper-packages, protobuf public imports or exposing internal package-APIs such a bad thing

nit: I don't think protobuf public imports need to be mentioned as a special secondary use case. They were designed for gradual code repair, as mentioned explicitly in both the internal design doc and even the public documentation, so they already fall under the umbrella of problems described by this issue. Also, I believe type aliases would be sufficient for implementing protobuf public imports. (The proto compiler generates vars, but they are logically const, so "var Enum_name = imported.Enum_name" should be sufficient.)

[niemeyer]

@Merovius Thanks for the productive response. Let me try to provide some context:

I still wonder what your reasoning here is. I understand the unit of atomicity to be a single commit. With that assumption, I simply do not understand why you are convinced that the deletion of a method or type can not first happen in separate,

Never said it cannot happen. Let me take a step back and restate more clearly.

We probably all agree that the end goal is two fold: we want working software, and we want to improve the software so we can continue working on it in a sane way. Some of the latter are breaking changes, putting it at odds with the former goal. So there's tension, which means there's some subjectiveness to where the sweet spot lies. The interesting part of our debate lies here.

One helpful way to search for that sweet spot is to think about human interventions. That is, once you do something that requires people to manually modify code to keep it working, inertia takes place. It takes a long time for the relevant portion of all dependent code bases to go through this process. We're asking busy people to do things that in most cases they'd rather not bother.

Another way to look at that sweet spot is likelyhood of working software. It doesn't matter how much we ask people to not use a deprecated method. If it's easily accessible and it solves their problem here and now, most developers will just use it. The common counterargument here is: _oh, but then it's their problem when it breaks!_ But that goes against the the stated goal: we want working software, not being right.

So, hopefully this provides some more insight into why simply moving a type seems unhelpful. For people to actually use that new type at its new home, we need human intervention. When people go over the trouble of manually changing their code, it's best to have an intervention that _uses the new type_ instead of something that will soon change again under their feet in the upcoming future. If we do go over the trouble of adding a language feature to help with refactorings, ideally it would allow people to gradually move their code _to that new type,_ not simply to a new home, for the reasons above.

[Merovius]

Thanks for the explanation. I think I understand your position better now and agree with your assumptions (namely, that people will use deprecated stuff no matter what, so providing any help possible to guide them to the replacement is paramount). FWIW, my naive plan to deal with this problem (no matter which solution for gradual repair we'll go with) is a go-fix like tool to automatically migrate code package-by-package in the deprecation period, but I freely admit hat I haven't yet tried how and if that works in practice.

[rogpeppe]

@niemeyer I don't believe your suggestion is workable without a serious disruption to the Go type system.

Consider the dilemma presented by this code:

package old
import "new"
type A adapts new.A
func (a A) NewA() {}

package new
type A struct{}
func (a A) OldA() {}

package main
import (
    "new"
    "old"
    "reflect"
)
func main() {
    oldv := reflect.ValueOf(old.A{})
    newv := reflect.ValueOf(new.A{})
    if oldv.Type() == newv.Type() {
        // The two types are equal, therefore they must
        // have exactly the same method set, so either
        // oldv doesn't have the OldA method or newv doesn't
        // have the NewA method - both of which imply a contradiction
        // in the type system.
    } else {
         // The two types are not equal, which means that the
         // old adapted type is not fully compatible with the old
         // one. Any type that includes either new.A or new.B will
         // be incompatible as one of its components will likewise be
         // unequal, so any code that relies on dynamic type checking
         // will fail when presented with the type that's not using the
         // expected version.
    }
 }

One of the current axioms of the reflect package is that if two types are the same, their reflect.Type values are equal. This is one of the foundations of the efficiency of Go's runtime type conversion. As far as I can see there is no way to implement the "adapts" keyword without breaking this.

[niemeyer]

@rogpeppe See the conversation with @rsc about reflection above. The two types are not the same, so reflect would just say the truth and provide details for the adapter when asked about it.

[rogpeppe]

@niemeyer If the two types are not the same, then I don't think we can support gradual code repair while moving a type between packages. For example, say we wanted to make a new image package that maintains type compatibility.

We might do:

package newimage
import "image"
type RGBA adapts image.RGB
func (r *RGBA) At(x, y) color.Color {
    return (*image.Buffer)(r).At(x, y)
}
etc for all the methods

Given the aim of gradual code repair, I think it's reasonable to expect that
an image created in the new package is compatible with existing functions
that use the old image type.

Let's assume for the sake of argument that the image/png package has
been converted to use newimage but image/jpeg has not.

I believe that we should expect this code to work:

img, err := png.Decode(r)
if err != nil { ... }
err = jpeg.Encode(w, img, nil)

but, since it does a type assert against *image.RGBA not *newimage.RGBA,
it will fail AFAICS, because the types are different.

Say we made the type assert above succeed, whether the type is *image.RGBA
or not. That would break the current invariant that:

reflect.TypeOf(x) == reflect.TypeOf(x.(anyStaticType))

That is, using a static type assertion wouldn't just assert the static type of a
value but sometimes it would actually change it.

Say we decided that was OK, then presumably we'd also need
to make it possible to convert an adapted type to any interface that any of its compatible
adapted types support, otherwise either new or old code would stop
working when converting to interface types that are compatible with the
type they're using.

This leads to another contradictory situation:

// oldInterface is some interface with methods that
// are only supported by the old type.
type oldInterface interface {
    OldMethod()
}
var x = interface{} = newpackage.Type{}
switch x.(type) {
case oldInterface:
    // This would fail because the newpackage.Type
    // does not implement OldMethod, even though we
    // we just supposedly checked that x implements OldMethod.
    reflect.TypeOf(x).Method("OldMethod")
}

Overall, I think that having two types that are both the same but different
would lead to a very hard to explain type system and unexpected incompatibilities
in code that uses dynamic types.

I support the "type X = Y" proposal. It's simple to explain and does not
disrupt the type system too much.

[bcmills]

@rogpeppe: I believe that @niemeyer's suggestion is to implicitly convert an adapted type to its base type, similar to @josharian's earlier suggestions.

To make that work for gradual refactoring, it would also have to implicitly convert functions with arguments of adapted types; in essence, it would require adding covariance to the language. That's certainly not an impossible task — plenty of languages do allow covariance, particularly for types with the same underlying structure — but it does add a lot of complexity to the type system, particularly for interface types.

That does lead to some interesting edge-cases, as you've noted, but they're not necessarily "contradictory" per se:

type oldInterface interface {
    OldMethod()
}
var x = interface{} = newpackage.Type{}
switch y := x.(type) {
case oldInterface:
    reflect.TypeOf(y).Method("OldMethod")  // ok
    reflect.TypeOf(x).Method("NewMethod")  // ok

    // This would fail because y has been implicitly converted to oldInterface.
    reflect.TypeOf(y).Method("NewMethod")

    // This would fail because accessing OldMethod on newpackage.Type requires
    // a conversion to oldInterface.
    reflect.TypeOf(x).Method("OldMethod")
}

[rogpeppe]

// This would fail because accessing OldMethod on newpackage.Type requires
// a conversion to oldInterface.

This still seems contradictory to me. The current model is a very simple one: an interface value has a well defined underlying static type. In the above code we infer something about that underlying type, but when we peek at the value, it doesn't look like what we've inferred. This is a serious (and hard to explain) change to the language in my view.

[rsc]

The discussion here seems to be winding down. Based on a suggestion by @egonelbre in https://github.com/golang/go/issues/16339#issuecomment-247536289, I've updated the original issue comment (at the top) to include a linked summary of the discussion so far. I will post a new comment, like this one, each time I've updated the summary.

Overall, it seems that the sentiment here is for type aliases rather than generalized aliases. Possibly Gustavo's adapter idea will displace type aliases, but possibly not. It seems a bit complex at the moment, although maybe by the end of the discussion a simpler form will be reached. I suggest that discussion continue for a little while longer.

[Merovius]

I am still not convinced that mutable global vars are "usually a bug" (and in the cases where they are a bug, the race detector is the tool of choice to find that kind of bug). I'd request that, if that argument is used to justify the lack of an extensible syntax, a vet-check is implemented that - say - checks for assignments to global variables in code not exclusively reachable by init() or their declarations. I'd naively think that this isn't particularly hard to implement and it shouldn't be much work to run it over - say - all godoc.org registered packages to see what the use cases for mutable global vars are and whether we do consider all of them bugs.

(I'd also like to believe that, if go grows immutable global vars, they should be part of const-declarations, because that's what they conceptually are and because that would be backwards-compatible, but I acknowledge that this will likely lead to complications around what kind of expressions can be used in array-types, for example and would need more thinking)

Re "Restriction? Aliases of standard library types can only be declared in standard library." -- notably, that would prevent the drop-in usecase for x/image/draw, an existing package which has expressed interest in using aliases. I could also very well imagine, for example, router packages or the like using aliases into net/http in a similar fashion (waves hands).

I also agree with the counter-arguments Re all the restrictions, i.e. I'm in favor of not having any of those.

[rsc]

@Merovius, what about mutable _exported_ global vars? It's true that an unexported global might be fine since all the code in the package knows how to handle it properly. It's less obvious that exported mutable globals ever make sense. We made this mistake ourselves a number of times in the standard library. For example there's no completely safe way to update runtime.MemProfileRate. The best you can do is set it early in your program and hope that no package you imported kicked off an initialization goroutine that might be allocating memory. You might well be right about var vs const, but we can leave that for another day.

Good point about x/image/draw. Will add to summary at next update.

I would very much like to assemble a representative corpus of Go code that we could analyze to answer questions like the ones you raise. I started trying to do this a few weeks ago and I ran into some problems. It's a bit more work than it seems like it should be, but it's very important to have that data set, and I expect we'll get there.

[jimmyfrasche]

@rsc your GothamGo presentation about this topic has been posted on youtube https://www.youtube.com/watch?v=h6Cw9iCDVcU and would make a good addition to the first post.

In the "What other issues does a proposal for type aliases need to address?" section it would be helpful to specify that the answer to "Can methods be defined on types named by alias?" is a hard no. I realize that goes against the decreed spirit of the section, but I've noticed that, in a lot conversations about aliases, here and elsewhere, there are people that immediately reject the concept because they believe that aliases would necessarily allow this and thus cause problems than it solves. It is implicit in the definition, but explicitly mentioning it would short circuit a lot of unnecessary back and forth. Though maybe that belongs in an alias FAQ in the new proposal for aliases, should that be the outcome of this thread.

[jimmyfrasche]

@Merovius any exported package-global mutable variable can be simulated by package-level getter and setter funcs.

Given version n of a package p,

package p
var Global = 0

at version n+1 getters and setters can be introduced and the variable deprecated

package p
//Deprecated: use GetGlobal and SetGlobal.
var Global = 0
func GetGlobal() int {
    return Global
}
func SetGlobal(n int) {
   Global = n
}

and version n + 2 could unexport Global

package p
var global = 0
func GetGlobal() int {
    return global
}
func SetGlobal(n int) {
   global = n
}

(Exercise left to the reader: you could also wrap access to global in a mutex in n + 2 and deprecate GetGlobal() in favor of the more idiomatic Global().)

That's not a fast fix, but it does reduce the problem so that only func aliases (or their present workaround) are strictly necessary for gradual code repair.

[jba]

@rsc One trivial use for aliases you left out of your summary: abbreviating long names. (Probably the only motivation for Pascal, which initially didn't have programming-in-the-large features like packages.) Although it's trivial, it is the only use case where unexported aliases make sense, so maybe worth mentioning for that reason.

[Merovius]

@jimmyfrasche You are correct. I do not like the idea of using getters and setters (just as I don't like to have them for struct fields) but your analysis is, of course, correct.

There is a point to be made about non-repair uses of aliases (e.g. making drop-in replacement packages), but I concede that it weakens the case for var-aliases.

[jimmyfrasche]

@Merovius agreed on all points. I'm not happy about it either but gotta follow the logic v☹v

[iand]

@niemeyer can you clarify how adapters would assist migrating types where both old and new have a method with the same name but differing signatures. Adding an argument to a method or changing the type of an argument seem like they would be common evolutions of a codebase.

[niemeyer]

@rogpeppe Note that this is exactly how it happens today:

type two one

This makes one and two independent types, and whether reflecting or under an interface{}, that's what you see. You can also convert between one and two. The adapter proposal above just makes that last step automatic for adapters. You may not like the proposal for multiple reasons, but there's nothing contradictory about that.

[niemeyer]

@iand As in the case of type two one, the two types have completely independent method sets, so there's nothing special about matching names. Before old code bases are migrated they would remain using the old signature under the previous type (now an adapter). New code using the new type would use the new signature. Passing a value of the new type into old code automatically adapts it because the compiler knows the latter is an adapter of the former, and thus uses the respective method set.

[rsc]

@niemeyer It seems like there is a lot of complexity hiding behind these adapters that is not fully specified. At this point I think the simplicity of type aliases weighs strongly in their favor. I sat down to list all the things that will need updating just for type aliases, and it's a very long list. The list would certainly be longer for adapters, and I still don't fully understand all the details. I'd like to suggest we do type aliases for now and leave a decision about the relatively heavier adapters to a later time, if you want to work out a full proposal (but again I am skeptical that there aren't dragons lurking there).

[rsc]

@jimmyfrasche Regarding methods on aliases, certainly aliases do not allow bypassing the usual method definition restrictions: if a package defines type T1 = otherpkg.T2, it cannot define methods on T1, just as it cannot define methods directly on otherpkg.T2. However, if a package defines type T1 = T2 (both in the same package), then the answer is less clear. We could introduce a restriction but there is not (yet) an obvious need for that.

[rsc]

Updated the top-level discussion summary. Changes:

• Added link to GothamGo video
• Added "abbreviating long names" as a possible use, per @jba.
• Added x/image/draw as argument against standard library restriction, per @Merovius.
• Added more text about methods on aliases, per @jimmyfrasche.

[rsc]

Design doc added: golang.org/design/18130-type-alias

As was the case a week ago, there still seems to be a general consensus for type aliases. Robert and I drafted a formal design doc, which I've just checked in (link above).

Following the proposal process, please post substantive comments on the proposal _here_ on this issue. Spelling/grammar/etc can go to the Gerrit codereview page https://go-review.googlesource.com/#/c/34592/. Thanks.

[Merovius]

I would like to have the "Effect on embedding" reconsidered. It limits the usability of type aliases for gradual code repair. Namely, if p1 wants to rename a type type T1 = T2 and package p2 embeds p1.T2 in a struct, they will never be able to update that definition to p1.T1, because an importer p3 might refer to the embedded struct by name. p2 then can't switch to p1.T1 without breaking p3; p3 can't update the name to p1.T1, without breaking with the current p2.

A way out of this would be, to a) in general limit any compatibility/deprecation-period promise to code that doesn't refer to embedded fields by name, or b) add a separate deprecation stage, so p1 adds type T1 = T2 and deprecates T2, then p2 deprecates referring to (say) s2.T2 by name, all importers of p2 will be repaired to not do that, then p2 makes the switch.

Now, in theory, the problem can recurse indefinitely; p4 might import p3, which itself embeds the type from p2; it would seem to me, that p3 also needs to have a deprecation period, to refer to the twice-embedded field by name? In that case, the innermost deprecation period becomes infinitesimal or the outermost becomes infinite. But even without considering the problem as recursive, it would seem to me, that b) would be pretty hard to time (the deprecation period of p2 would need to be fully contained in the deprecation period of p1. So if T is a "standard deprecation period", you'd have to choose at least 2T when renaming types, so that releases will line up).

a) also seems impractical to me; e.g. if a type embeds a *byte.Buffer and I want to set that field (or pass that buffer to some other function), there is simply no way to do that, without referring to it by name (except using struct initializers without names, which also loses compatibility guarantees :) ).

I understand the attractiveness of being compatible with byte and rune as aliases. But, personally, I'd place that secondary to preserving the usefulness of type aliases for gradual repairs. A (probably bad) example of an idea to get both would be, to, for exported names allow to use any alias to refer to an embedded field and for unexported names (inherently restricted to the same package, thus under more control of the author) keep the currently proposed semantics? Yes, I dislike this distinction too. Maybe someone has a better idea.

[jimmyfrasche]

@rsc re methods on an alias

If you have a type S that's an alias for type T, both defined in the same package, and you allow defining methods on S, what if T is an alias for p.F defined in a different package? While that should obviously fail as well, there are subtleties in enforcement, implementation, and readability of the source to consider (If T is in a different file from S, it's not immediately clear whether you can define a method on T by looking at the definition of T).

The rule—if you have type T = S, then you cannot declare methods on T—is absolute and it is clear from that single line in the source that it applies, without having to investigate the source of S, as you would in the alias of alias situation.

Further, allowing methods on a local type alias muddies the distinction between a type alias and a type definition. Since the methods would be defined on both S and T anyway, the restriction that they can only be written on one does not restrict what can be expressed. It just keeps things simpler and more uniform..

[jba]

@jimmyfrasche If we are writing type T1 = T2 and T2 is in the same package, then we are probably deprecating the name T2. In that case, we want as few occurrences of T2 in the godoc as possible. So we'd like to declare all methods as func (T1) M().

[jimmyfrasche]

@jba a godoc change to report the methods of an alias as being declared on that alias would fulfill that requirement without changing the readability of the source. In general it would be nice if godoc displayed the full method set of a type when aliasing and/or embedding is involved, especially when the type comes from another package. The problem should be solved with smarter tooling not more language semantics.

[bcmills]

@jba In that case, why wouldn't you just reverse the direction of the alias? type T2 = T1 already allows you to define methods on T1 with the same package structure; the only difference is the type name reported by the reflect package, and you can start the migration by fixing the name-sensitive call sites to be name-insensitive before adding the alias.

[mdempsky]

@jimmyfrasche From the proposal document:

"Since T1 is just another way to write T2, it does not have its own set of method declarations. Instead, T1’s method set is the same as T2’s. At least for the initial trial, there is no restriction against method declarations using T1 as a receiver type, provided using T2 in the same declaration would be valid."

Using p.F as a method receiver type is never valid.

[jimmyfrasche]

@mdempsky I was not very clear, but I did say it was invalid.

My point is that it's less obviously clear whether it is valid or not by just looking at that specific line of code.

Given type S = T, you also have to look at T to make sure it is not also an alias that aliases a type in another package. The only gain is complexity.

Always disallowing methods on an alias is simpler and easier to read and you don't lose anything. I don't imagine that a confusing case would arise very often, but there's no need to introduce the possibility when you don't gain anything that can't be handled better elsewhere or by a different but equivalent approach.

[bcmills]

@Merovius

if p1 wants to rename a type type T1 = T2 and package p2 embeds p1.T2 in a struct, they will never be able to update that definition to p1.T1, because an importer p3 might refer to the embedded struct by name.

It is possible to work around this problem today in many cases by changing the anonymous field to a named field and explicitly forwarding the methods. However, that would not work for unexported methods.

Another option might be to add a second feature to compensate. If you could adopt the method set of a field without making it anonymous (or with explicit renaming), that would allow the field name to remain unchanged even as the underlying type is changed.

Considering the declaration from your example:

package p2

type S struct {
  p1.T2
}

One compensating feature might be "field aliases", which would follow a similar syntax to type aliases:

package p2

type S struct {
  p1.T1
  T2 = T1  // field T2 is an alias for field T1.
}

var s S  // &s.T2 == &s.T1

Another compensating feature might be "delegation", which would explicitly adopt the method set of an anonymous field:

package p2

type S struct {
  T2 p1.T1 delegated  // T2 is a field of type T1.
  // The method set of S includes the method set of T1 and forwards those calls to field T2.
}

I think I prefer field aliases myself, because they would also enable another kind of gradual repair: renaming the fields of a struct without introducing pointer-aliasing or consistency bugs.

[jimmyfrasche]

@Merovius The main issue is when the type is renamed by an alias.

I haven't considered this in full—barely in passing, just a random thought:

What if you introduce an alias in your package that names it back and embed that?

I don't know if that fixes anything but maybe it buys some time to break the loop?

[Merovius]

@bcmills I didn't think of that workaround, thanks. I think, the caveat about unexported methods would seem (to me) to arise rarely enough in practice that it wouldn't sway my opinion in general (unless I don't fully understand it. Feel free to clarify, if you think that's useful). I don't think piling on more changes is justified (or a good idea).

[bcmills]

@Merovius The more I think about it, the more I like the idea of field aliases.

Forwarding the methods explicitly is tedious even if they're exported, and breaks other kinds of refactoring (e.g. adding methods to the embedded type and expecting the type that embeds it to continue to satisfy the same interface). And renaming struct fields also falls within the general umbrella of enabling gradual code repair.

[neild]

@Merovius

if p1 wants to rename a type type T1 = T2 and package p2 embeds p1.T2 in a struct, they will never be able to update that definition to p1.T1, because an importer p3 might refer to the embedded struct by name. p2 then can't switch to p1.T1 without breaking p3; p3 can't update the name to p1.T1, without breaking with the current p2.

If I understand your example, we have:

package p1

type T2 struct {}
type T1 = T2

package p2

import "p1"

type S struct {
  p1.T2
  F2 string // see below
}

I believe that this is just a specific example of the general case where we wish to rename a struct field; the same problem applies if we want to rename S.F2 to S.F1.

In this specific case, we may update package p2 to use p1's new API with a local type alias:

package p2

import "p1"

type T2 = p1.T1

type S struct {
  T2
}

This is not, of course, a good long-term fix. I don't think that there's any way around the fact that p2 will need to change its exported API to eliminate the T2 name, however, which will proceed in the same fashion as any field rename.

[atdiar]

Just a note concerning "moving types between packages". Isn't that formulation slightly problematic?

As far as I understand, the proposal allows to "refer" to an object definition that lies in another package via a new name.

It does not move the object definition, does it? (unless one writes code using aliases in the first place in which case, the user is free to change where the alias refer to, just like in the draw pkg).

[Merovius]

@atdiar Referring to a type in a different package can be used as a step in moving the type. Yes, an alias doesn't move the type, but it can be used as a tool to do so.

[atdiar]

@Merovius Doing that is likely to break reflection and plugins.

[Merovius]

@atdiar I'm sorry, but I don't understand what you are trying to say. Have you read the original comment of this thread, the article about gradual repairs linked therein and the discussion so far? If you are trying to add a so far not considered argument to the discussion, I believe you need to be clearer.

[ghost]

Finally, a useful and well-written proposal. We need type alias, I have big issues on creating a single API without type alias, so far, I have to write my code in a way I don't like so much to accomplish that. This should be included on go v1.8 but never is too late, so go ahead for 1.9.

[atdiar]

@Merovius
I'm explicitly talking about "moving types" between packages. It changes the object definition. For instance, in pkg reflect, some information is tied to the package an object was defined in.
If you move the definition, it may break.

[atdiar]

@kataras it's not really about good doc and comments, it's merely that type definitions should not be moved. As much as I appreciate the alias proposal, I'm wary that people think that they can just do that.

[Merovius]

@atdiar again, please read the article from the original comment and the discussion so far. Moving types and how to address your concerns are the primary concern of this thread. If you don't feel that Russ' article adequately addresses your concerns, please be specific about why his explanation isn't satisfying. :)

@kataras While I, personally, agree, I don't think it's particularly helpful, to simply assert how important we find this feature. There needs to be a constructive argument to be made to address people's concerns. :)

[atdiar]

@Merovius I've read the document. It doesn't answer my question. I think I've been explicit enough. It's related to the same issue that deterred us from having the former alias proposal implemented.

[Merovius]

@atdiar I, at least, don't understand. You are saying that moving a type would break things; the proposal is about how to avoid such breakages with gradual repair, by using an alias, then update each reverse dependency until no code uses the old type, then removing the old type. I don't see, how your assertion, that "reflection and plugins" are broken holds under these assumptions. If you want to question the assumptions, that has already been discussed.

I also don't see how any of the issues preventing aliases from entering 1.8 connects to what you said. The respective issues, to the best of my knowledge, are #17746 and #17784. If you are referring to the embedding issue (which could be interpreted as relating to breakages or reflection, though I'd disagree), then that is addressed in the formal proposal (though, see above, I believe the proposed solution merits more discussion) and you should be specific about why you don't believe it is.

So, I'm sorry, but no, you weren't specific enough. Do you have an issue number for "the same issue that deterred us from having the former alias proposal implemented" you are referring to, that relates to what you mentioned so far, to help understand? Can you give a specific example of the breakages you are speaking of (see examples for this upthread; give a sequence of packages, type definitions and some code and describe how it breaks when transformed as proposed)? If you want your concerns addressed, you really need to help others understand them first.

[atdiar]

@Merovius So in the case of transitive dependencies where one of these dependencies is looking at reflect.Type.PkgPath(), what happens ?
That's the same problem occurring in the embedding issue.

[Merovius]

@atdiar I'm sorry, I don't see how this is in any way an understandable concern, in light of the discussion in this thread so far and what this proposal is about. I will step out of this particular subthread now and give others, who might understand your objection better, the opportunity to address it.

[atdiar]

Let me rephrase it concisely:

The issue is about type equality given the fact that the type definition hardcodes its own location.
Since type equality can be and is tested at runtime, I don't see how moving types is so easy to do.

I'm simply raising a warning that this use case of "moving types" can be potentially breaking a lot of packages in the wild, at a distance. Similar worry with plugins.

(the same way changing the type of a pointer in a package would break a lot of other packages, if that parallel can make things clearer.)

[Merovius]

@atdiar Again, this issue is about moving types in two steps, by first deprecating the old location and updating the reverse dependencies, _then_ move the type. _Of course_ things will break if you just move types around, but this is not at all what this issue is about. This is about enabling a gradual, multi-step solution to doing that. If you have concerns that any of the solutions proposed here doesn't enable this multi-step process, please be precise and describe a situation, where no reasonable sequence of gradual repairing commits can prevent a breakage.

[rogpeppe]

@niemeyer

This makes one and two independent types, and whether reflecting or under an interface{}, that's what
you see. You can also convert between one and two. The adapter proposal above just makes that last
step automatic for adapters. You may not like the proposal for multiple reasons, but there's nothing
contradictory about that.

You can't convert between

 func() one

and

func() two

[atdiar]

@Merovius You cannot possibly consider changing all the importers of a code-repaired package that exist in the wild. And I'm not too keen on starting delving into package versioning on here.

To be clear, I am not against the alias proposal but the "moving types between packages" formulation which implies a use-case that is not provenly safe yet.

[rsc]

@jimmyfrasche re predictability of method-on-alias validity:

It's already the case that func (t T) M() is sometimes valid, sometimes invalid. It doesn't come up much because people don't push on these boundaries very often. That is, it works well in practice. https://play.golang.org/p/bci2qnldej. In any event, this is on the list of _possible_ restrictions. Like all possible restrictions, it adds complexity and we want to see concrete real-world evidence before adding that complexity.

[rsc]

@Merovius, re embedding names:

I agree that the situation is not perfect. However, if I have a codebase full of references to io.ByteBuffer and I want to move it to bytes.Buffer, then I want to be able to introduce

package io
type ByteBuffer = bytes.Buffer

_without_ updating any of the existing references to io.ByteBuffer. If all the places where io.ByteBuffer are embedded automatically change the name of the field to Buffer as a result of replacing a type definition with an alias, then I've broken the world and there's no gradual repair. In contrast, if the name of an embedded io.ByteBuffer is still ByteBuffer, then the uses can be updated one at a time in their own gradual repairs (possibly having to do multiple steps; again not ideal).

We discussed this at some length in #17746. I was originally on the side of the name of an embedded io.ByteBuffer alias being Buffer, but the above argument convinced me I was wrong. @jimmyfrasche in particular made some good arguments about the code not changing depending on the definition of the embedded thing. I don't think it's tenable to disallow embedded aliases completely.

Note that there is a workaround in p2 in your example. If p2 really wants an embedded field named ByteBuffer without referring to io.ByteBuffer, it can define:

type ByteBuffer = bytes.Buffer

and then embed a ByteBuffer (that is, a p2.ByteBuffer) instead of an io.ByteBuffer. That's not perfect either, but it means repairs can continue.

It's definitely the case that this is not perfect and that field renames in general are not addressed by this proposal. It could be that embedding should not be sensitive to the underlying name, that there should be some kind of syntax for 'embed X as name N'. It could also be that we should add field aliases later. Both seem like reasonable ideas a priori and both should be probably separate, later proposals evaluated based on real evidence of a need. If type aliases help us get to the point where lack of field aliases is the next big roadblock for large-scale refactorings, that will be progress!

(/cc @neild and @bcmills)

[rsc]

@atdiar, yes, it's true that reflect will see through these kinds of changes, and if code depends on the results from reflect it is going to break. Like the situation with embedding, it's not perfect. Unlike the situation with embedding, I don't have any answers except maybe code shouldn't be written using reflect to be quite that sensitive to those details.

[Merovius]

@rsc What I had in mind was a) forbid embedding both an alias and it's defining type in the same struct (to prevent ambiguity from b), b) allow to refer to a field by either name in source code, c) choose one or the other in the generated type information/reflection and the like (don't care which).

I would handwavingly claim, that this helps avoid the kind of breakages that I tried to describe, while also making a clear choice for the case where a choice is required; and, personally, I care less about not breaking code that relies on reflection, than code that doesn't.

I'm not sure right now whether I understand your ByteBuffer argument, but I'm also at the end of a long workday, so no need to explain further, if I find it unconvincing I'll respond eventually :)

[rsc]

@Merovius I think it makes sense to try the simple rules and see how far we get before introducing more complex ones. We can add (a) and (b) later if needed; (c) is a given no matter what.

I agree that maybe (b) is a good idea in certain circumstances, but maybe not in others. If you are using type aliases for the "structure a one-package API into multiple implementation packages" use case mentioned earlier, then maybe you don't want embedding of the alias to expose the other name (which may be in an internal package and otherwise inaccessible to most users). I hope we can gather more experience.

[atdiar]

@rsc

Perhaps adding package level information on aliasability to the object files could help.
(While taking into account whether go plugins have to keep working properly or not.)

[bcmills]

@Merovius @rsc

a) forbid embedding both an alias and it's defining type in the same struct

Note that in many cases this is already forbidden as a consequence of the way embedding interacts with method sets. (If the embedded type has a non-empty method set and one of those methods is called, the program will fail to compile: https://play.golang.org/p/XkaB2a0_RK.)

So adding an explicit rule forbidding double-embedding seems like it would only make a difference in a small subset of cases; doesn't seem worth the complexity to me.

[j7b]

Why not approach type aliases as algebraic types instead and support aliases to a set of types so we also get an empty interface equivalent with compile time type checking as a bonus, a la

type Stringeroonie = {string,fmt.Stringer}

[bcmills]

@j7b

Why not approach type aliases as algebraic types instead and support aliases to a set of types

Aliases are semantically and structurally equivalent to the original type. Algebraic datatypes are not: in the general case, they require additional storage for type-tags. (Go interface types already carry that type information, but structs and other non-interface types do not.)

[j7b]

@bcmills

This might be faulty reasoning, but I thought the problem could be approached as alias A of type T is equivalent to declaring A as interface{} and letting the compiler transparently convert variables of type A to T in scopes where variables of type A are declared, which I thought would be mostly linear compile-time cost, unambiguous, and create a basis for compiler-managed pseudotypes including algebraics using the type T = syntax, and possibly also allow implementing types like immutable references at compile time that as far as user code was concerned would just be interface{}s "under the hood."

Deficiencies in that train of thought would probably be a product of ignorance, and as I'm not in a position to offer a practical proof of concept I'm happy to accept it's deficient and defer.

[Merovius]

@j7b Even if ADT where a solution to one gradual repair problem, they create their own then; it's impossible to add or remove any members of an ADT without breaking dependencies. So, you'd in essence create more problems, than you'd solve.

Your idea of transparently translating to and from interface{} also doesn't work for higher-order types like []interface{}. And eventually you'll end up losing one of go's strengths, which is to give users control over the data layout and instead do the java thing of wrapping everything.

ADT are not the solution here.

[j7b]

@Merovius I'm pretty sure if an algebraic type construct includes renaming (which would be consistent with a reasonable definition of same) it is a solution, that interface{} can serve as a proxy for the form of compiler-managed projection and selection described, and I'm not sure how data layout is relevant nor how you're defining "higher-order" types, a type is just a type if it can be declared and []interface{} is just a type.

All that aside, I'm positive type T = has the potential to be overloaded in intuitive, useful ways beyond renaming, algebraic types and publicly immutable references seem the most obvious applications, so I hope the specification ends up stating that syntax indicates a compiler-managed meta or pseudo type and consideration is given to all the ways a compiler-managed type could be useful and the syntax that best expresses those uses. Since a new syntax doesn't need to concern itself with the set of globally reserved words when used as qualifiers something like type A = alias Type would be clear and extensible.

[bcmills]

@j7b

All that aside, I'm positive type T = has the potential to be overloaded in intuitive, useful ways beyond renaming,

I surely hope not. Go is (mostly) nicely orthogonal today, and maintaining that orthogonality is a good thing.

The way, today, that one declares a new type T in Go is type T def, where def is the definition of the new type. If one were to implement algebraic datatypes (a.k.a. tagged unions), I would expect them to follow that syntax rather than the syntax for type aliases.

[griesemer]

I like to throw in a different view point (in support) of type aliases, which may provide some insight into alternative use cases besides refactoring:

Let's step back for a moment and assume we didn't have regular old Go type declarations of the form type T <a type>, but only type alias declarations type A = <a type>.

(To make the picture complete, let's also assume that methods are somehow declared differently - not via association to the named type used as receiver, because we can't. For instance, one could imagine the notion of a class type with the methods literally inside and so we don't need to rely on a named type to declare methods. Two such types that are structurally identical but have different methods would be different types. The details are not important here for this thought experiment.)

I claim that in such a world we could write pretty much the same code that we write now: We use the (alias) type names so we don't have to repeat ourselves, and the types themselves make sure we use data in a type-safe way.

In other words, if Go would have been designed that way, we probably would have been fine, too, by and large.

More so, in such a world, because types are identical if they are structurally identical (no matter the name), the problems we have with refactoring now wouldn't have shown up in the first place, and there would be no need for any changes to the language.

But we wouldn't have a safety mechanism that we have in current Go: We wouldn't be able to introduce a name for a type and state that this should now be a new, different type. (Still, it's important to keep in mind that it is in essence a safety mechanism.)

In other programming languages, the notion of making a new, different type from an existing type is called "branding": A type get's a brand attached to it that makes it different from all the other types. For instance, in Modula-3, there was a special keyword BRANDED to make that happen (e.g. TYPE T = BRANDED REF T0 would create a new, different reference to T0). In Haskell, the word new before a type has a similar effect.

Going back to our alternative Go world, we might find us in the position where we have no problems with refactoring, but where we wanted to improve the safety of our code so that type MyBuffer = []byte and type YourBuffer = []byte denote different types so that we don't accidentally use the wrong one. We might propose to introduce a form of type branding for exactly that purpose. For instance, we might want to write type MyBuffer = new []byte, or even type MyBuffer = new YourBuffer with the effect that MyBuffer is now a different type from YourBuffer.

This is in essence the dual problem of what we have now. It just happens that in Go, from day one, we always worked with "branded" types as soon as they got a name. In other words, type T <a type> is effectively type T = new <a type>.

To summarize: In existing Go, named types are always "branded" types, and we lack the notion of just a name for a type (which we now call type aliases). In several other languages, type aliases are the norm, and one has to use a "branding" mechanism to create an explicitly new, different type.

The point is that both mechanisms are inherently useful, and with type aliases we finally get around to support them both.

[atdiar]

@griesemer The extension of that feature is the initial alias proposal which should ideally clean up refactoring. I'm afraid that only type aliases would create difficult refactoring edge cases due to its restricted scope.

In both proposals, I am left wondering whether collaboration from the linker shouldn't be required because the name is part of the type definition in Go as you've explained.

I'm not at all familiar with object code, so it's just an idea, but it seems that it is possible to add custom sections to object files. If by chance, it were possible to keep a kind of unrolled linked list, filled up at link time of type names and their aliases maybe that could help. The runtime would have all the information it needs without sacrificing separate compilation.

The idea being that the runtime should be able to dynamically return the different aliases for a given type so that error messages remain clear (since aliasing introduces a naming discrepancy between the running code and the written code).

An alternative to tracing aliasing use would be to have a concrete versioning story in the large, to be able to "move" object definitions across packages as was done for the context package. But that's a whole other issue altogether.

In the end, it's still a good idea to have left structural equivalence to interfaces and name equivalence to types.
Given the fact that a type can be considered an interface with more constraints, it appears that declaring an alias should/could be implemented by keeping a per-package slice of slices typename strings.

[rsc]

@atdiar I am not sure you mean what I do when you say "separate compilation". If package P imports io and bytes, then all three can be compiled as separate steps. However, if io or bytes changes, then P must be recompiled. It's _not_ the case that you can make changes to io or bytes and then just use an old compilation of P. Even in the plugin mode, this is true. Due to effects like cross-package inlining, even non-API-visible changes to the implementation of io or bytes change the effective ABI, which is why P must be recompiled. Type aliases do not make this problem worse.

[rsc]

@j7d, at a type system level, sum types or any kind of subtyping (as suggested by others earlier in the discussion) only help with certain kinds of uses. It's true that we can think of bytes.Buffer as a subtype of io.Reader ("a Buffer is a Reader", or in your example "a string is a Stringeroonie"). The problems happen when constructing more complex types using those. The rest of this comment talks about Go types but is speaking about their fundamental relationships at a subtyping level, not what Go the language actually implements. Go must implement rules consistent with the fundamental relationships, though.

A type constructor (a fancy way to say "a way to use a type") is covariant if it preserves the subtyping relation, contravariant if it inverts the relationship.

Using a type in a function result is covariant. A func() Buffer "is a" func() Reader, because returning a Buffer means you've returned a Reader. Using a type in a function argument is _not_ covariant. A func(Buffer) is not a func(Reader), because the func needs a Buffer, and some Readers are not Buffers.

Using a type in a function argument is contravariant. A func(Reader) is a func(Buffer), because the func needs only a Reader, and a Buffer is a Reader. Using a type in a function result is _not_ contravariant. A func() Reader is not a func() Buffer, because the func return a Reader, and some Readers are not Buffers.

Combining the two, a func(Reader) Reader is not a func(Buffer) Buffer, nor vice versa, because either the arguments don't work out or the results don't work out. (The only combination along these lines that works would be that a func(Reader) Buffer is a func(Buffer) Reader.)

In general, if func(X1) X2 is a (subtype of) func(X3) X4, then it must be that X3 is a (subtype of) X1 and similarly X2 is a (subtype of) X4. In the case of the alias use where we want T1 and T2 to be interchangeable, a func(T1) T1 is a subtype of func(T2) T2 only if T1 is a subtype of T2 _and_ T2 is a subtype of T1. That basically means T1 is the _same_ type as T2, not a more general type.

I used function arguments and results because that's the canonical example (and a good one), but the same happens for other ways to build complex results. In general you get covariance for outputs (like func() T, or <-chan T, or map[...]T) and contravariance for inputs (like func(T), or chan<- T, or map[T]...) and forced type equality for input+output (like func(T) T, or chan T, or *T, or [10]T, or []T, or struct {Field T}, or a variable of type T). In fact the most common case in Go, as you can see from the examples, is input+output.

Concretely, a []Buffer is not a []Reader (because you can store a File into a []Reader but not into a []Buffer), nor is a []Reader a []Buffer (because fetching from a []Reader might return a File, while fetching from a []Buffer must return a Buffer).

A conclusion from all this is that, if you want to solve the general code repair problem so that code can use either T1 or T2, you cannot do it with any scheme that makes T1 only a subtype of T2 (or vice versa). Each needs to be a subtype of the other - that is, they need to be the same type - or else some of these listed uses will be invalid.

That is, subtyping is not sufficient to solve the gradual code repair problem. This is why type aliases introduce a new name for the same type, so that T1 = T2, instead of attempting subtyping.

This comment also applies to @iand's suggestion two weeks ago of some kind of "substitutable types" and basically an expansion of @griesemer's reply from then.

[rsc]

Updated the top-level discussion summary. Changes:

• Removed TODO to update summary of adapter discussion, which seems to have faded out.
• Added summary of discussion of embedding and field renames.
• Moved summary of 'methods on aliases' to its own section out of design question list, expanded to include recent comments.
• Added summary of discussion of effect on programs using reflection.
• Added summary of discussion of separate compilation.
• Added summary of discussion of various subtyping-based approaches.

[atdiar]

@rsc concerning separate compilation, my comment is relative to whether type definitions need to keep a list of their aliases (which is not tractable at large scale, because of the separate compilation requirement) or each alias involves iteratively building a list of alias names following the import graph, all related to the given initial type name provided in the type definition. (and how and where to keep that information so that the runtime has access to it).

[rsc]

@atdiar There is no such list of alias names anywhere in the system. The runtime does not have access to it. Aliases do not exist at runtime.

[atdiar]

@rsc Huh, sorry. I'm stuck with the initial alias proposal in head and was thinking about aliasing for func (while discussing aliasing for types). In that case, there would be a discrepancy between the names in the code and the names at runtime.
Using the information in runtime.Frame for logging would need some rethinking in that case.
Nevermind me.

[Merovius]

@rsc thanks for re-summarizing. The embedded field name still irks me; all the workarounds proposed rely on permanent kludges to keep the old names around. Though the larger point in this comment, namely that this is a special case of renaming fields, which isn't possible either, convinces me that this should indeed be seen (and solved) as a separate problem. Would it make sense to open a separate issue for a request/proposal/discussion to support field renames for gradual repair (possibly addressed in the same go release)?

[rsc]

@Merovius, I agree that gradual code repair for field rename looks like the next problem in the sequence. To start that discussion I think someone would need to gather a suite of real-world examples, both so that we have some evidence that it's a widespread problem and also to check potential solutions against. Realistically, I don't see that happening for the same release.

[rsc]

Back from two weeks away. The discussion seems to have converged. Even the discussion update two weeks ago was fairly minor.

I suggest that we:

• accept the type alias proposal as the tentative solution for the problem laid out above,
  provided an implementation can be ready for people to try at the start of Go 1.9 (Feb 1).
• create a dev.typealias dev branch so that CLs can be reviewed now (January) and merged into master at the start of Go 1.9.
• make a final decision about keeping type aliases near the beginning of the Go 1.9 freeze (like we did for generalized aliases in the Go 1.8 cycle).

[jcao219]

+1

I appreciate the history of discussion behind this change. Let's say that it is implemented. Without a doubt, it will become a rather fringe detail of the language, rather than a core feature. As such, it adds complexity to the language and tooling disproportionate to its actual usage frequency. It also adds more surface area in which the language might be inadvertently abused. For that reason, being overly cautious is a good thing, and I'm happy that there has been a wealth of discussion so far.

@Merovius : Sorry for editing my post! I thought nobody was reading. Initially in this comment, I expressed some skepticism that this language change is needed when there already exist tools like the gorename tool.

[Merovius]

@jcao219 This has been discussed before, but surprisingly, I can't seem to find this quickly here. It is discussed at length in the original thread for general aliases #16339 and the associated golang-nuts threads. In short: This kind of tooling only addresses how to prepare the repairing commits, not how to sequence the changes to prevent breakages. Whether the changes are done by a tool or by a human is immaterial to the problem, that there currently is no sequence of commits that will not break some code or another (the original comment of this issue and the associated doc justify this statement more in-depth).

For more automated tooling (e.g. integrated into the go tool or the like), the original comment addresses this under the heading "Can this be a tooling- or compiler-only change instead of a language change?".

In conclusion, let's say that the change is implemented. Without a doubt, it will become a rather fringe detail of the language, rather than a core feature.

I'd like to express doubt. :) I do not consider this a foregone conclusion.

[jcao219]

@Merovius

I'd like to express doubt. :) I do not consider this a foregone conclusion.

I guess I meant that people who would use this feature will mainly be the maintainers of important Go packages with lots of dependent clients. In other words, it benefits those who are already Go experts. At the same time, it presents a tempting way to make code less readable to new Go programmers. The exception is the use case of renaming long names, but natural Go type names usually aren't too long or complex.

Just as in the case of the dot import feature, it would be wise for tutorials and docs to accompany their mentions of this feature with a statement on usage guidelines.

For instance, say I wanted to use "github.com/gonum/graph/simple".DirectedGraph, and I wanted to alias it with digraph to avoid typing simple.DirectedGraph, would that be a good use case? Or should this kind of renaming be restricted to unreasonably long names generated by things like protobuf?

[rsc]

@jcao219, the summary of the discussion at the top of this page answers your questions. In particular, see these sections:

• Can this be a tooling- or compiler-only change instead of a language change?
• What other uses might aliases have?
• Restrictions (the general notes starting that section)

To your more general point about Go experts vs new Go programmers, an explicit goal of Go is to make it easier to program in large codebases. Whether you are an expert is somewhat unrelated to the size of the codebase you are working in. (Maybe you are just starting out in a new project someone else started. You might still need to do this kind of work.)

[rsc]

OK, based on the unanimity / quiet here, I will (as I suggested last week in https://github.com/golang/go/issues/18130#issuecomment-268614964) mark this proposal approved and create a dev.typealias branch.

[ulikunitz]

The excellent summary has a section "What other issues does a proposal for type aliases need to address?" What are the plans to address those issues after the proposal has been declared as accepted?

[gopherbot]

CL https://golang.org/cl/34986 mentions this issue.

[gopherbot]

CL https://golang.org/cl/34987 mentions this issue.

[gopherbot]

CL https://golang.org/cl/34988 mentions this issue.

[rsc]

@ulikunitz re the issues (all of these quotes from the design doc assume 'type T1 = T2'):

1. Handling in godoc. Design doc specs out minimal changes to godoc. Once that's in we can see whether additional support is needed. Maybe, but maybe not.
2. Can methods be defined on types named by alias? Yes. Design doc: "Since T1 is just another way to write T2, it does not have its own set of method declarations. Instead, T1’s method set is the same as T2’s. At least for the initial trial, there is no restriction against method declarations using T1 as a receiver type, provided using T2 in the same declaration would be valid."
3. If aliases to aliases are allowed, how do we handle alias cycles? No cycles. Design doc: "In a type alias declaration, in contrast to a type declaration, T2 must never refer, directly or indirectly, to T1."
4. Should aliases be able to export unexported identifiers? Yes. Design doc: "There are no restrictions on the form of T2: it may be any type, including but not limited to types imported from other packages. "
5. What happens when you embed an alias (how do you access the embedded field)? The name is taken from the alias (the visible name in the program). Design doc: https://golang.org/design/18130-type-alias#effect-on-embedding.
6. Are aliases available as symbols in the built program? No. Design doc: "Type aliases are mostly invisible at runtime." (Answer follows from this but is not called out explicitly.)
7. Ldflags string injection: what if we refer to an alias? There are no var aliases, so this does not arise.

[gopherbot]

CL https://golang.org/cl/35091 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35092 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35093 mentions this issue.

[ulikunitz]

@rsc Many thanks for the clarifications.

Let's assume:

package a

import "b"

type T1 = b.T2

As far as I understand T1 is essential identical with b.T2 and is therefore a non-local type and no new methods can be defined. The identifier T1 is however re-exported in package a. Is this a correct interpretation?

[griesemer]

@ulikunitz that is correct

T1 denotes the exact same type as b.T2. It's simply a different name. Whether something is exported or not is based on it's name alone (has nothing to do with the type it denotes).

[rsc]

To make @griesemer's reply explicit: yes, T1 is exported from package a (because it is T1, not t1).

[gopherbot]

CL https://golang.org/cl/35099 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35100 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35101 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35102 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35104 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35106 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35108 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35120 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35121 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35129 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35191 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35233 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35268 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35269 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35670 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35671 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35575 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35732 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35733 mentions this issue.

[gopherbot]

CL https://golang.org/cl/35831 mentions this issue.

[gopherbot]

CL https://golang.org/cl/36014 mentions this issue.

[rsc]

This is now in master, in advance of Go 1.9 opening. Please feel free to sync on master and try things out. Thanks.

[davecheney]

Redirected from #18893

package main

import (
        "fmt"
        "q"
)

func main() {
        var a q.A
        var b q.B // i'm a named unnamed type !!!

        fmt.Printf("%T\t%T\n", a, b)
}

What did you expect to see?

deadwood(~/src) % go run main.go
q.A     q.B

What did you see instead?

deadwood(~/src) % go run main.go
q.A     []int

Discussion

Aliases shouldn't not apply to unnamed type. Their is no "code repair" story in moving from one unnamed type to another. Allowing aliases on unnamed types means I can no longer teach Go as simply named and unnamed types. Instead I have to say

oh, unless it's an alias, in which case you have to remember that it _could be_ an unnamed type, even when you import to from another package.

And worse, it will enable people to promulgate readability anti patterns like

type Any = interface{}

Please do not allow unnamed types to be aliased.

[bcmills]

@davecheney

There is no "code repair" story in moving from one unnamed type to another.

Not true. What if you want to change the type of a method parameter from a named to an unnamed type or vice-versa? Step 1 is to add the alias; step 2 is to update the types which implement that method to use the new type; step 3 is to remove the alias.

(It's true that you can do that today by renaming the method twice. The double-rename is tedious at best.)

And worse, it will enable people to promulgate readability anti patterns like
type Any = interface{}

People can already write type Any interface{} today. What additional harm do aliases introduce in this case?

[davecheney]

People can already write type Any interface{} today. What additional harm do aliases introduce in this case?

I called it an anti pattern because that is precisely what it is. type Any interface{}, because it the person _writing_ the code type something a little shorter, that makes a little more sense to them.

On the flip side, _all_ of the readers, who are experienced in reading Go code and recognise interface{} as instinctively as their face in a mirror, have to learn, and relearn, every variant of Any, Object, T, and map them to things like type Any interface{}, type Any map[interface{}]interface{}, type Any struct{} on per package basis.

Surely you agree that package specific names for commonplace Go idioms are a net negative for readability?

[bcmills]

Surely you agree that package specific names for commonplace Go idioms are a net negative for readability?

I do agree, but since the example in question (by far the most common occurrence of that antipatterrn that I've encountered) can be done without aliases, I don't understand how that example relates to the proposal for type aliases.

The fact that the anti-pattern is possible without type aliases means that we must already educate Go programmers to avoid it, regardless of whether aliases to unnamed types can exist.

[bcmills]

And, in fact, type aliases allow for the _gradual removal_ of that antipattern from codebases in which it already exists.

Consider:

package antipattern

type Any interface{}  // not an alias

type Widget interface{
  Frozzle(Any) error
}

func Bozzle(w Widget) error {
  …
}

Today, users of antipattern.Bozzle would be stuck using antipattern.Any in their Widget implementations, and there is no way to remove antipattern.Any with gradual repairs. But with type aliases, the owner of the antipattern package could redefine it like so:

// Any is deprecated; please use interface{} directly.
type Any = interface{}

And now the callers can migrate from Any to interface{} gradually, allowing the antipattern maintainer to eventually remove it.

[davecheney]

My point is there is no justification for aliasing unnamed types so
disallowing this option would continue to underling the unsuitability of
the practice.

The opposite, to allow aliasing of unnamed types enables not one, but two
forms of this anti pattern.

On Thu, 2 Feb 2017, 16:34 Bryan C. Mills [e-mail] wrote:

Surely you agree that package specific names for commonplace Go idioms are
a net negative for readability?

I do agree, but since the example in question (by far the most common
occurrence of that antipatterrn that I've encountered) can be done without
aliases, I don't understand how that example relates to the proposal for
type aliases.

The fact that the anti-pattern is possible without type aliases means that
we must already educate Go programmers to avoid it, regardless of whether
aliases to unnamed types can exist.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/18130#issuecomment-276872714, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAAcA6BGrFjjTi7eW1BPp7o81XIekbGXks5rYWr-gaJpZM4LBBEL
.

[griesemer]

@davecheney I don't think we have any evidence yet that being able to give an arbitrary type literal a name is harmful. This is also not an unexpected "surprise" feature - it has been discussed in detail in the design document. At this point it makes sense to use this for some time and see where it leads us.

As a counter-example, there are public APIs that use type literals only because the API doesn't want to restrict a client to a specific type (see https://golang.org/pkg/go/types/#Info for instance). Having that explicit type literal may be useful documentation. But at the same time it can be quite annoying to have to repeat that same type literal all over the place; and in fact be an impediment to readability. Being able to conveniently talk about an IntSet rather than a map[int]struct{} w/o being locked in to that one and only IntSet definition is a plus in my mind. That's where type IntSet = map[int]struct{} is exactly right.

Finally, I like to refer back to https://github.com/golang/go/issues/18130#issuecomment-268411811 in case you missed it. Unrestricted type declarations using = are really the "elementary" type declaration, and I'm happy that we finally have them in Go.

[nathany]

Perhaps type intSet = map[int]struct{} (not exported) would be a better way to use unnamed type aliases, but this sounds like the domain of CodeReviewComments and recommended programming practices, rather than limiting the feature.

[nathany]

That said, %T is a handy tool to see types when debugging or exploring the type system. I wonder if there should be a similar format verb that includes the alias? q.B = []int in @davecheney's example.

[bcmills]

@nathany How you you implement that verb? The alias information is not present at runtime. (As far as the reflect package is concerned, the alias is _the same type_ as the thing it is aliased to.)

[nathany]

@bcmills I thought that might be the case... 😞

I imagine static analysis tools and editor plugins are still in the picture to help work with aliases, so that's okay.

[minux]

On Feb 2, 2017 5:01 PM, "Nathan Youngman" [e-mail] wrote:

That said, %T is a handy tool to see types when debugging or exploring the
type system. I wonder if there should be a similar format verb that
includes the alias? q.B = []int in @davecheney
https://github.com/davecheney's example.

I think a better solution is to add a query mode to guru to answer this
question:

which are the declared aliases in the whole GOPATH (or a given package) for
this given type on command line?

I'm not worried about abuse of aliasing unnamed types, but potential
duplicated aliases to the same unnamed type.

[rsc]

@davecheney I added your suggestion to the "Restrictions" section of the discussion summary at the top. Like all restrictions, our general position is that restrictions add complexity (see notes above) and we would likely need to see actual evidence of widespread harm in order to introduce a restriction. Having to change the way you teach Go is not sufficient: any change we make to the language will require changing the way you teach Go.

As noted in the design doc and on the mailing list, we are working on better terminology to make explanations easier.

[rsc]

@minux, like @bcmills pointed out, alias information does not exist at runtime (completely fundamental to the design). There is no way to implement a "%T that includes the alias".

[minux]

On Feb 2, 2017 8:33 PM, "Russ Cox" [e-mail] wrote:

@minux https://github.com/minux, like @bcmills
https://github.com/bcmills pointed out, alias information does not exist
at runtime (completely fundamental to the design). There is no way to
implement a "%T that includes the alias".

I'm suggesting a Go guru (https://golang.org/x/tools/cmd/guru) query mode
for reverse alias mapping, which is based on static code analysis. It
doesn't matter if alias information is available at runtime or not.

[rsc]

@minux, oh I see, you are replying via email and Github makes the quoted text look like text you wrote yourself. I was replying to the text you quoted from Nathan Youngman, thinking it was yours. Sorry for the confusion.

[nathany]

Regarding terminology and teaching, I found the branded types background @griesemer posted quite informative. Thanks for that.

When explaining types and type conversions, baby gophers initially think I'm talking about a type alias, likely due to familiarity with other languages.

Whatever the final terminology, I could imagine introducing type aliases before named (branded) types, especially since declaring new named types is likely to come after introducing byte and rune in any book or curriculum. However, I do want to be mindful of @davecheney's concern to not encourage anti-patterns.

For type intSet map[int]struct{} we say map[int]struct{} is the _underlying_ type. What do we call either side of type intSet = map[int]struct{}? Alias and aliased type?

As for %T, I already need to explain that a byte and rune result in a uint8 and int32, so this is no different.

If anything, I think type aliases will make byte and rune easier to explain. IMO, the challenge will be knowing when to use named types vs. type aliases, and then being able to communicate that.

[griesemer]

@nathany I think it makes a lot of sense to introduce "alias types" first - though I wouldn't use the term necessarily. The newly introduced "alias" declarations are simply regular declarations that don't do anything special. The identifier on the left and the type on the right are one and the same, they denote identical types. I'm not even sure we need the terms alias or aliased type (we don't call a constant name an alias, and the constant value the aliased constant).

The traditional (non-alias) type declaration does more work: It first creates a new type from the type on the right before binding the identifier on the left to it. Thus the identifier and the type on the right are not the same (they only share the same underlying type). This is clearly the more complicated concept.

We do need a new term for these newly created types because any type can now have a name. And we need to be able to refer to them since there are spec rules referring to them (type identity, assignability, receiver base types).

Here's another way to describe this, which may be useful in a teaching environment: A type may either be colored or uncolored. All predeclared types, and all type literals are uncolored. The only way to create a new colored type is via a traditional (non-alias) type declaration which first paints (a copy of) the type on the right with a brand-new, never-before used color (stripping the old color, if any, entirely in the process) before binding the identifier on the left to it. Again, the identifier and the (implicitly and invisibly created) colored type are identical, but they are different from the (differently colored or uncolored) type written down on the right.

Using this analogy, we can reformulate various other existing rules as well:

• A colored type is always different from any other type (because each type declaration uses a brand-new, never-before used color).
• Methods may only be associated with receiver base types that are colored.
• The underlying type of a type is that type stripped off all its color.
  etc.

[nathany]

we don't call a constant name an alias, and the constant value the aliased constant

good point 👍

I'm not sure if the coloured vs. uncoloured analogy is easier to understand, but it does demonstrate that there is more than one way to explain the concepts.

Traditional named/branded/coloured types certainly require more explanation. Especially when a named type can be declared using an existing named type. There are fairly subtle differences to keep in mind.

type intSet map[int]struct{} // a new type with an underlying type map[int]struct{}

type myIntSet intSet // a new type with an underlying type map[int]struct{}

type otherIntSet = intSet // just another name (alias) for intSet, add methods to intSet (only in the same package)

type literalIntSet = map[int]struct{} // just another name for map[int]struct{}, no adding methods

It's not insurmountable though. Assuming this lands in Go 1.9, I suspect we'll be seeing 2nd editions of several Go books. 😉

I regularly refer to Go spec for the accepted terminology, so I'm very curious what terms are chosen in the end.

[bcmills]

We do need a new term for these newly created types because any type can now have a name.

Some ideas:

• "distinguished" or "distinct" (as in, can be distinguished from other types)
• "unique" (as in, it is a type different from all other types)
• "concrete" (as in, it is an entity that exists in the runtime)
• "identifiable" (as in, the type has an identity)

[griesemer]

@bcmills We've been thinking about distinguished, unique, distinct, branded, colored, defined, non-alias, etc. types. "Concrete" is misleading because an interface can be colored as well, and an interface is the incarnation of an abstract type. "Identifiable" also seems misleading because a "struct{int}" is just as identifiable as any explicitly (non-alias) named type.

[bcmills]

I would recommend against:

• "colored" (in non-programming contexts the phrase "colored types" carries strong racial-bias connotations)
• "non-alias" (it's confusing, since the target of the alias may or may not be what was formerly called a "named type")
• "defined" (aliases are defined too, they're just defined to be aliases)

"branded" could work: it carries a "types as cattle" connotation but that doesn't strike me as intrinsically bad.

[jimmyfrasche]

Unique and distinct seem like the stand out options so far.

They're simple and understandable without a lot of additional context or knowledge. If I didn't know the distinction, I think I'd at least have a general sense of what they imply. I can't say that about the other choices.

Once you learn the term it doesn't matter, but a connotative name avoids unnecessary barriers to internalizing the distinction.

[rsc]

This is the definition of a bikeshed argument. Robert has a pending CL at https://go-review.googlesource.com/#/c/36213/ that seems perfectly fine.

[gopherbot]

CL https://golang.org/cl/36213 mentions this issue.

[LionNatsu]

I want to bring up the issue of go fix again.

To be clear that I am not suggesting 'take down' the alias. Maybe it is some thing useful and suitable for other jobs, that is another story.

It's something very important IMO that the title is about moving type. I have no wish to perplex the issue. Our aim is to deal with a kind of interface changes in a project. When we come to a change on interface, it is not true that we hope all the users use these two interface (old & new) as the same eventually, and that is why we say 'gradual code repair'. We hope that users remove/change the usage of the old one.

I still consider tool as the best method to repair the code, something like the idea which @tux21b suggested. For example:

$ cat "$GOROOT"/RENAME
# This file could be used for `go fix`
[package]
x/net/context=context
[type]
io.ByteBuffer=bytes.Buffer

$ go fix -rename "$GOROOT"/RENAME [packages]
# -- or --
# use a standard libraries rename table as default
$ go fix -rename [packages]
# -- or --
# include this fix as default
$ go fix [packages]

The only reason @rsc say no here is that changes will affect other tools. But I think it's not true in this work flow: if there is an out-of-date package (e.g. a dependency) uses the deprecated name/path of package, e.g. x/net/context, we can fix the code at first, just like the doc says how to migrate code to new version, but not hard-coding, via a configurable table in text format. Then you may use any tools whenever you like as same as Go of the new version. There is a side-effect: it will modify code.

[rsc]

@LionNatsu, I think you are right, but I think that's a separate issue: should we adopt conventions for packages to explain to potential clients how to update their code in response to API changes in a mechanical way? Perhaps, but we'd have to figure out what those conventions are. Can you open a separate issue for this topic, pointing back at this conversation? Thanks.

[gopherbot]

CL https://golang.org/cl/36691 mentions this issue.

[crawshaw]

With this proposal at tip, I can now create this package:

package safe

import "unsafe"

type Pointer = unsafe.Pointer

which allows programs to create unsafe.Pointer values without importing unsafe directly:

package main

import "safe"

func main() {
    x := []int{4, 9}
    y := *(*int)(safe.Pointer(uintptr(safe.Pointer(&x[0])) + 8))
    println(y)
}

The original alias declarations design doc calls out this as explicitly supported. It is not explicit in this newer type alias proposal, but it works.

On the alias declaration issue the rational for this is: _"The reason we allow aliasing for unsafe.Pointer is that it's already possible to define a type that has unsafe.Pointer as underlying type."_ https://github.com/golang/go/issues/16339#issuecomment-232435361

While that's true, I think allowing an alias of unsafe.Pointer introduces something new: programs can now create unsafe.Pointer values without explicitly importing unsafe.

To write the program above before this proposal, I would have to move the safe.Pointer cast into a package that imports unsafe. This may make it a bit harder to audit programs for their use of unsafe.

[bcmills]

@crawshaw, couldn't you have just done this before?

package safe

import (
  "reflect"
  "unsafe"
)

func Pointer(p interface {}) unsafe.Pointer {
  switch v := reflect.ValueOf(p); v.Kind() {
  case reflect.Uintptr:
    return unsafe.Pointer(uintptr(v.Uint()))
  default:
    return unsafe.Pointer(v.Pointer())
  }
}

I believe that would allow exactly the same program to compile, with the same lack of import in package main.

(It wouldn't necessarily be a valid program: the uintptr-to-Pointer conversion includes a function call, so it doesn't meet the unsafe package constraint that "both conversions must appear in the same expression, with only the intervening arithmetic between them". However, I suspect it would be possible to construct an equivalent, valid program without importing unsafe from main by making use of things like reflect.SliceHeader.)

[dr2chase]

Seems like exporting a hidden unsafe type is just another rule to add to the audit.

[crawshaw]

Yes, I wanted to point out that directly aliasing unsafe.Pointer makes code harder to audit, enough so that I hope no one ends up doing so.

[griesemer]

@crawshaw Per my comment, this was also true before we had type aliasing. The following is valid:

package a

import "unsafe"

type P unsafe.Pointer

package main

import "./a"
import "fmt"

var x uint64 = 0xfedcba9876543210
var h = *(*uint32)(a.P(uintptr(a.P(&x)) + 4))

func main() {
    fmt.Printf("%x\n", h)
}

That is, in package main, I can do unsafe arithmetic using a.P even though there's no unsafe package and a.P is not an alias. This was always possible.

Is there something else you are referring to?

[crawshaw]

My mistake. I thought that didn't work. (I was under the impression that the special rules applied to unsafe.Pointer would not propagate to new types defined from it.)

[griesemer]

The spec is actually not clear on this. Looking at the implementation of go/types, it turns out that my initial implementation required unsafe.Pointer exactly, not just some type that happened to have an underlying type of unsafe.Pointer. I just found #6326 which is when I changed go/types to be gc compliant.

Perhaps we should disallow this for regular type definitions and also disallow aliases of unsafe.Pointer. I can't see any good reason for allowing it and it does compromise the explicitness of having to import unsafe for unsafe code.

[griesemer]

I created https://github.com/golang/go/issues/19306.

[bradfitz]

This happened. I don't think anything remains here.