Go: proposal: spec: represent interfaces by their definition and not by package and name

[metakeule]

An interface is a collection of method names and signatures.

Say, we have:

package a

type Writer interface {
    Write([]byte) (int, error)
}


type WriterTo interface {
     WriteTo(Writer) (int64, error)
}

type WriterTo2 interface {
    WriteTo(w interface{Write([]byte) (int, error)} (int64, error)
}


From the perspective of the one writing an implementation, the a.Writer
and io.Writer are the same: As long as the Write method has the right
signature they are both fulfilled. 

However a.WriterTo and io.WriterTo are not, simply because they refer
to Writers of different packages although they have the same definition. Even WriterTo2
would only be compatible with io.WriterTo if
they later had an anonymous interface too.

see http://play.golang.org/p/kOH11TOlsE

Since interfaces are all about reusable code this behaviour must be 
considered an artifact of the way, interfaces are represented and treated during
compilation.

Since interfaces have no methods and no values, their only value is to help the
programmers via the type constraints. 

From the implementation perspective a.WriterTo, io.WriterTo and a.WriterTo2 must be
treated as being the same, a contraint to have a method WriteTo(w
interface{Write([]byte) (int, error)} (int64, error).

The names (and packages) of the interfaces should not be relevant for the compiler.
Instead the definition should be the representation and the names just shortcuts to
refer to them.

[ianlancetaylor]

Comment 1:

_Labels changed: added go2, languagechange, repo-main, release-none._

_Status changed to LongTerm._

[griesemer]

Comment 2:

See also issue #8691.

_Status changed to Thinking._

[gopherbot]

Comment 3 by [email protected]:

While admittedly this issue comes up rarely in regular code, as a data point, I run into
this issue quite a bit.  I've built an RPC stub compiler that takes an API specified in
my own language, compiles it, and then generates RPC stubs in different languages.  Go
is the core generated language.
As a simplified example, let's say the user tells me to generate stubs for a RPC
function "Print" that streams int32 from client to server, and streams strings from the
server back to the client.  Here's the Go API that I generate for the user on the client
side.
type PrintClientStream interface {
    // SendStream returns the send side of the client stream.
    SendStream() interface {
        Send(item int32) error // Places an item on the output stream
        Close() error // Indicates no more items will be sent
    }
    // RecvStream returns the receiver side of the client stream.
    RecvStream() interface {
        Advance() bool // Stages an item to be retrieved via Value
        Value() string // Returns the staged item
        Err() error // Returns errors
    }
}
Obviously the choice of "int32" and "string" are up to the user, as is the RPC function
name.  The reason SendStream() and RecvStream() return anonymous interfaces is because
of this issue.  A client may need to use two different services that define different
APIs, but result in the same streaming types; client sends int32 and receives string. 
I'd like them to be able to write a single function that operates over the two different
PrintClientStream interfaces.  But if I named the return types from SendStream and
RecvStream, they'd lose this ability.
I also have a straw man proposal.  
I understand the concepts of type identity and assignability as defined in the language
spec:
http://golang.org/ref/spec#Properties_of_types_and_values
I also appreciate the consistency of the current type identity rules wrt named types: a
named and an unnamed type are always different.  Whenever you see a "type" declaration
in a source file, you know that a distinct type is being defined.
Let's say we changed the language spec to say that type identity for interfaces ignores
the name.  I.e. interface names are simply a convenient shorthand, but for type identity
purposes, it's as if every interface were anonymous.  The rules for comparing these
interfaces remains the same; it's an order-agnostic comparison of method names and their
function types, and unexported methods in different packages are always different.
I think this is a nice and consistent way to specify this feature.  The main downside I
see is that it's yet another special-case in the language spec.  But interfaces are
already special; we already have the special-case in our value assignability rules.
<wishful thinking>
If you haven't tuned out so far, here's a thought.  Why wait until Go2 to incorporate
this feature?  My rationale: it seems unlikely that anyone was actually relying on the
existing semantics.  After all the compiler has ensured that all static type checks obey
the existing rules, and we're simply relaxing those rules so that more cases are allowed.
But for full disclosure, I do recognize that if we change this in Go1, existing code can
notice a change in behavior.  After all we provide type assertions in the language, as
well as the reflect package.  E.g.
http://play.golang.org/p/OscKQCdDOL
That's where my wishful thinking comes in.  I can't think of a reasonable way someone
would be using the existing semantics; my assertion is that if they're relying on
something this subtle, their code deserves to be broken (and I'm only half-kidding). 
But I can see that if we strictly follow the Go 1 compatibility rules, we simply cannot
change this feature, regardless of how much the code that relies on this deserves to be
broken.
Thoughts?

[maxnordlund]

This reminds me of the strawman to add typed objects to JavaScript, which considers two struct types equivalent iff they have the same fields (name and type) in the same order.

[mattn]

I'm thinking main.Writer should be treated as different as io.Writer. Because WriterTo have different signature of argument. I prefer to use embed interface of io.Writer in this case.

http://play.golang.org/p/IklkWaIvWG

[metakeule]

@mattn I want to be able to write a function that can receive any of the WriterTo interfaces.

[evanphx]

A quick note on this issue for anyone that is still watching it: Go "does the right thing" if the 2 inner interfaces are declared anonymous. That means the compiler and runtime already support the ability to do the right thing, it's just being overly strict if the interface has a name. Seems that a simple change, to ignore any given name when type checking 2 interfaces, would easily solve this issue.

[evanphx]

Here shows that if the interfaces are declared anonymous Go "does the right thing": http://play.golang.org/p/jpbd2325sj

[griesemer]

Removing Go2 label in light of use cases pointed out by #16209. Perhaps this (or some other solution) needs consideration before Go2.

[extemporalgenome]

Interfaces are already special enough that I think this would be warranted (one could argue the same for function types, so that first-order functions aren't pinned to type name either).

This may change/break existing programs, though. It's possible that some type assertions which fail now would succeed after this change.

[puellanivis]

I'm not sure this won't break existing programs, though. It's possible that some type assertions which fail now would succeed after this change.

Not sure how. If the two interfaces have identical methods, then all type assertions between the two should be knowably and provably valid…

I discussed this on the other bug #16209, in all, the worst thing possible is that two unnamed struct types might become assignable where they were not before, i.e.

var a struct { ctx context.Context } = struct { ctx x/net/context.Context }{ ctx: myctx }

But this seems like REALLY weird code…

Otherwise, you have unnamed function types, which would accept either interface the same way when called, but then that's kind of exactly the desired behavior for context.Context moving from x/net/ into the mainline library.

Finally, unnamed interfaces assign between identical interface definitions already…

[puellanivis]

https://play.golang.org/p/NPkhXi0VuD

Type assertions already go through fine:
if _, ok := a.(A).(B).(A); ok {

and the type switch tests for “implements interface” not “is actually typed this interface”

As such, a in the example will type switch to A or B depending on which case is presented first. (A in the first type switch, B in the second.)

[mdempsky]

@puellanivis Under the proposal, this code would change behavior:

type U1 interface{}
type U2 interface{}

var u interface{} = (*U1)(nil)
_ = u.(*U2)

In Go 1, this panics because *U1 and *U2 are different types.

If the proposal is accepted, the code would succeed because *U1 and *U2 are now identical types.

[puellanivis]

In Go 1, this panics because *U1 and *U2 are different types.

A) don't use pointers to interfaces… it's pretty much redundant, and almost certainly not what you want to do.

B) your code panics even if you attempt: _ = u.(U1) instead.

https://play.golang.org/p/alO15c93b-

And it STILL fails even if you change the var decl to var u U1 instead of var u interface{}

And then change the nil to 0, and both assertions succeed no panics.

[mdempsky]

A) don't use pointers to interfaces… it's pretty much redundant, and almost certainly not what you want to do.

That may be, but they're still part of Go 1 and they do have valid uses.

B) your code panics even if you attempt: _ = u.(U1) instead.

That's because u is the nil interface value, which always causes the type assertion to fail. See Type assertions [emphasis added]:

For an expression x of interface type and a type T, the primary expression

x.(T)

asserts that x is not nil and that the value stored in x is of type T.

[puellanivis]

Quoting the Go1 guarantee:

It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification.

As a subset of all code accepted by Go1 any code that compiles and runs without panic will, _unchanged_ continue to compile and run without panic if this proposal were implemented.

Next, according to specs:

In other words, even though the dynamic type of x is known only at run time, the type of x.(T) is known to be T in a correct program.

Note that Go1 only guarantees for programs that “run correctly.” And by spec, having a type assertion that will knowably at compile-time panic a type-assertion is, by definition, not a correct program.

Thus, the only case we're REALLY dealing with here is:
if u2, ok := u.(*U2); ok { /* this code will begin to start running */ } else { /* instead of this code */ }

First, let's set aside that this code is wrong wrong wrong, and the wrongiest wrong that ever wronged.

u2 will with this proposal be a valid pointer to an interface U2, which is actually—by design—identical to a pointer to an interface of U1. That is, to get at anything other than methods of U1 or U2, you would STILL have to type assert the pointed-to value into something usable.

Thus, the code will actually start working in a way that is guaranteed to work correctly. Because whether u points to an interface of type U1, or U2, then (_u).(U2), and (_u).(U1) would work if the pointer were not nil (otherwise nil-pointer deref) or the value of the pointed-to interface is not nil.

Namely, there is no code where treating type *U1 and *U2 as identical could possibly cause “broken” behavior. After all, they were already “meta-semantically” identical…

So, as noted, this actually makes MORE code correct, and is a strict super-set of all currently correct code.

var u interface{} = (*U1)(nil)

I just have to say, breaking this out into what it is actually saying; this is making a variable u, which is an untyped empty interface, which is set with the concrete type to an empty interface, and a value of concrete value of nil.

This code kinda made me throw up a little in my mouth…

[mdempsky]

Note that Go1 only guarantees for programs that “run correctly.” And by spec, having a type assertion that will knowably at compile-time panic a type-assertion is, by definition, not a correct program.

Your interpretation of "correct" here is stricter than intended: correct Go programs are allowed to panic.

Happy to discuss further if you disagree, but let's move it to golang-dev. It's tangential to this proposal.

[metakeule]

I want to add that if this feature was implemented, it would allow a whole new class of packages/libraries to require no imports.

Let's say you follow the paradigm of receiving an interface and returning an implementation when designing your functions and methods.

Now you are writing package z which should be compatible to the interface A in package x and B in package y. ATM you would need to reference x.A and y.B directly and therefor need to import packages x and y. Your package becomes dependent of them.

With the proposed change you could simply copy the definitions of the interfaces A and B to your library z and use them as parameter types or embed them to structs without having to import x or y. This way your package z has no dependencies and the users of your package z might combine it with x and y and at a later point swap x or y out and replace them with compatible versions without breaking any dependency of z.

All in all this could lead to a better and more decoupled library ecosystem based on compatible interfaces. This is the most important argument for the change IMHO.

[puellanivis]

In other words, even though the dynamic type of x is known only at run time, the type of x.(T) is known to be T in a correct program.

This isn't a matter of “correct programs aren't allowed to panic” it's that a type assertion that is KNOWN to always panic, is not a correct program. But as well…

The exact error values that represent distinct run-time error conditions are unspecified.

Thus, there is no possible correct code that can depend upon a type assert and only a type assert throwing a panic. Any deferred function recovering run-time panics must by necessity handle _all_ possible run-time panics. (like instead of let's say nil-pointer dereference)

Otherwise, they are—by necessity—depending upon unspecified behavior.

[bcmills]

It's not just programs that panic today that will break.

Consider a program that uses named versions of interface{} as variant types and unpacks them using type-switches. Relaxing interface equality will cause the switch to take the wrong branch, and the program (which previously worked) will fail when that branch tries to type-switch to unpack the variant parameter.

A toy example: https://play.golang.org/p/Lrql97NtI6
It currently runs successfully, but with the proposed change would panic.

It may be that no such programs exist in practice, but I don't think it's correct to claim that the proposed change is strictly backward-compatible in theory.

[mdempsky]

@bcmills I think you mean with the proposed change it won't compile. The Go compilers don't allow duplicate cases in a type switch. (Edit: And the Go spec was updated to recognize this in 3d81d4adc9419e2fcba5888ab074d3f17eb5eb03.)

[bcmills]

Oh, that could be. At any rate, it works today and it wouldn't work with the proposed change; the details of the non-working are mostly irrelevant.

(You could imagine an equivalent program written using reflection that moves the failure from compile-time to run-time.)

[puellanivis]

The entire code you pasted strikes me as broken.

First, it's a bad design, because you can't type switch between Uint or Int, nor could you possibly type assert between the two. Both of which one is likely to expect to be able to do.

The ONLY way this design works is by passing around anonymous function signatures that take named interfaces. Even a named function type fails: https://play.golang.org/p/ox1KiNdZ7r

Now, if you had used named function types to begin with, then the anonymous functions would still work as would the specific named-type function.

If you want to say that this highly-contrived code constitutes a suicide-pact on the Go1 compatibility guarantee… fine… have at it… enjoy 1.7 and all the people complaining how you broke x/net/context.Context vs context.Context, which is far more likely and larger and more visible impact than breaking type switches on anonymous function types…

[puellanivis]

Reflection DeepEqual's documentation:

“ 141 // Func values are deeply equal if both are nil; otherwise they are not deeply equal.”

Realized all of this is irrelevant.

[mdempsky]

It's a questionable design, but it's valid Go 1.

Also, reflect.DeepEqual's documentation is talking about function value equality, not function type identity.

[puellanivis]

Fine, you've convinced me we can't change this in any Go 1.x release…

So, I guess we're stuck being incapable of writing reusable code for gRPC, that allows us to transition from x/net/context.Context to context.Context without breaking everyone's code in one fell swoop… because at some point, it's going to switch to context.Context, and everyone will need to change their code at that time, and all their code before will still be using x/net/context.Context, because it's the most compatible Go code… and it's what the function signatures of the server interfaces for gRPC take, so no one can switch earlier without breaking everything in the first place.

We're literally stuck in a Catch 22 with moving context out of the experimental tree. We would like people to switch to using context, but because gRPC uses x/net/context.Context, everyone is going to have to keep x/net/context imports around, just to keep compatible with it. And gRPC can't just change to context, because then everyone using x/net/context won't be able to compile, and if they do, then since 1.6 has no context inbuilt library, you can't write code that compiles both on Go 1.6 and 1.7 unless you use x/net/context.

All because we've decided that keeping the uncommon, nearly-impossible brokenly designed code compatible is more important than the common use case. #slowclap

I know, “you can complain about this when you write your own language”, so yeah, I'm forking Go… because I think this design decision is a bad choice… seems weird to keep a fork that's only going to have like 10s of lines difference from the mainline, but hey, whatever… at least merge conflicts are going to be unlikely…

[cespare]

@puellanivis I don't see how that is relevant. The Go 1 compatibility promise applies to the Go language and its standard library, not everything written in Go.

[mdempsky]

@puellanivis First, your tone is coming off as very antagonistic. I'd appreciate if you reflected on your approach and going forward presented your arguments in a more constructive manner.

With that out of the way, I've never said we can't address this issue _in some way_ for Go 1. That's not even my decision. I've only clarified that the code samples presented are valid Go 1.

I've also not ruled out the possibility of finding some other solution to gRPC's use of x/net's Context. E.g., the type alias proposal (#16339) would be a generic solution to that. I can envision other non-generic proposals if it becomes really problematic; e.g., maybe the compiler could treat "golang.org/x/net/context" as a _path_ alias for "context".

Lastly, like @cespare points out, backwards compatibility for gRPC is technically outside the Go 1 compat guarantee (though that doesn't mean we should cavalierly ignore the problem either).

[puellanivis]

I've already made a reasoned and non-antagonistic argument. But no one seems to care about that. People appear to be more interested in keeping neigh-impossible, contrived, and broken-in-design code working than ensuring that common-case code moving forward can be written in a sensible manner, rather than with giant hacks everywhere…

It's not a matter of gRPC being part of the Go1 compat guarantee; it's about ensuring that common and widely used code can be written compatibly in a forward-looking sensible way, rather than breaking the common case, just because someone can contrive some goat-offal code that wouldn't work after this proposal would be enacted.

First, your tone is coming off as very antagonistic.

Welcome to Open Source, if you don't like it, there's the door. I mean, if you weren't listen to me before, then why should I care about my rhetoric now? What are you going to do? Doubly reject my argument? Or reject it more sternly?

There's literally nothing you can do to me that hasn't already been done… even banning me from participating in the project doesn't matter at this point… why would I stay a part of a community that won't consider reasonable arguments in favor of technicalities and horrible, _horrible_ code?

[velovix]

@puellanivis Compatibility vs improvement is a hard choice to make. Go has already made that choice. The Go compatibility promise has been cited as one of the biggest reasons for its large-scale adoption, and breaking it sets a bad president. I understand your argument about usability, but exceptions to the compatibility promise were decided long ago, and this doesn't fit any of them, IMO.

Welcome to Open Source, if you don't like it, there's the door.

This might be true for the Linux kernel or the Vim dev mailing list, but in my experience, the Go community is nothing like those places. That's a hugely positive thing.

[ianlancetaylor]

@puellanivis No decisions have been made about this proposal. People are presenting data, just as you are. We can not ignore the fact that the change would not be strictly backward compatible. We can decide to make the change anyhow, but we can not ignore it.

Regarding tone, this issue tracker is covered by the Go community code of conduct (https://golang.org/conduct). Please be respectful. Thanks.

[bcmills]

@puellanivis You're right that the snippet I linked would be pretty bad code to write in the first place. That doesn't necessarily mean that there aren't more reasonable cases that boil down to essentially the same problem, nor does it imply that there aren't Go users somewhere relying on code like that.

As Ian says: we can't ignore the fact that the change wouldn't be strictly backward-compatible. It's at least a useful input to the decision, and to how we communicate the result of that decision in either direction.

[mewmew]

From @griesemer in https://github.com/golang/go/issues/16209#issuecomment-230850590:

@zellyn I'm going to remove the Go2 label on #8082 so it's on the radar. Please add your comments there. Concentrating comments on one proposal will show community support better.

As issue #16209 has been closed as a duplicate of this issue, would you mind updating the title of this issue to include the proposal: prefix and add the relevant proposal tag? This will make it easier for the community to become involved in the ongoing discussion. As this proposal also solves many of the problems related to move of the x/net/context package, it would be very interesting to discuss this proposal in tandem with the alias proposal (#16339).

[mewmew]

I'd be very curious to find out if this proposal may resolve an issue I've hit quite a few times in the past, when wanting to return a concrete type from a method, but being forced to return an interface instead simply to satisfy the type equality requirements of another interface.

A breif example is given below, which is taken more or less verbatim from an ongoing effort to implement a pure library for interacting with LLVM IR in Go [1].

(Full GoDoc for the LLVM IR types package.)

package types

type Type interface {
   Equal(u Type) bool
}

// An IntType represents an integer type.
type IntType struct {
   // Integer size in number of bits.
   size int
}

// Size returns the integer size in numer of bits.
func (t *IntType) Size() int {
   return t.size
}

// Equal reports whether t and u are of equal type.
func (t *IntType) Equal(u Type) bool {
   if u, ok := u.(*IntType); ok {
      return t.size == u.size
   }
   return false
}

(Full GoDoc for the LLVM IR value package.)

package value

import "foo/types"

//  A Value represents a computed value that may be used as an operand of other values.
type Value interface {
   // Type returns the type of the value.
   Type() types.Type
}

(Full GoDoc for the LLVM IR constant package.)

package constant

import (
   "math/big"

   "foo/types"
)

// An Int represents an integer constant.
type Int struct {
   val *big.Int
   typ *types.IntType
}

// NOTE: The return type is types.Type, as required to satisfy the value.Value
// interface, but returning *types.IntType would be preferable.
func (c *Int) Type() types.Type {
   return c.typ
}

In the example given above, what I would really like to return from the Type method on *constant.Int is *types.IntType rather than types.Type, as this would allow direct access to the Size method. In other words, I would like to return the concrete type rather than the interface type it satisfies, while still satisfying the value.Value interface, which requires a Type() types.Type method.

Has anyone else had to work around similar issues?

Edit: I realize that this example has been covered in the FAQ: Why doesn't type T satisfy the Equal interface?. My question is then, if this proposal were to be implemented, would that also allow for the above use-case (and consequently mandate an update of the FAQ entry)?

[kostya-sh]

As this proposal also solves many of the problems related to move of the x/net/context package, it would be very interesting to discuss this proposal in tandem with the alias proposal (#16339).

context.Context and x/net/context.Context are defined differently because context.CancelFunc type (e.g. returned from WithCancel method) is not the same type as x/net/context.CancelFunc. So I am not sure that this proposal as it is can help with move of the x/net/context package.

[atdiar]

Putting that here for Go2, but I think that implementing such a proposal will add a restriction to the implementation of variant types as bounded interfaces if we ever wished to have them implemented.

It would mean than two compatible interfaces can never be used in a variant since the pattern matching (i.e. exhaustive type switch) would be undecidable.
That can be a fair choice to make, independently of the implementation of this proposal, if one wants to avoid deep nested levels of abstraction anyway.

Just leaving that here for history.

Re this exact proposal, it is indeed backward-incompatible. And as @kostya-sh mentionned, the issue is superseded by the issue of package/namespace identification. (not really an issue to me, works as intended).

[bcmills]

@atdiar

It would mean than two compatible interfaces can never be used in a variant

That's already true today: type-switches for interface types match based on the method set. You can only usefully switch on interfaces if there is some method in the first interface that isn't in the second. For the "variant types as bounded interfaces" use-case, you need to either deal exclusively with concrete types or accept that constraint on the interface types.

The same workaround - adding a possibly-unexported method to the first interface - works regardless of this proposal.

[atdiar]

@bcmills You're right https://play.golang.org/p/TgiUz7BdYH

I guess it should only be possible to use concrete types (and variants which do not intersect) for an implementation of variant types (as bounded interfaces).

Extending the reasoning to the current interfaces, my point is that the compiler should probably have been able to warn about undecidable overlapping cases instead of relying on source order.

Alternatively, if we consider that source order shall be the norm, the program you wrote above should still be decidable if we adopted the current proposal (the first case would be triggered)

I guess that it can be corrected in Go2 if it makes sense. I don't think that it can be in Go1 unfortunately (unless declared a bug).
(some people might have relied on this behaviour)

[dlsniper]

From my understanding by reading this and https://github.com/golang/go/issues/16209 this should be labeled as a Bug not Language change, no?

[bcmills]

@dlsniper No. The proposed change would require a change to the spec, the proposed behavior is not the current behavior of any mainline implementation, and the change is not strictly source-compatible with the Go 1 spec. It is not a "bug" in the sense that we typically mean within the Go project.

[dlsniper]

Well, it's an implementation bug, as you have also said, that's what I mean. The either the language spec needs to be updated to reflect the current implementation or the current implementation should be fixed to reflect the specifications imho.

[puellanivis]

@ianlancetaylor “No decisions have been made about this proposal.” Just because no formal decision has been made does not mean that I haven't changed my mind. People have presented a reasoned argument that due to the Go 1 compat promise, this change cannot be made without breaking _some_ possible code. As such, I now agree that the mainline Go 1.x cannot implement this proposal without violating that compat promise, which is a no go.

@mewmew “My question is then, if this proposal were to be implemented, would that also allow for the above use-case (and consequently mandate an update of the FAQ entry)?” No, it would not allow for your desired use-case. The proper solution to what you're trying to do is to type-assert to type.Int from type.Type.

@kostya-sh “… because context.CancelFunc type (e.g. returned from WithCancel method) is not the same type as x/net/context.CancelFunc” but the CancelFunc type does not actually appear in either Context interface. As such regardless of these those two named function types being and remaining different, both Contex interfaces are still identical.

@dlsniper “it's an implementation bug” This is not an implementation bug. The current behavior is strictly in conformance with the specification. It is difficult to—in anyway meaningful way—call this a “bug”.

[Merovius]

The potential stability breakage that people are discussing is mainly a red herring in and off itself, IMHO: Pretty much every change to the stdlib breaks compatibility in theory and no one had ever trouble doing that. I brought this up before and people rightly pointed out, that there is a tradeoff to be made between strict, theoretical stability and the ability to evolve.

So, I'd encourage people to find actual, existing code that gets broken (maybe the BigQuery dataset can help with that, though it's probably a complicated query). Because that's the only way to consider this tradeoff in an informed matter.

Personally, I'd like to see this proposal if we deem it at all possible, for all the reasons mentioned above by others.

[bcmills]

@Merovius

Pretty much every change to the stdlib breaks compatibility in theory and no one had ever trouble doing that.

The compatibility policy specifically says:

Under some circumstances, such as when the type is embedded in a struct along with another type, the addition of the new method may break the struct by creating a conflict with an existing method of the other embedded type. We cannot protect against this rare case and do not guarantee compatibility should it arise.

There is a difference between "breaking compatibility" and "breaking compatibility in a way prohibited by the Go 1 policy". This proposal does the latter, not just the former. That's a significant difference from most (all?) previous changes in the language spec since Go 1.

[Merovius]

@bcmills that section was added a year ago, long after the release of go 1. So it doesn't really disprove my point.

[puellanivis]

that section was added a year ago, …

Right, under the specific exception noted here:

Specification errors. If it becomes necessary to address an inconsistency or incompleteness in the specification, resolving the issue could affect the meaning or legality of existing programs. We reserve the right to address such issues, including updating the implementations. Except for security issues, no incompatible changes to the specification would be made.

Removing func/type/var/const at package scope

This has not been done with a single Golang stdlib change. Even when those things have been obsoleted. c.f. os.SEEK_XXX. “Deprecated: Use io.SeekStart, io.SeekCurrent, and io.SeekEnd.”

Adding a method to an interface

This is also not done. This is why there is a type flag.Value and a type flag.Getter

Removing a method from an interface

Also not done, as noted above.

Also, Golang stdlib interfaces are designed to be tight. That is, they do not implement more functions than is absolutely necessary. That is why there is io.Reader, io.ReadCloser, io.ReaderAt, etc

Adding a field to a struct
Adding a method to a type

Already mentioned above as an exception to the breaking compat guarantee.

Removing a field from a struct
Removing a method from a type

Also never done. src/archive/zip/struct.go: CompressedSize uint32 // Deprecated: Use CompressedSize64 instead.

Changing a function/method signature

Never done in Go1. This example is a struct of functions, but still, same idea. If you need change the signature of a function/method, you introduce a new, more specifically named function, and then make the deprecated calls use the new signature under the hood:

        // DialContext specifies the dial function for creating unencrypted TCP connections.
        // If DialContext is nil (and the deprecated Dial below is also nil),
        // then the transport dials using package net.
        DialContext func(ctx context.Context, network, addr string) (net.Conn, error)

        // Dial specifies the dial function for creating unencrypted TCP connections.
        //
        // Deprecated: Use DialContext instead, which allows the transport
        // to cancel dials as soon as they are no longer needed.
        // If both are set, DialContext takes priority.
        Dial func(network, addr string) (net.Conn, error)

This severely limits the kind of changes you can do to your API if you want to claim backwards compatibility.

It limits the form, but not the actual changes you can make. But the advantage of needing only run a compat compile test on your code for the new version to make sure it doesn't horrible break things has a very, very strong advantage here. And is worth the occasional, split interface… and besides now the interface is much more specific about the contract it fulfills, rather than be a ridiculous horrible mess of every idea someone had at the time, but are we ever going to use them? Who knows.

[Merovius]

I commented here, because in the past it has been pointed out to me that an all-or-nothing "there could be a theoretical program that might get broken by this" approach isn't the one chosen by go and because that exact approach is the one used to block this language change. I was hoping to steer this debate in a more nuanced and productive direction, not to pick nits.

Everyone agrees, that this change would constitute a breakage in the sense of the go 1 compatibility guarantee. Yet, there have been plenty of such breakages in the past (with the versions of the compatibility guarantee published at that time) and a nuanced approach was advocated for and advertised then.

Even if the final decision is going to be "we are not going to make that breakage", I still think talking about real-world breakages that could occur due to this change would be useful. At the very least much more useful than the straight refusal to even debate the notion of making an exception to stability.

@puellanivis I'm sorry, but my intention of linking to that wasn't to debate that post again, but to give a written justification of my statement that stability has been broken in the past. Which it has. That is not debatable. Saying "but only because the guarantee we gave was wrong" isn't a valid argument that we did not violate the guarantee given.

[puellanivis]

I'm not particularly looking to debate this either. But Im on the pro-change side, and I still regardless disagree with the very premises that you advance.

… written justification of my statement that stability has been broken in the past. Which it has. That is not debatable.

I'm going to need actual concrete examples, which you have not provided. Meanwhile, as I covered, none of these concerns have actually been done.

If you want me to agree that Go 1.x broke go1 backwards compat, then I need concrete examples, not just “theoretically, it could have broken if we made these changes.”

Saying "but only because the guarantee we gave was wrong"
“Methods. As with struct fields, it may be necessary to add methods to types”

So, the specification already covered struct fields. As such, the clarification made to the specification was not unprecedented. There was _already_ an exception for adding things to embedded structs. The change you linked to simply clarified further that “Methods are also subject to the same exception as Fields.”

This requires rewriting the fundamental rule about when two types are identical. From “If they're named, only if they're part of the same typedecl, otherwise different.” to “except if it's an interface, in which name will never be considered.” That is a big brand new whole exception to type handling that doesn't exist for any other type.

[awalterschulze]

One example of something that it would break would be really great.

[mdempsky]

@Merovius As the discussion on that issue explains, the de facto policy was already that it was okay to add methods to standard types, and was something we've done in every Go release. The compat document was updated to better reflect existing practice/intentions, not to suddenly allow new incompatibilities.

@awalterschulze I gave an example earlier in the discussion: https://github.com/golang/go/issues/8082#issuecomment-231457565

[awalterschulze]

great example @mdempsky

Your example should work with type aliasing ( https://github.com/golang/go/issues/16339 ), but somehow I don't think its enough for the other examples.

[Pomyk]

How about limiting this change to non empty interfaces? Ugly but would break way less stuff.

[ghasemloo]

A general advice I have heard in go is to have functions accept interfaces but to return and pass concrete types from/to them. To me this proposal sounds like we want to pass interfaces to functions in place of concrete types. Might worth considering what are the use cases for this proposal. How common are the cases where this proposal would have made better go code?

[metakeule]

@ghasemloo for a common case where this proposal would allow decoupling of packages (which I would consider beneficial, esp. regarding the package management chaos that we currently have), please read my comment above: https://github.com/golang/go/issues/8082#issuecomment-231479060

[mewmew]

@ghasemloo

Might worth considering what are the use cases for this proposal. How common are the cases where this proposal would have made better go code?

https://github.com/golang/go/issues/8082#issuecomment-236474953 presents one such use case.

[ianlancetaylor]

Now that we have type aliases, a package writer can get the same effect in at least some cases by defining an alias rather than defining an interface with the same methods.

That said, this proposal would permit packages to define identical interface types without importing any common package.

The crux of this issue is changing the definition of type identity. Currently named types are identical if they come from the same type definition. This proposal suggests changing that for interface types: interface types will be identical if they have identical method sets.

[mewmew]

Now that we have type aliases, a package writer can get the same effect in at least some cases by defining an alias rather than defining an interface with the same methods.

@ianlancetaylor From my understanding, even using type aliases, it is still not possible in https://github.com/golang/go/issues/8082#issuecomment-236474953 to use

func (c *IntType) Type() *types.IntType {
   return c.typ
}

to satisfy the value.Value interface, but rather,

func (c *Int) Type() types.Type {
   return c.typ
}

has to be used. Please point me in the right direction if I'm wrong.

Cheers,
/u

[bcmills]

@mewmew That seems orthogonal to the proposal.

The point of this proposal is to consider interfaces with identical definitions to be the same type, presumably because they are already mutually assignable. However, *types.IntType and types.Type are _not_ mutually assignable: a types.Type is not assignable to a *types.IntType without a conversion.

[ianlancetaylor]

@mewmew I agree with @bcmills. That requires a kind of covariant typing, which is not in this proposal.

[mewmew]

@bcmills and @ianlancetaylor Ok, thanks for the clarification. Was hoping to remove a category of run-time type assertions. Not sure if the sum-type proposal (#19412) could be a better candidate then, for this specific use case. Or if it something else entirely.

[puellanivis]

I definitely think there still is value in this proposal. Assignment already contains an exceptional behavior for interfaces (in that it checks to see if the interface is assertable), and semantically the idea behind interfaces is that they should define strictly and solely only a set of methods that a value implements.

The idea that all type Stringer interface { String() string } types are the same type fundamentally continues to hold a compelling argument. Interfaces are kind of a distilled… essense(?) of “Duck Typing”. So, the idea that a Duck isn’t an Ente, because they have different names, even though they both quack like a duck, walk like a duck, and are in fact ducks… I dunno… Both names describe and define ducks… are they not semantically identical then?

But as noted, yes, there remains always still the exception of the empty interface, in that really the only thing available to define them in any way is the name. And then there are additionally all of the other caveats pointed out earlier in the thread…

“Good” Go code shouldn’t break if we tweaked this definition, because it should already be treating interfaces as duck types. But the question is—as always when redefining behaviors—would we be accepting that _some_ code likely will break… and I think before Go 2.0, that answer is no.

If we hype up the change in definition and behavior for Go 2.0, and people are anticipating it, and they can prepare for it… we will still end up with Photoshop porting out of Carbon at the last minute… but, at least we would have taken the most responsible path towards adoption… ?

[neild]

Does this proposal differ from defining every interface type as a type alias? If so, how?

What does the following print?

type I1 interface{}
type I2 interface{}

t1 := reflect.TypeOf((func(I1))(nil))
t2 := reflect.TypeOf((func(I2))(nil))

fmt.Println(t1)
fmt.Println(t2)
fmt.Println(t1 == t2)

[griesemer]

@neild There may possibly be minor technical differences (e.g., how an interface is printed, either by name or by its literal, depending on how this would be implemented), but my understanding of this proposal is that if we would write every interface type definition as a alias type declaration, we'd get the same effect. The problem is of course that you can't do that with pre-existing code that you don't control but may depend on.

Interfaces already behave essentially like the proposal suggests for operations such as assignments, type assertions, equality operations, etc. because there we have special rules in place.

But as @ianlancetaylor pointed out, this proposal is really affecting the definition of type identity for interfaces, which comes into play when we have other (non-interface) types composed of interfaces: A function type func(x Stringer) currently is different from func(x interface{String() string}). The proposal would make those identical.

[jimmyfrasche]

@griesemer wouldn't this also have to extend to interfaces such as

type I interface {
  RecursivelyReferences(I)
}

which cannot be expressed with type aliases?

[griesemer]

@jimmyfrasche I'm not sure I understand the question. Do you mean something like

type T = interface{
   f(T)
}

except that we wouldn't have to write the = ? (I guess in this case, we're talking about the non-interface type of f(T)).

[bcmills]

This compiles today (https://play.golang.org/p/cnLlSHpwRz):

type I interface {
    SetFrom(I)
}

As an alias, it fails to compile with invalid recursive type alias I (https://play.golang.org/p/CoesgI8bCs):

type I = Interface {
    SetFrom(I)
}

So that's one of the “minor technical differences”: interfaces can be recursive, but aliases (at the moment) cannot.

[bcmills]

Upon further reflection, the fact that aliases cannot be recursive, combined with the fact that mutually-assignable interface types are not identical, is probably going to be a significant headache for me.

I'm trying to write a code generator that can wrap C++ APIs as Go APIs, and I had planned to use aliases of interface types to represent template instantiations (which will need to be defined in every Go package that wraps a C++ API that instantiates the template).

I have to use type aliases rather than defined interface types so that the instantiated interfaces are mutually-assignable with the same instantiations in other wrapper packages, even if they include methods that refer to _other_ template instantiations. (For example, consider std::vector<std::unique_ptr<T>> for some non-copyable, non-movable type T.)

Unfortunately, the prohibition on recursive aliases means that I can't use that approach to wrap any C++ member function that refers to its own type. Notably, that includes all assignment operators.

(On the other hand, my problem could be addressed by allowing recursive aliases, which would be strictly compatible with Go 1 but perhaps more difficult to implement in practice.)

[griesemer]

@bcmills Let's not digress from the proposal at hand. Yes, the above example is currently not accepted, but it's not clear if that's "just" and implementation issue. At least I don't see why it couldn't be accepted.

[bcmills]

Right, I don't mean to digress. My point is that there are several possible solutions to the problem I have, and this proposal is one of them.

(Accepting mutually-recursive aliases would be another. Allowing aliases of struct types to define methods would be a third, although that seems less likely to happen.)

[metakeule]

When it comes to recursive interface definitions, I would propose, that

type A interface {
   SomeThing(A)
}

and

type B interface {
   SomeThing(B)
}

would result in the same (nameless) internal representation.

[cznic]

Let's go exponential?

[griesemer]

@metakeule The internal representation doesn't have to be the same, or nameless; the implementation may choose whatever representation is suitable. The point of this issue is that the rules for identity for interfaces would have to be changed in the spec.

[metakeule]

@griesemer Yes, you are right.

I chose "internal representation" as a helper to illustrate the point, that when checking for identity there would be some representation that is not part of the language but has some meaning or checks for "self-ness" if that is a word; an argument that is the surrounding interface itself would not be considered the interface name, but this special placeholder "self".

[evanphx]

I'd still love for this to be fixed for Go 2!

[DeedleFake]

This proposal seems to be missing the Proposal label. Is that intentional?

[bvk]

I came across this problem recently when I am trying to implement marshaling/unmarshaling objects to/from key-value stores. I think solution to the original problem in comment 1 would be good to have in Go 2. Below is my interface design for marshaling objects into key-value stores. Any other solutions to my problem are also much appreciated. Thanks.

The Problem - Cyclic Dependencies between interfaces wouldn't compile.

type Encoder interface {
    Encode(name string, object KVMarshaler) error
}
type KVMarshaler interface {
    MarshalKV(enc Encoder) (key string, value []byte, status error)
}

My Solution 1 - Break the cycle using expanded anonymous interface.

type Encoder interface {
  Encode(name string, object interface{MarshalKV(Encoder)(string,[]byte,error)}) error
}
type KVMarshaler interface {
  MarshalKV(enc Encoder) (key string, value []byte, status error)
}

Problem 2 with Solution 1 - Following TreeEncoder doesn't compile because Encode signature doesn't satisfy the Encoder interface with the anonymous interface.

type TreeEncoder struct {
  ...
}

func (te *TreeEncoder) Encode(name string, object Marshaler) error {
   ...
}

Solution for Problem 2 - This works. Since number of encoder types would be far fewer than number of object types that must implement marshaling methods, I am going with this for now.

type TreeEncoder struct {
  ...
}
func (te *TreeEncoder) Encode(name string, object interface{MarshalKV(string,Encoder)(string,[]byte,error)}) error {
...
}