Go: proposal: Go 2: allow defining new methods on non-local imported types

[adnsv]

Purpose

Allow defining new methods on non-local imported types.

Description

Proposed is a new core language feature which allows to extend functionality of types imported from other packages by binding new methods.

For example:

package main

import "foreign" 

// assuming foreign has type A that does not support the required interface methods

// String implements stringer interface for foreign.A
func (a *foreign.A) String() string {
    // read public fields in 'a' or use it's public methods
}

// UnmarshalJSON implements JSON unmarshaller for foreign.A 
func (a *foreign.A) UnmarshalJSON(b []byte) error {
    // write to 'a' public fields or use it's public methods
}

func main() {
    a := &foreign.A{...}
    s := a.String()
    ...
}

Presently this kind of functionality may partially be achieved by struct-wrapping (embedding) or type-aliasing the imported type, but it is hard to maintain all the imported features without lots of boilerplate code.

For example:

• If the imported type has chainable methods where it returns a reference to it's original 'self', wrapping or type aliasing will not work.

• Passing wrapped or aliased values as parameters may require explicit casting

• If the imported type is used anywhere as a struct member or an array element, the whole enclosing type would need to be rewritten

Since the memory layout of the type does not change, it seems like all these efforts can be reduced by allowing binding new methods. In order to avoid issues with other packages, the new binding should probably be effective only within the scope of the package where it was declared.

This kind of functionality was added in Delphi (and FreePascal) as Record Helpers, which turned out to be quite useful, albeit with a bit awkward syntax.

It looks like Go version could be implemented with a very straightforward syntax.

[ianlancetaylor]

Dup of #21401?

[adnsv]

Yes, missed that one. For the most part it's the same thing, but I have a few points compared to the original #21401:

1. Important! Locality, scope, and outside visibility of changes should only be limited to the package where this new method is declared. The goal is to makes things easier to use _within_ your package; for all other packages things should remain unchanged. I believe, this addresses many of the concerns in the original thread.

2. Syntax does not require any radical grammatical changes. Majority of parsers/highlighters/linters/etc. will work as is or with minimal updates.

3. Keeping full functionality of chainable methods (which you will loose with wrapping and aliasing), from my point of view, is an important advantage that was not mentioned in #21401.

4. It improves "syntactic symmetry" :)

• allowed now: func (a T) Foo(b pkg.T) (ret pkg.T)
• not allowed: func (a pkg.T) Foo(b pkg.T) (ret pkg.T)

[randall77]

the new binding should probably be effective only within the scope of the package where it was declared.

It seems weird that when passing a value around, its methods mysteriously disappear (and maybe reappear?) when passed between packages.

How do you do that? Take this example:

package a
type I interface { Foo() }
func callFoo(i I) {
   i.Foo()
}

package b
func (t c.T) Foo() { // c is some third package with a type T
    println("Foo in b")
}
func main() {
    var t c.T
    t.Foo() // prints "Foo in b"
    var i a.I = t // succeeds because t has a Foo method
    a.callFoo(i) // allowed, because i is of type a.I
    // callFoo then prints "Foo in b" also.
}

You want something to fail here, but I'm not sure which step.
Also, same question but replacea.I with the anonymous type interface { Foo() } everywhere.

[adnsv]

I think, for your example, it should show the same behavior as a wrapped/embedded version:

package a
type I interface { Foo() }
func callFoo(i I) {
   i.Foo()
}

package b
func (t c.T) Foo() { // c is some third package with a type T
    println("Foo in b")
}

type W struct { c.T } // wrapper
func (w W) Foo() {
    println("Foo in b")
}

func main() {
    var t c.T
    var w W

    t.Foo() // prints "Foo in b"
    w.Foo() // ditto

    var i a.I = t // succeeds because t has a Foo method
    var j a.I = w // also succeeds because w also has a Foo method

    a.callFoo(i) // allowed, because i is of type a.I
    // callFoo then prints "Foo in b" also.

    a.callFoo(j) // same output here...
}

[adnsv]

Here is how the logic of this kind of binding could be implemented:

1. For all local func declarations that bind to "foreign.T":
2. Internally define an "invisible" type struct local_T { foreign.T }
3. Treat the local declarations from #0 as methods of local_T instead of foreign.T
4. Observe that local_T and foreign.T have the same memory layout and can be type-casted and assigned in both directions
5. Get somebody really smart to work on #5 and #6
6. For other local declarations and functions that refer to foreign.T: substitute (or cast) the types of local parameters/returns values that refer to foreign.T with local_t (where it is possible and where it makes sense)
7. Figure out what to do with type reflection/switches/assertions
8. Come up with a good explanation why the Capitalized versions of the new methods are not publically visible

[deanveloper]

package foreign

type T struct

func (t T) String() {
    return "foreign"
}

package main

import "foreign"

func (t foreign.T) String() {
     return "main"
}

func main() {
    println(foreign.T{}.String())
}

One of 3 things can happen here, all of them are bad:

• "main" is printed
  
  
  
  • This is bad because this complicates the type system. This allows "overriding" of foreign.T's functions, which can cause significant problems, especially now that there is _no way_ to call foreign.T.String.
  
  
  • Also, what if the signatures do not match?
  
  
• "foreign" is printed
  
  
  
  • This is bad because the local environment declares a function that overrides T's, but it is impossible to call, instead it should not compile.
  
  
• The code does not compile
  
  
  
  • This means that adding methods to a type is now a breaking change. For instance:
  
  
  • Let's assume foreign.T (v1.5) does not implement fmt.Stringer
  
  
  • You want it to implement fmt.Stringer, so you add the extension to implement it
  
  
  • The author realizes it should implement fmt.Stringer, so they add a String() string method and increase the version to v1.6
  
  
  • You pull the change, assuming that it will not break your code, but it does, because foreign.T.String() is already defined.
  
  

[krhubert]

I do not like the foreign syntax but I think there is a need for allowing extendability for types. I see the code like this very often:

type APlus struct {
  a.A
}

func (a *APlus) SomeExtraMethod() {}

The standard example is with Unmarshal/Encoder.

I think there is a need for such extensions but nobody comes up with some syntax that would satisfy community and do not lead to errors.

• forbid to overwrite existing Method (foreign syntax)
• add syntax for struct for allowing extensions

type A extendable struct {
}

• ???

For sure the topic appears many times and I think it just needs more discussion. Some survey, code meta analyise (someone did it for if err != nil { return err } patter, but I can't find it right now) will help to make a final decision.

[deanveloper]

I do not like the foreign syntax but I think there is a need for allowing extendability for types. I see the code like this very often

With the way I used "foreign", it was just a package specifier. github.com/user/foreign would probably be the fully-qualified package, foreign.T is the type. It's just standard Go syntax...

forbid to overwrite existing Method (foreign syntax)

Again, this isn't a good idea. We run into the semantic versioning problem. See my previous comment under "The code does not compile"

We already have two features that allow "extending" (embedding) a struct inside of another one though, I really don't think we need another.

package foreign

type T struct{}

func (t T) String() {
    return "foreign"
}

package main

import "github.com/user/foreign"

type OurT struct {
    foreign.T
}

func (t OurT) String() {
     return "main"
}

func main() {
    // Now whether we use foreign.T or OurT is very well defined and visually easy to understand

    t := foreign.T{}
    ourT := OurT{t}
}

We don't need an entire language construct to save a few lines of code (that you only have to write once) and a _single_ line of code (that you'd use every time)

[adnsv]

Perhaps I should better explain what I meant by the loss of "chainability" in the original post:

package main

import (
    "fmt"
    "strconv"
)

type OrigT struct{}

func (ot *OrigT) Print(s string) *OrigT {
    fmt.Printf(s)
    return ot
}

// if OrigT comes from another package, we can't customize
// it directly, need to embed...
type CustomizedT struct { OrigT }

func (ct *CustomizedT) PrintInt(i int) *CustomizedT {
    ct.OrigT.Print(strconv.Itoa(i))
    return ct
}

func main() {
    ot := OrigT{}
    ot.Print("Hello").Print(" World!\n") // works

    ct := CustomizedT{}
    ct.PrintInt(1).PrintInt(2) // works
    ct.Print("A").Print("B")   // also (deceptively) works
    ct.Print("A").PrintInt(2)  // does not work because the first print still returns *OrigT
}

The only workaround I am aware of involves rewriting all the original methods binding them to (*CustomizedT) and returning *CustomizedT instead of *OrigT:

func (ct *CustomizedT) Print(s string) *CustomizedT {
    ct.OrigT.Print(s)
    return ct
}

This is an unnecessary boiler plate code and does not contribute to overall code elegance.

[deanveloper]

That issue could be solved by wrapping the resulting ct.Print("A") in CustomizedT{}. This may not be elegant, but it does make sure that the type system stays rigid.

This pattern also pops up when using types with underlying number types, like time.Duration:

CTF:

x := 5

fiveS := 5 * time.Second // Good!
fiveS = x * time.Second // BAD: x (int) is not a time.Duration!

// Workaround:
fiveS = time.Duration(x) * time.Second

Also, how would reflect handle extension methods?

Also, the chaining pattern isn't super common anyway. It pops up in a few places like math/big, but because of things like multiple-return (which makes a method call unchainable) and semicolon insertion (which prevents multi-line chains from looking nice), chaining in Go is a pretty uncommon practice.

Not to mention that embedding only breaks the cf.Print().PrintInt() case and neither of the others...

It really seems like the only justification to add this language feature (which might vastly increase confusion AND that we (basically) already have with embedding) is an extremely niche case

[ianlancetaylor]

Thanks for the suggestion, but we aren't going to do this. This is not the sort of language that Go is.