Go: proposal: Go 2: let := support any l-value that = supports

[bradfitz]

(Pulling this specifically out of #377, the general := bug)

This proposal is about permitting a struct field (and other such l-values) on the left side of :=, as long as there's a new variable being created (the usual := rule).

That is, permit the t.i here:

func foo() {
    var t struct { i int }
    t.i, x := 1, 2
    ...
}

This should be backwards compatible with Go 1.

Edit: clarification: any l-value that = supports, not just struct fields.

/cc @griesemer @ianlancetaylor

[robpike]

I think instead we should aim to eliminate redeclaration, which becomes much less compelling if we can get to a smoother error handling model. Won't happen soon though.

Remove features rather than add them.

[ianlancetaylor]

I think that eliminating redeclaration is a good path, but I'm not sure it affects this proposal. This basically says that you can write

    // Declare err, assign to s.f and err.
    s.f, err := F()

[josharian]

Why only struct fields? slice[i], array[i], map[k]?

Remove features rather than add them.

Removing restrictions can be a net increase in simplicity, if they result in a more uniform application of rules. This proposal would reduce some differences between what can be on the LHS/RHS of = and :=. (Although it is not obvious to me whether describing that set of differences gets easier or harder.)

[bradfitz]

Yeah, sorry, I oversimplified. Any L-value that works with =.

[go101]

And *pointer.

If redeclaration is removed, then a if block would become:

    if var x, y = f(); x == y {
        ...
    }

good? Personally, I can accept it.

[griesemer]

@go101 This is not about removing ":=" (which we definitively want to keep), but about the ability to redeclare a previously declared variable (which one can only do using ":=").

[networkimprov]

@robpike, how would you mitigate the millions (?) of lines of code that would be broken by eliminating assignment redeclaration?

[bradfitz]

@networkimprov, slightly off topic, but when we do remove language features, the plan is outlined in https://github.com/golang/proposal/blob/master/design/28221-go2-transitions.md ... language features can be removed (for a user declaring a certain language version), but we can't change language semantics and silently alter programs.

But this isn't a bug about removing language features. This is strictly about increasing the number of programs that are accepted and making := behave more like =.

[beoran]

I think :=is already too powerful and complex as it stands. While now, due for error handling I also use it in the form newvar, err := somefunc(), it's not very clean that only some of the variables are newly defined while others merely get assigned . If the proposals for better error handling make that form unneccessary, then we can upgrade the millions of lines of existing code like that with go fix.

In my mind 'x :=' should be semantic sugar for 'var x ='. Allowing more complex expressions on the LHS only adds to the confusion :=can create, which is why I respectfully ask that this proposal be rejected.

[pam4]

Possible duplicate of #6842 (but #6842 only talks about fields).

I don't see this as a feature, but rather something about allowing a more general behavior.
In the code: var x int; x, y := f(), in Go terminology we say that we are redeclaring x, but programmers are more likely to think in terms of reusing the previously declared x.

The := behaves like a var for y, because var y would be allowed there, but it behaves like regular = for x, because var x would not be allowed there.
The second case could be made to include other kinds of l-values for the same principle, without adding complexity.

But if this proposal is accepted, we really don't need redeclarations any more, because we can just turn the variables we want to reuse into expressions by enclosing them in parentheses.
This would not just spare us a few long declarations. It would finally make it clear, in a multivariable :=, which variables are new and which are not (imho effectively solving #377).

[bradfitz]

Oh, yes, this is #6842. But #6842 was closed, folded into #377. Amusingly, I was asked to create this bug because #377 was too crowded and hard to discuss.

[josharian]

One interesting comment from #6842 worth duplicating here, by @ascheglov:

A slightly different case is when it happens in the if statement: if x.f, ok := f(); !ok { You usually want that ok variable visible only inside that if statement, and you don't want to declare it in outer scope.

There is indeed a bit of a scope problem there.

[bradfitz]

@josharian, this proposal doesn't change what would happen to ok in that code.

What's the scope problem? That it permits assigning to x.f where x is not in a private scope specific to that if body?

[josharian]

Yeah. That’s a significant behavior change from before.

(The alternative is to have the assignment to x.f be temporary to that scope, and reverted afterwards, which would just be weird. But would also be analogous in some ways what would happen with a shadowed variable: https://play.golang.org/p/BtdQSLQGh-e.)

[bradfitz]

I guess I don't see that as an important or significant behavior change. That's behaving exactly as this bug is about.

[pam4]

Yeah. That’s a significant behavior change from before.

@josharian, you are probably making some assumption about programmer expectations that I can't see (also I don't see any compatibility issue).

Obviously in the code: *f(x), y := g() I would not expect anything strange to happen to f or x; it wouldn't make any sense.

The principle for := would be: if you cannot make a var of an element, then such element is just assigned to.
If you think of redeclared variables as if they were "just assigned to" (the effect is the same), this principle already describes current := behavior.

var x int
x, y := f() // "var x" not allowed here -> just assign to x

[josharian]

Yes, I was thinking about programmer expectations. I find the if x.f, ok := f(); !ok { example surprising, but perhaps others don't. That's fine.

[networkimprov]

To a dev used to javascript, these would seem to add an element to a container (EDIT: and declare a variable). I think @beoran has a point.

t.m,  x := f()
s[i], x := f()  // slice
m[k], x := f()  // map; really can add an element :-)

[bradfitz]

@networkimprov, I don't follow.

[DeedleFake]

I think he meant that there's a difference between a struct field, a slice index, and a map index on the left-hand side of a :=. A struct field and a slice index both require the field and index to actually exist, although the struct field is checked at compile-time and the slice index is checked at runtime. A map index, on the other hand, adds something to the map if you assign to an index that doesn't already exist. Conceptually, there is a difference here, and since := is a combo declaration and assignment I think he's saying that he thinks it's confusing that only one of those actually creates something new rather than reassigning an existing item.

I don't think it makes a whole lot of difference, however. The three cases would, I assume, work exactly the same way with := if they were allowed alongside a new variable declaration as they already do with =.

The JavaScript reference is probably in reference to the fact that, in JavaScript, objects are actually maps with a few extra features and you can assign to an array element that doesn't exist yet, which automatically fills in the rest of the array. If you tell someone who's used to JavaScript that := is a declaration and then also allow those assignments on the left-side, as proposed here, they _may_ find it confusing that it doesn't work the same as in JavaScript. I highly doubt it would be much of an issue, however. They also have to learn how a type system works, along with anything else that's different, so...

[networkimprov]

@bradfitz, mixing assignment and declaration can be confusing. The assignments in the stmts I listed could appear to be declarations affecting a container.

It's allowed for x, err := f() because that's a pure declaration in some cases, and because there isn't a scheme to dispatch errors to handlers in Go1.

[bradfitz]

With this proposal, it's obvious from visual inspection that the proposed new LHS forms are not declarations if they have any punctuation at all.

[baryluk]

So, I kind of stumbled on this entire non-name x.y on left side of :=, when I tried:

reply.Data, found := someMap[request.Key]
if !found { return notFoundError }

In this particular case I do not care what happens when the key is not found, and I am fine with reply.Data being overwritten or be an empty of some sorts (in my case Data is of a struct type). However, some people might have an expectation that if the value was not found, it is not being assigned to reply.Data:

a[k], err := f(k)

It is very common for f(k) to return no or empty data and error, on error, and in many cases user do not want assignment to a[k] to be done on error. However, adding rules for this will make semantic riddled with possible special cases, which is not a good thing.

As much as I would like to have such functionality, error handling first should be tackled in the first place, and then this topic covered later, otherwise it might not align with general handling techniques.

PS. I do not even know what exactly is going to happen here right now, and I simply avoid this construct, because I do not care to remember all spec details all the time:

var found bool
a[k], found = b[k]

[networkimprov]

There are good arguments for _eliminating_ assignment from := statements (so they're purely declarative).

The Go 2 Draft Design for Error Handling suggests that assignment by := be dropped. Its model hides error return values -- the primary reason that assignment by := was adopted.

Go takes _one of __eight__ paths_ for a, b := f(), as each var is defined, assigned, or shadowed. One cannot tell which without searching prior code, so it's easy to write something incorrect. Dropping assignment cuts the number of paths to four.

[pam4]

@networkimprov, I assume that by "assignment" you mean "redeclaration" (I also think of it as an assignment, but in Go terminology it's a redeclaration).

I think dropping redeclarations would only make sense if this proposal is also accepted. Otherwise why have := at all?
Redeclarations are the defining characteristic of :=. It's not type inference; many users forget that var has type inference too.
Without redeclarations, := would be no more than a shorthand to save exactly 3 characters
(var a, b = f() -> a, b := f()).
Conciseness is good, but I don't think these 3 characters really make a difference.
Dropping redeclarations would be just as backward incompatible as dropping := altogether, but it would make for a more painful transition (old style code harder to spot and more confusion).
I think @ianlancetaylor motivation for closing #29081 is pretty weak if dropping redeclarations is ok (EDIT: wrong, I forgot about for/if/switch init statements).

However, if we trade redeclarations for something better (i.e. this proposal), we can make := useful again, taking full advantage of type inference without giving up explicitness.

a, b := f()
a, c := f()   // comp. error: "a redeclared in this block"
(a), d := f() // assign to a; new d
{
    (a), d := f() // assign to a (from the outer block); new d (shadowing)
}

There's nothing special about parenthesized identifiers, they are normal expressions. As @bradfitz put it: "it's obvious from visual inspection that the proposed new LHS forms are not declarations if they have any punctuation at all".

I also don't see any interference with the error handling draft.
I still like the colon-prefix syntax better (backward compatible), but I would settle for this one and be happy with it.

[networkimprov]

Related: #31064 - cmd/vet: require explicit variable shadowing

[baryluk]

There are good arguments for eliminating assignment from := statements (so they're purely declarative).

I can agree with that. I wouldn't be against removing := from the language, if it helps resolve some ambiguities.

Without redeclarations, := would be no more than a shorthand to save exactly 3 characters
(var a, b = f() -> a, b := f()).

I didn't know that this even possible. Or maybe I did know, but didn't make connection. To a person coming from C / C++ background, it feels in var a, b = f(), only b would be assigned/initialized. Sure, after second glance it is obvious, there must happen something to a to, but it is not obvious if it will get initializer from f(), or compiler will complain about lack of one. :)

I also don't see any interference with the error handling draft.

Technically there is no interfering, but because error handling and 'err' reassigning is extremal common pattern related to this bug, the error handling is driving a design here, but that is not good idea, if the entire error handling draft thingy will remove all the need for reassignment of err.

I still like the colon-prefix syntax better, ...
I just checked it. I like it, but doesn't feel Go-like, and is a bit esoteric. I am not a fan of having symbols like colon be used often - it makes code visually noisy (Just look at Perl or some crazy C++).

[DeedleFake]

I just checked it. I like it, but doesn't feel Go-like, and is a bit esoteric. I am not a fan of having symbols like colon be used often - it makes code visually noisy (Just look at Perl or some crazy C++).

I'd actually thought of this syntax before. Interesting to see that it was proposed all the way back in 2010.

I agree that it makes it a bit noisy, but I think the general idea here isn't the specifics of using a colon but more the idea of marking the variables being created, rather than trying to get the language to guess by using a different assignment operator. The colon's just an extension of the existing colon-equals syntax. I think that the potential of fixing all of the problems with shadowing is worth the slight noisiness increase.

[pam4]

Technically there is no interfering, but because error handling and 'err' reassigning is extremal common pattern related to this bug, the error handling is driving a design here, but that is not good idea, if the entire error handling draft thingy will remove all the need for reassignment of err.

Yes, err reassigning is an extremely common pattern, but I don't think removing such pattern is going to significantly reduce the problem's importance.

A Go function or method can return multiple values, but currently there's no way to mix declarations and non-declarations in the same LHS tuple.
As a result, if some but not all of such return values need new variables, you are stuck with 2 non-optimal solutions:
1) use temporary variables, which introduce noise and naming problems;
2) give up type inference, which is also intended to reduce noise.

Even with redeclarations, this situation is already common enough to be annoying for me, in fact I often find myself rearranging code in ways that I shouldn't, just because I don't want to give up type inference.
In a language with type inference, why should I ever need to write the type of a variable that I'm making just to store a returned value? (Unless I want a different type, of course.)

Redeclarations are a bad solution to this problem: insufficient, because the problem is still there, just less frequent, and harmful, because they come with their own set of problems.

If we introduce the new error handling design and drop redeclarations, it wouldn't be harmful anymore, but it would still be insufficient (even more so, for example the "comma ok" idiom would not be covered).

The problem can really be solved only by a mean to explicitly request assignment or declaration independently for different elements of the same LHS tuple. This proposal is one, the colon-prefix syntax is another, and there are many variations.

Everyone has its own (partly subjective) opinion about how they looks visually, but they give you important and concise information about the user's intentions.
Conversely writing a type that could have been inferred, really is just noise because it doesn't tell anything useful.

[deanveloper]

I'm not sure if I like the idea of wrapping parentheses around assignments, the main reason being that it would be hard to search for if someone wanted to figure out what it meant. For instance if someone saw:

x, ok := m["foo"]

// ...

y, (ok) := m["bar"]

How would they figure out what (ok) means? I had a similar problem with JavaScript the other day, when I saw code similar to the following:

var foo = 5

// ...

return { foo, bar }

I was very confused as to what { foo, bar } meant. Obviously it's a JavaScript object, but in my eyes it was shorthand for { "foo": undefined, "bar": undefined } or something similar. In reality, it is shorthand for { "foo": foo, "bar": bar }.

I wasn't able to do a search for it either. It's a niche feature so most searches for different ways to declare javascript objects don't go over it. That's really my only issue in allowing something like y, (ok) := m["bar"].

[miladrahimi]

It's important to have a solid definition for each element in the syntax. The definition of := is clear and some hacks that change its behavior shouldn't come in.

[SaleProperly]

Personally, I love this idea and think it makes the code more readable.

var err
s.x, err = f()

now becomes

s.x, err := f()

I think this behavior is predictable, readable, and expected. Given that we can already redeclare/reassign in other more limited circumstances.

It would be clear to the reader that something on the left is being declared as we are using :=

It would also be clear that s.x (or any other more complicated expression) is being reassigned.

IMO, this allows more code to be written using an expected go idiom.

[bogdanpetrea]

This problem has a nice solution somebody proposed in 2010, in the previously mentioned issue: https://github.com/golang/go/issues/377#issuecomment-66049402
I also had the same idea recently, so I guess it wouldn't be unintuitive. It's basically:

func foo() {
    var t struct { i int }
    t.i, :x = 1, 2
    ...
}

[go101]

Is it good to retire the := token? like

var t.i, x = 1, 2
var y, t.i = 1, 2

var a, b = f()   // new a and new b
var (a), c = f() // assign to a; new c
var *&a, d = f() // assign to a; new d

[ianlancetaylor]

@go101 Let's please discuss that elsewhere. This is a specific proposal for changing one aspect of how := works. Thanks.

[go101]

@ianlancetaylor
Do you mean it should be discussed in https://github.com/golang/go/issues/377?

If the answer is yes. I think the current issue is more general than https://github.com/golang/go/issues/377.
If we can denote a re-declared identifier in a non-pure-identifier form (such as (id) and *&id),
then the problem described in https://github.com/golang/go/issues/377 can be converted to the one described here.

[ianlancetaylor]

This issue is a very specific and narrow proposal: permit general l-values on the left hand side of :=, just as they are permitted on the left hand side of =, while still requiring the := statement to declare a new variable.

Let's please keep this issue to discussing only that and nothing else. Thanks.

In particular, this issue is not about giving special meaning to expressions like (id) and *&id when used on the left hand side of :=.

Issue #377 is for general discussion about changes to :=.

[scorsi]

There's just one thing that I dislike, here the example:

type A struct {
  x int
}

func myErrorFunc() (int, error) {
  return 0, fmt.Errorf("oups")
}

func doSomething(a *A) error {
  a.x, err := myErrorFunc()
  if err != nil {
    return err
  }
  return nil
}

func main() {
  a := A{x: 42}
  err := doSomething(&a)
  if err != nil {
    // do something with a.x but... it was changed and we lost our data...
  }
}

To fix this issue, we could do:

if tempX, err := myErrorFunc(); err != nil {
  return err
} else {
  a.x = tempX
}

We'll finally fallback to the old syntax or create temporary structs/slices/maps to avoid altering data in case of errors...

[icholy]

@scorsi If you assign a value to a variable, you will lose the old value. I don't see how that's specific to this proposal.

x := 42
var err error
if x, err = strconv.Atoi("bad"); err != nil {
    // do something with x but... it was changed and we lost our data...
}

[mdempsky]

This seems like a popular issue. Do folks have examples of real world code that this would benefit?

I frequently need to use = instead of := because the latter wouldn't scope variables how I intended. But I can't think of any times I've had to do it because := doesn't support assigning to struct fields / array elements / etc.

My experience is most multi-valued functions include a return value that provides information about the other returned values; e.g., an error or ok bool. Further, that when calling these functions, I want to check this value before storing the other values somewhere.

I'm curious to hear about others' experiences here, and any common use cases I'm not aware of.

--

From an implementation point of view, this seems easy enough.

I haven't seen any mention of what should happen if a redeclared variable appears in an LHS expression. For example:

p := new(int)
{
    p, *p := new(int), *p + 1
    println(*p)
}
println(*p)

Does this program compile? If so, what does it print?

My assumption would be LHS expressions should be evaluated in the same scope as RHS expressions (i.e., based on identifier bindings before they're re-bound). That is, it would be equivalent to:

p0 := new(int)
{
    var p1 *int
    p1, *p0 = new(int), *p0 + 1
    println(*p1)
}
println(*p0)

and thus print 0 1.

[icholy]

My experience is most multi-valued functions include a return value that provides information about the other returned values; e.g., an error or ok bool. Further, that when calling these functions, I want to check this value before storing the other values somewhere.

@mdempsky It comes up often when parsing something into a struct. If an error is encountered, the whole thing is thrown away anyway.

func readThing(r *bufio.Reader) (*Thing, err) {
    var t Thing
    if t.A, err := readA(r); err != nil {
        return nil, err
    }
    if t.B, err := readB(r); err != nil {
        return nil, err
    }
    return &t, nil
}

[pam4]

@mdempsky

I frequently need to use = instead of := because the latter wouldn't scope variables how I intended.

IMHO, that's the most important problem this proposal would solve (more details above; if I misunderstood what you mean, perhaps you could provide an example).

file := os.Stdin
if arg != "-" {
    (file), err := os.Open(arg)
    if err != nil {
        //...
    }
    defer file.Close()
}

It is also a better alternative to the infamous "redeclaration" mechanism of :=, and would pave the way for its future removal:

a, ok := f()
if !ok {
    //...
}
b, (ok) := g()

(Again, there is nothing special about parenthesized identifiers, they already work on the LHS of an =. The use of parenthesized identifiers in this fashion is no exception to the basic rule of this proposal: naked identifiers are declared, everything else is assigned to.)

But I can't think of any times I've had to do it because := doesn't support assigning to struct fields / array elements / etc.

It happens to me from time to time. @icholy use case is a good example. Sometimes assigning to a field/element before checking the error is not a problem. And there are multi-valued functions that always succeed.

Does this program compile? If so, what does it print?

I don't see a reason for compiling to fail, and I agree with your assumption. It seems consistent with existing evaluation rules.

[icholy]

@pam4 does that syntax introduce any parsing ambiguities?

[mdempsky]

Thanks for the example use cases.

@pam4:

IMHO, that's the _most_ important problem this proposal would solve (more details above; if I misunderstood what you mean, perhaps you could provide an example).

No, sorry, I just missed that detail. So to restate this:

This proposal is about extending the LHS of := statements to include any assignable expression. Bare identifiers ("no punctuation" as Brad put it) would continue to have their current behavior, but other expressions would work as in normal assignment statements.

Assignment statements already allow LHS expressions to be parenthesized; e.g., (a) = 1 is valid. So (a), b := f() would therefore also be valid under this proposal. Here a would never be (re)declared (because it involves punctuation), but b would always be declared. (Or there would be a compiler error if b already exists in scope, since := must declare at least one variable).

I think that makes sense. I also agree it would apply to the use case I mentioned. So instead of writing something like:

var f *os.File
if foo {
    var err error
    if f, err = os.Open("foo"); err != nil { ... }
}

I could write:

var f *os.File
if foo {
    if (f), err := os.Open('foo"); err != nil { ... }
}

That looks pretty funny to me at the moment, but maybe it would grow on me. (I've always preferred the x := x trick for working around the scoping issue with go statements inside of loops, even though most other folks seem to prefer go func(x) { ... }(x).)

--

@icholy:

does that syntax introduce any parsing ambiguities?

I'd have to try actually changing one of the parsers to be sure, but I don't immediately see any reason this proposal would cause problems.

[gopherbot]

Change https://golang.org/cl/250037 mentions this issue: cmd/compile, go/parser: relax ":=" statements

[mdempsky]

Yeah, no parser ambiguity from this proposal. The Go parsers already handle = and := identically, and only complain about non-identifiers on the LHS of := later on.

I uploaded a super rough proof-of-concept CL of the current proposal in case anyone wants to play with the idea and maybe demonstrate more real world use cases. It has cmd/compile and go/parser support, so "go fmt" and "go build" should work. It doesn't have go/types support yet, so "go vet" (and thus "go test") will fail.