Go: proposal: Go 2: add new type of variadic functions with limited number of parameters and default values

[kpym]

What version of Go are you using (go version)?

$ go version
go version go1.15 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Any

What did you expect to see?

I try to add a parameter to existing function without breaking the backward compatibility. Like we can add a variadic parameter. For example I would like to be able to declare

func (re *Regexp) ReplaceAllString(src, repl string, n int = -1) string {
  ...
}

What did you see instead?

Actually if for example I want to solve #40198 by adding an additional integer parameter to the existing ReplaceAllString function that limit the number of replacements, and to keep it backward compatible, I can do this by using a variadic function like this

func (re *Regexp) ReplaceAllString(src, repl string, an ...int) string {
  var n int = -1 // the optional parameter
  if len(an) > 0 {
    n = an[0]
  }
  ...
}

but this is ugly! (because using more than one optional parameter is possible but useless and meaningless in this case)

So my proposition is to consider new type of variadic functions where the last parameters are optional with default values like for example:

func test(a, b string, c, d int = 1, 2)  

This type of functions works like standard variadic function but with two additions :

• the number of optional parameters is limited (in this example 2),
• there is a default value for the missing parameters.

In this "limited" version all optional parameters are of the same type (like in the classical variadic functions), but may be we can consider even more general situation where the optional parameters can have different types.

[ianlancetaylor]

Go has so far rejected default values for function parameters, as they are a form of function overloading. Go's attitude is that rather than overload functions, use different function names.

Note that adding a parameter of any sort to regexp.ReplaceAllString, even one with a default value, would not be permitted by the Go 1 compatibility guarantee, as it would break a program with code like

var F func(*regexp.Regexp, string, string) string = (*regexp.Regexp).ReplaceAllString

[kpym]

@ianlancetaylor I have to main arguments :
1) I can already simulate function overloading by using variadic functions (as shown in my question). So not heaving default values do not prevent from this.
2) Use different function names is good practice, but in some situations it can became obstacle. For example in #40198 the main argument against introducing the n parameter was that this will add another six methods to the library.

The argument of "different parameters => different names" was used in Go 1 to not introduce generic types, but finally this was reconsidered. Similarly, the default values of the variadic functions can be reconsidered perhaps in Go 2. ;)

[yazver]

Variadic functions requires optional types and this syntax:

func (v *Value) Convert(to string?, from string?) {
    if from, ok := to?; ok {
        v.Value = from // *from
    }
    ...
}

v.Convert(from: "some value")

[ianlancetaylor]

Based on the discussion above, and the emoji voting, this is a likely decline. Leaving open for four weeks for final comments.

[kpym]

Sorry to insist.

I have heard @ianlancetaylor's argument that this can't be used for backward compatibility because introducing an optional parameter change the function signature, and I agree with that (even if in 99% we do not assign a function to variable before to use it).

But, I haven't seen for the moment a valuable argument against this syntax change. For the moment I maintain my position that this syntax will make the code more secure and more readable. Using variadic functions to introduce optional parameters is possible, and is already used in projects, but is ugly and non secure in some sens, because allowing many parameters when we need only one or two is not a good practice.

Here are two examples of the same code.

Working example with variadic functions (the optional parameter is n int = -1):

func FunctionWithOptioanlN(s string, an ...int) {
  var n int = -1 // the optional parameter
  // check the number of parameters
  if len(an) > 1 {
    // throw an error 
    ...
  }
  // check if the optional parameter is present
  if len(an) > 0 {
    n = an[0]
  }
  ...
}

and the same with the proposed syntax:

func FunctionWithOptioanlN(s string, n int = -1) {
  ...
}

In both cases we can call FunctionWithOptioanlN("test") and FunctionWithOptioanlN("test", 1). But if we call FunctionWithOptioanlN("test", 1, 2) in the first code the argument 2 will generate an error at execution time, but in the case of the proposed syntax this should throw a compilation error : too many arguments.

I think that we do not have to discuss about the readability of the two codes.

So how the introduction of this syntax makes the go code worst compared to what we can already do with the variadic functions ?

[ianlancetaylor]

In my opinion, you are arguing for the ability to do something that Go explicitly discourages. Go intentionally does not support overloading: https://golang.org/doc/faq#overloading. You are saying that it is possible to simulate overloading uses variadic functions, which is true. But we really do need variadic functions, and it would be pointless to add restrictions from using them to simulate overloading. In Go, simulating overloading with variadic functions is bad style; instead of overloading, use two different function names. So the fact that variadic functions can simulate overloading is not in itself an argument for adding a different way to do overloading.

[kpym]

@ianlancetaylor if I understand well there are two main arguments against function overloading:
1) Method dispatch is simplified if it doesn't need to do type matching as well. Matching only by name and requiring consistency in the types was a major simplifying decision in Go's type system.
2) Experience with other languages told us that having a variety of methods with the same name but different signatures was occasionally useful but that it could also be confusing and fragile in practice.

The argument 1) looks to me as the main argument (because is 1, and because is objective). But this argument is not contradicted by my proposal, one more time, because it is already available in the variadic functions. I do not ask for real overloading where we have to check the arguments types at runtime and decide which function to call. What I propose can be seen as syntactic sugar over variadic functions, but actually is even simpler than variadic functions because everything is decided at compile time. If we call (in my previous example) FunctionWithOptioanlN("test") the compiler simply replace this call by FunctionWithOptioanlN("test", -1) instead of raising "not enough arguments" error.

About the argument 2):

• The real function overloading, when we have to decide at runtime which function to call depending on the arguments types, could be really confusing and fragile, that's true. But omitting the last arguments is not the same thing: that types are always the same, the signature is the same and everything can be checked at compile time.
• Go is designed to have default values for all types. When we create a structure we can initialize only some of the fields and let the other with their default values. Why should we keep the functions out of this "by defaults" strategy ?
• I'm not sure that the Go designers would be opposed to a variable number of arguments if they had seen from the beginning how unreadable some libraries can become because of the huge number of almost identical functions.
• All this point to the fact that argument 2 is very subjective. And being such I can understand that not everybody agree with me ;)

In conclusion, I'm not asking to introduce real function overloading. I think, that what I propose:

• Has very small compilation coast.
• Preserves Go out of complicated function dispatching at runtime.
• Keeps the function signature strictly related to the function name.
• Allows to write very readable code.
• Same as generic types, helps to keep the number of functions in the libraries as small as possible.
• Is compatible with "by default values" strategy in Go.

[ianlancetaylor]

Yes. It's subjective. Go style is not use variadic arguments to simulate optional arguments. Go style is to not have optional arguments. Instead of having optional arguments, write two different functions with different names. Again, yes, this is subjective. But we aren't going to change to language solely to support a different coding style. As has often been said, Go is an opinionated language.

[kpym]

@ianlancetaylor it is a little bit too late to say "you should not use variadic functions for optional parameters", IMO. They are everywhere, because the programmers need them (this is why they exist in so many languages), and because go allows this (by a "bad" use of variadic functions).

I have quickly checked some of the most popular go projects on GitHub and I found this.

From github.com/gogf/gf:

func Create(safe ...bool) RWMutex {
  mu := RWMutex{}
  if len(safe) > 0 && safe[0] {
    mu.RWMutex = new(sync.RWMutex)
  }
  return mu
}

From github.com/yuin/goldmark:

func (m *markdown) Convert(source []byte, writer io.Writer, opts ...parser.ParseOption) error

You can pretend that these do not exist, or you can try to make this better, it's up to you ;)

[griesemer]

@kpym You have mentioned earlier that what you are proposing "can be seen as syntactic sugar", so it's not really true that "programmers need them". Programmers may want them, but people want lots of things. Default values for arguments in particular can make code harder to read because at a call site those default values are not visible. There are certainly situations where it can make sense, but the basic philosophy of Go has always been to leave away what does not carry its weight sufficiently. (And based on this argument, we probably should remove features from the language, not add more.)

With respect to libraries that have become unwieldy or unreadable because of the large number of functions: This is largely a matter of a library's design and naming conventions. Note that the real complexity lies in the varies combinations of arguments possible, and that doesn't go away with default arguments. If anything, having lots of functions exposes this complexity. Using default arguments to "hide" the complexity is certainly questionable.

Function invocation is a space where plenty of "innovation" or "improvement" is possible, but it doesn't change the fact that at the very bottom we're just calling a function with arguments. Better to leave it at exactly that.

Finally, I note that this proposal has not received much support. I am ok with closing this.

[ianlancetaylor]

There was further discussion, but no change in consensus.

[kpym]

I understand. Thank you for taking the time to consider my proposal.