Dotty: New implicit parameter & argument syntax

[odersky]

Motivation

The current syntax for implicit parameters has several shortcomings.

1. There can be only one implicit parameter section and it has to come at the end. Therefore
   normal and implicit parameter types cannot depend on other implicit parameters except
   by nesting in an inner object with an apply method.
2. The syntax (implicit x: T, y: U) is a bit strange in that implicit conceptually scopes over
   x and y but looks like a modifier for just x.
3. Passing explicit arguments to implicit parameters is written like normal application. This
   clashes with elision of apply methods. For instance, if you have

def f(implicit x: C): A => B

then `f(a)` would pass the argument `a` to the implicit parameter and one has to write 
`f.apply(a)` to apply `f` to a regular argument.

Proposal

• Introduce a new symbolic delimiter, ?(. This is one token, no spaces allowed between the ? and the (.

• Write implicit parameter definitions with ?( instead of (implicit. E.g.

  def f(x: Int)?(ctx: Context) = ...
  

  instead of

  def f(x: Int)(implicit ctx: Context) = ...
  

• Explicit arguments to implicit parameters have to be enclosed in ?(...). E.g.

  f(3)?(ctx)
  

• There can be several implicit parameter sections and they can be mixed with normal parameter sections. E.g.

  def f ?(ctx: Context)(tree: ctx.Expr) = ...
  

  Problems

• ? can be already used as an infix operator. So the meaning of (a)?(b) would change but
  only if no space is written after the ?, which should be rare.

• ? can be part of a symbolic operator. In this case the longest match rule applies. So
  def #?(x: T) defines an operator #? with a regular parameter list. To define # with
  an implicit parameter list an additional space is needed: def # ?(x: T).

[smarter]

Explicit arguments to implicit parameters have to be enclosed in ?(...)

I like this because it would prevent sneaky and hard to diagnose bugs like https://github.com/lampepfl/dotty/commit/5225f00ab3300bac62a467ecc532c275e02e7f43

But I worry that newcomers will think that ??? and ? are related, even though one is very unsafe and the other completely safe, I don't have a better suggestion except maybe renaming ??? to !!! :).

[retronym]

This would still lack the expressive power to skip an implicit parameter section that is immediately followed by another implicit param section.

The workaround would be the same for as the way we selectively provide implicit params within a single section, by using implicitly.

def foo ?(a: Int) ?(b: Int) = 0
foo ?(1)?(2) // explicitly providing a and b
foo ?(implicitly)?(2) // explicitly providing b

[odersky]

@retronym Good point. I think the workaround is fine.

[Blaisorblade]

Hi! I like that you address this problem :-)

Should you want to bikeshed syntax, I'd consider using characters that are reserved — my proposal happened to use [[...]] (inspired by [...] for type arguments, which also create optional parameter lists) and should avoid the above problems with ?.
http://blaisorbladeprog.blogspot.de/2013/01/flexible-implicits-from-agda-for-scala.html

This would still lack the expressive power to skip an implicit parameter section that is immediately followed by another implicit param section.

The workaround would be the same for as the way we selectively provide implicit params within a single section, by using implicitly.

Slight issue with this workaround: you might need many implicitly to skip many arguments, and adding implicit parameters would break more source code.

What about using allowing to use named parameters to skip implicit sections? That's the solution in Agda, and it easily scales to longer lists of arguments.
While having 10 implicit arguments might (should) be less common in Scala than Agda, a more SCALAble language doesn't hurt, and I trust shapeless to run into arbitrary limits to scaling.

def foo ?(a: Int) ?(b: Int)?(c: Int)?(d: Int) = 0
foo?(d = 2) //explicitly providing `d`

[odersky]

@Blaisorblade The named argument trick looks neat. I'd like to check how hard it would be to add it.

[ctongfei]

So it would be "xs sorted ?(order)"? A little bit ugly I think.

I prefer using an additional keyword for introducing implicit parameters such as by or under but probably that'll not be backward-compatible.

Declaration:
def sorted under (T: Order[T]) = ???
def map[B](f: A => B) under (sc: SparkContext) = ???
Usage:
xs sorted under order
xs map f under sparkContext

[ctongfei]

This ? syntax also cannot express the following currently expressed by implicit:
{ implicit r => ... }

[propensive]

As an alternative, how about reusing the period (.) instead of ?? There would be no risk of it meaning anything else.

implicit val str: String = "x = "
def foo(x: Int).(y: String) = y+x
foo(0) // "x = 0"
foo(1).("Value: ") // "Value: 1"

Also, to which implicit block would context bounds be appended? Could we say, an entirely new block appended immediately after the type parameters, or would that be more confusing than a new block at the end?

Perhaps, given this new lightweight syntax, the type parameter block is the wrong place to be defining context bounds. Much as I like the context-bound syntax, assuming an anonymous parameter syntax, there's not _that_ much difference between,

def foo[T: Typeclass](t: T) = ...

and,

def foo[T](t: T).(_: Typeclass[T]) = ...

versus the original,

def foo[T](t: T)(implicit ev: Typeclass[T]) = ...

and I think the argument could quite easily be made that the additional clarity at the definition site (especially given potential confusion about which implicit block contains the context bounds) would be a net benefit.

Generally, I'm very supportive of the original proposal, though. :+1:

[mdedetrich]

Like the proposal, I disagree with the ? syntax though. ? implies optional, or maybe unknown, it just seems kinda random in the context of implicits.

In fact, I don't really see the problem with having stuff like

def f(x: Int) implicit (ctx: Context) = ...

and

f(3) implicit(ctx) 

or maybe

f(3) implicitly(ctx)

Note that I have just replaced ? with implicit/implicitly and just added whitespace to make it look cleaner.

p.s. I am kinda a fan of the implicit keyword because its easier to spot with your eye and its not confused with anything else. If we really want a symbol, than I think something like % would be better, since I don't think its used anywhere else and its also visually easy to spot. Not really a fan of @propensive idea of using a period (.), since its really hard to spot and easy to visually confuse with method invocation.

# could also work, I mean its already used for type projections but in context of implicit its somewhere else (you could also argue that they are somewhat related)

[smarter]

? implies optional, or maybe unknown, it just seems kinda random in the context of implicits.

Well, implicits are sort-of-optional arguments, the ? is also used in Haskell for implicit parameters: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#implicit-parameters

[mdedetrich]

Well, implicits are sort-of-optional arguments, the ? is also used in Haskell for implicit parameters: https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/glasgow_exts.html#implicit-parameters

Most other mainstream languages tend to use ? to refer to an optional value, or as a safe way to deal with null. I can sought of see the connection in logic, however I would prefer to prefix the ? infront of the type, rather than using it inplace of the keyword, i.e. (? a: A, ? b: B) instead of ?(a: A,b: B).

I guess my biggest grip is more how it visually/syntactically looks

[DarkDimius]

I like the proposal, but I'm not sure about the syntax, as ? is also a potential shortcut for OR type with null.
I don't like the look of signature:

def f[A, B]?(a: ?A, b: ?B)

[sirinath]

Add an optional :- separator for implicit. E.g. (implicit:- x: T, y: U). :- means applies to all. If it is optional then omitting it will give the old syntax. Similarly you can have (x, y -: U) if you want.

Another option is to make implicitness part of the typesystem when you add effects. E.g. (x: T @ {implicit}, y: U @{implicit}) or (x: T, y: U -: @{implicit}). Assuming @{} how you specify effects and tag types (as called in Nim Language) / type extensions / annotated type (as called in X10 language).

[sirinath]

Also implicits can simply become annotations than a baked in language feature.

[nicolasstucki]

I assume that this change in syntax should also be reflected on lambdas with implicits. @odersky do you have a proposed syntax for those cases? In particular consider a method that has a parameter that is a lambda which receives one implicit parameter. Would that clash with the Or type withnull mentioned by @DarkDimius?

[sirinath]

Another way might be (x: T = _, y: U = _) where _ means implicitly resolved. This way ? can be reserved for or with Null. E.g. (x: T? = _, y: U? = _) and also { x: X = _ => .... }

[stuhood]

How about allowing use of a keyword in the default syntax?

def meth(arg1: String, arg2: StringContext = implicit): String

On Aug 19, 2016 1:51 AM, "Suminda Dharmasena" [email protected]
wrote:

Another way might be (x: T = _, y: U = _) where _ means implicitly
resolved. This way ? can be reserved for or with Null. E.g. (x: T? = _,
y: U? = _)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lampepfl/dotty/issues/1260#issuecomment-240963978,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAC2lFAbyaTJyh4d7ErJfAxz9kkOmEGJks5qhW6KgaJpZM4IhPF4
.

[Blaisorblade]

@stuhood Most proposals are about implicit parameter lists—but all parameters in the same list are of the same "implicitness", unlike in your example. Going in that direction seems be more problematic though I admit I lack ATM a specific example of problems with it.

[sirinath]

I think new syntax should allow to pick and choose what is implicit.

[OlivierBlanvillain]

A bit late to the party, here is my proposal for explicitly providing implicit arguments: remove this option from the language. Instead, implicit arguments could be provided explicitly using the following:

(Assuming implicit I => O is the syntax for an implicit function of arity one)

implicit class ExplicitlySyntax[I, O](f: implicit I => O) {
  def explicitly(i: I): O = {
    implicit val ii: I = a
    f.apply
  }
}

Example of usage:

def f(i: Int)(implicit j: Int)(k: Int)(implicit l: Int): Int = i + j + k + l

f(1)(3) // i = 1, k = 3, j & l implicitly resolved

f(1).explicitly(2)(3).explicitly(4) // i = 1, j = 2, k = 3, l = 4

I think this would play nicely with removing parameter blocks with multiple implicits from the language, which would also fix point 2 of the original motivations. (otherwise explicitly would have to be defined for several arities...)

[Blaisorblade]

@OlivierBlanvillain

I think this would play nicely with removing parameter blocks with multiple implicits from the language,

If you do that, how would you migrate existing code to the proposed language?

The original proposal is also confusing in that regard, but since code migration is an important concern, I assumed the old syntax would still be supported with the current semantics... I don't immediately see how in your proposal. Unless you do all those changes on the new and improved implicit syntax.

[sirinath]

Code migration is an issue but shouldn't you make dotty as an opportunity to improve and and fix problems in a breaking way. Let as long as Scalafix can fix something breaking will not be an issue.

[OlivierBlanvillain]

@Blaisorblade Turning f(implicit a: Int, b: Int) into f(implicit a: Int)(implicit b: Int) should be a straightforward syntaxtic rewrite, and multiple (trailing) implicit parameter lists looks like a simple change in scalac (see https://github.com/scala/scala/pull/5108). Expliciting implicit parameters needs semantic understanding, but shouldn't be too hard either:

f(Nil)(ctx)

becomes

{
  implicit val $c: Context = ctx
  f(Nil)
}

Which hopefully is equivalent and cross compile with both scalac/my proposal.

[sirinath]

I still feel better to represent f(implicit a: Int, b: Int) as f(a: Int = ?, b: Int = ?) or f(a: Int = _, b: Int = _) or f(a: Int = implicit, b: Int = implicit)

[odersky]

Turning f(implicit a: Int, b: Int) into f(implicit a: Int)(implicit b: Int) should be a straightforward syntaxtic rewrite

I thought we'd keep the syntax f(implicit a: Int, b: Int), so no rewrite of function definitions would be necessary.

The migration story for .explicitly is more difficult. We first have to teach the compiler to accept it, because right now it would do the wrong thing. I.e.

 f(a).explicitly(b)

would _enforce_ that an implicit argument is passed to f(a) instead of _preventing_ it. I would hope that we can do this under some "future" mode in scalac. dotty would still accept arguments passed to implicit parameters, but only under -language:Scala2. The rewrite tool would insert .explicitly whereever this is needed.

So, if we can get scalac to accept .explicitly we have a migration path.

[stuhood]

@stuhood Most proposals are about implicit parameter lists—but all parameters in the same list are of the same "implicitness", unlike in your example. Going in that direction seems be more problematic though I admit I lack ATM a specific example of problems with it.

@Blaisorblade : My feeling is that aligning implicit arguments with default arguments as @sirinath suggested (although not directly relevant to the discussion of multiple parameter lists) would be a cohesive change. In particular, the semantics of default arguments (needing to be defined at the end of a parameter list) match. The ability to pass implicit arguments by name would also be cohesive.

Additionally, in a huge number of cases, defining an implicit would not require a second parameter list. You'd only define a second parameter list if it gave the callsite syntax you desired. I suspect that a vast majority of currently multi-arg-list functions would not need to be any longer.

[sirinath]

Dotty should be take as an opportunity to make breaking changes to improve.

[propensive]

@stuhood The thing that I think would be difficult to reconcile (though maybe not impossible) is the calculations of LUBs for unconstrained types mentioned in more than one parameter in the same parameter block, when implicit resolution is involved too. In Scala 2.x, types mentioned in implicit parameter blocks get fixed parameter-by-parameter, left-to-right. It's often very useful to walk this delicate path to get desirable type inference, though that's not to say alternative ways of achieving the same thing wouldn't be possible too...

[sirinath]

I am not sure about the internals or if relates to this specific case. by Guy Steele mentions how it was done differently in this in comparison to Scala: https://www.youtube.com/watch?v=EZD3Scuv02g

Aside from this I am really hoping that Dotty borrows the operator overloading described in the above.

[propensive]

Hi @sirinath - sorry I'm not able to watch the Guy Steele video now, so hopefully I'm not answering the wrong question, here...

It's very much a subtlety of the language, but it's not "internals", as it's a useful feature to users of the language.

To look at this differently, separate parameter blocks offer two distinct things (aside from alternative syntax at the call site):

• a separate parameter block (implicit or explicit) permits references to values specified in previous parameter blocks, and
• the parameters in the same explicit parameter block are typechecked together.

I would want to make sure that any new implementation offers at least the same power.

[smarter]

the parameters in the same explicit parameter block are typechecked together.

Note that this is already pretty different in dotty, we solve constraints "as late as possible but not too late", which means in particular that we can delay solving constraints until several parameter blocks have been typechecked. See https://www.youtube.com/watch?v=YIQjfCKDR5A

[propensive]

Yes, I remember we talked about this about a year ago... ;) I really need to experiment with it to get a feel for it. I may need to learn to love the new inferencer...

[sirinath]

I am not an expert in the implementation side but please try to watch the video also. It might generate more ideas on how to improve the state of the art.

[nafg]

Passing explicit arguments to implicit parameters is written like normal application. This
clashes with elision of apply methods.

First let me say that while this is a problem, I feel like some of the solutions are worse than the problem. The whole idea of implicit parameters is that they are first and foremost ordinary parameters, except that they can be "inferred" so to speak -- similar in some ways to type inference and argument defaults. I feel like making it harder to supply implicit parameters explicitly breaks that elegance.

How about a very simple solution -- allow writing empty parentheses for implicit lists. For instance:

def m(a: A)(implicit b: B, c: C)(d: D)(implicit e: E)

m(new A)()(new D)    // supply b, c, and e from the implicit scope

This would make implicits consistent with default arguments. In fact it's almost like (e: E = implicitly[E]) except that the implicit is resolved at the call site. (Maybe if implicitly was an inlined macro this could be made actually true?)

The syntax (implicit x: T, y: U) is a bit strange in that implicit conceptually scopes over
x and y but looks like a modifier for just x.

That is annoying but really doesn't justify breaking code, making valid identifiers magically "sometimes-keyword"s, etc.

[stuhood]

@odersky : Would you mind responding to @sirinath's proposed syntax from https://github.com/lampepfl/dotty/issues/1260#issuecomment-240963978? It really feels like it would be a huge unification, would remove the need for multiple implicit parameter lists entirely, and removes all concerns around eta expansion and additional callsite syntax.

[joan38]

I'd like also to see arguments from the main people on @sirinath and @stuhood proposal:
https://github.com/lampepfl/dotty/issues/1260#issuecomment-262139822
I didn't see anyone showing cons to these proposals.

[nicolasstucki]

A simple cons is that you would not be able to support implicit arguments with default values.

[soronpo]

A simple cons is that you would not be able to support implicit arguments with default values.

Well, it's still possible to allow f(a: Int = implicit myDefaultValue, b: Int = implicit), so a is an implicit argument with th default value myDefaultValue, and b is an implicit argument without a default value.

[soronpo]

I should mention, though, that explicitly annotating every implicit argument with the word implicit is very verbose, which is a con for me. Also a block that has mixed implicit and explicit arguments will be difficult to use. It is better to come up with a way to annotate the whole block. I think @mdedetrich's https://github.com/lampepfl/dotty/issues/1260#issuecomment-223608870 is best syntax proposal yet.

[Blaisorblade]

From Gitter May 11, 2018 4:35 AM:

@Glavo: If I want to have a function with both erased and ordinary implicit parameters, what should I do?

@Blaisorblade: @Glavo wow, seems we'd need lampepfl/dotty#1260 for that

That's because also erased scopes over whole blocks, so you'd need to write

def f(implicit x1: T1)(implicit erased x2: T2) = ...

which doesn't work because that's two implicit argument lists. And I don't see a way to handle this case without solving the general issue. @nicolasstucki thoughts?

[nicolasstucki]

Currently there is no way to support this. The plan is to wait for support of multiple implicit parameters lists in the future.

[lihaoyi]

I think @sirinath's and @stuhood's ideas on unifying default params & implicits are worth considering. As I understand it, the core of the idea is this:

Implicits as Defaults

A function defined as:

def f(a: Int, implicit b: Int, implicit c: Int)

Would be called via:

f(1) // b and c provided implicitly, if in scope
f(1, b = 2) // b provided explicitly, c provided implicitly
f(1, b = 2, c = 3) // b and c provided explicitly

As you can see, this provides a neat, already-familiar syntax for anyone who wants to provide a subset of implicit params while letting the others get inferred: something that is impossible/awkward under status quo Scala. Also note the symmetry with how Scala (and every other programming language under the sun) already treats default params:

def f(a: Int, b: Int = 2, c: Int = 3)
f(1) // b and c provided by defaults
f(1, b = 2) // b provided explicitly, c provided by default
f(1, b = 2, c = 3) // b and c provided explicitly

Effectively, implicit parameters become a special case of default params: rather than the default being hardcoded, the default is implicitly resolved at the call-site, but otherwise the call-site syntax & semantics are identical.

Implicit params with defaults would also compose nicely:

def f(a: Int, implicit b: Int = -1, implicit c: Int = -1)

Where you would be able to call f(1) without an implicit Int in scope, and b and c would be assigned to default values. With an implicit Int, that value would be used for b and c. Or you could pass b and c explicitly. This is in fact exactly the current semantics in status-quo Scala!

Implicits-as-defaults unifies two currently similar-but-awkwardly-different ways of eliding, or manually providing, values for function params: implicits & default params. It does so in a way that makes implicit params more familiar to anyone coming from any other language that supports default params: Python, Javascript, C#, C++, etc..

Implicits-as-params also decouples "implicit params" with "multiple parameter lists" entirely. Nevertheless, you can still have implicit params in a separate parameter list, as is the status quo requirement:

def f(a: Int)(implicit b: Int, implicit c: Int)

But you can have it as part of an existing parameter list (as shown above) and still have it be resolve implicitly. Implicits-as-defaults neatly generalizes the status quo implicit-param-list-must-always-come-last requirement: current code using implicits-always-come-last will continue to work unchanged[1].

Use Case

Consider a subprocess function, that takes in a list of command-line args, a working directory, and spawns a subprocess:

def subprocess(args: Seq[String], cwd: String) = ???

subprocess(Seq("ls"), "/home/ubuntu")

A library author may want to make the cwd have a default value: they may think "most" commands the user will want to run in the same directory they started the process in:

def subprocess(args: Seq[String], cwd: String = new java.io.File("").toAbsoluteFile().toString) = ???

subprocess(Seq("ls"))) // default cwd 
subprocess(Seq("ls"), "/home/ubuntu") // explicitly pass in cwd

However, at a later date, they may decide that it's better if the "default" is configurable, but yet still want the user to be able to override it on a case-by-case basis. In the status quo, and under the top-level proposal above, that will require re-writing all call-sites to:

subprocess(Seq("ls"))) // implicit cwd 
subprocess(Seq("ls"))("/home/ubuntu") // passing in an implicit explicitly, status quo
subprocess(Seq("ls")).explicitly("/home/ubuntu") // passing in an implicit explcitly, top-level proposal

However, when you think about it, in both cases both the library author and user want the same thing: a default value, that the user can override. Does the fact that the default value is configurable really make such a big difference that the user should go and re-write all their callsites mechanically into a different syntax?

Using implicits-as-defaults, this would look like:

def subprocess(args: Seq[String], cwd: String) = ???

subprocess(Seq("ls"), "/home/ubuntu")
subprocess(Seq("ls"), cwd = "/home/ubuntu")

```scala
def subprocess(args: Seq[String], cwd: String = new java.io.File("").toAbsoluteFile().toString) = ???

subprocess(Seq("ls"))) // default cwd
subprocess(Seq("ls"), cwd = "/home/ubuntu") // explicitly pass in cwd

```scala
def subprocess(args: Seq[String], implicit cwd: String) = ???

subprocess(Seq("ls"))) // implicit cwd 
subprocess(Seq("ls"), cwd = "/home/ubuntu") // passing in an implicit explicitly

With implicits-as-defaults, the identical syntax in these three code samples properly reflects the identical semantics that a user wants to convey: that they sometimes want the value to be inferred, and sometimes they want to provide it explicitly. Whether the inferred value is hardcoded, computed in the default value getter, picked up from the receiver of the subprocess method or it is resolved from the enclosing scope implicitly, isn't really that important w.r.t. the intent of the code.

(Note that this is a real issue I've hit several times; ammonite-ops code is full of ugly curried function calls like %%('git, "rev-parse", "HEAD")(pwd) that have no real reason to be curried, except to make implicit-passing possible)

Compatibility

Implicits-as-default-params can be specced out to be compatible with both the status quo Scala syntax/semantics* and the @odersky's proposal's call-site semantics, with two minor tweaks:

• Parameter lists for which every parameter is implicit can be elided. i.e.

  def f(a: Int)(implicit b: Int, implicit c: Int)
  f(1) // b and c resolved implicitly
  

  This will allow backwards-compatibility[1] with the existing Scala 2.x implicit-parameter-list syntax & semantics, rather than forcing people to call it via f(1)() as would a naive reading of the above description.

• Implicit parameters cannot be passed positionally:

  def f(a: Int, implicit b: Int, implicit c: Int)
  f(1, 2, 3) // invalid
  f(1, b = 2, c = 3) // OK: implicits passed explicitly
  f(1) // OK: implicits resolved implicitly
  

  This will allow for interspersed implicit-and-non-implicit params:

  def f(a: Int, implicit b: Int, c: Int)
  f(1, 2) //a = 1, b resolved implicitly, c = 2
  f(1, b = 3, 2) //a = 1, b explicitly passed as 3, c = 2
  

  Note that keyword-only/non-positional params are themselves not a new idea, being common in Python as well as languages like Ruby or Javascript where passing arguments as dictionaries is commonplace. This should be the only tweak necessary in order to make possible all of the semantics that @odersky describes above.

Conclusion

To sum up, implicits-as-defaults would:

• Allow us to get all the semantics of @odersky's original proposal above

• Provide 100% semantic backwards compatibility with the status-quo Scala semantics and syntactic backwards compatibility with the status-quo Scala syntax, needing only a trivial[1] syntactic migration.

• It decouples implicit parameters with currying, separate parameter lists, and other confusing-to-newcomers language features that they have no business being coupled to and tend to make learning about implicits more confusing than is truly necessary

• While at the same time associating them with default parameters, a language feature that they are very similar to (both in semantics & in use case) and that we can expect newcomers to be already familiar with

• Be instantly familiar to the people already using Scala! They don't know what ?() or .explicitly are, but they know you can mark parameters as implicit and you can pass default arguments explicitly. No re-training required

• Provides a neat, already-understood syntax/semantic for providing a subset of implicit params explicitly.

• It would make implicit parameters easier to learn, not by adding special syntax & features, but by removing unnecessary coupling & arbitrary restrictions that hide the fact that implicits really aren't that different from default/named parameters, a feature everyone already knows & loves.

I'm not going to pretend that it won't be a lot of work tweaking the spec/parser/typechecker/inferencer/ASTs/etc. to make this implicits-as-defaults work, but as far as I can tell this proposal gives us the best of both worlds, with zero migration-costs, and some huge additional benefits in learnability of implicit parameters for newcomers to the language. So if we're going put in the effort to do anything, we might as well do this.

[1] You would need a trivial syntactic transform to prepend the implicit keyword to every argument in an implicit parameter list, but that's straightforward. Even a regex would probably do. Note that we cannot preserve the existing "every parameter after the implicit keyword is itself implicit" property without making it impossible to interleave implicit/non-implicit params.

Or we could throw new syntax at it to make the new implicit-per-argument semantic opt-in, making this proposal truly zero-migration, and leaving the existing def foo(implicit x: Int) syntax to remain with it's current meaning of "all params in list are implicit". def foo(x: Int = implicit) or def foo(x: Int = implicit 123) as described above, or def foo(x: Int = implicitly) all seem plausible,

[sirinath]

@lihaoyi Great proposal! Well thought though.

Perhaps something to consider add to the above is when passing implicit parameters positionally in some cases with some rules:

Basically you can add 3 more rules

• pass all arguments then you can pass positionally, i.e., number of arguments passed matchers the number of arguments of the function including implicit and non implicit. The implicit arguments can appear anywhere. The number of arguments match the number of parameters. This is a special case of the last item.
• pass all non implicit arguments then you can pass positionally, i.e., the number of non implicit arguments matchers the number of arguments passed hence all implicit arguments are inferred. The implicit arguments can appear anywhere. The number of arguments match the number of non implicit parameters.
• if all implicit arguments are to the end of the parameter list, last few which are not provided are inferred while 1st few arguments are taken explicitly. The implicit arguments can appear only towards the end of the parameter list. The number of arguments greater or equal to the number of non implicit parameters, while all implicit parameters are at the end of the list. This is a special case of item below:
• If all the implicits are not grouped to the end of the parameter list, then all parameters upto the last group of parameters are present then also arguments can be provided positionally. E.g. there 5 params with 3 implicits out of 2 are in the last position and the 3rd from the last is a non implicit. In this case if 3 or more of the arguments are provided you can pass the parameters positionally. The number of arguments greater or equal to the number of non implicit parameters plus the implicit arguments not appearing at the end of the parameter list.

I would believe most of the uses will confirm to the above rules.

We cannot allow positional arguments if all implicits are not to the end of the parameter list and only part of the implicit parameters are provided in which case we do not know what parameters are inferred.

[buzden]

@lihaoyi, I can't catch how do you suggest to pass explicitly a parameter to a function like

def f(x: Int) = { (implicit ctx: Context) => ??? }

Do you mean that this name for the context variable should be expressed in the type of returned value to make me able to call it like f(5)(ctx = whatever)?

[LPTK]

How hard would it be to have implicit resolution in default arguments be delayed to call sites instead of definition site?

def foo(x: Int = implicitly) = x

The above currently looks for an Int implicit when type-checking foo, but it would be more useful if it delayed the implicit resolution to call sites. This would not be fully backward-compatible, but I doubt many people currently use implicit resolution in default arguments, though I could be wrong.

This way, what @lihaoyi suggests could be supported with only a minor semantic change.

---

EDIT to add: there is also an easy migration story to make sure old code retain the same meaning, which is to extract all default arguments to definitions (that's is the way they are compiled anyways):

// the above, when considered as legacy code, would be migrated to:
def foo$default$0: Int = implicitly
def foo(x: Int = foo$default$0) = x

[acjay]

@LPTK

I doubt many people currently use implicit resolution in default arguments

FWIW, I've done this before, to achieve "configurable defaults". Pulling this example out of nowhere, but if you had a function that was:

def mapAsync[B](f: A => B)(implicit parallelism: Int = 2)

You could always call it with one parameter list, regardless of whether you provide the implicit or not. Not sure if this is the same situation you were talking about.

[Milyardo]

I'm personally not a fan of default implicits, so a proposal that use an assignment like @sirinath suggested with def foo(n: Int = ?) appeals to me since it prevents the same parameter from being simultaneously implicit and having a default.

[LPTK]

@acjay I think your example would be unaffected by my proposal. To be clear, I don't advocate for getting rid of the status quo way of doing things (with a keyword and an additional parameter list).

You can still encode your use case as:

def mapAsync[B](f: A => B)(parallelism: Int = implicitlyOr(2)) = ...

where we have:

def implicitlyOr[A](default: A)(implicit a: A = default) = a

or if we don't want to use the legacy way of declaring implicits, we can also define the above as:

def implicitlyOr[A](default: A)(a: Defaulted[A] = implicitly) = a match {
  case Default => default
  case ImplicitValue(v) => v
}
trait Defaulted[+A]
case object Default extends Defaulted[Nothing]
case class ImplicitValue[A](value: A) extends Defaulted[A]
object Defaulted extends LowPriorityDefaulted {
  implicit def apply[A](a: A = implicitly) = ImplicitValue(a) }
class LowPriorityDefaulted { implicit def default = Default }

[Ichoran]

Let's allow any parameter block to contain zero or more implicits, and use ;; as a separator (which lexically cannot be there now, and the double symbol makes it really easy to parse visually). Regular parameters go before; implicit parameters go after. To supply those parameters, you must use ;; at the call-site too.

• We recover the existing behavior as long as we allow (implicit a: A, b: B, ...) to mean (;; a: A, b: B, ...).
• We can allow but deprecate un-;;ed calls to all-implicit parameter blocks to ease migration.
• Defaults are easy; they work like they do now. Just = myDefault.
• Calling by position is also fine.
• Calling a subset explicitly by name is fine too.
• For a verbose alternative, allow ; implicit instead of ;;, with the leading ; elidable if there is nothing else. (This is the way in which I propose to allow (implicit a: A, b: B ...) to mean (;; a: A, b: B ...).)
• Typeclasses go into their own parameter block at the end (as they do now) unless the last parameter block ends with implicits, in which case they are appended into that block.
• This all can be ported into Scala 2. If implicit resolution is too complicated with mixed blocks like that, the syntax can still be ported in with the restriction that only the last parameter block can contain implicits.

Before:

xs.sorted(3)  // I wanted element at index 3 but I got type mismatch?!
xs.sorted(myOrder)(3)  // Okay

During transition:

xs.sorted(3)  // Warning: bare passing of implicit parameters is deprecated.  Try ( ;; 3).
  // (Then error about scala.math.Ordering)
xs.sorted(myOrder)(3)  // Also warning
xs.sorted( ;; myOrder)(3)  // Works
xs.sorted(implicit MyOrder)(3)  // Fine too, more explicit

After transition:

xs.sorted(3)  // Works!
xs.sorted( ;; myOrder)(3)  // Also works!
xs.sorted(implicit myOrder)(3)  // Works too!

[propensive]

Thanks for the proposal, @lihaoyi! I would love to hear more about how typechecking could work, particularly with respect to dependently-typed implicits. The ability to interleave implicit and explicit parameters would be very nice, but I think it introduces quite a bit more complexity than the proposal suggests (or at least, details which need to be ironed out).

[odersky]

Very thought provoking proposal! It's true that implicits and defaults have overlapping use cases. As @acjay notes, it is already possible to combine them, like this:

def mapAsync[B](f: A => B)(implicit parallelism: Int = 2)

I am dubious about using named parameters as a disambiguation tool though. First, implicits coming from context bounds [A: Ord] or implicit function types don't have a named parameter. So if we want to pass explicit arguments to them we'd need a different scheme. Second, even for normal implicits you might get into a situation like this:

    class C { def apply(x: T): T }
    def foo(implicit x: T): C

Then foo(x = t) is again ambiguous. So it seems named arguments by themselves are too weak and fragile as a disambiguation mechanism.

There's also an important difference between default parameters and implicits that seems to have been glossed over so far: they behave differently under partial application. Given:

implicit val i: Int = 2
def f(implicit x: Int): Int
def g(x: Int = 1): Int

we have:

f   maps to   f(2)             : Int
g   maps to   (x: Int) => g(x) : Int => Int

[acjay]

@propensive My understanding is that today, parameter lists are typechecked one at a time, and so dependent types or values have to be fully resolved in the previous parameter list:

@ case class X(x: Int = 5)(implicit val y: Int = x) 
defined class X
@ X().x 
res3: Int = 5
@ X().y 
res4: Int = 5

This is certainly pretty easy to reason about, but it's also quite limiting. Just this week, I wanted to do something where a parameter's default value depended on implicitly available context. I'll try to simplify the essence of it to be bite-sized:

case class ReifiedOperation(
  arg: String, 
  blockingStrategy: BlockingStrategy = Block(timeout)
)(
  implicit timeout: Int
)

sealed trait BlockingStrategy
case object DontBlock extends BlockingStrategy
case class Block(timeout: Int) extends BlockingStrategy

val defaultOperation = ReifiedOperation("this is normal")
val specialOperation = ReifiedOperation("i'm special", blockingStrategy = DontBlock)

This doesn't work, for reasons I'm sure we all understand.

• The point Jon made brings up the question of whether it's possible to make parameter list type checking more flexible? Could types and values be resolved in a more dynamic order, as long as it's possible to infer a DAG of dependencies?

It seems like it would solve my modeling issue. Although, with Li Haoyi's idea, I'd probably do:

case class ReifiedOperation(
  arg: String, 
  implicit timeout: Int,
  blockingStrategy: BlockingStrategy = Block(timeout)
)

• So, that raises another follow-up question. What are the rules for exposure as val in case class and for use in the synthesized unapply method?

Let me end by saying I've long pined for keyword args to have the same level of syntactic support as positional args, which could be a side-effect of exploring this proposal.

[gbersac]

@lihaoyi when I began scala, this his how I expected implicits to work. The power of this proposal is that it feels so natural. Great proposal !

On the other hand, the proposition of implicits with the ? keyword feel alien to me. Why a question mark to represent implicit ? Why not a hashtag or anything else ? It would required to google it to understand this synthax.

[LPTK]

@odersky

First, implicits coming from context bounds [A: Ord] or implicit function types don't have a named parameter. So if we want to pass explicit arguments to them we'd need a different scheme. Second, even for normal implicits you might get into a situation like this:

class C { def apply(x: T): T }
def foo(implicit x: T): C

Then foo(x = t) is again ambiguous.

I see a very simple solution to that: an implicit argument list can only be omitted if it's the last specified argument list.

So foo(x = t) has type C (sets foo's implicit x to t), and if you wanted to actually call C's apply method you'd write either foo.apply(x = t) as is currently required, or foo()(x = t) – which is enabled by @lihaoyi's proposal, and solves the issue of apply elision failure. It's also very intuitive IMHO.

What's more, I think the rule above is backward compatible: in current Scala, foo(x = t) _also_ means what it does under @lihaoyi's proposal plus this rule.

[lihaoyi]

@odersky some responses:

First, implicits coming from context bounds [A: Ord] or implicit function types don't have a named parameter. So if we want to pass explicit arguments to them we'd need a different scheme.

Here's one scheme that fits well with my proposal: you can pass implicit parameters positionally, only if you use the implicit keyword.

def foo(a: Int, implicit b: String, c: Boolean) = ???
foo(1, true) // b is implicit
foo(1, b = "lol", true) // b is explicitly provided
foo(1, implicit "lol", true) // b is explicitly provided

That will allow both passing arguments positionally while also allowing inference of interleaved implicit and non-implicit parameters.

Another alternative, is to give up interleaved-implicit-and-non-implicit parameters-within-single parameter-list: then we could just require that implicit parameters always occur after non-implicit parameters in a parameter lists:

def foo(a: Int, b: String, implicit c: Boolean) = ??? // implicits must come at the end
foo(1, "lol") // b is implicit
foo(1, "lol", c = true) // b is explicitly provided
foo(1, "lol", true) // b is explicitly provided

There are probably other schemes we could come up with to allow passing in implicit parameters positionally, if we really wanted to explore that design space. I think both of the above syntaxes look a lot more natural than foo(1, true).explicitly("lol")

Second, even for normal implicits you might get into a situation like this. Then foo(x = t) is again ambiguous.

I think this is a problem we can solve by decree. For example, we may say that given:

    class C { def apply(x: T): T }
    def foo(implicit x: T): C

Then foo(x = t) returns C, and if you want to apply it twice you use foo(implicitly)(x = t) or foo.apply(x = t). This is not unlike what we already have, and I think wouldn't be too surprising.

This point is a bit like PEG parsers v.s. context-free-grammars: when there's ambiguity it's a bit arbitrary to pick one parse over the other, but as long as it's simple & predictable, it may be good enough or even better than an unambigious parse based around clever constraint-solving.

here's also an important difference between default parameters and implicits that seems to have been glossed over so far: they behave differently under partial application.

To be more precise, currently implicits behave differently under currying partial application, and there's an arbitrary restriction on using implicits for non-curried partial application:

// non-curried partial application with default arguments
def foo(i: Int, b: Boolean = true) = ???
f(1)

// non-curried partial application with implicits (doesn't work at all in status quo)
def foo(i: Int, implicit b: Boolean) = ???
f(1)

My proposal makes implicits and defaults behave the same under non-curried partial application (the second example above). Making them behave the same under non-curried partial application means someone learning about implicits doesn't need to learn about currying at the same time. Maybe for people with a Haskell/Scheme/OCaml background are already familiar, but multiple-argument-list currying is very foreign to people from Java/C#/Python/Javascript backgrounds (see e.g. the perpetual confusion around parametrized-decorators in Python).

And ultimately, "i want to pick a default parameter value from the local scope" and "I want to use multiple sets of parentheses when i call this function" are entirely orthogonal concerns

We could preserve the existing syntax & semantics (eliding entire argument lists, etc.) for curried implicit parameter lists, whether for compatibility or for other reasons, without impacting the main body of the proposal.

[LPTK]

@lihaoyi

if you want to apply it twice you use foo(implicitly)(x = t) or foo.apply(x = t).

I don't understand why not foo()(x = t), as I suggested above. If you allow def f(a: Int, implicit b: Int) to be called as f(1), then for sure you would also allow def f(implicit b: Int) to be called as f(), no?

Also, I missed what your rationale was for disallowing passing implicit arguments positionally. Why not simply use exactly the same rules as are currently in place for default arguments?

// current Scala:
def f(x: Int=0, y: Int) = x + y
f(0) // not enough arguments
f(y = 0) // ok
f(0, 1) // ok

// under new proposal:
def g(x: Int = implicit, y: Int) = x + y  // or in your proposed syntax (implicit x: Int, y: Int)
g(0) // not enough arguments
g(y = 0) // ok
g(0, 1) // ok

[odersky]

@lihaoy OK. As far as I see it then, the proposal does not affect the curried implicits of current Scala since by-name parameters are not a good disambiguation mechanism. We still need something else, and foo(x).explicitly(y) is a possible candidate which has the advantage that it fits the current syntax well.

The question is then, should we add another way to define implicits which makes them similar to default parameters? If we do, we have to solve the problem that implicits and defaults behave differently with respect to partial application. Here's a problematic scenario. Say you have

def f(x: Int = 1, implicit y: C)

Then f() is f(1, implicitly[C]). Fine.

Now say you want to turn the first parameter into an implicit as well, since you need more flexibility:

def f(implicit x: Int, implicit y: C)

Now f() is illegal, or, depending on its result type, means something completely different! You have to write f instead. I don't think we should open the door to surprises like this. The other possibility would be to reconcile implicits and defaults by changing the behavior of default parameters. I.e. given a function like

def g(x: Int = 1)

g() would be illegal and instead g would expand to g(1). But that would be a backwards incompatible change. And it would open another discrepancy where I cannot eta expand g to
x => g(x) anymore just because g has a default argument. Altogther this looks at least as bad as auto-unit insertion, which we just dropped.

I also wanted to raise another red flag. All the examples we gave in this thread are extremely bad code since they pretend it's OK to define an implicit for a common type like Int. Most default arguments do have common types, but implicits should never have them. So it looks like moving from default parameters to implicits should specifically not be seamless and require some effort from the programmer.

[odersky]

While we are discussing implicit definition syntax, here's another thought. Currently when we write

    def f(x: Int)(implicit c: C) = ... implicitly[C] ...

we get "two-way" implicitness - the second argument of f is synthesized, and c is in turn available as an implicit in the body of f. It would be nice if there was a way to disentangle these two functions. One possible syntax to express this would be to also allow implicit in front of a parameter list:

   def f(x: Int) implicit (c: C) = ... c ...

This would still synthesize the second argument but c would have to be referred to explicitly in the body of f. As far as I can see, this change could subsume the functionality of MacWire as a dependency injection mechanism. Just set up your program like this:

    class C_1 implicit (C_1_1, ..., C_1_m1)
    ...
    class C_n implicit (C_n_1, ..., C_n_mn)

Here C_1, ..., C_n are the component classes, that each have a subset of the C_i as dependencies. Dependencies are expressed in a implicit parameter clause. Auto-wiring is then done like this:

  {
    implicit val c_1: C_1 = new C_1
    ...
    implicit val c_n: C_n = new C_n

    (c_1, ..., c_n)
  }

The reason why we do not want to do this with current implicits (and the reason why MacWire exists) is that we do not want to pollute the implicit namespace of the bodies of our components C_i with the dependent components. MacWire is implemented with the kind of macros that won't be supported in Scala 3, so it would be good to find a language-level alternative.

I am bringing this up here because it would also avoid the problem that implicits require curried parameter lists. (Although in fact I am not sure that's a strong argument. For better or for worse, curried parameter lists are pretty ubiquitous in Scala. You need them not just for implicits but also for better type inference, or for passing arguments in {...}).

/cc @adamw

[odersky]

Note: If we do adopt

def f(x: Int) implicit (c: C) = ...

then the current definition syntax

def f(x: Int)(implicit c: C) = ...

can be treated as syntactic sugar for

def f(x: Int) implicit (c$: C) = { implicit val c: C = c$; ... }

[propensive]

@LPTK No, the idea is that you should never be able to supply an implicit parameter "by accident" at the call site. It should be necessary that it's distinguished in some way.

[LPTK]

the idea is that you should never be able to supply an implicit parameter "by accident" at the call site. It should be necessary that it's distinguished in some way

Is this actually a central goal of the design? At least it's not part of the original motivation explained in the opening message of this github issue. But I may be missing something (e.g., discussions happening offline). Though I can see that having to write foo()(...) is a minor annoyance compared to just foo(...), I don't know if it's that important in practice.

[lavrov]

@lihaoyi Interleaving implicit and non-implicit parameters doesn't work with current implementation of implicit function types which are encoded like this:

trait ImplicitFunction1[-T0, R] extends Function1[T0, R] {
  override def apply(implicit x: T0): R
}
implicit (T0) => R

In the case of def foo(x: Int, implicit ctx: Context) = ... how would you represent it as a function type?

[acjay]

@odersky

Now say you want to turn the first parameter into an implicit as well, since you need more flexibility:

def f(implicit x: Int, implicit y: C)

Now f() is illegal, or, depending on its result type, means something completely different!

Hmm, just to make sure I'm understanding, this is because implicit parameter lists can currently be completely elided, but we're not going be allowing no-arg and zero-arg calls to be interchangeable anymore. Is that right? If so, then in the spirit of unifying the syntax of implicit and default args, these behaviors should be rectified somehow.

While we are discussing implicit definition syntax, here's another thought. Currently when we write

   def f(x: Int)(implicit c: C) = ... implicitly[C] ...

we get "two-way" implicitness - the second argument of f is synthesized, and c is in turn available as an implicit in the body of f. It would be nice if there was a way to disentangle these two functions. One possible syntax to express this would be to also allow implicit in front of a parameter list:

I often use an idiom where services are modeled by case class, and I use implicits for my wiring, as you say. In this case, I have to put val by implicits that I want to be propagated to the inner scope. So, in this particular case, there's already a way to differentiate one-way and two-way implicits. Could this be expanded by requiring val for two-way implicits elsewhere?

Lastly, on a meta level, one question folks seem to be dancing around a bit is whether it makes sense to continue to think of implicits positionally at all.

(I do get that you're largely talking about things that are additive to today's Scala, and I'm talking about breaking changes.)

[odersky]

Hmm, just to make sure I'm understanding, this is because implicit parameter lists can currently be completely elided, but we're not going be allowing no-arg and zero-arg calls to be interchangeable anymore. Is that right?

Sort of. They never were interchangeable. There was a one-way automatic conversion from one to the other, but that one's gone now as well.

I believe the driving force here is that we want to be very clear about to which parameter(s) arguments are passed to. Implicits are hard enough, no need to throw an additional puzzler in the mix.

[odersky]

I often use an idiom where services are modeled by case class, and I use implicits for my wiring, as you say. In this case, I have to put val by implicits that I want to be propagated to the inner scope. So, in this particular case, there's already a way to differentiate one-way and two-way implicits. Could this be expanded by requiring val for two-way implicits elsewhere?

Can you show an example? I don't see how having val or not would affect the scope.

[lihaoyi]

We still need something else, and foo(x).explicitly(y) is a possible candidate which has the advantage that it fits the current syntax well.

Yep that seems right!

I also wanted to raise another red flag. All the examples we gave in this thread are extremely bad code since they pretend it's OK to define an implicit for a common type like Int. Most default arguments do have common types, but implicits should never have them. So it looks like moving from default parameters to implicits should specifically not be seamless and require some effort from the programmer.

This is true, but can be mitigated mechanically: rather than having the implicit be Int, have the implicit be FooInt with an implicit constructor:

case class FooInt(value: Int)
object FooInt{
  implicit def create(value: Int) = FooInt(value)
}

This gives you the nice default-parameter syntax along with the nice specific type for implicit-resolution. It's a pattern I use pretty commonly (e.g. from String to fansi.Str, from Int to sourcecode.Line). It's a bit of boilerplate, but not too bad overall

[acjay]

@odersky Ah, you're right. I forgot the real reason I had needed val. In my case, is is because my services sometimes extend abstract traits, in a mini cake-pattern sort of way, and those traits sometimes need implicits. So I define them as abstract implicit members in the trait, which can only be fulfilled in a case class by using val.

[ryantheleach]

Sorry in advanced for my inexperience. But 'implicitly[Foo[A]]' inside a function/method body is essentially summoning something in implicit scope, without it being Available in the visible function definition right?

So, that's essentially a hidden, anonymous parameter (it's not named until you name it) that's not immediately obvious?

If we call the concept of summoning a parameter in this form, without following implicit resolution, 'anonymous parameter passing' in that, it 1. Has no name/keyword 2. Has no position in the function definition. Then what would an anonymous parameter without implicits look like? (In that it's required to be passed in (has no default value defined) what would syntax look like to pass it in?

I think if that was introduced orthogonal to implicit resolution, then you could have default values, implicits, and anonymous parameter passing all orthogonal to each other.

[LPTK]

@ryantheleach inside a method body, implicits are resolved based on the _static scope_ of the body, so they do not behave like parameters. The resolution won't change depending on the call sites.

[gabriel-bezerra]

@odersky 's suggestion for implicit in and out of the parameter list reminds this (badly named) suggestion on Scala Contributors:
https://contributors.scala-lang.org/t/more-on-duality-and-homonyms-in-the-language/1775

[nicolasstucki]

This was implemented in #5458 and then the syntax changed in #5825