Dotty: Implicit conversion to context function does not work

[djneades]

Minimized code

import scala.language.implicitConversions


type Fn = Int => Int

given Conversion[String, Fn]:
  def apply(s: String): Fn = (i : Int) => i + s.length

def f(fn: Fn): Int = fn(10)


type ContextFn = Int ?=> Int

given Conversion[String, ContextFn]:
  def apply(s: String): ContextFn = (i : Int) ?=> i + s.length

def g(cfn: ContextFn): Int = cfn(using 10)


@main def main() =
  println(f("hello")) // compiles
  println(g((i : Int) ?=> i + 5)) // compiles
  println(g(99)) // compiles, but should it?
  println(g("hello")) // fails to compile, but should

Output

Compilation with Dotty 0.27.0-RC1 fails with the following error:

[error] -- [E007] Type Mismatch Error: C:\Git\Overload\src\main\scala\Main.scala:24:12 -
[error] 24 |  println(g("hello")) // fails to compile, but should
[error]    |            ^^^^^^^
[error]    |            Found:    ("hello" : String)
[error]    |            Required: Int
[error] one error found
[error] (Compile / compileIncremental) Compilation failed

Expectation

1. I may be misunderstanding how context functions are supposed to work, but I would have expected the println(g("hello")) line to compile as there is an implicit conversion available to convert the "hello" String to the required ContextFn. The analogous println(f("hello")) compiles and works as expected.

2. I did not expect the error message Required: Int, as the required type for the argument to g is Int ?=> Int, not Int.

3. Finally, and this is likely related to the previous point, I did not expect the println(g(99)) line to compile, yet it did. An analogous println(f(99)) (not shown in the above code sample) does not compile, which is what I would expect.

[bishabosha]

Edit: didn't make sense what was written before

[LPTK]

An implicit (or context) function type appearing in expected type position will always be treated as an implicit lambda binding the context, and the expected type effectively used will be its RHS (here Int). Therefore, I think an implicit conversion will never be triggered based on an expected implicit function type, which is by design. I don't think there is a way around this.

[djneades]

@LPTK Thank you very much for the explanation, which I greatly appreciate.

I do find this behaviour surprising as, after all, _I_ can clearly see that the method I am calling expects a context function, and I can also see that the compiler knows how to construct something suitable from the actual argument using the appropriate conversion. It doesn’t therefore seem (at least to a naïve user such as me!) overly unreasonable to expect that the compiler should be able to realize the intended behaviour.

The current behaviour is also rather disappointing. First, because it seems to be a significant irregularity in the Scala 3 language, whereas Scala 3 generally tries to remove such irregularities (with considerable success). Second, because it places some unhelpful limits on the sorts of things one can do along the lines of the _Example: Builder Pattern_ given in the current Dotty documentation. For example, this limitation would seem to make it impossible to have the compiler convert a plain String instance into a context function that could be used to call the builder. (That was basically what I was trying to make work, but this issue bit me, as did #10016). This issue is not the end of the world, as there are alternative approaches but, again, the limitation seems surprising, and means that the overall resulting builder is likely to be a little less pleasing than might otherwise have been possible.

Perhaps, if the current behaviour is genuinely intended, the feature documentation for both Context Functions and Implicit Conversions could be updated with an explanation for why what I was trying to do here does not (and cannot) work? That may at least spare future users of these features a degree of puzzlement.

Once again, thank you for the explanation. I am _really_ liking Scala 3 thus far, even with these little hiccups. I am finding it to be a huge improvement over Scala 2, and I can’t wait for a production-quality release.

[OlivierBlanvillain]

Implicit conversions to context functions seams like an anti-pattern and not something we would not like to support, so I will close this for now.

Something that's perhaps hidden between the lines of our documentation is that we tried really hard to get rid of implicit conversion altogether, but it's not possible for legacy reasons. We tried to put conversions behind a flag, then behind a flag at use site, then deprecate the convenient syntax in favor of instances of the Conversion class, each time hoping to make users more and more guilty for using that terrible feature, but it's never enough...

[djneades]

@OlivierBlanvillain Thank you for the clarity, that is very helpful. I know it is early days for the Scala 3 documentation, so it is understandable if not everything is stated as clearly as may have been intended. I can confirm that the message regarding your having tried to get rid of implicit conversions altogether is _completely_ hidden between the lines of the documentation :-) In fact, I’ve just re-skimmed the page on implicit conversions, and I see not even the merest hint of that message there.

May I suggest that, if the Scala 3 language designers do not wish people to use a feature, it would be extremely helpful for everyone if the documentation were to _clearly_ say that?

Whilst on the subject of documentation, I shall repeat here my request that the non-support for implicit conversion to context functions also be plainly stated.

Thank you again for your help.

[neko-kai]

hoping to make users more and more guilty for using that terrible feature, but it's never enough...

As a library author I do not plan to shy away from -in companion object- implicit conversions / magnet pattern post 3.0, even if they were to remain completely hidden behind a flag – a note to add the language flag to all builds would be enough to unblock users and the added API ergonomics are worth it.

It's likely that union types could solve some of the use cases that I'm using magnet pattern for currently, but they also lack the open extensibilty aspect of being able to add new conversions for user-owned types – or for the library author to execute arbitrary code on conversion-site by making the conversion a macro.

[OlivierBlanvillain]

I don't want to (re)open the depate on implicit conversion, but don't worry @neko-kai the conversions are not going away :)

May I suggest that, if the Scala 3 language designers do not wish people to use a feature, it would be extremely helpful for everyone if the documentation were to clearly say that?

It's all there, in the documentation of the import:

Why keep the feature? Implicit conversions are central to many aspects of Scala’s core libraries.
Why control it? Implicit conversions are known to cause many pitfalls if over-used. And there is a tendency to over-use them because they look very powerful and their effects seem to be easy to understand. Also, in most situations using implicit parameters leads to a better design than implicit conversions.

Maybe we should link users to that documentation every time they try to use the feature, just like scalac does:

$ scala
Welcome to Scala 2.12.10 (OpenJDK 64-Bit Server VM, Java 11.0.8).
Type in expressions for evaluation. Or try :help.
scala> implicit def s(in: String): Int = 1
<console>:11: warning: implicit conversion method s should be enabled
by making the implicit value scala.language.implicitConversions visible.
This can be achieved by adding the import clause 'import scala.language.implicitConversions'
or by setting the compiler option -language:implicitConversions.
See the Scaladoc for value scala.language.implicitConversions for a discussion
why the feature should be explicitly enabled.
       implicit def s(in: String): Int = 1
                    ^
s: (in: String)Int

Whilst on the subject of documentation, I shall repeat here my request that the non-support for implicit conversion to context functions also be plainly stated.

That would definetly be helpful, if someone has time to contribute here I'm sure it would get merged. However I'm not sure the issue you raised would be the priority, just to give one example, does the Scala spec even mention the fact that implicit conversion don't chain?

[djneades]

@OlivierBlanvillain Thank you for the further comment, much appreciated. I don’t wish to consume any more of anyone’s time and attention than I already have on this feature as I am content with the decision to reject this issue. However, it may be helpful for anyone working on the Scala 3 documentation if I push back a little on your latest comment. Thus, please take the rest of what I say in this comment as being purely for that purpose; no further response here is necessary.

As I read it, the documentation on the import that you quote cautions users not to _overuse_ implicit conversions, which makes perfect sense for a feature with pitfalls. However, it falls far short of suggesting that the language designers think that implicit conversions are a ‘terrible feature’. In fact, the import documentation’s statement that ‘Implicit conversions are central to many aspects of Scala’s core libraries’ suggests rather the opposite, making implicit conversions sound like a fundamental feature that is absolutely necessary for the implementation of the core of the Scala ecosystem (and thus, presumably, for other libraries as well).

I should add here that I have never understood the requirement to use a feature import as indicating that a feature should not be used, but rather as an indication to the user to ensure that one first knows what one is doing.

In any case, the documentation you cite does not seem to reflect the clarity of your earlier comment, neither does the Dotty page on Implicit Conversions.

I do take your point about the Scala specification.

[djneades]

This discussion answers many of my questions: Can We Wean Scala Off Implicit Conversions?