Dotty: Cannot define a path-dependent Conversion instance

[neko-kai]

minimized code

class Named[S <: String with Singleton](s: S)

object Named {
  given Conversion[(s: String), Named[s.type]] {
    def apply(s: String): Named[s.type] = new Named(s)
  }

  // this is ok:
  // implicit def convert(s: String): Named[s.type] = new Named[s.type](s)

  "abc": Named["abc"]
}

expectation

Expected being able to define an instance of Conversion equivalent to implicit def convert(s: String): Named[s.type]. However the definition above fails to parse!

[error] example.scala:22:20: an identifier expected, but '(' found
[error]   given Conversion[(s: String), Named[s.type]] {
[error]                    ^
[error] example.scala:28:3: Found:    String("abc")
[error] Required: Named[String("abc")]
[error]   "abc": Named["abc"]
[error]   ^
[error] two errors found

[ashwinbhaskar]

can I pick this up?

[ashwinbhaskar]

@neko-kai can you assign this to me?

[neko-kai]

@ashwinbhaskar I can't, i'm not a dotty developer and don't have any permissions. Maybe @smarter could? (sorry for bothering)

[bishabosha]

@ashwinbhaskar I assigned you

[ashwinbhaskar]

@neko-kai @bishabosha thank you:)

[smarter]

Conversion[(s: String), Named[s.type]]

This syntax does not make sense in general (what happens if the second type parameter appears in a context where s is not bound ?) so we can't support it just for Conversion. What we could possibly support is something like:

  given Conversion[(s: String), Named[_]] {
    def apply(s: String): Named[s.type] = new Named(s)
  }

But that would also require some special-casing of Conversion. Since we want to discourage people from using conversions, this might not be worth it. If it is, then a different syntax (like given conversion (s: String): Named[s.Type] = ... ?) might be more appropriate, but that's a tall order at this point...

[neko-kai]

@smarter

This syntax does not make sense in general (what happens if the second type parameter appears in a context where s is not bound ?) so we can't support it just for Conversion

IMHO it can conceivably make sense for all subtypes of Function* where these type parameters appear in appropriate positions with appropriate variance.

But that would also require some special-casing of Conversion.

AFAIU the special case would appear only in Implicits.scala? (i.e. use refined apply signature for search & result, not type parameters) That seems completely fine to me, Conversion is already special-cased there.

Since we want to discourage people from using conversions, this might not be worth it

shapeless, akka, singleton-ops, spire etc. afaik use dependent conversions. An even more absolutely massive amount of libraries use implicit def macro conversions. IMHO this will come up again and it will be easier to support earlier before the codebase becomes more mature. Also scalafix rules to rewrite conversions cannot be made comprehensive until Conversion reaches parity with implicit def.

[smarter]

AFAIU the special case would appear only in Implicits.scala? (i.e. use refined apply signature for search & result, not type parameters) That seems completely fine to me, Conversion is already special-cased there.

Yes, it's not too bad.

shapeless, akka, singleton-ops, spire etc. afaik use dependent conversions.

Good point, could you link to some examples ?

[neko-kai]

@smarter

Off the top:

shapeless:

• whitebox dependent macro conversion: https://github.com/milessabin/shapeless/blob/452158eda991a6d145089563149313de044d4764/core/src/main/scala/shapeless/singletons.scala#L42
• super-dependent implicit conversion: https://github.com/milessabin/shapeless/blob/452158eda991a6d145089563149313de044d4764/core/src/main/scala/shapeless/poly.scala#L248

sbt:

• implicit (+overriden) def macros eve-ry-wh-ere

My company uses both dependent and macro conversions in open-source and closed-source as well.

I can find more usages if you ask again for more proofs of usage.

[julienrf]

This syntax does not make sense in general (what happens if the second type parameter appears in a context where s is not bound ?) so we can't support it just for Conversion.

If the second type parameter appears in a context where s is not bound, then this is a compilation error.

But I think this syntax does make sense, in general. We already have it for Function1. Indeed, the following type-expression is valid: (a: A) => a.B, and conceptually it desugars to Function1[(a: A), a.B].

[ashwinbhaskar]

@julienrf The syntax does not work for Function1 as well. This gives an error.

trait Foo
    type Baz
    val baz: Baz

val bazExtractor: Function1[(f: Foo), f.baz] = ???

Did you mean that if it is made to work for Function1 then it would work for Conversion as well?

[julienrf]

Did you mean that if it is made to work for Function1 then it would work for Conversion as well?

Yes, we could maybe generalize the path-dependent syntax to other types than Function1.

[ashwinbhaskar]

@smarter @julienrf What is the consensus on this? Should we allow such a syntax?

[dwijnand]

I can find more usages if you ask again for more proofs of usage.

@neko-kai Do you remember where Akka used these? I'd be interested to check that out.

[dwijnand]

sbt:

• implicit (+overriden) def macros eve-ry-wh-ere

~The only dependent conversions I could find in sbt is~ (edit: derp, actually not dependent)

implicit def applicativeInstance[A[_]](implicit ap: Applicative[A]): Instance.Aux[A]

Most of the implicit defs are method enrichments (implicit def fileToRichFile(file: File): sbt.io.RichFile) or straight, non-dependant, conversions (implicit def projectToLocalProject(p: Project): LocalProject). Please correct me if I'm wrong, but I think sbt could live with a clunkier alternative for its applicativeInstance.

[smarter]

implicit def applicativeInstanceA[_]: Instance.Aux[A]

That doesn't look like a dependent method type to me, there's no parameter or result type which refers to a previous term parameter

[neko-kai]

@dwijnand @smarter
Of __path-dependent__ and only __path-dependent__ conversions there are:

akka-http's Directive.scala has two 1 2
There's another in FieldMagnet
Another in OnSuccessMagnet
Another in ParamMagnet
Two more in akka-parsing: 1 2

For a total of 7 in akka-http.

@dwijnand

Most of the implicit defs in sbt are method enrichments

I mentioned implicit def _macros_ for sbt. Why did I intentionally conflate dependent conversions and macro conversions here?

1. Because whitebox macro conversions can be used to implement dependent conversions, which is what happens in shapeless.Witness implementation. (admittedly sbt's macro conversions are blackbox)
2. Macro conversions are also hurt by Conversion, because macro methods must override a concrete method (because Conversion is now a value that can be upcast and "lose" it's macro status). Because of that to define a macro conversion you have to define two traits now:

trait Fixture0[A] extends Conversion[Int, A] {
  def a: A
  override def apply(i: Int): A = i match {
    case 0 => a
    case _ => throw new AssertionError("Runtime error: access only from 0 not any other number!")
  }
}

trait Fixture[A] extends Fixture0[A] {
  override inline def apply(i: Int): A = inline i match {
    case 0 => a
    case _ => error("Compile error: access only from 0 not any other number!")
  }
}

[nicolasstucki]

Instead of writing a Conversion as

Conversion[T, U]

we could use

Conversion[T => U]

which also allows dependent conversion

Conversion[(x:T) => x.U]

Here, Conversion would be an opaque type

opaque type Conversion[+F <: Nothing => Any] = F

See #8523 for more details

[SethTisue]

8523 was merged — was this ticket left open on purpose (because of @neko-kai's caveat at https://contributors.scala-lang.org/t/proposal-changes-to-implicit-conversions/4101/15), or just by accident?

[smarter]

https://github.com/lampepfl/dotty/pull/8523 does not change the current implementation, it's a test file showing a prototype of what a changed implementation could look like.