Dotty: Ergonomics of partially explicit delegates

[milessabin]

If we have a method with multiple given parameters and we want to pass one or more of them _explicitly_ at some call site we currently have to explicitly pass _all_ of them,

object Test {
  trait MyThing[F[_]]
  trait Monad[F[_]]

  def foo[F[_]: MyThing: Monad] = ???

  class Foo[T]
  object Foo {
    delegate for Monad[Foo] = ???
    delegate for MyThing[Foo] = ???
  }

  val local: MyThing[Foo] = ???

  foo[Foo] given (local, the[Monad[Foo]])
}

But it seems like we ought to be able to do better. Do the explicit givens have to be ordered and complete? Could we support usage like this,

foo[Foo] given local

at call sites and have the required Monad[Foo] instance inferred? Supporting this seems to me to be of a piece with the the desire to support anonymous delegates.

[SystemFw]

Just wanted to quickly say that supporting this pattern would make a pretty big difference to the whole cats-effect/typelevel ecosystem and style of code.
I'll add a real world example which can hopefully serve as motivation. It's slightly simplified from what you would actually see but it should make the point.

trait Db[F[_]] {
   def get: F[User]
}
object Db {
  // note the argument and the `F[Thing[F]]` shape
   def create(c: Config): IO[Db[IO]]
}

The idea is that you pass the Db algebra implicitly (as a capability), but then create it explicitly. I call this _implicit call sites_.
Most of the code then looks like this:

def userName[F[_]: Db: Functor]: String = Db[F].get.map(_.name)

But now we have a dilemma in our main, when we need to instantiate this.
One way would be:

Db.create(config).flatMap { db =>
   foo(db, Functor[IO]) // we also need to pass Functor explicitly
}

Another would be:

Db.create(config).flatMap { implicit db =>
   foo[IO]
}

The latter works quite well except when you need to create a few of this algebras (like Db, Kafka or w/e), at which point:

• for does not support binding things implicitly (you need better-monadic-for)
• an explicit flatMap with a tuple requires to then make an implicit for each element of the tuple

---

Ideally, the ability to pass an argument in given explicitly, while letting the others be inferred would completely solve this problem, and make a whole bunch of code a lot more pleasant.

[jdegoes]

@SystemFw

That's an abuse of type classes, IMO, since for a given F[_], you can have infinitely many valid Db. The parameter should be passed explicitly; or with implicit function types (separate from the constraint list); or with reader.

I think you'll find in every case you want to do this, there are infinitely many potential values, and hence, co-opting delegate machinery for this purpose is an anti-pattern.

That said, _if_ there were a feature like this, it would strike me as similar to named parameters with defaults; or named types (foo[A = Int]); and would hopefully be similar enough so that users knowledge of one would help them learn the others.

[nafg]

Wasn't this one of the main problems with implicits in the first place,
that was supposed to be the reason for completely reinventing them?

On Thu, Jun 20, 2019, 8:48 AM Miles Sabin [e-mail] wrote:

If we have a method with multiple given parameters and we want to pass one
or more of them explicitly at some call site we currently have to
explicitly pass all of them,

object Test {
trait MyThing[F[_]]
trait Monad[F[_]]

def foo[F[_]: MyThing: Monad] = ???

class Foo[T]
object Foo {
delegate for Monad[Foo] = ???
delegate for MyThing[Foo] = ???
}

val local: MyThing[Foo] = ???

foo[Foo] given (local, the[Monad[Foo]])
}

But it seems like we ought to be able to do better. Do the explicit givens
have to be ordered and complete? Could we support usage like this,

foo[Foo] given local

at call sites and have the required Monad[Foo] instance inferred?
Supporting this seems to me to be of a piece with the the desire to support
anonymous delegates.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/lampepfl/dotty/issues/6717?email_source=notifications&email_token=AAAYAUEDS3LZGNPWM2D4R6LP3N4CDA5CNFSM4HZSVLCKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G2WIMEQ,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAYAUGG5Y26KIJVPNT2YADP3N4CDANCNFSM4HZSVLCA
.

[SystemFw]

@jdegoes Overall, I think I disagree (open to changing my mind though). As long as the usage is "eliminating" the constraint, I find the above exactly equivalent to having a newtype over Kleisli, with much less ceremony

[SystemFw]

class Foo[T]
  object Foo {
    delegate for Monad[Foo] = ???
    delegate for MyThing[Foo] = ???
  }

  val local: MyThing[Foo] = ???

  foo[Foo] given (local, the[Monad[Foo]])

btw, I'd probably not do this where there is actually an existing delegate for MyThing[Foo] : if I plan to use implicit call sites I don't make anything implicit in the companion object, you can pass it around implicitly but you have to create it explicitly, which avoids issues with picking up the wrong instance by accident, and makes the point about multiple potential instances for F moot

[jdegoes]

@SystemFw I'll give you some (what I find to be compelling) examples later.

[odersky]

In the hopefully rare cases where this matters one has already two ways to address this:

1. Curried given clauses:

   def foo[F[_]] given MyThing[F] given Monad[F] = ???
   ...
   foo given local
   

2. Local delegates:

   def foo[F[_]: MyThing: Monad] = ???
   
   {  delegate local for MyThing[Foo] = ???
    foo[Foo]
   }
   

I propose to try these out at large scale first before deciding we need something else.

[jdegoes]

@SystemFw

Here are some examples of the power of ordinary records:

1. Updating a single function via copy (e.g. replacing Db#update)
2. Updating functions or nested modules with lenses (lenses allow you to drill and perform pinpoint modifications)
3. Record combinators (e.g. Db[F] => Db[F] that adds logging), AKA decorators

All of these operations are made much simpler when operating at the level of values.

It's like Arbitrary vs Gen. It's super easy to compose and manipulate Gen but awkward and painful to do the same with Arbitrary.

If the type doesn't define the instance, then it's best to just use records, IMO.

[SystemFw]

well, I can tell you that there is at least one decently (but obviously not predominantly) common style of code (final tagless within the typelevel ecosystem) where this isn't rare at all.

@odersky I'd like to ask a question about 1) however, if you don't mind :)

What's the interaction with context bounds?
In other words, would this work?

def foo[F[_]: Monad] given MyThing[F] = ???

val local: MyThing[Foo] = ???

foo given  local

Number 2) unfortunately suffers from the same problems local implicits suffer in Scala 2, it's pretty awkward to use where the creation of MyThing returns F[MyThing[F]] (cannot implicitly bind in for, and flatMap { implicit does not work well when you have more than one of these things)

Thank you for your time :)

[SystemFw]

@jdegoes I find all of your points entirely orthogonal tbh.
I could use a record like you propose and pass it implicitly.
What I'm trying to explore here is ways in which I can do things differently from what I would do in Haskell, using the fact that instances are values.

It's like Arbitrary vs Gen. It's super easy to compose and manipulate Gen but awkward and painful to do the same with Arbitrary.

Except the distinction between Gen and Arbitrary is completely spurious in Scala, you can just pass Gen implicitly (I actually don't think it's worth it in the case of Gen tbh, but it is in other cases like the example I'm describing above)

[jdegoes]

@SystemFw IMO delegates should not be abused to pass things that should be passed manually (even automatically with reader). It's like the old implicit defaultPort: Int anti-pattern that used to dominate early Scala code bases. If there exist many valid and useful implementations, then it should be passed as an ordinary parameter. That's just my opinion obviously.

[SystemFw]

Well, as I said I'm just _exploring_, I'm open to changing my mind (in fact, I haven't even fully made up my mind yet).
I think the crucial detail is this: it's ok to _pass_ things implicitly as long as you only _make_ their instances implicit close to their final call site (what I call implicit call sites in scala 2), or pass them explicitly when you want to instantiate them at the top of the call chain (if curried given plus context bounds work, I'd be happy with that).
Doing this prevents you from picking up the wrong thing from the implicit scope by accident, and I consider _that_ to be the problem, not the implicit passing.

Another way to frame the question is what makes passing things with Reader and then using run any different than passing things implicitly and using given.

[OlivierBlanvillain]

My take on this is that this is not really an issue given that explicitly passing implicit arguments in user code should be the exception and not the norm. And if it isn't, it's probably good practice to stay away from the "given call site syntax" and have explicit overloaded methods instead:

/** The foo method. */
def foo[F[_]: MyThing: Monad] = ???

/** If you want to explicit pass MyThing, use this one! */
def foo[F[_]: Monad](m: MyThing[F]) = {
  implicit val a: MyThing[F] = m
  foo
}

[SystemFw]

@OlivierBlanvillain Just so we are on the same page, the point of my question is _not_ to have some usages of foo be implicit, and other explicit.
It's to have things like:

trait Db[F[_]]
object Db {
   def create: F[Db[F]]
}

be passed implicitly throughout the call chain, except when they need to be created (which is at the top of the call chain).
Declaring them as implicit just before the call is what I do in Scala 2, and it suffers from the drawbacks I mention here https://github.com/lampepfl/dotty/issues/6717#issuecomment-505142056.

It's fine if Dotty does not improve on the situation, but ideally I'd like some motivation for

And if it isn't, it's probably good practice to stay away from the "given call site syntax"

Tangentially, do you know if this would work?

def foo[F[_]: Monad] given MyThing[F] = ???
val local: MyThing[Foo] = ???
foo given local

[jdegoes]

@SystemFw

The mere act of grabbing an implicit value, composing it, and making it implicit again for a lower-scope is exceedingly awkward, involving a certain amount of mandatory boilerplate and ceremony, and difficult to wrap one's mind around (indeed, if properly done, a user is not interacting with or even explicitly aware of type class instances at all, due to extension methods).

So while theoretically possible, value-level composition is not going to happen very often on instances, even in cases where it's beneficial, as seen in real world code bases.

Compare to code using reader or kleisli, for example, which leverages value composition _all the time_. Theoretically, we might be able to do the same thing using instances, but practically speaking, it's not happening due to ergonomics and mindsets.

I think the crucial detail is this: it's ok to pass things implicitly as long as you only make their instances implicit close to their final call site

I think what you propose is much more sensible (defining instances 'right before' entry application), but consider the pedagogical and institutional implications: following this philosophy, it's OK to create implicit defaultPort: Int as long as you create the implicit "close" to the entry application point. That does not seem like a good idea under any circumstances because of what it leads to.

• How do you define "close"? The same method? The same class? The same module?
• Developers will place the implicits in packages or objects to reuse construction logic. Now you've got to import your implicits, and you're back in implicit hell.

In a large team, it's easier to say and enforce "no local implicits and no orphans". These simple rules can be placed into a linter and they result in a large improvement on the quality of code. They result in people using values for things that should be values, and implicits to reveal structure on types (Db[F] is _not_ structure on a type F, merely an F interface for a database). They result in clearer and cleaner ways of thinking about structural properties versus interfaces.

I think in every single case you want "partial explicit delegates", it's actually an abuse of delegates to provide a feature that works better, is taught more easily, and can be enforced more reliably in large code bases, with just ordinary values.

Your example written using modules:

class MyService[F[_]](db: Db[F]) {
  def userName[F[_]: Functor]: F[String] = db.get.map(_.name)

  ...
}

It's very obvious and easy to manipulate db as an ordinary value, e.g.:

class MyService[F[_]](db0: Db[F]) {
  val db = databaseLogger(db0)

  def userName[F[_]: Functor]: F[String] = db.get.map(_.name)

  ...
}

And of course, module-oriented programming is a strength of Scala, plays well with large applications, and is just one of many ways to solve this problem—without adding new language features that (IMO) carrot developers toward "delegate abuse".

[SystemFw]

The mere act of grabbing an implicit value, composing it, and making it implicit again for a lower-scope is exceedingly awkward, involving a certain amount of mandatory boilerplate and ceremony, and difficult to wrap one's mind around (indeed, if properly done, a user is not interacting with or even explicitly aware of type class instances at all, due to extension methods).

Heh, but that's the starting point of the discussion: can we make it less awkward? If you could just say given and pass it explicitly, this point becomes moot. Also note that as I said a few times, the main usage is _not_ to take a value in the explicit scope and manipulate it (although that's useful as well), but create a value that _isn't_ yet in the implicit scope and make it implicit for the rest of the call chain.

Theoretically, we might be able to do the same thing using instances, but practically speaking, it's not happening due to ergonomics and mindsets.

Which is why I'm asking if we can improve the ergonomics (and btw, it _is_ happening).

How do you define "close"?

Again, this is a problem with the _current_ way of doing this (declaring implicit closely), if you could just say given thing this is also moot.

Developers will place the implicits in packages or objects to reuse construction logic

Again, for my use case the creation is effectful (F[Thing[F]])) so they _cannot_ physically do this.

---

I'd like to reiterate that I'm not dead set arguing for this, I'd just like to explore the space, but most of the answers seem to reply to say "it's an abuse" "it's not good practice" without very strong arguments as to _why_ this is.

Assuming you can say given thing, in the scenario that I've described above, what are actual technical problems that can arise?

Note that I do find @odersky original answer reasonable: maybe the use case is not considered important enough, or the current tools are deemed sufficient, but I think that's a slightly different question, what I'm asking is mostly "would this work well, and if not, why" in the specific context I have mentioned

[jdegoes]

My 1 sentence response is that the distinction between delegates and interfaces is good and useful (not just _syntactical_, but _semantic_), and conflating the two muddles thinking, complicates teaching, poses a challenge for linting, and often (if not inevitably) deteriorates the quality of code; and such conflation should be discouraged and not encouraged, especially given that Scala has very good facilities already for dealing with delegates and interfaces, in their own separate ways.

We don't need to write combinators to take a monad and return another monad; or rather, the combinators we write are at the level of delegate machinery (monad transformers). It makes sense for _properties_ of types to be delegates, because they reflect the _actual structure of those types_. We do need to write combinators for interfaces (c.f. _decorator pattern_, monoids, other binary operators). It makes sense for interfaces to be values, because they do not reflect the structure of types, but rather, ad hoc patterns of usage around those types for which we need a layer of indirection.

That's just an opinion, of course, and I'm sure if there _were_ partial explicit delegates, I have no doubt you would effectively use them to write high-quality code; I'm just not sure language-level changes are necessary and I cringe at the thought of seeing conflation between delegates and interfaces. I would very much _not_ like to see the many faces of launchServer given defaultPort in Scala 3.x code bases...

Btw what did you think of the module-oriented factoring above? A lot of people are using it with good success.

[SystemFw]

Btw what did you think of the module-oriented factoring above? A lot of people are using it with good success.

I'm actually using that myself about 60% of the time :)
I appreciate I'm coming across in this issue like I'm very strongly opinionated about it, but I'm actually not.

I'm going to withdraw from this issue for the time being, but I'd like to end with this:
what's the point of having implicits and delegates be values, if we cannot use them as such? Every other alternative I could do in Haskell as well, and it does work there, but can we do anything differently or better?

P.S. If anyone knows the answer to my small question about foo[F[_]: Monad] given Thing, that would be appreciated :)

[OlivierBlanvillain]

P.S. If anyone knows the answer to my small question about foo[F[_]: Monad] given Thing, that would be appreciated :)

It works the way you expect :)

[SystemFw]

Thanks @OlivierBlanvillain :)

I think that's actually enough for me to have things work.