Chapel: Visibility of "tertiary" methods?

[vasslitvinov]

A method is always visible if it is public and is declared in the lexical scope where the type is declared, according to our visibility rules. This includes both primary and secondary methods. The rationale is that those are made by the type author, so they are "part of the type".

How about "tertiary" methods, i.e. secondary methods declared outside of the type definition's module, "after the fact," so to speak? For example:

module Rdef {
  record R {}
  proc R.secondary() { }
}
module Rter {
  use Rdef;
  proc R.tertiary() { }
}
/* This module provides an enhanced version of 'R'. */
module Rmake {
  use Rdef, Rter;
  proc makeR() {
    return new R(); 
  }
}
use Rmake;
makeR().secondary();  // always visible
makeR().tertiary();   // also visible?

The motivation for allowing makeR().tertiary() is that the author of makeR explicitly wants to make tertiary() available to its users.

The primary challenge with allowing this idiom is its use in third-party libraries, for example:

module Library {
  proc drawme(arg) {
    writeln(arg.secondary());
    writeln(arg.tertiary());
  }
}

module User {
  use Rmake, Library;
  drawme(makeR());
}

Another challenge is mixing R instances from different sources, ex.:

module Rmake2 {
  .... R and a different proc R.tertiary() are visible ....
  proc makeR2() return new R();
}

module User {
  use Rmake, Rmake2, Library;
  drawme(if condition() them makeR() else makeR2());
}

The leading solutions (exluding the "mixing" scenario) are:

(WR) Have makeR() return a wrapper record that defines tertiary() and forwards the rest to R. (TODO: review visibility of forwarded methods.) Cons: lost type equality between the result of _makeR()_ and _R_.

(CG) Have drawme() use constrained generics, requiring tertiary() on its argument. Cons?

[bradcray]

The motivation for allowing makeR().tertiary() is that the author of makeR explicitly wants to make tertiary() available to its users.

If tertiary is the enhancement to R, what is the rationale for why makeR is in a different module from Rter?

Cons: lost type equality between the result of makeR() and R.

I think this type equality only really matters if R is intended to be a visible rather than opaque type; and if it's intended to be a visible type, then I would expect Rter and/or Rmake to public use the modules defining R and R.tertiary rather than privately use-ing them.

[vasslitvinov]

if it [R] is intended to be a visible type, then I would expect Rter and/or Rmake to public use the modules defining R and R.tertiary rather than privately use-ing them

Sure. However, if this solution is available, then we do not need the rule for visibility of secondary methods either.

The fact that we require the additional visibility "hook" for secondary methods tells me that there are situations in which this solution is not available.

I believe those situations arise when instances of R are passed around before we invoke a drawme() on them. In those situations secondary and tertiary methods are equally "in trouble", visibility-wise. For example:

module Cdef {
  use Rmake;
  class C {
    var f = makeR();
  }
}

module Cmake {
  use Cdef;
  proc makeC() return new C();
}

module Cuse {
  use Cmake;
  makeC().f.secondary();
  makeC().f.tertiary();
  // or even something like draw(makeC())
  // that invokes arg.f.secondary/tertiary()
}

In these situations it does not matter whether R is opaque. Also it does not matter whether R.tertiary() is defined in module Rter then used into Rmake or is defined in module Rmake directly.

[mppf]

Sure. However, if this solution is available, then we do not need the rule for visibility of secondary methods either.

The fact that we require the additional visibility "hook" for secondary methods tells me that there are situations in which this solution is not available.

Something about this isn't making sense to me. I thought that in the context of this discussion, "secondary method" means only those methods defined in the module defining the type when those methods are defined outside of the type (as in record R { } proc R.secondary() { }). The reason that "secondary method" and "primary method" are treated the same is that the choice of defining the method inside of record R or not is one we leave up to preference of the type author without impacting program behavior.

We have been talking about what use M only R means for a type like record R defined in a module M. Or use N only R when N adds tertiary methods. It's my understanding that in both cases it would make R itself available as well as any methods defined in that module.

[bradcray]

The fact that we require the additional visibility "hook" for secondary methods

What "additional visibility hook" are you referring to here?

[vasslitvinov]

The fact that we require the additional visibility "hook" for secondary methods tells me that there are situations in which this solution is not available.

By "visibility hook" I mean the rule that, upon resolving a method call, visible candidates include the methods visible in the scope of the type definition.

We added this rule because in some scenarios we found it impossible - or perhaps impractical - to get to those methods using use declarations.

This issue anticipates users wanting to access "tertiary" methods in similar scenarios.

[bradcray]

This issue anticipates users wanting to access "tertiary" methods in similar scenarios.

Right, so I think our current thinking is that the author of a tertiary method can't impose their methods on other users without those users opting into them by use-ing the module that defines them, either directly or indirectly (via a public use). This seems both reasonable to me ("that capability wasn't part of the original type's definition") and safe (in the sense that it prevents a bad actor to add new capabilities to a type without a client of that type opting into it by using their module, again directly or indirectly). As mentioned before, an alternative for someone who truly wants to extend a type while keeping everything about the type name opaque is for that person to create their own type that leverages the original type (e.g., by wrapping it) rather than secretly extending the type without a client being aware of it. And if they don't need to keep it opaque, then changing most of the use statements in your original example into public use statements would permit the modules to work as desired.

To me, this all seems like a sound, reasonable starting point for us to use today, and I'd be inclined to wait for users that want something more to express that argument and why they think it's reasonable rather than trying to anticipate everything that a user may want.