Chapel: [design] import statement, a la Rust

[lydia-duncan]

This issue is a counterproposal to #13831, based on behavior in Rust.

What Rust Does

For the following examples, M is a module that contains another module N, which contains a symbol x and a symbol y.

Qualified Example

To access x in a qualified manner, there are a couple of strategies. The main one is:

use crate::M::N;

We’ll talk about the other strategy later

Unqualified Example

To access x in an unqualified manner, there are three strategies. The first of these strategies uses the syntax we saw earlier, for when you only want to access x and not any other symbol in N

use crate::M::N::x;

Rust allows non-modules to be used at the end of a use statement precisely for this purpose.

If you want all the symbols in a module to be usable in an unqualified manner, you can use *, e.g.

use crate::M::N::*;

If you only want some of the symbols to be available in this way, you can specify a list of symbols like this:

use crate::M::N::{x, y};

This is where the second strategy for enabling qualified access comes in. All of the above strategies for unqualified access only enable unqualified access; they don’t enable qualified access. However, in the last option you can add a reference to the module which contains those symbols using self, e.g.

use crate::M::N::{x, y, self};

This enables accessing x and y both in a qualified and an unqualified fashion, the same behavior as supported by Chapel's use statements today.

Module Paths

Module paths in Rust as used by use statements are usually full paths and must start with a crate (where crate is used for the current crate). E.g.

pub mod M {
  pub mod N {
     pub const x: i32 = 3;
  }
  pub mod Other {
    pub fn foo() {
        use crate::M::N;
        println!("{}", N::x);
    }
  }
}
fn main() {
    use crate::M::Other::foo;
    foo();
}

Even though Other is a sibling module to N, you can't just write use N;, you have to have something to indicate where to find N. This doesn't have to be the full path, though, it can be relative by specifying either self or super. E.g.

pub mod M {
  pub mod N {
     pub const x: i32 = 3;
  }
  pub mod Other {
    pub mod N {
       pub const x: i32 = 3;
    }
    pub fn foo() {
        use super::N; // When N is a sibling module
        use self::N::x; // When N is a submodule
        println!("{} {}", N::x, x);
    }
  }
}
fn main() {
    use crate::M::Other::foo;
    foo();
}

Re-Exports

Rust's use statements are private by default and can be declared public using the keyword pub, which allows them to be accessed via another use in the same manner as was made available. E.g.

pub mod M {
  pub mod N {
     pub const x: i32 = 3;
  }
  pub mod Other {
    pub use super::N::x; // Enables unqualified access to x in Other and Other::x
    pub use super::N; // Enables qualified access to x in Other, and Other::N::x
    pub fn foo() {
        println!("{}", x);
    }
  }
}
fn main() {
    use crate::M::Other;
    Other::foo();
    println!("{} {}", Other::x, Other::N::x);
}

The Proposed Chapel Strategy

Similarly to Rust’s use statements, the Chapel import statements will trade off between specifying qualified or unqualified access. When no only list is present, the symbols of the module specified will be made available via qualified access. When an only list is present, the symbols of the module will only be made available via unqualified access.

Here's an example for accessing a module's symbols in a qualified manner:

module M {
  var x: int;
  var y: bool;
}
module User {
  proc main() {
    import M;
    writeln(M.x); // writeln(x) would not work because only M is brought it, not its symbols
  }
}

Here's an example for accessing M's symbols in an unqualified manner only:

// same definition of M
module User {
  proc main() {
     import M only x, y;
     writeln(x);
     writeln(y);
     // writeln(M.x); // does not work because M itself is not brought it
  }
}

Key differences from #13831

• Enables import to bring in unqualified access to more than just a module itself.
• Drops except
  
  
  
  • This means that users have to explicitly list every symbol they want included for unqualified access. We probably should enable * to be used in the only list so that full unqualified access is not a burden
  
  
• only list applies to unqualified access instead of limiting what qualified access is available
  
  
  
  • This gives the user less control over qualified access. However, it's entirely possible that most users won't want to limit what can be accessed in a qualified manner, so maybe it won't be missed.
  
  
  • It also might be confusing to have the presence of the only list impact whether we are enabling qualified or unqualified access, when that is not the case for use statements.
  
  

Similarities to #13831

• Still enables as for renaming a symbol
• Still enables public and private imports

Open questions

In addition to the ones raised by #13831, the following are specific to this strategy

• Rust has a notion of self within {}. We could allow this in only lists if we wished to allow specifying both qualified and unqualified access to a module's symbols in a single statement. However, the use of this keyword would maybe lead to confusion about how renaming a symbol impacts its qualified access (as pointed out in this comment) which not enabling the use of self would avoid.
• Currently we support naming a module relative to your current scope in addition to from the top level. However, Rust requires either a super or a self keyword to do so, otherwise everything has to be specified from the top level. Should we follow that precedent here?

[mppf]

Since this proposal is inspired by Rust module system's rules but not its exact syntax, I would expect it to use this instead of self because that is the corresponding keyword in Chapel AFAIK.

[BryantLam]

I'm okay with this proposal. It all sounds reasonable.

It also might be confusing to have the presence of the only list impact whether we are enabling qualified or unqualified access, when that is not the case for use statements.

This isn't an issue to new users that are taught to only learn import statements.

Is re-exporting the contents of M implemented as public import M only *? If so, that's clever!

Regarding only-self/this: an interim compromise might be to allow import M only {this, x, y} if there are no renames. Or possibly accept the potential for user confusion on renames like Rust does.

Regarding relative-scope module naming: I would not want to specify super or self all the time, provided:

1. the syntax form is available should a user prefer using that style, and
2. the rules are extremely clear regarding what module is picked when ambiguities occur, preferably defaulting to not allowing an ambiguously named module (i.e., compilation error for ambiguity between same-scope module, submodule, or top-level module). The rules should synchronize with #13524.

I think I would prefer:

• to always look down a relative point in the module tree (only see submodules)
• supermodules should never be reachable without super.
• like supermodules, a top-level package should only be reachable with some top-level keyword like package (notional e.g., import package std.BlockDist, std.CyclicDist, std.{List, Set})

Otherwise, some of the error conditions would be:

• submodule is found today, but someday in the future, a conflicting top-level package is added with the same name and now your program doesn't compile or is hijacked
• top-level module is found today, but someday in the future, a conflicting submodule is added with the same name
  
  
  
  • This would be more likely in the case of a code refactoring where an older version of the code is inadvertently chosen because it was at same or top-level scope versus a refactoring effort to move it into a submodule.
  
  

Historically, absolute-path module naming was added to Rust because the rules for when a module was picked via relative-from-here vs. top-level were confusing to a lot of users. I'd like to avoid that problem if absolute-path module naming is not enforced in Chapel.

[mppf]

Regarding only-self/this: an interim compromise might be to allow import M only {this, x, y} if there are no renames. Or possibly accept the potential for user confusion on renames like Rust does.

What potential for confusion? Are you bringing up the fact that in Chapel this might be a method name?

[BryantLam]

@mppf -- No. For background, I suggested a behavior in https://github.com/chapel-lang/chapel/issues/10799#issuecomment-562376157 that later took back and proposed the opposite in https://github.com/chapel-lang/chapel/issues/14738#issuecomment-573251368 with @lydia-duncan's response in https://github.com/chapel-lang/chapel/issues/14738#issuecomment-575271738.

The potential for confusion is my originally suggested behavior that I was proposing but have now rescinded.

To be more clear, #10799 was to figure out the behavior of variable renaming inside a module on a use statement. I originally suggesting something like:

use M only x as x2;
writeln(x2);
//writeln(M.x); // Error. Illegal because it was renamed.
writeln(M.x2);

but realized that you can't do that if you take the only logical choice of figuring out how to decompose use statements into their equivalent import statements. That behavior would suggest that there needs to be an import statement that can rename the contents of a module, which is admittedly less useful and more confusing than it is worth. Part of the confusion that Lydia mentioned and I reaffirmed is whether a user would find this behavior confusing (like I mistakenly believed):

use M only x as x2;
writeln(x2);
writeln(M.x); // same result as x2
//writeln(M.x2); // Error. M.x2 doesn't exist.

if use statements were to be equivalent to:

import M only this, x as x2;

[bradcray]

Recapping a bit of what I've said on other issues / emails:

The more I think about import designs, the more I'm inclined to go with an approach like this one, though using syntax that's a bit closer to Rust's, where some sample import statements (not intended to necessarily work together in a single program) might look like:

import M;
import myLongModuleName as M;
import M.N;
import M.{x, y, z};
import M.{x as Mx, y as My, z as Mz};
import M.{x, y, z, this};
import M.this;  // presumably?

I'm also advocating that we not support import M.*; or import M except x; in order to have the import statement be as explicit as possible about what's being made available within scope. Those who want to be less precise should use the use statement instead.

[BryantLam]

What's the proposed behavior for import M.this? Would it be equivalent to import M? I'm generally not a fan of alternative syntax and would prefer if import M.this wasn't valid because it reads somewhat confusingly when this is used on its own like that.

An equivalent to import M.{this, x, y, z} would be import M, M.x, M.y, M.z.

[mppf]

@BryantLam - would you want import M.{x, y, z, this}; to be allowed? This is just trying to correspond to the Rust feature.

Leaving out the this from the implementation to start seems fine to me - since one could write import M.{x, y, z}; import M; import M, M.x, M.y, M.z as you said.

[BryantLam]

I'd be okay without it. My opinion is that the intent is more clear if it was written as import M, M.{x, y, z} anyway (this statement and other examples should be in the documentation!).

It can also be added later if users want it. It's likely better to add it after #14904 is resolved.

[lydia-duncan]

Personally I don't see the need to support import M.this - I'd implement import M.{x, this}, but the form with only .this is just redundant with import M. I don't think anyone would want to write it in practice

[bradcray]

I agree import M.this; is redundant. I only added it at the last minute for orthogonality. It seems to me that anything you can put within the list using curly brackets should be able to be named as a singleton without the curly brackets as well. I'm also fine with skipping this in the initial draft and adding it later (or never if nobody misses it).

[lydia-duncan]

I think at this point I can say that we're going to follow the Rust proposal for import statements, replacing only list syntax with more Rust-like .{} lists to avoid the clarity issue Greg pointed out. This means that:

• import M; will enable M.x (aka qualified access to M's symbols)
• import M.x; will enable x (aka unqualified access to the symbol x defined in M)
• import M.{x, y}; will enable x and y (aka unqualified access to the symbols x and y defined in M)
• import M.N.O; will enable O.x (aka qualified access to O's symbols without always having to use M.N.O at the front)
• import M as N; will enable N.x (aka qualified access to all of M's symbols but using the name N)
• import M.x as mx; will enable mx (aka unqualified access to x but using the name mx
• import M.{x as mx, y}; (aka unqualified access to M.x and M.y, calling them mx and y respectively)

It sounds as though we will not support import M.*; for the moment (this would mean unqualified access to all public symbols in M if we were to implement it). We may revisit this later if users request it, but will encourage use statements in its place.

We can support things like import M.{x, y, this}; to enable x, y, and M.<all public symbols in M> (aka unqualified access to x and y, and qualified access to M's symbols), but will wait to do so until there is user demand.

We're going to wait to decide on relative imports of nested modules (e.g. importing M.N from within M) based on discussion around submodules-as-files.

For re-exporting symbols, the general consensus is to use public import as opposed to alternate syntax or strategies.