Rescript-compiler: Namespace discussion

[chenglou]

I've collected some feedbacks and thoughts regarding our namespace solution so far:

• Despite the fact that the namespace option has been turned on by default in the bsb templates, it seems that every few weeks there's a new blog post coming out that warns of global file clashes and/or say that we don't have a namespacing system. It's a somewhat recurrent topic in Discord as well.
• We don't seem to have a plan to transition to "namespace": true by default in bsb in the medium term.
• There's also the issue that folks really want to put bindings into the top-level module, e.g. MyLibrary.foo, but our current namespace system doesn't allow it.
• The lack of the latter feature also means that folks can't transition toward using the namespace feature in in bsconfig.json as easily. This is a feature in rather high demand (cc @rickyvetter, @jaredly)
• Even though it's well-documented, it seems that folks also get confused that namespacing is for third-party consumption and doesn't affect the current library.
• There are some small hard-to-repro bugs regarding namespace staleness.
• Confusions around the name of the module vs package.json name vs bsconfig.json name vs npm scoped name.

I'm creating this issue because I feel that we should have a cleaner experience regarding namespacing. As an alternative, bsconfig also happens to have the feature sources.public. I'm wondering if we can tweak that as an alternative to namespace. Usage:

{
  "name": "my-app",
  "sources": {
    "dir" : "src",
    "public": ["MyApp"]
  }
}

In src/MyApp.re:

let a = 1;
module Utils = Utils;

Usage in another project:

{
  "name": "your-app",
  "bs-dependencies": ["my-app"]
}

let myValue = MyApp.a; /* 1 */
let log = MyApp.Utils.log;

This currently fails to compile because Utils from my-app project is hidden, although it's aliased from MyApp.re.

This is apparently not trivial to implement. The benefits is that you get top-level bindings for free, and you get to expose more than one thing. Additionally, this by itself solves all of the above pitfalls of our current solution.

[stroiman]

I've experienced two issues with the namespacing.

• Scoped packages gets awkward package names.
• If same package is implicitly imported multiple times, the versions must be compatible.

Scoped package names

Package @ScopeName/PackageName is imported as ScopeNamePackageName, which becomes quite awkward. More natural package names would be:

• PackageName IMHO most natural, but at risk of namespace collisions with other packages.
• ScopeName.Package.Name

This awkward namespace name is also the reason I didn't enable namespaces for my published scoped packages.

Multiple imports of same package

I have package @stroiman/respect which depends on package @stroiman/async for helping managing async code. This is purely an implementation detail - there is nothing in the public interface of respect that refers to async.

If I use respect from my application, and my application also depends on async, but in an incompatible version - I am unable to compile.

In pure node.js, this sort of dependency would not be a problem, as NPM itself happily installs both versions:

myapp/node_modules/@stroiman/async
myapp/node_modules/@stroiman/respect/node_modules/@stroiman/async

Suggestion

If you could overwrite the root namespace on import, I think that could solve both issues (namespace collision of similarly named packages, and having multiple incompatible versions)

The package name of the imported package would merely be a _default_ namespace, but could be overwritten in the bs-dependencies/bs-dev-dependencies section.

"name": "@stroiman/respect",
"namespace": true,
"bs-dependencies": [
  { "name": "@stroiman/async",
    "root-namespace": "Imports.Async" 
    // The root-namespace should be prefixed with this package's name, to
    // respect any overwrites by whoever is importing this package.
  }

This way, the two versions of async, located in two different folders on disc, are compiled into two separate root namespaces.

[jaredly]

Here's an idea for how to implement it. Currently, we generate a "top level module" from the .mlmap file, which just aliases all of the contained modules. In BsReactNative, it's called BsReactNative.cm[ijt], and contains something similar to this:

module Geolocation = BsReactNative-Geolocation
module Dimensions = BsReactNative-Dimensions
... etc

(it's generated at the AST level, so it's OK that that code wouldn't parse).
This map file is then opened at the top of every module, so that they can access the other modules by normal names.
This map file is then the "public api" for the module, and users to BsReactNative.Geolocation etc.

I propose that we name the file BsReactNative-map.cmi (or similar), and then have the "main module" that we want to be exposed, instead of being named BsReactNative-Main.cmi, just BsReactNative.cmi. Then clients can just use that as the public API.
Thoughts @chenglou @bobzhang?

[bobzhang]

relevant to #2215

[cullophid]

This would be a great improvement.

[yawaramin]

Has there been any recent development on namespacing? I've been doing it manually on a couple of projects, but it would be nice if the npm scope auto-generated a top-level module for each scope in the _consuming_ project, like https://github.com/BuckleScript/bucklescript/issues/2405#issuecomment-368902124 :-)

It might be helpful to see how Dune is doing it. IIUC, it copies all the source files into the _build directory (the equivalent of lib) and renames them as necessary. For example, if we have a project name foo, and file src/Bar.re, it would get copied to (e.g.) lib/bs/src/Foo__Bar.re (I think we would have to use two underscores as module path separators because some people might be using single-underscore as part of a module name).

If we also have a project scope @ns, the file would get copied to lib/bs/src/Ns__Foo__Bar.re before the build. And then we could auto-gen (as mentioned above) the files lib/bs/src/Ns.re with a module alias module Foo = Ns__Foo; and lib/bs/src/Ns__Foo.re with module Bar = Ns__Foo__Bar;. Just to reiterare, these transformations would happen in the _consuming_ package's build. As usual, we would .gitignore anything in lib.

Essentially, the usual src/ directory would just become a _template_ of the actual source directory. There would be other considerations with this, like what happens to in-source builds and what to name the generated JS files.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.