Rescript-compiler: Use NPM package scope as top-level module and package name as nested module

[yawaramin]

Current Behaviour

The BuckleScript auto-generated top-level module (with "namespace": true) is derived from concatenating the NPM scope and package names, e.g. @foo/bar gets top-level module FooBar.

Expected Behaviour

A package scope and name should be used as the components of a two-level namespace, e.g. @foo/bar should result in a top-level module Foo, with a nested module Bar, with the package's module all contained inside Foo.Bar.

Justification

Semantically maps module names to NPM scopes and package names, keeping exposed module names self-contained and understandable. E.g.,

• @reasonml-community/bs-express could be renamed to @reasonml-community/express, resulting in module path ReasonmlCommunity.Express
• Encourages competition for the best bindings--folks could publish under their scopes and get nice module paths like Person1.Express and Person2.Express.

[bobzhang]

if this is the major use case how about given @scope/xx generate name Xx?

[yawaramin]

That could lead to more frequent name clashes, I fear. E.g. if I want to use @foo/utils and also @bar/utils, I'll get two Utils modules. The ideal is to keep the existing behaviour (i.e. I would get FooUtils and BarUtils) _except_ if the package author wants to override the default i.e. they know what they are doing, maybe they know that instead of FooUtils the top-level module should really be AmazingUtils. I.e. they are taking responsibility for potential name clashes.

In the case of the @reasonml-community scope, this is a bit of a special case because we're hoping to collect type declarations of NPM ecosystem packages under this scope, so it would be ideal to access the bindings with the same names as the NPM modules themselves, e.g. LeftPad.leftPad.

[bobzhang]

how about using short names, like @bs/xx or @re/hh, I am not a big fan of providing customizations without clear benefits. It introduces non-trivial cost since user always need to look up bsconfig.json to tell two names

[yawaramin]

Not sure I understand, are you proposing we get the NPM scopes @bs and @re? I don't know how scopes work exactly but AFAIK they're tied to GitHub org names?

Btw, agree with you that custom namespace might be a new headache for a user looking to use a lib. I don't have a good answer right now (I realise 'look up bsconfig.json is not a good answer), so maybe we should close this for now.

[TheSpyder]

npm scopes aren’t tied to GitHub; in the case of the standard npm registry they’re usernames on npmjs.com :)

bs and re are both taken, though. The only way to use them now is not publish to the standard npm registry.

[yawaramin]

Hm, actually, maybe the scope name should be the top-level namespace, with the package name one level below that. Scopes are obviously namespaces, now that I think about it...

[yawaramin]

@kMeillet @bobzhang @TheSpyder I've repurposed this issue into what I think is a better idea, FYI in case you want to upvote or downvote :-)

[bobzhang]

@yawaramin There is some technical challenges that the namespace is closed, you can not add new modules to existing namespace

[yawaramin]

@bobzhang oh, that's true. Sad :-(

But, how about this: separate the scope and package names with an underscore in the generated top-level module name, e.g.: ReasonmlCommunity_Express, Glennsl_BsJson, etc.

This provides better readability and separation between the scope and package names.

[ryb73]

@yawaramin This might be a tangential feature request, how about making the namespace name configurable? E.g. instead of namespace: true the config would be namespace: "ReasonmlCommunity_Express". That way it's up to the opinion of the programmer vs opinion of the framework

[yawaramin]

@ryb73 hi, this issue actually started out as exactly that feature request, but after discussing with Bob we agreed it would lead to more confusion and potential name clashes, see https://github.com/BuckleScript/bucklescript/issues/2405#issuecomment-354121144

[wokalski]

@yawaramin I see your concerns.

IMO, the clear benefit is that people don't want to have those weird namespaces so they don't namespace their libraries at all. An example from the top of my head is webapi which would have BsWebapiIncubator namespace. Same goes for bs-json. Also, I have a library which is scoped on npm and it'd have this weird WokalskiVow namespace.

I realise it's a "stupid" concern because you can simply alias a module but for some reason we don't do that. Also, let's suppose I advise people to alias the namespace. Wouldn't it beg the question, why you can't modify it? If that's the case, doesn't this make custom namespaces worthwhile?

EDIT: I think that for modules under scopes the namespace name is pretty obvious to infer. i.e. it's intuitive that @wokalski/vow will have Vow namespace.

[yawaramin]

@wokalski good question. We can of course turn off namespacing and use a custom module structure, like I'm doing with https://github.com/yawaramin/bs-webapi/tree/master/src to emulate the Scope_Packagename namespace (in my case Yawaramin_BsWebapi).

Yeah I would really recommend aliasing modules that you're going to use. The more successful Reason/OCaml become, the more we'll need proper scoping and namespacing. Ultimately I don't think we can escape from the same 'wall of imports' style that you see in other languages, we'll just maybe do it a bit better:

module Vow = Wokalski_Vow
module Web = Yawaramin_BsWebapi
module React = Facebook_React
(* and so on *)

[wokalski]

@yawaramin I'm not sure if I made myself clear. What you just wrote above reinforces my point. If people are going to alias (or even be advised to alias) modules anyway, there should probably be a way to modify the namespace.

[yawaramin]

@wokalski oh, OK. I was working off the assumption that modules are closed after definition. We can't add submodules after the fact. See https://github.com/BuckleScript/bucklescript/issues/2405#issuecomment-354996488

[wokalski]

@yawaramin I realise that. I'm referring to custom namespaces. i.e. "namespace": "Webapi" (in your bs-webapi case). IIUC it's not complex to implement.

[yawaramin]

@wokalski I think it would be:

{
  "namespace": "yawaramin",
  "name": "bs-webapi",
  ...
}

But I think I'm starting to get your point. So e.g., if I'm using the packages @facebook/reason-react and @facebook/foo, my local BuckleScript project should automatically create a top-level module for each unique scope, e.g. Facebook, and populate it with modules for each package in the scope, e.g. Facebook.ReasonReact and Facebook.Foo?

That could work...

[wokalski]

No. I mean a solution as simple as setting the namespace to the namespace field. Without taking anything else into account (i.e. without taking the scope into account). IMO name clashes are less important.

[alexfedoseev]

Current namespace implementation has one more con: user have to deal with at least 2 levels deep modules.

/* Current ReasonReact API */
ReasonReact.stringToElement("string");

/* Namespaced version */
ReasonReact.OneMoreLevel.stringToElement("string");

Quick thought about concept of the entry point of the package (somewhat similar to "main" field in package.json). E.g. in bsconfig.json author defines main module and only this module is exposed to outside world. And author decides what's the shape of the public module API: it's possible to include modules or just expose modules / values etc.

include Make;

/* or */
module Make = Make;

let thing = 42;

In case of single module package bsconfig.json just points to this single module.

[wokalski]

@alexfedoseev I think there's an issue for that. @glennsl has raised this issue on multiple occasions.

[glennsl]

Here you go: https://github.com/BuckleScript/bucklescript/issues/2215

[alexfedoseev]

@wokalski Oh I just landed here from discussion on Discord. Do you have a link (can't find it in this repo) so I can follow mentioned issue as well?

[alexfedoseev]

@glennsl Thanks! Yeah, this problem is really pumping 😁

[gilbert]

What's the status on this? Is custom namespaces a planned feature?

[yawaramin]

Closing this in favour of further discussion over at #2688