Graaljs: Inconsistent filesystem parsing of modules when commonjs is enabled

[pmlopes]

Assume you have a custom FileSystem implementation, when eval'ing the script:

import { hi } from 'verticle4';
hi();

Your custom implementation will receive a call to parsePath(String) with the value:

Full dump of calls to parsePath:

parsePath(verticle4)
parsePath(/home/paulo/.m2/repository/org/graalvm/regex/regex/20.1.0)
parsePath(/home/paulo/.m2/repository/org/graalvm/js/js/20.1.0)
parsePath(verticle4)
parsePath(verticle4)

As expected, as this is the module resource name on the script above.

However if you create your context with:

 final Map<String, String> options = new HashMap<>();
    options.put("js.commonjs-require", "true");
    builder
      .allowExperimentalOptions(true)
      .options(options);

And eval again the same script, (note that the script is esm, so this should have no impact), the following parsePath() calls are:

parsePath()
parsePath()
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/es4x/node_modules)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/node_modules)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/node_modules)
parsePath()
parsePath(/home/paulo/Projects/node_modules)
parsePath()
parsePath(/home/paulo/node_modules)
parsePath()
parsePath(/home/node_modules)
parsePath()
parsePath(//node_modules)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/es4x/node_modules/verticle4)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/es4x/node_modules/verticle4/package.json)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/node_modules/verticle4)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/es4x/node_modules/verticle4/package.json)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/node_modules/verticle4)
parsePath()
parsePath(/home/paulo/Projects/reactiverse/node_modules/verticle4/package.json)
parsePath()
parsePath(/home/paulo/Projects/node_modules/verticle4)
parsePath()
parsePath(/home/paulo/Projects/node_modules/verticle4/package.json)
parsePath()
parsePath(/home/paulo/node_modules/verticle4)
parsePath()
parsePath(/home/paulo/node_modules/verticle4/package.json)
parsePath()
parsePath(/home/node_modules/verticle4)
parsePath()
parsePath(/home/node_modules/verticle4/package.json)
parsePath()
parsePath(/node_modules/verticle4)
parsePath()
parsePath(/node_modules/verticle4/package.json)

In my specific case this fails as my FileSystem is implementing the classic typescript resolution model, when a module is not relative (meaning it doesn't start with ./, ../ or /) then if the system property baseUrl is defined it shall prefix the path given for parsing. So the file is never found in the later case as the path to parse is always relative.

[eleinadani]

Thanks for opening this issue @pmlopes,

with js.commonjs-require, the module loader looks for the 'verticle4' module using node.js' path resolution, which essentially tries to scan for node_modules folders and package.json. Of course, in your case the lookup mechanism will fail and never invoke parsePath in your virtual FS.

I think that we could change the way modules are loaded under js.commonjs-require by always invoking the customized FS if a module identifier is not found in a node_modules folder -- As a result, you would get the following invocations trace (for the example above):

parsePath(/home/paulo/Projects/reactiverse/es4x/es4x/node_modules)
parsePath(/home/paulo/Projects/reactiverse/es4x/node_modules)
parsePath(/home/paulo/Projects/reactiverse/node_modules)
// ... many more, and then:
parsePath(verticle4)

would this approach be OK for you?

[pmlopes]

Yes. That would work as at some point I'd get the name as in the import statements and would return the true location for it.

[eleinadani]

We added this feature in this commit and should land soon in our nightly build. Please let us know if this works for you!