K6: Problem in importing chain of modules

[kkeranen]

As per your documentation one needs to add ".js" to paths when importing external modules.

I'm trying to import modules that try to import other modules (I explained this also in comment of another issue), and only way I can get this to work is to add ".js" into import statements in the whole import chain. However, this is very hard for two reasons: 1) some modules are 3rd party modules and 2) our modules are compiled to JavaScript from TypeScript.

Am I doing something wrong, or is using of chained modules (test script importing module1, then module1 importing module 2 and so on) really so hard? It's so hard that I have not been able to complete the work yet and I've started to consider alternatives to k6.

[na--]

You don't need to append .js at the end of a file, for example import { some_func } from './module'; where module is a file that has export function some_func() { /* ... */ } works.

I think the issue here may be because k6 doesn't read the index.js if you try to import something from a folder. I'm struggling to find where that's defined in some sort of official ES modules spec, but without much luck. Looking at this stackoverflow answer, it seems like it may actually be officially undefined, but de-facto standard accepted behavior.

Seems like it should be easy enough to implement if we want to support the "mainstream" behavior of reading the index.js file if users import a folder... Though I don't think supporting node's next step (Parse X/package.json, and look for "main" field.) would be necessary at this point...

@robingustafsson, thoughts? @kkeranen, since I'm not terribly familiar with the whole huge JS ecosystem, any corrections, examples or documentation links related to this would be much appreciated!

[na--]

Related/duplicate issue: https://github.com/loadimpact/k6/issues/475

[kkeranen]

Thank you @na for reply. However, I don't see that this is related to index.js. Let me explain what I'm trying to do. We are using libraries originally written in TypeScript and then compiled to JavaScript. So, one of the simple cases was using guid-typescript npm package and importing it to our main script.js by
import { Guid } from "guid-typescript";

This didn't work. Got runtime error:

The file "guid-typescript" couldn't be found on local disk ...

I got this working by changing that line to:
import { Guid } from "../node_modules/guid-typescript/dist/guid.js";

Note that if I remove ".js" from that line, it doesn't work anymore, so this is not working:
import { Guid } from "../node_modules/guid-typescript/dist/guid";

I can't apply that kind of fix to all other cases that are more complex, containing imports in chains.

[na--]

Hmm that should still be mostly doable. When I look at node's documentation on module loading, if X is not a file, it tries to load X.js next. We need to account for the k6 support for remote modules (GH issue, docs) that allows users to import scripts from HTTPS URLs, but supporting points 1-3 from node's module loading algorithm should be doable with the following caveats:

• we can't support binary node modules (i.e. points If X.node is a file, load X.node as binary addon. and If X/index.node is a file, load X/index.node as binary addon.)
• not sure if we should support package.json parsing at this point - it probably wouldn't be too complicated to do, but it may lead users to believe we directly support node modules in k6, which we still wont
• as mentioned, we still won't actually support most node.js libraries (since a lot of them will depend on node core modules that we don't have in k6) and the few node.js libraries that we could support would still have to pass through browserify first...
• support for point 4 in that algorithm would be a lot more tricky and potentially error-prone, both for implementation by us (especially in combination with the current support for remote modules) and in the sense that it could trick users we support node libraries when we don't... so I think if we ever have support for it, it would be at a later time than points 1-3
• we have to make sure that our error messages when users try to import a node.js core module (i.e. something like import { readFile } from 'fs';) are very explanatory to reduce the amount of user confusion...

Basically the big challenge here is to support a lot of the npm dependency management and resolution functionality in combination with k6-specific features like importing scripts from URL, and without actually confusing users or fully supporting node.js libraries...

[na--]

@kkeranen, as a workaround until we improve the k6 module loader to support some of the module resolution used by node.js, can you try to use some project that bundles script dependencies in a single file and use that? As I said, I'm not very familiar with the whole JS ecosystem, but IIRC browserify does that and there seem to be other projects that just merge modules like this and this.

[kkeranen]

The merge module tools you linked seem to combine nested package.json files into one merged package.json, so I don't see how they could help.

Then the k6 requirement to use .js ending in local import paths has been verified by others. For example, see liclac's second comment.

[kkeranen]

I've tried to strip down all dependencies to progress in my work, but there are still some basic things like creating GUID that I can't use in easy way because import chain doesn't work.

[micsjo]

You need a GUID or an UUID4? If you need a UUID4 here you go:

/**
 * Fast UUID generator, RFC4122 version 4 compliant.
 * @author Jeff Ward (jcward.com).
 * @license MIT license
 * @link http://stackoverflow.com/questions/105034/how-to-create-a-guid-uuid-in-javascript/21963136#21963136
 **/
var UUID = (function() {
    var self = {};
    var lut = []; for (var i=0; i<256; i++) { lut[i] = (i<16?'0':'')+(i).toString(16); }
    self.generate = function() {
      var d0 = Math.random()*0xffffffff|0;
      var d1 = Math.random()*0xffffffff|0;
      var d2 = Math.random()*0xffffffff|0;
      var d3 = Math.random()*0xffffffff|0;
      return lut[d0&0xff]+lut[d0>>8&0xff]+lut[d0>>16&0xff]+lut[d0>>24&0xff]+'-'+
        lut[d1&0xff]+lut[d1>>8&0xff]+'-'+lut[d1>>16&0x0f|0x40]+lut[d1>>24&0xff]+'-'+
        lut[d2&0x3f|0x80]+lut[d2>>8&0xff]+'-'+lut[d2>>16&0xff]+lut[d2>>24&0xff]+
        lut[d3&0xff]+lut[d3>>8&0xff]+lut[d3>>16&0xff]+lut[d3>>24&0xff];
    }
    return self;
  })();

[ppcano]

in combination with k6-specific features like importing scripts from URL

@na-- , @MStoykov

This is a nice feature but I think JS developers are not expecting this API.

The dynamic import proposal is the most similar to remote modules that I have found on the JS ecosystem and will likely be used by Node.

To me, I would be totally fine with dropping the support of this feature if it provides a benefit to the future of the k6 module resolution system.

[na--]

As I mentioned in https://github.com/loadimpact/k6/issues/475#issuecomment-764746931, we're very unlikely to adopt the node module resolution algorithm in k6. If someone needs nested NodeJS modules with k6, using webpack (or any of the other bundlers) to resolve and bundle them should be very easy: https://github.com/k6io/template-es6

Dynamic import() is a bit trickier, since we don't really have async/await (https://github.com/loadimpact/k6/issues/779) or even event loops (https://github.com/loadimpact/k6/issues/882) yet. We'd probably like to support it eventually, but we're a long ways off... :disappointed: However, as seen in the k6 compatibility mode docs, if someone needs to more dynamically load JS files now, k6 has a require(). It just doesn't follow the NodeJS module resolution algorithm...