Typescript: External module resolution logic

[vladima]

Problem

Current module resolution logic is roughly based on Node module loading logic however not all aspects of Node specific module loading were implemented. Also this approach does not really play well with scenarios like RequireJS\ES6 style module loading where resolution of relative files names is performed deterministically using the base url without needing the folder walk. Also current process does not allow user to specify extra locations for module resolution.

Proposal

Instead of using one hybrid way to resolve modules, have two implementations, one for out-of-browser workflows (i.e Node) and one for in-browser versions (ES6). These implementations should closely mimic its runtime counterparts to avoid runtime failures when design time module resolution succeeded and vice versa.

Node Resolution Algorithm

Resolution logic should use the following algorithm (originally taken from Modules all toghether):

_require(X) from module at path Y_

If exists ambient external module named X {
  return the ambient external module 
}
else if X begins with './' or '../' or it is rooted path {
  try LOAD_AS_FILE(Y + X, loadOnlyDts=false)
  try LOAD_AS_DIRECTORY(Y + X, loadOnlyDts=false)
}
else {
  LOAD_NODE_MODULES(X, dirname(Y))
}
THROW "not found"

function LOAD_AS_FILE(X, loadOnlyDts) {
  if loadOnlyDts then load X.d.ts 
  else { 
    if  X.ts is a file, load X.ts
    else if X.tsx is a file, load X.tsx
    else If X.d.ts is a file, load X.d.ts
  }
}

function LOAD_AS_DIRECTORY(X, loadOnlyDts) {
  If X/package.json is a file {
    Parse X/package.json, and look for "typings" field.
    if parsed json has field "typings": 
    let M = X + (json "typings" field)
    LOAD_AS_FILE(M, loadOnlyDts).
  }
  LOAD_AS_FILE(X/index, loadOnlyDts)
}

function LOAD_NODE_MODULES(X, START) {
  let DIRS=NODE_MODULES_PATHS(START)
  for each DIR in DIRS {
    LOAD_AS_FILE(DIR/X, loadOnlyDts=true)
    LOAD_AS_DIRECTORY(DIR/X, loadOnlyDts=true)
  }
}

function NODE_MODULES_PATHS(START) {
  let PARTS = path split(START)
  let I = count of PARTS - 1
  let DIRS = []
  while I >= 0 {
    if PARTS[I] = "node_modules" CONTINUE
    DIR = path join(PARTS[0 .. I] + "node_modules")
    DIRS = DIRS + DIR
    let I = I - 1
  }
  return DIRS
}

RequireJS/ES6 module loader

• If module name starts with './' - then name is relative to the file that imports module or calls require.
• If module name is a relative path (i.e. 'a/b/c') - it is resolved using the base folder.

Base folder can be either specified explicitly via command line option or can be inferred:

• if compiler can uses 'tsconfig.json' to determine files and compilation options then location of 'tsconfig.json' is the base folder
• otherwise base folder is common subpath for all explicitly provided files

Path mappings can be used to customize module resolution process. In 'package.json' these mappings can be represented as JSON object with a following structure:

  {
    "*.ts":"project/ts/*.ts",
    "annotations": "/common/core/annotations"
  }

Property name represents a pattern that might contain zero or one asterisk (which acts as a capture group). Property value represents a substitution that might contain zero or one asterisk - here it marks the location where captured content will be spliced. For example mapping above for a path 'assert.ts' will produce a string 'project/ts/assert.ts'. Effectively this logic is the same with the implementation of locate function in System.js.

With path mappings in mind module resolution can be described as:

for (var path in [relative_path, relative_path + '.ts', relative_path + "d.ts"]) {
    var mappedPath = apply_path_mapping(path);
    var candidatePath = isPathRooted(mappedPath) ? mappedPath : combine(baseFolder, mappedPath);
    if (fileExists(candidatePath)) {
        return candidatePath
    }
}
return undefined

With path mappings it becomes trivial to resolve some module names to files located on network share or some location on the disk outside the project folder.

{
    "*.ts": "project/scripts/*.ts",
    "shared/*": "q:/shared/*.ts"
}

Using this mapping relative path 'shared/core' will be mapped to absolute path 'q:/shared/core.ts'.

We can apply the same resolution rules for both modules and tripleslash references though for the latter onces its is not strictly necessary since they do not implact runtime in any way.

[vladima]

@mprobst and @jrieken - can you please check if this proposal covers your scenarios?

[danquirk]

Need to cross reference with comments here too https://github.com/Microsoft/TypeScript/issues/247

[mprobst]

One question:
"2. If X begins with './' or '/' or '../' [...]"

What does it mean when X is e.g. '/foo/bar' and you calculate Y + X? Do you resolve X against Y, e.g. like a browser would resolve a URL, so that '/foo/bar' would result in an absolute path?

Regarding the path mapping, this approach would not quite solve our problem. What I want to express is that a source file should first be searched in location A, then in location B, then C, and so on. That should be true for _all_ source files, not just for a specific subset (as you do with the pattern matching). The source code of the including file should not care where the included file is located.

I presume the harder problem is establishing what the path that is searched in those locations is exactly. If a file is loaded relative to a base URL, we could first resolve all paths relative to that file's relative URL to the base, and then use the result to look up in the search paths.

Given a file Y, a require('X'):

1. let path be resolve(Y, X). Question here: default to relative paths or default to absolutes?
2. for each include path i (passed on command line, current working directory is the implicit first entry)
   
   
   
   1. let effective path be resolve(i, path)
   
   
   2. if effective path exists, return effective path
   
   
   3. continue
   
   
3. throw not found.

The initially passed 'Y' from the command line would also be resolved against the include/search paths.

For example, for a file lib/a.ts and running tsc -I first/path -I /second/path lib/a.ts in a path /c/w/d, where a.ts contains a 'require("other/b")', the locations searched for b would be, assuming default to absolute paths:

1. /c/w/d/other/b.ts (+.d.ts, +maybe /index.ts and /index.d.ts)
2. /c/w/d/first/path/other/b.ts
3. /second/path/other/b.ts

If you specified ./other/b, the locations searched would be:

1. /c/w/d/lib/other/b.ts (+.d.ts, +maybe /index.ts and /index.d.ts)
2. /c/w/d/first/path/lib/other/b.ts
3. /second/path/lib/other/b.ts

This would allow us to "overlay" the working directory of the user over an arbitrary number of include paths. I think this is essentially the same as e.g. C++ -I works, Java's classpath, how the Python system search path works, Ruby's $LOAD_PATH etc.

[mprobst]

... oh and obviously, I mean this as a suggestion to be incorporated into your more complete design that also handles node modules etc.

[vladima]

What does it mean when X is e.g. '/foo/bar'?

Yes, module name that starts with '/' is an absolute path to the file

I think path mappings can solve the problem if we allow one entry of it to be mapped to the set of locations

{
    "*": [ "first/path/*", "/second/path/*" ] 
}

Having this update module resolution process will look like:

var moduleName;
if (moduleName.startsWith(../) || moduleName.startsWith('../')) {
    // module name is relative to the file that calls require
   return makeAbsolutePath(currentFilePath, moduleName); 
}
else {
    for(var path of [ moduleName, moduleName + '.ts', moduleName + '.d.ts']) {
        var mappedPaths = applyPathMapping(path);
        for(var mappedPath in mappedPaths) {
            var candidate = isPathRooted(mappedPath) ? mappedPath : makeAbsolute(baseFolder, mappedPath);
            if (fileExists(candidate)) {
                  return candidate;
            }
        }
    }
}
throw PathNotFound;

Note:
I do see a certain value of having a path mappings, since it allows with a reasonably low cost easily express things like: part of files that I'm using are outside of my repository so I'd like to load the from some another location. However if it turns out that all use-cases that we have involve remapping of all files in project and path mappings degrade to just include directories - then let's use include directories.

Out of curiosity, do you have many cases when code like require('../../module') should be resolved in include directories and not in relatively to file that contains this call?

[mprobst]

Re the code example, I think even relative paths should be resolved against the mapped paths. Imagine you have a part of your code base in a different physical location (repository), but still use the conceptually relative path. In general, I think it might be a good idea to have a logical level of paths that get resolved, and then those are matched against physical locations, but the two concepts are orthogonal otherwise - that is, you can have relative or absolute logical paths mapping to any physical location.

Our particular use case is that we currently exclusively use absolute rooted include paths (require('my/module') where my is resolved to the logical root of the source repository). Relative paths could be useful if you have deep directory structures, but would need to be clearly marked so that there's no ambiguity, e.g. by using ./relative/path.

I see that your example of path mappings is strictly more powerful, but at least from where I stand, I think include directories cover all we need, and might be simpler for tooling to understand & implement. YMMV.

[basarat]

Would be great if typescript.definition was supported. https://github.com/Microsoft/TypeScript/issues/2829

I am open to different suggestions if you want.

[csnover]

For path mapping AMD/ES6 could you follow the syntax already used by AMD paths common configuration? It maps module ID prefixes, from most-specific to least specific, so e.g.:

paths: {
  'foo': '/path/to/foo',
  'foo/baz': '/path/to/baz'
}

'foo/bar/blah' -> '/path/to/foo/bar/blah.{ts,d.ts}'
'foo/baz' -> '/path/to/baz.{ts,d.ts}'

In so doing, this leaves open the possibility of the compiler being able to simply consume the same AMD configurations used by an app at runtime, instead of introducing an incompatible equivalent syntax.

[3nsoft]

I will give my vote for having typescript field in package.json, into which declarations can be placed, or some other configs, relevant to compiler.

This will keep things simpler (no yet another config), and it will clearly indicate that package can be used with TS.

[3nsoft]

When package produces single entity (ES6' export default), I assume, declaration file, hopefully generated by compiler, will be something like

exports = {
    foo: ...;
    bar: ...;
}

[mhegazy]

When package produces single entity (ES6' export default), I assume, declaration file, hopefully generated by compiler, will be something like

@3nsoft, i am not sure i understand the question/comment

[3nsoft]

@mhegazy I have in mind package that is a single object or function. How will this be done?
The exports = Foo; is analogy from declarations like

declare module 'q' {
    exports = Q;
}

In given setting the declare module 'q' { is no longer needed, and we are left with exports = Foo;, I guess. I do not know if such line is currently valid, or if something else will be chosen.
I just want to point out this use case, analogous to ES6's export default. As having every single-function module to export function makeFactory(): Factory; is not very convenient.

[mhegazy]

Typescript has a non-ES6 export syntax export = id. so your module would look like:

declare module 'q' {
    function makeFactory(): Factory;
    export = makeFactory;
}

and you would import it as:

import q = require("q");
var f = q();

[poelstra]

@vladima For the Node case, would it make sense to automatically detect the typing to use based not only on a typings property, but also on the main field?

LOAD_AS_DIRECTORY(X)

1. If X/package.json is a file,
   
   
   
   • Parse X/package.json
   
   
   • If there is a "typings" field
     
     
     
     
     
     • let M = X + (json 'typings' field)
     
     
     
     • LOAD_AS_FILE(M)
     
     
     
   
   
   • If there is a "main" field
     
     
     
     
     
     • let M = X + (json 'main' field, with optional ".js" extension stripped off)
     
     
     
     • LOAD_AS_FILE(M)
     
     
     
   
   
2. If X/index.ts is a file, load X/index.ts. STOP
3. If X/index.d.ts is a file, load X/index.d.ts. STOP

This way, I won't have to specify e.g.:

{
  // ...
  "main": "./dist/foo.js",
  "typings": "./dist/foo.d.ts"
}

but a simple main suffices.

BTW FWIW, I think I also prefer typings over typescript.definitions, as it seems more people (Flow) are using the .d.ts format, so it's not necessarily Typescript-specific.

[3nsoft]

@poelstra I'd recall Python's wisdom of "explicit is better".
It is better to explicitly have types-related field(s).

If others are using .d.ts, and format becomes non-Typescript-specific, then _it is_ a good reason to have a separate typescript place, which compiler will be _assured_ is related to TS, and, therefore, can be reliably used.

Here are other fields, which might be useful in the future:

• typescript.src :string
• typescript.src-maps :string

These and other goodies will benefit from having an agreed-upon place in package.json, i.e. typescript object in package.json

[poelstra]

I think it would be useful to consider adding an extra lookup to the Node case, to get smooth support for non-Typescript packages too (i.e. all the good stuff on DefinitelyTyped).

Didn't want to highjack this thread, so created #2839 for it. Curious what you guys think about it, though.

[basarat]

If X.d.ts is a file, load X.d.ts. STOP

I didn't notice the .d.ts stuff in there. Thanks @poelstra for the example : https://github.com/basarat/typescript-node/tree/master/poelstra2 :+1: !

[basarat]

Having looked through @poelstra's example I definitely agree about calling it typings. This is because it is _different_ from typescript.definition. typescript.definition would import the stuff into the global namespace (and that is a bad idea) whereas typings is used to resolve a require call and do a _local_ import of the exported stuff.

[basarat]

@vladima Can you have a look at this small repo: https://github.com/basarat/typescript-node-test so that I have a clear understanding. According to the spec it would work right?

The ts:

import mylib = require('mylib');
console.log(mylib.something());

here the require call will eventually resolve to ../node_modules/package.json's typings : https://github.com/basarat/typescript-node-test/blob/master/node_modules/mylib/package.json#L5 and therefore mylib == everything exported from this file : https://github.com/basarat/typescript-node-test/blob/master/node_modules/mylib/dist/index.d.ts

export interface SomeOldInterface {
    foo: string;
}
export declare function something(): SomeOldInterface;

(based on @poelstra's sample and my understanding of this spec)

[vladima]

@basarat yes, your reading of the spec is correct.
@poelstra this is an interesting idea - added it into the proposal

[basarat]

@vladima awesome!

[poelstra]

@vladima cool, thanks! :+1:

@basarat I 'emulated' the resolution logic of this thread by hand-editing the import statements in poelstra2, which leads to a fully working example, including working hover-info in Atom TS!

See poelstra2poc in https://github.com/basarat/typescript-node/commit/366ebb5cdf18d0193b93cbc285de913066a0dffd

[frankwallis]

In SystemJS, plugin-typescript already resolves referenced .d.ts files using the SystemJS mappings, and this works quite well. It lets you do /// <reference path="react/type-definitions/Immutable.d.ts" and that resolves to "./jspm_packages/github/facebook/[email protected]/type-definitions/Immutable.d.ts".

One current niggle is that the editor cannot resolve these references in the same way, so I have to add 'jspm_packages/*_/_.d.ts' to tsconfig.json in order to get the intellisense working. If there was a place where I could register the same directory mappings with the source-editor then everything would work seamlessly and the red squiggles under these references would go away.

Also if you have a workflow in which you do not distribute compiled JavaScript, but compile and bundle TypeScript split across multiple packages, then this sort of reference file resolution becomes essential.

[basarat]

@frankwallis IDE's shouldn't need reference comments anymore : http://blog.icanmakethiswork.io/2015/02/hey-tsconfigjson-where-have-you-been.html

That said. I need to study jspm module resolution logic more to emulate the same for .d.ts files

[MicahZoltu]

:+1: for this in particular:

Path mappings can be used to customize module resolution process. In 'package.json' these mappings can be represented as JSON object...

Something I didn't see called out here (though I may have missed it) is that explicit mappings should take precedence over all other mechanisms of mapping. If I do import { Foo } from 'foo'; and I have a typing with a key foo, whatever that points at should be used and the other search paths should not be used. The exception to this would be if the import started with ./, ../, or /. Also, an exact match should take precedence over a pattern match. So if I have fo* key and foo key, foo should be used, not fo*.

This becomes particularly important in large projects where avoiding name collisions can become difficult. If the mapping of all names to files on disk is consolidated in one place, it becomes much easier to ensure that you don't have a collision and it gives you a way to deal with it when it happens.

Mappings also make refactoring directory structures much easier as the project grows. Often, small projects put everything in the same folder, but then as the project grows you start to spread out. If I extract one of my dependencies out into a separate project I would like to only have to update the change in one place (the mapping file) rather than in every .ts file that imports it.

Personal Note:
The number one reason I don't fully switch over to TypeScript (from ES6) is dependency management. The first step to resolving that problem is making it so tools can provide information to tsc that will teach it where to find definition files. From there, dependency management tool authors (such as JSPM or TSD) can start adding support for updating the appropriate config files when a dependency is installed.

[basarat]

stable link to the locate function : https://github.com/ModuleLoader/es6-module-loader/blob/8eb0d559a4666c1f26b2555b004185a4bd3c525e/src/system.js#L214-L257

[basarat]

A sample implementation : https://github.com/TypeStrong/atom-typescript/commit/da8bb1d1e35fbbbf3068b744c123f1e013261b25 that does everthing for node_modules except for the typings stuff as I don't see the rational for that considering there is no way to have tsc send .js to one location and .d.ts to another location.

Expect a PR soon.

[MicahZoltu]

It seems like devDependencies should be ignored. If I pull in a devDependency, its contents should not be eligible for compilation. If I want a d.ts from a devDependency, it should be moved to a regular dependency.

[MicahZoltu]

What is the motivation behind starting out with so many options? If only one option were provided, TS package developers would generate packages in the expected format. This would make it so tool authors (including TS compiler) would have an easier time of coding the logic necessary to do TS lookup and troubleshooting would be simplified.

I propose that the TS maintainers pick one place in an NPM module to store the definition file(s) and only support that initially. Unless there is compelling reason to start with lots of options (such as backwards compatibility), it seems like there is little to be gained and much to be lost.

Personally, my preference is to only support loading from main in package.json and use whatever file is mapped there, but with a .d.ts suffix (note: I am also personally against distributing TS files for anything other than source-map debugging).

[basarat]

If I pull in a devDependency, its contents should not be eligible for compilation

Disagree. These files are available for consumption in nodejs + these modules _should be usable by typescript code you are authoring in your package_. Otherwise you would be _forced_ to use .js for dev stuff.

[basarat]

I propose that the TS maintainers pick one place in an NPM module to store the definition file
Personally, my preference is to only support loading from main in package.json and use whatever file is mapped there

Agree. That is exactly what https://github.com/Microsoft/TypeScript/pull/3147 implements ;)

[MicahZoltu]

That is exactly what #3147 implements ;)

Unless I am misreading the PR, you added support for resolving from a number of locations. I am proposing that there is only one location. Either the file is there or it isn't. There is no fallback strategy.

[MicahZoltu]

These files are available for consumption in nodejs + these modules should be usable by typescript code you are authoring in your package. Otherwise you would be forced to use .js for dev stuff.

Dependencies are shipped to package consumers (libraries) and deployed to servers (web applications). Dev dependencies are not. So the question is, should you always ship d.ts files that you depend on to build.

In the case of shipping a library, it seems like you _should_ ship the d.ts file because there is a good chance that your user will need it if they are referencing you as a typescript library.

In the case of deploying a server, shipping the dependency would likely get you the source maps.

If your library users are regular JS users, then the definition dependencies are not needed. Unfortunately, at the time you deploy to npm you can't know whether your user will be JS or TS.

[poelstra]

Personally, my preference is to only support loading from main in package.json and use whatever file is mapped there

I generally agree.
However, maybe the intent was to support a case where (e.g. for 'documentation' purposes) a different .d.ts file is shipped, as mentioned in #2568.

[poelstra]

@zoltu I would certainly like to be able to use TS packages in e.g. my mocha tests.

Dev dependencies are not. So the question is, should you always ship d.ts files that you depend on to build.

The solution proposed in this thread main solves handling of 'native' TS packages, where the .d.ts files are shipped with the (dev-)dependency itself. So no, no need to ship any .d.ts file for these with your package. When I install the devDepencies, I'll get the typings, when I don't, no problem.

I do see what you mean though, for shipping/not-shipping of .d.ts files for non-TS packages (such as mocha itself), but that may be more related to #2839.

[poelstra]

@basarat Hadn't noticed your image in https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-101486950 before

That bar.d.ts file directly in node_modules (i.e. not in a bar subfolder) may very well solve most cases of #2839 too.

I could e.g. npm install --save bluebird, then put bluebird.d.ts directly in node_modules (or, better yet, let a tool like tsd download the correct typings file and store it there).

No need to 'bundle' the typings in my packages anymore, no typings folder, tsd can e.g. symlink all occurrences of node.d.ts to one single location, nice!

This only really works if such typings are indeed just single-file, but given that this 'directly-in-node_modules'-trick would likely only be used for non-native TS packages, that may be just fine.

[basarat]

I could e.g. npm install --save bluebird, then put bluebird.d.ts directly in node_modules (or, better yet, let a tool like tsd download the correct typings file and store it there)

Good idea. Neverthless if this PR gets merged adding a lookup in _another directory_ like typings is a trivial duplication of this code : https://github.com/Microsoft/TypeScript/blob/07c9ea01b4384c78e81315119c50be251ed21032/src/compiler/program.ts#L261-L265

            // Also look at all files by node_modules
            if (!found) {
                found = forEach(supportedExtensions,
                    extension => getNameIfExists(normalizePath(combinePaths(combinePaths(containingFile, "node_modules"), moduleName)) + extension));
            }

[troyji]

My gut tells me that the line of thinking here is going in the wrong direction. The TypeScript compiler should not be conforming fundamental routines to third-party ones (Node, RequireJS). I strongly believe that the compiler should end up with one single way, TypeScript's way, of resolving dependencies. Creating customized resolution techniques to accommodate every system that comes along is a train wreck waiting to happen. Furthermore, it does not seem to be necessary to solve this problem.

I do not believe the proposed perspective, 'in-browser resolution vs out-of-browser resolution', is the correct one to be analyzing. I think a more appropriate perspective to consider is 'compile-time resolution vs run-time resolution'. The current resolution system assumes that the resolution at both compile-time and run-time are similar. This happens to be true most of the time with a system like Node (out-of-browser) because both compile-time and run-time resolution largely depend on well-established file-system paths. This falls apart rapidly in an 'in-browser' scenario where the resolution begins to happen across resources - when paths are commonly transformed variations of the underlying file-system paths. An important observation to be made from this perspective is that compile-time resolution logic is exactly the same (file-system based) no matter what third-party system is being used. It is the run-time resolution that differs.

For example, lets say that we have typescript files 'scripts/A.ts' and 'scripts/B.ts'. With an out-of-browser system, we can generally expect the files to exist at the same location during both compile-time and run-time - inside the 'scripts' folder. However, with an in-browser, cross-resource system, we are likely to see the run-time path be a transformation of its compile-time path. The files may be sitting on the file system in the 'scripts' folder, but may be requested as '/cats/A' and '/dogs/B'. Or they may be published to the file system in '/cats/A' and '/dogs/B' but need to be requested as simply 'A' or 'B'. Note that out-of-browser systems could just as well require a transformed resolution between compile-time and run-time, but tend not to.

So how can we solve this problem? A simple, but painful solution would be to define both a compile-time path and a run-time path in scenarios where they will be different.

import A = require({compile: "/scripts/A", run: "/cats/A"});

More complex but less painful, I think the default system should provide a mechanism for providing the compiler with a list of standard file-system paths (including support for globbing) ordered by priority, to be used for compile-time resolution; as seen in traditional systems (if it ain't broke, don't fix it). To support transforming import paths to an appropriate run-time variant, perhaps there could be optional compilation transforms for paths in the same way that AMD and CommonJS are optional transforms for modules. For example, a RequireJS transformation could offer to normalize import paths to a baseUrl during compilation. Worst case, a one-to-one mapping could be provided for each path or sub-path. Interestingly, most out-of-browser systems would require no transformation options, since their compile-time and run-time paths are likely identical.

Minimally, I think the focus should be on getting compile-time resolution perfected first - this is a fundamental concern for the compiler. At least then us using in-browser systems can handle run-time path transforms with external mechanisms without having to restructure applications into something TypeScript can compile. Getting transformations to run-time paths is a nice-to-have that can be invested in later.

[troyji]

Here is a sample of what a TypeScript configuration file may look like that would support passing module paths (compile-time locations) and path transforms (to generate run-time paths) to the compiler.

{
    //compile-time module locations (in order of priority)
    moduleLocations: [
        "/absolute/path/directory/", //absolute path
        "../relative/path/directory/", //relative path
        "/globbing/**/directory/*{.ts,.d.ts}", //globbing support
        "specificFile.ts", //individual file
        { "MappedModule" : "MyModule.ts" } //module path
    ],
    //transformation to generate run-time module locations
    pathTransforms: [
        { "/originalPath/resource" : "/transformedPath/resource" } //resource transform
        { "/originalSubPath/*" : "/transformedSubPath/*" } //subpath transform
        { "/[Rr][Ed][Gg][Ee][Xx]Path/(\subPath)/(\*.ts)", "/RegEx/\1/Replace/\2" }, //regex path transform (backreference support)
        { "/functional/transform/required/resource", "GlobalRuntimePathTransformFunctionName" } //run-time global transform function
    ]
}

[basarat]

Regarding the path mapping, this approach would not quite solve our problem. What I want to express is that a source file should first be searched in location A, then in location B, then C, and so on.

@mprobst Is there a runtime today that uses customized path mappings _and_ does this? AFAIK (based on my limited 24hour research) SystemJS doesn't provide support for it right now and a particular * can only map to a single destination https://github.com/systemjs/systemjs/wiki/Basic-Use If I am wrong can you share an example path mapping. I'd like to get feature request merged and closed :rose:

[mprobst]

@basarat the use case is having your source code import locations (import {x} from 'foo/bar';) independent from your source code layout.

E.g. you might have some tool generating source code (parser generator or so) into a build directory as build/my/lib/generated_file. You want to use that code from a file in src/my/lib. You should be able to write just import {x} from './generated_file';, the physical layout of your code shouldn't impact the conceptual import.

There are many other use cases along those lines, e.g. having a partial checkout of your source code repo with some files in different locations, or developers having slightly different local directory layouts. I think every other programming language I know supports include paths that are sequentially searched for this reason, from C++ to Java to Ruby to Python.

[basarat]

the use case is having your source code import locations (import {x} from 'foo/bar';) independent from your source code layout.

@mprobst I definitely get the use case. But my question is different and I've rephrased for clarity :smile: :

Is there a JavaScript runtime today (SystemJS / AMD / CommonJS / something else) that uses customized path mappings _and_ does this?

Basically I'd be happy to code up an module resolution logic in TypeScript if its already there in an existing JS runtime that TypeScript should support. If not then I think runtime support should be done _before_ such a proposal is merged into TS or hold up a node_modules resolution system as I'd like to see people publishing easily to NPM :)

[troyji]

@basarat
JavaScript runtimes do not have a need for such a feature because it is a compile-time concern. TypeScript needs support for special compile-time scenarios like the one @mprobst is describing because it is a compiled language.

Please see my discussion above for a more thorough explanation.

[troyji]

In my discussion above, I recommend that TypeScript should handle module loading at compile-time strictly through a list of prioritized module paths to be passed into the compiler. You may think this approach to be too naive in the case where pre-compiled modules are being imported and the dependency chain must be identified within the context of the target module loading system. I maintain that it is not the TypeScript compiler's responsibility to understand every module loading system. To handle this responsibility, a tool that understands how to walk the target dependency chain and generate file paths would be inserted into the tool chain, in front of the compiler.

Benefits

• compiler compile-time module loading is kept simple and clean.
• support for new or changing module loading engines can be inserted into the toolchain without changes to the compiler.
• multiple module loading techniques can easily be supported in one compilation pass instead of enforcing a single system be used.
• this is a well known and proven technique used by compilers for decades.

[poelstra]

Node.js/NPM is a very common use-case (and even browser packages are starting to switch to NPM for their distribution), so although the compiler wouldn't have to understand every module loading system, it makes a lot of sense to effortlessly support the NPM scenario.

The implementation by @basarat is fairly trivial and requires no configuration.
It can be seen as a more specific implementation of @troyji's and @mprobst's more generic and configurable proposals, which also seem more complex and therefore may need more discussion.

My suggestion would be to get the very common case of NPM working first, then decide whether (and how) we should go the extra mile on making it completely loader-agnostic too.

[mprobst]

@basarat I'm not aware of a module system for JavaScript that works exactly as I describe it. Node arguably does something similar with the lookup in node_modules, though doesn't allow customizing locations.

But: in the browser, many projects have something like this in disguise: during their asset pipeline/delivery, they move around JavaScript files into well known locations, such that when the browser requests them, they get served from there, or they concatenate files from different locations. This is all very ad hoc because many (most) projects don't use a JS module system, but it satisfies the same need. This is usually not a problem because your web serves and/or copying files around glues it all back together, but the TypeScript compiler runs on your file system, not on what your web server serves.

If you can come up with a system that doesn't allow customization but works for most/all users without forcing them into an awkward source layout, more power to you. But I don't believe that's realistic. Not everyone will use npm as their library distribution mechanism, and people will need flexibility in their path layouts.

Just to be clear: it'd be great to make progress and properly support the npm use case, it's much needed. But I do think we still need the multiple inclusion locations. Maybe these issues are orthogonal.

Also to be clear: while interesting, I think @troyji's proposal is rather complex. I'd suggest to canonicalize the path of a requested module relative to the root (e.g. resolving ./foo from bar/baz to bar/foo, and then searching for that path relative to a list of directories (e.g. for ['dirA', 'dirB'] search dirA/bar/foo, then dirB/bar/foo).

[troyji]

@poelstra
I will refer to the last response from @mprobst. It captures the spirit of my response. I will also add that I agree NPM scenarios should seamlessly be supported by the tooling, I just disagree that it belongs within the compiler itself. I also disagree with prioritizing the NPM scenario over the fixing loader issues that affect everyone.

I am not hung up on my proposed implementation, only the goals that it aims to accomplish. Alternatives are highly welcome and encouraged.

I sincerely hope that the strategy of tightly conforming the core product to third party design is reconsidered. Technology and attitudes towards products change way too quickly - anchoring TypeScript to them could cause it to sink when the tide changes. Give them first class support through tooling? Absolutely. Force TypeScript into their mold? Absolutely not.

[mmv]

@mprobst is right on in that there are many more use cases than using TypeScript within standard nodejs package development workflows. I don't think it's that uncommon to have several projects that are all built in their own folders but bundled together as a final build step.

@basarat requirejs allows configuring custom paths for module location resolution.

[MicahZoltu]

_TL;DR:_ Coupling tsc to npm is likely to cause pain in the future. node_modules support can be achieved through tooling plus the more general solution provided in the second half of this issue.

Technology and attitudes towards products change way too quickly - anchoring TypeScript to them could cause it to sink when the tide changes. Give them first class support through tooling? Absolutely. Force TypeScript into their mold? Absolutely not.

This mirrors my sentiments, though a little more strongly than I actually feel. NodeJS and NPM are tightly coupled, they ship together and you cannot use one without the other. TypeScript on the other hand is not tightly coupled with NPM.

The second half of this proposal I am on board with. It doesn't couple TypeScript to any dependency management system, though it gives tooling a way to bind TypeScript to any module/dependency system. The logic encapsulated in the first half of this proposal (the NPM specific stuff) can be resolved through the second half of the proposal + some tooling.

As an example, as part of my build process I could have a pre-compile step that generates the appropriate mapping file using the exact logic that this proposal recommends. All I would have to do is throw that in front of my compile task and I would get node_modules support without any tight coupling.

As far as real risks go, it seems likely that at some point NPM will change its module structure (they have to if they eventually want to offer a better Windows experience). When that happens, at best TypeScript will break until it is updated at which point the TypeScript compiler will need to retain support for both the old and the new module system side by side. Alternatively, TypeScript could end up using the wrong files for definitions (this seems unlikely).

[nycdotnet]

I'm really happy with the recent conversation. I would just like to throw this scenario out there with the hopes that the smart people working on this will consider it and ensure that the new functionality addresses it. This has been a major pain point in TypeScript for me, even from back in the 0.9.x alpha days, and I'm glad the team is finally putting energy toward addressing it.

(This is my most recent example of several over the last two years.)

I currently maintain grunt-ts which is a Grunt wrapper for tsc published to npm. Grunt-ts uses csproj2ts which is a separate npm package that I wrote to encapsulate the Visual Studio .csproj file parsing logic that grunt-ts uses.

For most folks doing work on grunt-ts, they probably wouldn't need to work on csproj2ts, so the d.ts file that I created for it when I last released it to npm will be fine to use. However, for me, I'd like to have TypeScript use the actual source code of csproj2ts (or at least a definition file compiled from it) that was generated from my dev copy of csproj2ts in a different folder on my machine. That way I can make changes to the shape of code in csproj2ts and immediately have the changes reflect in my grunt-ts project. Right now there's no good way to achieve this. (When I say project, I don't even mean VS project - I also mean anything hooked into a tsconfig.json file such as Atom)

A related issue was discussed in #1753 and I made a comment about how it's still too much of a hassle to make TypeScript code _truly_ modular https://github.com/Microsoft/TypeScript/issues/1753#issuecomment-71023610 . It's not that you _can't do it_, it's just a pain. I know JavaScript isn't C#, but "UI + Server + common library" is a bedrock pattern useful across most any modern software ecosystem, yet today's TypeScript still forces you to plumb this yourself. Of course #17 comes into this as well.

This very simple capability would be delightful, and it's eluded TypeScript developers since the beginning. I think it's a bit of a shock to C# devs coming into TS that this _really isn't supported_ by TypeScript yet (without having to build your own scripts to copy definitions around), especially considering @ahejlsberg 's involvement. It took me months to realize and to finally accept "no, actually this just _isn't_ supported, Steve, and you have to write the plumbing yourself for copying definitions around".

So anyway - thanks to the TypeScript team for looking into this further and hopefully addressing it in 1.6. You're all awesome! In my opinion, removing the incorrect assumption baked-in to the language that _compile time type definitions MUST be 1:1 with runtime references_ would eliminate one of the few remaining major pain points in the language for people writing medium-complexity projects.

[Jaeiya]

@nycdotnet :+1: I think you've described how I've felt about typescript forever, since moving from a C# background myself. I'm getting tired of fighting the convention, or lack thereof. I'm happy to see this finally getting properly addressed :smile:

[mprobst]

Here's a very concrete example of stuff that's hard to achieve right now:

Imagine you have this folder structure:

project/
  build/ # all compiled code goes here
  src/
    a/
      parser.y # YACC grammar for something
      code.ts # My source code using parser

You have some build process that takes parser.y and generates parser.ts from it. Now your source tree looks like this:

project/
  build/
    a/
      parser.ts
  src/
    a/
      parser.y # YACC grammar for something
      code.ts # My source code using parser

Ideally I'd also like to write import {Parser} from './parser'; in code.ts.

How do I run tsc such that it picks up both parser.ts and code.ts, and generates output in build/a/parser.js and build/a/code.js? --rootDir cannot be either src or build, and --outDir gets the wrong structure if I pass the project root into it (plus the relative import doesn't work). Obviously mixing generated output files with by source code is icky for all kinds of reasons.

[MicahZoltu]

I believe a definition map would handle that situation. Your map would sit at project root and contain: "parser" : "build/a/parser.ts"

Your import would be import { Parser } from 'parser'; (no ./ prefix).

[mprobst]

@Zoltu yes, but it's a rather inelegant solution if I have to special case a mapping for each build artefact, isn't it? And it's hard to understand for developers as they'd have to trace back what parser is mapped to in this context. Multiple --rootDirs would fix both of that quite easily.

[MicahZoltu]

As I have been suggesting in this issue and others, it is not the compilers job to add elegance and simplicity to all user scenarios, that is the job of other tools. The compiler, in my opinion, should be a very explicit step in the build process and make no guesses about what a user intends.

If you want the compiler to find all of your .ts files in build/a/ then you can add a pre-compile step that generates the map for you by searching that directory for .ts files. Basically, any logic for auto-finding files should be in a separate tool, not the compiler.

I personally don't think parser.y should output to build, it should output to src. However, the beauty of a map file is that you can have your project setup in whatever way makes sense to you and I can setup my project in a way than makes sense to me. Neither of us have to worry about the compiler doing magic that adversely effects our setup because that magic is in a tool that I can opt in to or not... rather than the compiler that I must include.

[mprobst]

@Zoltu true, you can emulate many other behaviours with a definition mapping. And if you generate it, you can do even more. But then you have to write a program to generate it, or just configure the map, and then your IDE, and all other tooling around TypeScript. Developers will have to understand the mapping and its rules, and all TypeScript tooling will have to implement them.

Include paths are a really simple solution that covers a lot of different use cases, and is also well supported by all build tools out there (because include paths are the standard in more or less every other language).

What's the motivating use case that works with a definition map but doesn't with include paths? Is it worth the complexity?

[MicahZoltu]

What's the motivating use case that works with a definition map but doesn't with include paths?

Difficulty debugging odd behavior. I have been in situations with C/C++ using include paths where I would get some bizarre behavior either at compile time or runtime that was incredibly difficult to troubleshoot and would end up being that a file was being included from an unexpected location due to some kind of name collision. Versioning also becomes more complicated and harder to solve explicitly when using include paths because the file names tend to not contain version information (C# solved this by making version part of the assembly name) so if your include path(s) contain two versions of the same library it is ambiguous which one will be chosen.

@mprobst If you really like include paths, they can actually be implemented directly with the mapping file using one of the proposals (https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-79702125):

{
    "*": ["project/src/*.ts", "project/build/*.ts", "project/src/*.d.ts", "project/build/*.d.ts"]
}

This will effectively result in include path behavior, the mapping file would not need to be updated for each file. I would personally not build my projects this way, but does it solve your desire for the include path pattern?

In an earlier post you mentioned that you wanted relative paths resolved as well, using the above mapping file you could import { Foo } from 'a/foo' from anywhere in your project and it would resolve to one of:

• project/src/a/foo.ts
• project/build/a/foo.ts
• project/src/a/foo.d.ts
• project/build/a/foo.d.ts

If you intended to explicitly reference a file relative to the current file then you would instead do import { Foo } from './a/foo' which would only look for foo.ts and foo.d.ts in a folder named a that sits next to the current source file. Again, I wouldn't personally use this but I believe it gives the flexibility you are looking for.

[csnover]

I asked this earlier but I’ll ask again differently/more explicitly: Why are new syntaxes being invented in this ticket for module mapping instead of using ones that already exist from the AMD spec (paths, map)? What doesn’t the existing established standard solve when it comes to explicitly specifying (i.e. not relying on a filesystem walker) where to resolve absolute module IDs that requires this inventing?

[MicahZoltu]

@csnover I don't use AMD so I can't say much about it but looking at those links it appears that it doesn't support any kind of globbing? Also, what benefit would we gain by copying AMD keys/values since the AMD map doesn't sit in a well defined file (unless I am missing something)? Perhaps you just suggesting that we do something similar to what AMD does, not that we try to share a file with them?

It appears that the AMD paths solution is just a set of include directories. As I have stated above, I'm generally against include directories because it can lead to unexpected behavior. That being said, the non-npm part of this proposal (plus some additions in the comments) can provide include path functionality if you like with globbing, without forcing the user to use include-path-like functionality.

As far as the AMD map solution goes, I do like that it offers different mappings for different modules. I do believe that globbing support would be necessary to make it really work, otherwise one (or a tool) would have to maintain the entire collection of files even though in most cases they will all route to the same place. I would still want support for multiple map files so that a dependency can setup its own mappings and I don't have to modify my mappings with my dependency information.

[nycdotnet]

The AMD style of modules is designed to be used in browser scenarios. As such, globbing really doesn't make sense since it's very unusual for web servers to permit enumeration of paths.

I don't know about other AMD implementations, but at least with Require, the mapping configuration is done by passing an object literal to a config function. (Take a peek at some example RequireJS config calls). As such, there's not really a mapping "file" - just something that gets passed-in at Runtime.

[MicahZoltu]

Sorry, when I said globbing I didn't mean search resolution. Just the ability to do

"foo*": "path/to/bar*"

Then if I did import from 'foobaz'; it would resolve to path/to/barbaz.(d.)ts

[csnover]

@Zoltu It works exactly like that, it’s based on path segments, longest wins.

Paths (filesystem lookup target):

{
  paths: {
    'foo': 'built/foo',
    'foo/bar': 'http://remote/bar'
  }
}

module 'foo/baz' is loaded from '/built/foo/baz.'
module 'foo/bar/blah' is loaded from 'http://remote/bar/blah.'

Map (module ID remapping):

{
  map: {
    'foo/blah': {
      'bar/baz': 'foo/baz'
    },
    'foo': {
      'bar': 'other/bar'
    },
    '*': {
      'baz': 'other/baz'
    }
  }
}

module 'foo/blah' or 'foo/blah/other' requests 'bar/baz', gets 'foo/baz' instead
module 'foo' or 'foo/bar' requests 'bar', gets 'other/bar' instead
any other module requests 'baz', gets 'other/baz' instead

Filesystem lookup and module ID remapping are separated because you can preload modules into the module system (this is how a build works), which means that the paths lookup logic would never be used, but you may still want to redirect the module being loaded (for example if you are adding a wrapper around some other code or trying to fix a bug in another module by duck punching).

[MicahZoltu]

@csnover For the paths, can you do non-directory matching?

{
    paths: {
        'foo': 'built/foo'
    }
}

import { Foo } from 'foo'; // resolves to built/foo.dts or built/foo.ts

All of the examples I have seen have the looked up module inside a folder and the folder is what is mapped, not the file.

[csnover]

@Zoltu yes.

[nycdotnet]

And the key thing is it might allow physical location of module definitions to be different than their virtual "namespacing" ("mycompany/lib1", "mycompany/lib2") and further permits runtime references to be in different physical locations. All virtual and configurable.

[subesokun]

Not sure if this fits the discussion here but I'm currently facing an issue with systemjs. With systemjs you can import a variety of formats (es6, es5, ts, text, ...) so when I try to do something like the following I get compile errors although they're valid systemjs imports.

import template from './template.html';
// error TS2307: Cannot find module './app.html'.

import template from './template.html!text';
// error TS2307: Cannot find module './app.html!text'.

Is it somehow possible to tell the TS compiler/linter to ignore these kind of imports?

[mhegazy]

@subesokun the compiler needs to know the type of template

declare module "template.html!text" {
    /// type of template
}

then your importing code can just use that.

import template from 'template.html';

[paleo]

About path mapping, is it possible to make recursive definitions? It would help to make sub-projects:

project/
    subproject1/
        moduleA/
            MA.js
        modulePaths.json
    moduleB/
        MB.js
    modulePaths.json

File project/subproject1/modulePaths.json:

{
    "basePath": "[current-directory]",
    "paths": {
        "moduleA": "moduleA/MA"
    }
}

File project/modulePaths.json:

{
    "include": ["subproject1/modulePaths.json"],
    "basePath": "[current-directory]",
    "paths": {
        "moduleB": "moduleB/MB"
    }
}

[milkshakeuk]

@mhegazy, could you give a more verbose example of your response to @subesokun on how to inform the compiler of the template?

import template from './template.html!text';
// error TS2307: Cannot find module './app.html!text'.

does this need to declared in a d.ts file i.e.

// File: templates.d.ts
declare module "template.html!text" {
    /// type of template -- what would be in here?
}

followed by

// File: view.ts
/// <reference path="../Interfaces/IView" />
/// <reference path="./BaseView" />
/// <reference path="../../../../typings/templates.d.ts" />
import IView from '../Interfaces/IView';
import BaseView from './BaseView';
import template from 'template.html';
import Handlebars from 'handlebars';

export default class View extends BaseView implements IView {
    setTemplate(){
        this.template = Handlebars.compile(template);
    }
}

and does this need to be done for each of the html partials we want to import though the text plugin or is there a more generic approach for this sort of import?

[basarat]

FWIW as a workaround I've done : https://github.com/TypeStrong/atom-typescript#packagejson-support

When #2338 lands the user will be able to delete the typescript.definition from their package.json and everything will continue to _just light up_ perpetually.

[frankwallis]

@milkshakeuk something like this should work I think

declare module "template.html!text" {
   var __html__: string;
   export = __html__;
}

also see #2709

[milkshakeuk]

@frankwallis, @basarat thanks for the responses hopefully typescript will includea nice non-manual approach for this in a future release.

Typescript + systemjs is making for a fun time :+1:

[poelstra]

It seems things have become a bit stuck on whether to 'just' support node_modules, or create a more generic and flexible path-mapping system.

Would it be feasible to e.g. go for the implementation that @basarat provided a while ago for the node case, then later expand that into a generic solution? A bit like what's been done for the 'exclude' property first, versus complete globbing support later?

@vladima Could you maybe share an update on the status and the TS team's view on this?

[frankwallis]

Just adding the resolve method to the Host will enable most scenarios, personally I would like to see the resolve method called for referenced declaration files as well as imports as this will give more flexibility in how the declaration files are delivered.

Ideally the resolve method would be asynchronous and return a promise but I don't think that is currently possible.

[mhegazy]

@vladima started on fixing up the module resolution, and enabling a better experience there. @basarat has contributed the node resolution piece, we still need to work out the language service part of the story.

@vladima then got busy with some release-1.5 perf/memory bugs, he is finally done and should be able to get back on that.

However, I am not sure i see the module resolution work would fit into this, other than @frankwallis suggestion of having a the host override the resolve method, which is doable, but means that there are some complications involved in specifying these resolver extensions in different ways the compiler is invoked (e.g. command line, system js, language service, etc..)

The real issue is that the TypeScript compiler expects an import to map to a module, it uses the module declaration to understand the type of the import. in this case you want to say something like "*!text" is always a string. there are two issues, 1. there is no conciseness between different loaders on whether this comes first "template.html!text" or last "text!template.html" and 2. what about other loader extensions, e.g. requirejs plugins.

So far the compiler has taken an agnostic look at these, and treated the module names an opaque strings. thus my recommendation to use an ambient external module declaration, which @frankwallis explained in https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-113400736.

The obvious issue here is you have to add one of these for each resource you want to load.

The only semi-reasonable solution i can think of is adding a way to tell the compiler to shut up and do not bug you any longer, as explained in https://github.com/Microsoft/TypeScript/issues/2709#issuecomment-91968950

[milkshakeuk]

@mhegazy Could there not be a plugin system for the reference which allows you to specify how the reference path gets resolved in a similar style to how you can now give AMD module references a name for variable resolution.
This way the compiler can get the information it needs and not complain that the module doesn't exist

/// <reference path="templates/nav.html!text" resolver="path/to/reference/path/resolver"/>

Is this a feasible suggestion or am I just talking rubbish?

[mhegazy]

@milkshakeuk it is reasonable. Making it extensible is part of the changes @vladima is making. The question is how to surface that, i would argue you do not want to type this /// <reference> tag for every template, possibly you want to specify some rule/resolver for every extension once throughout your project.

[frankwallis]

I really like the proposal in https://github.com/Microsoft/TypeScript/issues/2709#issuecomment-91968950 for the following reasons:

• it makes importing primitives like strings very easy, e.g.

import myTemplate: string from "./my-template.html"

• it lets you import modules as 'any' when you are just prototyping/hacking

import someCoolLibrary: any from "cool-new-thing-without-a-dts"

• if for deployment reasons you have to use a module name which does not match the official name in definitely-typed you can do that, e.g.

import angular: ng.IAngularStatic from "my-angular-override" or
import angular: ng.IAngularStatic from "./angular-patched.js" or
import angular: ng.IAngularStatic from "https://cdnjs.cloudflare.com/ajax/libs/angular.js/1.4.1/angular.js"

• it lets you bypass creating a reference file for some locally defined type, e.g.

import myConfig: IConfig from "my-view-config.json"

[guybedford]

I was pointed to this issue from the jspm chat room, only just found out about it now. Would it be possible to make the resolution algorithm itself hookable in TypeScript? I certainly agree the Node resolution should be the default, but trying to work out other fixed types of resolution mechanisms at this point seems like they will only be left lacking.

[MicahZoltu]

:+1: to https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-114989607 In particular, I would like to be able to run the TypeScript compiler with a plugin (perhaps through a compiler option) which would result in some third-party-authored module resolution system being used _instead of_ the built-in one. In particular, if I am using JSPM, I want to be able to _turn off_ node resolution because node resolution could result in incorrect resolutions.

[vladima]

Yes, current intention to push module resolution responsibilities to the host and provide a number of predefined implementations to handle popular scenarios

[subesokun]

I'd like to see a hookable resolution algorithm in combination with https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-114989607. Meaning that TS transforms a call like import * as foo: IFoo from "foo"; figuratively spoken into its plain form import * as foo from "foo";, let it resolve by some external resolver and then assign the type foo to this result.

If the resolver is smart enough it could also add on-the-fly the needed type declarations like the following example in case of importing a string via template.html!.

// resolver would return
declare module "template.html!" {
   var __html__: string;
   export = __html__;
}
export ...;

This way you could avoid adding the /// <reference> tag manually for every template or declaring manually the foo: IFoo type when importing it (or?).

Actually I'd be awesome if the /// <reference> tag could be also made hookable so that you can intuitively write

/// <reference path="somemodule" />    // --> resolves the *.tsd file
import {something} from 'somemodule';   // --> resolves the *.ts file

and a resolver like jspm + SystemJS plugin could potentially resolve both of them. Maybe it would be even possible to create a jspm endpoint for http://definitelytyped.org/tsd/ to get this smoothly working.

Just writing down my thoughts :)

[masaeedu]

@vladima Could you clarify whether the mappings for ES6 resolution scenarios go in the tsconfig file or a package.json?

[johnnyreilly]

I'm not too close to the detail of this issue but I was wondering if the npm 3 changes have any bearing?

https://github.com/npm/npm/releases/tag/v3.0.0

[basarat]

@johnnyreilly thanks for bringing that up. No, it does not impact how node_modules resolution works. So will not have any impact of what is proposed / discussed here :rose:

Side note: npm3 only changes how npm install works ;)

[johnnyreilly]

Thanks for clarity @basarat!

[basarat]

Worth linking here : https://github.com/Microsoft/TypeScript/pull/4154 (and a previous https://github.com/Microsoft/TypeScript/pull/3147 as a sample of using 4154 to add node_modules support)

[mhegazy]

Node resolution has been merged in #3147, #4154, and #4352. This should be in the TypeScript 1.6 release.

paths support as well as baseDir support is not in yet. possibly we need to split this into a different issue, an close this one.

[domenic]

Hmm. Very important clarification. What you cite as "RequireJS\ES6 style module loading" is _not_ ES6. It is just RequireJS. The specification for browser module loading is up in the air but it definitely will not match what you outlined in the OP and what RequireJS does.

[mhegazy]

@domenic this is correct. one thing to note, resolution happens based on the module format you are asking the compiler to emit to, e.g. amd, commonjs, system.. since there is not really an engine with ES6 module loader implementation available, users have to target one of the existing module systems. the compiler will follow the node resolution logic, if you are targeting commonjs and will try to emulate require resolution logic if you are targeting amd and so on and so forth.

[domenic]

Cool. As long as it is not marketed or enumerated as ES6 module resolution in any way, but instead CommonJS or AMD or SystemJS, we should be fine.

[drjokepu]

I realise that there's supposed to be a different issue for this, but I can't find it, so I'll add this here: Support for base URLs or additional search paths for module resolution would be very useful for larger applications.

In a larger ASP.NET MVC application you typically have area-level scripts as well reusable components that sit on a global level. Currently, to pull in a global external module from an area-level script, you have to do something like this for every global dependency you need:

import foo = require('../../../Scripts/foo');

... which is not very nice at all.

[alexdresko]

Pardon my ignorance.. I've thoroughly enjoyed trying to understand this entire thread... but what I'm most interested in knowing is whether or not requirejs-ish resolution will be supported. It's not exactly clear to me in this thread. Side note: @basarat is, like, one of the smartest people I've ever not really met, so I totes trust his judgement. :)

[basarat]

what I'm most interested in knowing is whether or not requirejs-ish resolution will be supported. It's not exactly clear to me in this thread

Will be, but isn't yet. This thread has (over time) become focused on node_modules as NPM is really a big target to be missing first class support (which is now there!). Perhaps this should be closed and a new issue thread should be created as @mhegazy suggested.

PS: thanks for the kind words! :rose:

[mpawelski]

It would be nice to at least know where you put this requirejs support on Roadmap. Are you planning to implement this for example in 1.8 version?

We are thinking about using TS in our current solution that heavily use requirejs modules with module path mapping and I believe this feature is crutial for easy migration from JavaScript to TypeScript.

[vladima]

Summary

Node resolution was implemented in shipped in TypeScript 1.6. Its specifics are tracked by separate issues. Remaining work here is related only to path mappings so I'm closing this issue in favor of #5039.

[mhegazy]

@mpawelski #5039 is now tracking that and is added to the roadmap

[Pajn]

What is the current status for a package that exports multiple modules?
For example I want to add typings for both my-package and my-package/react.
I can't seem to use declare module...

[avesus]

Module resolution should work on the underlaying JavaScript level, and not on transpiled language. Stop doing wrong work. Babel never trying to traverse dependencies, that's the purpose of a dependencies bundler which will work on compiled JavaScript level.

WebPack and TypeScript guys really do very bad design choices about separation of concerns and isolation of responsibilites.

[basarat]

Babel doesn't do type checking. To check imports you need to know their location 🌹

[masaeedu]

@avesus Module resolution is a platform/runtime concern, not a language concern. ES6 specifies a syntax for describing and importing modules, but it does not specify a uniform module loader or module resolution strategy that is implemented by all environments where JS runs. While NodeJS and browsers do not yet natively implement a module loader, TypeScript will need to emit different stuff to support different module loaders.

[avesus]

Dear Asad,

NodeJS natively implements module loader. And I mean not LOADING process,
but MODULES COMPOSITION process. De-facto standard for that - NPM Node
modules.

And that standard is f_ly simple: EXPORT YOUR JAVASCRIPT to get it easily
imported by require() or import declaration specified in f_ng JAVASCRIPT
code. Not in TypeScript, not in CoffeeScript, not in ClosureScript.

This f**ng ClosureScript and TypeScrypt allow to export more than one
module from a single file.

Mr. Asad, you have to know that Node Modules system along with NPM was best
invention in world of composability.

When you write import 'module.ts'; and it's compiled into LOADING of a
typescript file instead of JavaScript, that's a crap.

Babel compiles any JSX and other files without trying to directly load
dependencies AS LANGUAGE OF DEPENDENT CODE, but transforms them to
require(JAVASCRIPT).

Exporting multiple modules from a single typescript file enforces to create
multiple JavaScript files - this breaks npm/node modularity.

When WebPack (f_*ng too) allows you to import css, coffee, JSX and a lot
of other s_t, YOUR FILE CANNOT BE COMPILED INTO A MODULE WITH EXPORTS.
Trying to import external dependencies EVEN ON A FILE LEVEL WITHIN THE SAME
PROJECT HIERARCHY in the language of implementation instead of JavaScript
makes your project fragile AND INCOMPATIBLE WITH NODE ECOSYSTEM.

Hope @sindresorhus and @tj agree with me. It will be very interesting to
hear expert point of view on the topic of corruption of NPM/Node best parts
of the world's best practices made by Microsoft who has no idea what NPM
ecosystem is or intentionally trying to break it and mr. Sokra who trying
to solve all problems in the world supporting importing of any s**t thru
loaders system.

Mighty Lamers can break all best things made by professionals. Angular2
along with TypeScript teams have to bear a lot of responsibility thinking
about what influence they made. Supporting AMD modules? SystemJS? Importing
TypeScript in import declarations instead of JavaScript?

You're lamers and you're trying to break NPM world of JavaScript magic.

On Tuesday, July 5, 2016, Asad Saeeduddin [email protected] wrote:

@avesus https://github.com/avesus Module resolution is a
platform/runtime concern, not a language concern. ES6 specifies a syntax
for describing and importing modules, but it does not specify a uniform
module loader or module resolution strategy that is implemented by all
environments where JS runs. While NodeJS and browsers do not yet natively
implement a module loader, TypeScript will need to emit different stuff to
support different module loaders.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-230627045,
or mute the thread
https://github.com/notifications/unsubscribe/AD9MI5VDAv6g3baOL4vMT69XtR0OB00zks5qSuGigaJpZM4DuGom
.

Best Regards,
Ivan Borisenko

[hdavidzhu]

@avesus I understand where you are coming from, however we don't have to get so passionate and personal over something like this. We're all just trying to make the best solutions for developers.

[mhegazy]

@avesus this is a forum for technical discussions and issue reporting for a programming language and a compiler, and not a political or social forum. The way you express your opinions and the language you have used in this thread are not inducive to a constructive discussion. If you want to contribute to this project, and have an interest in future of the TS/JS tooling, please refrain from using such language, and avoid directing insults to the community members and/or the core team.

[avesus]

I apologise for super emotional tone of course. Hope future of npm will be
better because of the very high role TypeScript developers have got and how
they can influence the future of web development.

Only one simple prayer: please be responsive and smart. Think more before
making decisions and when do explain to community.

I support the language itself and have nothing against angular or webpack
in their principles, but what they do with modularity is hell.

Sorry for the emotional tone.

On Wednesday, July 6, 2016, Mohamed Hegazy [email protected] wrote:

@avesus https://github.com/avesus this is a forum for technical
discussions and issue reporting for a programming language and a compiler,
and not a political or social forum. The way you express your opinions and
the language you have used in this thread are not inducive to a
constructive discussion. If you want to contribute to this project, and
have an interest in future of the TS/JS tooling, please refrain from using
such language, and avoid directing insults to the community members and/or
the core team.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-230848544,
or mute the thread
https://github.com/notifications/unsubscribe/AD9MI0bvI-BacJw1JVvk0EPz5sI9O8_qks5qS-jBgaJpZM4DuGom
.

Best Regards,
Ivan Borisenko

[avesus]

"Babel doesn't do type checking. To check imports you need to know their
location" - and that's great. It is necessary to invent another way to do
type checkings of imported TypeScript files (my advice is: never import
TypeScript files directly, follow the Babel's approach).

On Wednesday, July 6, 2016, Ivan Borisenko [email protected] wrote:

I apologise for super emotional tone of course. Hope future of npm will be
better because of the very high role TypeScript developers have got and how
they can influence the future of web development.

Only one simple prayer: please be responsive and smart. Think more before
making decisions and when do explain to community.

I support the language itself and have nothing against angular or webpack
in their principles, but what they do with modularity is hell.

Sorry for the emotional tone.

On Wednesday, July 6, 2016, Mohamed Hegazy <[email protected]

@avesus https://github.com/avesus this is a forum for technical
discussions and issue reporting for a programming language and a compiler,
and not a political or social forum. The way you express your opinions and
the language you have used in this thread are not inducive to a
constructive discussion. If you want to contribute to this project, and
have an interest in future of the TS/JS tooling, please refrain from using
such language, and avoid directing insults to the community members and/or
the core team.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Microsoft/TypeScript/issues/2338#issuecomment-230848544,
or mute the thread
https://github.com/notifications/unsubscribe/AD9MI0bvI-BacJw1JVvk0EPz5sI9O8_qks5qS-jBgaJpZM4DuGom
.

Best Regards,
Ivan Borisenko

Best Regards,
Ivan Borisenko

[avesus]

What I advice is to compare modules idea with object files / linker system in C world. TypeScript - compiler. Webpack - linker.

If disable TypeScript modules resolution, each *.ts file will be compiled to a *.js file and that *.js file will import dependencies. It allows superfast *.ts files compilation.

The command line I use:

 tsc --watch --isolatedModules --pretty --skipDefaultLibCheck --listFiles --target es5 \
   --moduleResolution node --module commonjs --inlineSourceMap --inlineSources \
   --noResolve --jsx react --removeComments --strictNullChecks \
   --experimentalDecorators --emitDecoratorMetadata \
   --project . --outDir . --rootDir src

It is necessary to have ambient declarations, for example,

• ambient.d.ts:

declare module 'angular2/core' {
  var Component:any;
  export { Component };
}

declare module 'angular2/platform/browser' {
  var bootstrap: any;
  export { bootstrap };
}

declare var module: any;
declare var require: any;

declare interface Window {
  MSStream: any;
  webkitURL: any;
  Worker: any;
}

Declare reference in *.ts files:

/// <reference path="ambient.d.ts"/>

So, package-name/src/index.ts becomes package-name/index.js and package-name/src/lib/abc.ts becomes package-name/lib/abc.ts. And if index.ts imports abc.ts, when bundling it shouldn't be imported directly into the *.ts file but referenced from index.js.

It allows to author files written in multiple different languages within one package and export correct require()'able node exports consumable by any JavaScript code.

All I ask from community is to support this approach widely because it is base of npm modularity and success.