Ts-loader: Combined type definitions (.d.ts) output

[mblandfo]

I'm writing a typescript library for use with other typescript projects. Running ts-loader I got a .d.ts file (in the wrong directory, but that's already a tracked issue).

This .d.ts file contained import statements to import classes from local library files (these would not be shipped) instead of the actual class declarations I needed.

I found a npm plugin that fixes this: https://www.npmjs.com/package/declaration-bundler-webpack-plugin. This plugin has a bug due to a webpack change, but I have pasted a fixed version here: http://pastebin.com/ypqed086

I think that should be the default declarations file behavior.

[JasonKleban]

what is the bug that it fixes? Is it:

            var lines = data.split("\n");
                            ^

TypeError: Cannot read property 'split' of undefined
    at DeclarationBundlerPlugin.generateCombinedDeclaration (...node_modules\decl
aration-bundler-webpack-plugin\plugin.js:44:29)

[mblandfo]

Yes, that is the bug.

It appears the change I made was to change this line:

var data = declarationFile._value;

into this:

var data = declarationFile._value || declarationFile.source();

So presumably webpack changed from using _value to using .source(). I don't think declaration-bundler-plugin has a github or anything (it's just on npm?) I did email the author back in 2016 but he never responded. Can ts-loader be fixed to not need this plugin? It'd be really nice if ts-loader just did this and didn't have to rely on declaration-bundler-plugin at all!

Otherwise I guess someone can make a declaration-bundler-plugin2 package or something.

[johnnyreilly]

@mblandfo - if you'd like to submit a PR I'll take a look

[mblandfo]

@johnnyreilly it looks like this wouldn't be hard to do, like in https://github.com/TypeStrong/ts-loader/blob/master/src/instances.ts at the bottom plugins are getting added, so the fixed version of declaration-bundler-plugin could probably be added there.

I started to write a PR but get errors running tests. I did this:

git clone https://github.com/TypeStrong/ts-loader.git
cd ts-loader
npm install
npm run build
npm run test

and got test failures such as this:

    1) should have the correct output
    2) should work with transpile


  0 passing (6s)
  2 failing

  1) simpleDependency should have the correct output:

      Uncaught AssertionError: patch0/output.txt is different between actual and expected
      + expected - actual

And then npm printed out a ton of "npm ERR!" lines, which I think were just because there were failing tests.

[mblandfo]

Looking at this more, I think the proposal here is to add a fixed version of DeclarationBundlerPlugin into ts-loader, supporting the options (out, moduleName), and have it only be off by default, since having declarations in separate files is a valid use case. Typescript itself can be run in a mode that outputs a single file and declaration file (the outFile flag), but then, of course, you lose the power of webpack.

So, maybe it looks like this in your webpack config:

{
    test: /\.ts$/,
    loader: 'ts-loader',
    options: {
        declarationBundle: {
            out: 'dist/MyApp.d.ts',
            moduleName: 'MyApp'
        }
    }
}

[johnnyreilly]

I don't really want to create a dependency in ts-loader upon another plugin if possible. If it's a separate plugin then presumably it can be used as such in the webpack.config.js already.

Don't worry about failing tests though. Our test pack is "quirky". Feel free to submit a PR and keep hacking on it until it gets to a good place. Tests will run on Travis and appveyor against each commit

[johnnyreilly]

(and I'm happy to advise!)

[mblandfo]

Oh, I didn't mean create a dependency on that plugin. The issue is that the DeclarationBundlerPlugin is broken due to a webpack api change and no longer maintained. It's also very short, so I was actually planning to just copy the fixed version of the source into ts-loader, with the flag to turn it on. Does that sound ok?

[johnnyreilly]

It's probably fine yes - submit it and I'll take a look.

[JasonKleban]

I'm trying to port Framework7 to typescript and see if the project will accept it. But even with the above patch for DeclarationBundlerPlugin (plugin.js:43:var data = declarationFile._value || declarationFile.source();) to allow it to run through, I'm still getting export default _default; for most of the modules in the combined d.ts. output.

Can that be fixed without altering the structure of the library to not use named exports? Or can it be addressed by this addition to ts-loader?

[mblandfo]

Without the DeclarationBundlerPlugin, ts-loader just outputs a bunch of .d.ts files that refer to each other. Maybe you can just use those? Or, I guess, hand write your own.

The DeclarationBundlerPlugin pretty much just concatenates things, so, named exports are probably not something it could easily handle. Typescript by itself (no webpack) has an option compilerOptions.outFile which if set will combine to a single js and .d.ts, but they require compilerOptions.target to be "AMD" or "System".

Also, for DeclarationBundlerPlugin, I think file export names must match file names.

[JasonKleban]

Yes, this is the situation I'm hoping can be addressed. We'd all certainly need the d.ts files to be generated as part of the automated production pipeline. TypeScript can output single js, d.ts, and sourcemaps in limited configurations, but webpack still needs to be in the mix for other assets and any other post-processing of the generated scripts. Framework7 releases es6 and umd modules as single files. Multiple definition files to accompany them is undesirable. Typescript by itself isn't an good option anyway because webpack or other similar automation is still required for the other assets.

I've already made definitions manually for Framework7 v1 (I'm now looking ahead to v2) so the definitions can be made.

What's the real technical challenge in combining these definitions in webpack & ts-loader alongside the js and sourcemaps? Maybe I could help with it if someone could catch me up. What non-trivial rewriting has to happen? I don't think there's anything unexpressible since it can be done accurately manually.

[mblandfo]

I think you said you got a fixed version of DeclarationBundlerPlugin working on your project, that's probably the best place to start. See if you can modify that to do what you want. If you get something that solves the general cases for ES6 modules or whatever module system, that'd be great! I'm not sure how it would work, like if you can just parse all the import/export/require declarations or what.

It'd be nice to see what the typescript people think https://github.com/Microsoft/TypeScript/pull/5090

[nilpath]

@mblandfo I noticed that your PR didn't make it into the ts-loader did you ever get around to publish your own declaration file plugin that works?

[mblandfo]

No, but here's the code:

In webpack.config.js:

var DeclarationBundlerPlugin = require('./somepath/fixed-declaration-bundler-webpack-plugin');
...
    plugins: [
        new DeclarationBundlerPlugin({
            moduleName: 'MyModule',
            out: './MyModule.d.ts'
        })
    ],

somepath/fixed-declaration-bundler-webpack-plugin.js

var DeclarationBundlerPlugin = (function () {
    function DeclarationBundlerPlugin(options) {
        if (options === void 0) { options = {}; }
        this.out = options.out ? options.out : './build/';
        this.excludedReferences = options.excludedReferences ? options.excludedReferences : undefined;
        if (!options.moduleName) {
            throw new Error('please set a moduleName if you use mode:internal. new DacoreWebpackPlugin({mode:\'internal\',moduleName:...})');
        }
        this.moduleName = options.moduleName;
    }
    DeclarationBundlerPlugin.prototype.apply = function (compiler) {
        var _this = this;
        //when the compiler is ready to emit files
        compiler.plugin('emit', function (compilation, callback) {
            //collect all generated declaration files
            //and remove them from the assets that will be emited
            var declarationFiles = {};
            for (var filename in compilation.assets) {
                if (filename.indexOf('.d.ts') !== -1) {
                    declarationFiles[filename] = compilation.assets[filename];
                    delete compilation.assets[filename];
                }
            }
            //combine them into one declaration file
            var combinedDeclaration = _this.generateCombinedDeclaration(declarationFiles);
            //and insert that back into the assets
            compilation.assets[_this.out] = {
                source: function () {
                    return combinedDeclaration;
                },
                size: function () {
                    return combinedDeclaration.length;
                }
            };
            //webpack may continue now
            callback();
        });
    };
    DeclarationBundlerPlugin.prototype.generateCombinedDeclaration = function (declarationFiles) {
        var declarations = '';
        for (var fileName in declarationFiles) {
            var declarationFile = declarationFiles[fileName];
            var data = declarationFile._value || declarationFile.source(); // FIXED!

            var lines = data.split("\n");

            var i = lines.length;
            while (i--) {
                var line = lines[i];
                //exclude empty lines
                var excludeLine = line == "";
                //exclude export statements
                excludeLine = excludeLine || line.indexOf("export =") !== -1;
                //exclude import statements
                excludeLine = excludeLine || (/import ([a-z0-9A-Z_-]+) = require\(/).test(line);
                //if defined, check for excluded references
                if (!excludeLine && this.excludedReferences && line.indexOf("<reference") !== -1) {
                    excludeLine = this.excludedReferences.some(function (reference) { return line.indexOf(reference) !== -1; });
                }
                if (excludeLine) {
                    lines.splice(i, 1);
                }
                else {
                    if (line.indexOf("declare ") !== -1) {
                        lines[i] = line.replace("declare ", "");
                    }
                    //add tab
                    lines[i] = "\t" + lines[i];
                }
            }
            declarations += lines.join("\n") + "\n\n";
        }
        var output = "declare module " + this.moduleName + "\n{\n" + declarations + "}";
        return output;
    };
    return DeclarationBundlerPlugin;
})();
module.exports = DeclarationBundlerPlugin;

[rupeshtiwari]

Hi @mblandfo sorry to write on closed ticket.
I was also getting can not find 'split' of undefined error when I used your fixed-declaration-bundler-webpack-plugin class it fixed that error.
Your solution fixes the error that i am getting however my question is why we have not added your fix in ts-loader declarationbundle plugin ?

[mblandfo]

I did email the maintainer but never got a response

[pmunin]

@mblandfo here is a bit cleaner version:

//fixed-declaration-bundler-webpack-plugin.js:

const DeclarationBundlerPlugin = require('declaration-bundler-webpack-plugin');

let buggyFunc = DeclarationBundlerPlugin.prototype.generateCombinedDeclaration;
DeclarationBundlerPlugin.prototype.generateCombinedDeclaration = function (declarationFiles) {
    for (var fileName in declarationFiles) {
        let declarationFile = declarationFiles[fileName];
        declarationFile._value = declarationFile._value || declarationFile.source();
    }
    return buggyFunc.call(this, declarationFiles);
}

module.exports = DeclarationBundlerPlugin;

[rupeshtiwari]

Hi Everyone is providing there own fix that is very helpful.
But my question is when officially we are going to fix it. Because I am also using the `'declaration-bundler-webpack-plugin' Here is my webpack.config

const DeclarationBundlerPlugin = require('declaration-bundler-webpack-plugin');
const path = require('path');
const DIST = path.resolve(__dirname, 'dist');
module.exports = {
  mode: 'production',
  entry: {
    starter: path.resolve(__dirname, './src/index.ts'),
  },
  output: {
    path: DIST,
    library: 'mylib',
    libraryTarget: 'commonjs',
    filename: 'mylib.js',
  },
  resolve: { extensions: ['.ts'] },
  module: {
    rules: [
      {
        test: /\.ts$/,
        exclude: [/node_modules/],
        loader: 'ts-loader',
      },
    ],
  },
  devtool: 'source-map',
  plugins: [
    new DeclarationBundlerPlugin({
      moduleName: 'mylib',
      out: DIST,
    }),
  ],
};

And I am getting below error. Cannot read property 'split' of undefined

(node:5072) DeprecationWarning: Tapable.plugin is deprecated. Use new API on `.hooks` instead
Unhandled rejection TypeError: Cannot read property 'split' of undefined
    at DeclarationBundlerPlugin.generateCombinedDeclaration (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\declaration-bundler-webpack-plugin\plugin.js:44:30)
    at Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\declaration-bundler-webpack-plugin\plugin.js:25:45
    at AsyncSeriesHook.eval [as callAsync] (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10),
<anonymous>:7:1)
    at AsyncSeriesHook.lazyCompileHook (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\Hook.js:154:20)
    at Compiler.emitAssets (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compiler.js:358:19)
    at onCompiled (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compiler.js:225:9)
    at hooks.afterCompile.callAsync.err (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compiler.js:547:14)
    at _err0 (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10), <anonymous>:11:1)
    at Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\ts-loader\dist\after-compile.js:28:9
    at AsyncSeriesHook.eval [as callAsync] (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10),
<anonymous>:7:1)
    at AsyncSeriesHook.lazyCompileHook (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\Hook.js:154:20)
    at compilation.seal.err (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compiler.js:544:30)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10),
<anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\Hook.js:154:20)
    at hooks.optimizeAssets.callAsync.err (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compilation.js:1296:35)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10),
<anonymous>:6:1)
    at AsyncSeriesHook.lazyCompileHook (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\Hook.js:154:20)
    at hooks.optimizeChunkAssets.callAsync.err (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\webpack\lib\Compilation.js:1287:32)
    at _err0 (eval at create (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\tapable\lib\HookCodeFactory.js:32:10), <anonymous>:11:1)
    at Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\uglifyjs-webpack-plugin\dist\index.js:287:11
    at step (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\uglifyjs-webpack-plugin\dist\uglify\Runner.js:94:11)
    at Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\uglifyjs-webpack-plugin\dist\uglify\Runner.js:117:20
    at tryCatcher (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\util.js:16:23)
    at Promise._settlePromiseFromHandler (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:512:31)
    at Promise._settlePromise (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:569:18)
    at Promise._settlePromise0 (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:614:10)
    at Promise._settlePromises (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:694:18)
    at Promise._fulfill (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:638:18)
    at Promise._resolveCallback (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:432:57)
    at Promise._settlePromiseFromHandler (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:524:17)
    at Promise._settlePromise (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:569:18)
    at Promise._settlePromise0 (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:614:10)
    at Promise._settlePromises (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:694:18)
    at Promise._fulfill (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:638:18)
    at Promise._resolveCallback (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:432:57)
    at Promise._settlePromiseFromHandler (Z:\rupesh\rnd\webpack-library-ps\lib-pj\node_modules\bluebird\js\release\promise.js:524:17)

[tim-codes]

bump on this issue, can it get added to ts-loader? Seems the most sensible approach