Xterm.js: Source maps not parsing with Webpack

Created on 9 Aug 2017 · 6Comments · Source: xtermjs/xterm.js

Receiving parse errors when bundling project with Webpack. In particular, it does not look like the source-map-loader Webpack loader is correctly parsing the source maps in the project.

Details

Webpack Version: 3.5.1
OS version: Ubuntu 17.04
xterm.js version: 2.9.2

Steps to reproduce

Relevant webpack configuration

devtool: 'source-map',

    module : {
        rules : [
            { 
                test: /\.tsx?$/, 
                use: ['awesome-typescript-loader']},
            {
                test: /\.scss$/,
                use: [
                    {
                        loader: "style-loader"
                    }, {
                        loader: "css-loader"
                    }, {
                        loader: "sass-loader"
                    }
                ]
            },
            {
                test: /\.css$/,
                use: [
                    {
                        loader: "style-loader"
                    }, {
                        loader: "css-loader"
                    }
                ]
            },
            {
                test: /\.woff($|\?)|\.woff2($|\?)|\.ttf($|\?)|\.eot($|\?)/,
                use: ['url-loader']
            },
            {
                test: /\.(jpe?g|png|gif|svg)$/i,
                use: [
                    'url-loader',
                    'image-webpack-loader'
                ]
            },
            {
                test: /\.js\.map$/,
                use: {
                    loader: 'source-map-loader'
                },
            },
            {
                test: /\.js$/,
                use: {
                    loader: 'source-map-loader'
                },
            }
        ]
    }

Example of importing into React Typescript project

class Home extends React.Component {

    componentDidMount() {
        var term = new Terminal();
        // Exposes the terminal in an existing DOM object.
        term.open(document.getElementById('xterm-container'));
    }

    render() {
        return (
            <div id="xterm-container">
            </div>
        );
    }
}

Error messages when running webpack

WARNING in ./node_modules/xterm/lib/addons/search/search.js.map
Module parse failed: /home/jack/Projects/C290/website/node_modules/source-map-loader/index.js!/home/jack/Projects/C290/website/node_modules/xterm/lib/addons/search/search.js.map Unexpected token (1:10)
You may need an appropriate loader to handle this file type.
| {"version":3,"sources":["../../../src/addons/search/search.ts"],"names":[],"mappings":";;AAIA,+CAA8C;AAQ9C,CAAC,UAAU,KAAK;IACd,EAAE,CAAC,CAAC,UAAU,IAAI,MAAM,CAAC,CAAC,CAAC;QAIzB,KAAK,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;IACzB,CAAC;IAAC,IAAI,CAAC,EAAE,CAAC,CAAC,OAAO,OAAO,KAAK,QAAQ,IAAI,OAAO,MAAM,KAAK,QAAQ,CAAC,CAAC,CAAC;QAIrE,MAAM,CAAC,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC,CAAC;IACjD,CAAC;IAAC,IAAI,CAAC,EAAE,CAAC,CAAC,OAAO,MAAM,IAAI,UAAU,CAAC,CAAC,CAAC;QAIvC,MAAM,CAAC,CAAC,aAAa,CAAC,EAAE,KAAK,CAAC,CAAC;IACjC,CAAC;AACH,CAAC,CAAC,CAAC,UAAC,QAAa;IAOf,QAAQ,CAAC,SAAS,CAAC,QAAQ,GAAG,UAAS,IAAY;QACjD,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;YACxB,IAAI,CAAC,YAAY,GAAG,IAAI,2BAAY,CAAC,IAAI,EAAE,QAAQ,CAAC,2BAA2B,CAAC,CAAC;QACnF,CAAC;QACD,MAAM,CAAgB,IAAI,CAAC,YAAa,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;IAC1D,CAAC,CAAC;IAQF,QAAQ,CAAC,SAAS,CAAC,YAAY,GAAG,UAAS,IAAY;QACrD,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;YACxB,IAAI,CAAC,YAAY,GAAG,IAAI,2BAAY,CAAC,IAAI,EAAE,QAAQ,CAAC,2BAA2B,CAAC,CAAC;QACnF,CAAC;QACD,MAAM,CAAgB,IAAI,CAAC,YAAa,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC;IAC9D,CAAC,CAAC;AACJ,CAAC,CAAC,CAAC","file":"search.js","sourceRoot":"."}
 @ ./node_modules/xterm/lib/addons ^\.\/.*$
 @ ./node_modules/xterm/lib/xterm.js
 @ ./client/hof-content.tsx
 @ ./client/hof.tsx
 @ ./client/app.tsx
 @ ./client/main.tsx
 @ multi (webpack)-dev-server/client?http://localhost:3000 webpack-dev-server/client?http://localhost:3000/ ./client/main.tsx

WARNING in ./node_modules/xterm/lib/addons/search/SearchHelper.js.map
Module parse failed: /home/jack/Projects/C290/website/node_modules/source-map-loader/index.js!/home/jack/Projects/C290/website/node_modules/xterm/lib/addons/search/SearchHelper.js.map Unexpected token (1:10)
You may need an appropriate loader to handle this file type.
| {"version":3,"file":"SearchHelper.js","sources":["../../../src/addons/search/SearchHelper.ts"],"names":[],"mappings":";;AAgBA;IACE,sBAAoB,SAAc,EAAU,4BAAiC;QAAzD,cAAS,GAAT,SAAS,CAAK;QAAU,iCAA4B,GAA5B,4BAA4B,CAAK;IAK7E,CAAC;IAQM,+BAAQ,GAAf,UAAgB,IAAY;QAC1B,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QAED,IAAI,MAAqB,CAAC;QAE1B,IAAI,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC;QAC3C,EAAE,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,CAAC,CAAC;YAEjD,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;QAC7D,CAAC;QAGD,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,QAAQ,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,CAAC,EAAE,EAAE,CAAC;YACtF,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;gBACX,KAAK,CAAC;YACR,CAAC;QACH,CAAC;QAGD,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;gBAClC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;gBACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;oBACX,KAAK,CAAC;gBACR,CAAC;YACH,CAAC;QACH,CAAC;QAGD,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;IACpC,CAAC;IAQM,mCAAY,GAAnB,UAAoB,IAAY;QAC9B,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC;YAC/B,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QAED,IAAI,MAAqB,CAAC;QAE1B,IAAI,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC;QAC3C,EAAE,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,cAAc,CAAC,CAAC,CAAC;YAEnD,QAAQ,GAAG,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC;QAC/D,CAAC;QAGD,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,QAAQ,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC;YACvC,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;YACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;gBACX,KAAK,CAAC;YACR,CAAC;QACH,CAAC;QAGD,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,GAAG,CAAC,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC,SAAS,CAAC,IAAI,GAAG,CAAC,EAAE,CAAC,GAAG,QAAQ,EAAE,CAAC,EAAE,EAAE,CAAC;gBACtF,MAAM,GAAG,IAAI,CAAC,WAAW,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC;gBACnC,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;oBACX,KAAK,CAAC;gBACR,CAAC;YACH,CAAC;QACH,CAAC;QAGD,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,MAAM,CAAC,CAAC;IACpC,CAAC;IAQO,kCAAW,GAAnB,UAAoB,IAAY,EAAE,CAAS;QACzC,IAAM,UAAU,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QACtD,IAAM,eAAe,GAAG,IAAI,CAAC,4BAA4B,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC;QAC1F,IAAM,SAAS,GAAG,IAAI,CAAC,WAAW,EAAE,CAAC;QACrC,IAAM,WAAW,GAAG,eAAe,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC;QACvD,EAAE,CAAC,CAAC,WAAW,IAAI,CAAC,CAAC,CAAC,CAAC;YACrB,MAAM,CAAC;gBACL,IAAI,MAAA;gBACJ,GAAG,EAAE,WAAW;gBAChB,GAAG,EAAE,CAAC;aACP,CAAC;QACJ,CAAC;IACH,CAAC;IAOO,oCAAa,GAArB,UAAsB,MAAqB;QACzC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;YACZ,MAAM,CAAC,KAAK,CAAC;QACf,CAAC;QACD,IAAI,CAAC,SAAS,CAAC,gBAAgB,CAAC,YAAY,CAAC,MAAM,CAAC,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QACzF,IAAI,CAAC,SAAS,CAAC,UAAU,CAAC,MAAM,CAAC,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC;QAC3E,MAAM,CAAC,IAAI,CAAC;IACd,CAAC;IACH,mBAAC;AAAD,CA3HA,AA2HC,IAAA;AA3HY,oCAAY","sourceRoot":"."}
 @ ./node_modules/xterm/lib/addons ^\.\/.*$
 @ ./node_modules/xterm/lib/xterm.js
 @ ./client/hof-content.tsx
 @ ./client/hof.tsx
 @ ./client/app.tsx
 @ ./client/main.tsx
 @ multi (webpack)-dev-server/client?http://localhost:3000 webpack-dev-server/client?http://localhost:3000/ ./client/main.tsx

The project works fine if I redirect the main option in the package.json to dist/xterm.js and also works if I delete the search.js.map and SearchHelper.js.map files. However, I would ideally like to preserve the package's integrity to maintain portability to other devices.

help wanted typenhancement

Source

jclangst

👍2

Most helpful comment

Thanks for the tip. I changed that but it still didn't work. However, I think I know why it breaks.

The /lib/xterm.js file has variable dependencies in the Terminal.loadAddon function (line 477). The way that webpack works is to go ahead and bundle together all possible resolutions to the variable dependency; in this case it tries to include every file in the the addons folder.

It gets tripped up on the .min.js files because webpack normally only accesses these files through their corresponding .js (or other format) file.

My dumb brain said that obviously we should process the .js.map files with the source-map-loader but that's really just for .js files, hence the error.

Changing this

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'source-map-loader'
                },
            },

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'file-loader'
                },
            },

will just bundle the file as-is without trying to process it in any way.

It just tripped me up because apparently this is a pretty uncommon problem since noone really intentionally loads .js.map files with a require statement directly. Hence the 2 hours of fruitless google searches on the issue.

Now the real question is how to get it so that the addon files don't get bundled by webpack and cause unnecessary bundle bloat and funky errors. I don't know if it would be better to do a PR or if there is some webpack config option that I am missing.

jclangst on 10 Aug 2017

👍2

All 6 comments

Try changing this:

 { 
  test: /\.tsx?$/, 
  use: ['awesome-typescript-loader']},

 { 
  test: /\.(tsx|ts)?$/, 
  use: ['awesome-typescript-loader']},

Or try ignoring the node_modules folder(which should be done anyway):

 { 
  test: /\.(tsx|ts)?$/, 
  exclude: /node_modules/,
  use: ['awesome-typescript-loader']},

lucat1 on 9 Aug 2017

Thanks for the tip. I changed that but it still didn't work. However, I think I know why it breaks.

It gets tripped up on the .min.js files because webpack normally only accesses these files through their corresponding .js (or other format) file.

My dumb brain said that obviously we should process the .js.map files with the source-map-loader but that's really just for .js files, hence the error.

Changing this

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'source-map-loader'
                },
            },

            {
                test: /\.js\.map$/,
                use: {
                    loader: 'file-loader'
                },
            },

will just bundle the file as-is without trying to process it in any way.

jclangst on 10 Aug 2017

👍2

@jclangst thanks for reporting this. A PR is definitely welcome if there is a transparent way of fixing this.

Just mentioning that we are using the v3 branch, as base now.

parisk on 10 Aug 2017

I had the very same issue, even though I wasn't using source-map-loader. I was able to get rid of the warnings by using the webpack ignore-loader as seen in this Stackoverflow post. This is the line I included in module.rules:

{ test: /node_modules.+xterm.+\.map$/, loader: 'ignore-loader' },

Possibly this aproach could also be used to prevent webpack from bundling the other addons.

jnodev on 31 Aug 2017

🎉1

Here's another possible solution: use webpack.IngorePlugin

new webpack.IgnorePlugin(/^\.\/.*js.map$/ ,/.*xterm\/lib\/addons/),

KagamiChan on 19 Sep 2017

I'm consolidating all bundling/loadAddon related issues into https://github.com/sourcelair/xterm.js/issues/1018, please comment on that issue if you have thoughts :smiley: