Rxjs: UMD and/or FESM bundles - now tractable with 6.0.0

[kylecordes]

RxJS version:

6

Expected behavior:

Ship a small number of UMD and/or FESM bundles, one for each of the small number of import targets under RxJS 6.

Actual behavior:

For the moment (v6 beta) it looks like there is just a single UMD bundle provided, which can't support multiple important targets. Please many small files, suitable for application-level bundling.

Additional information:

This thing has been discussed in the past, but there was never a tenable solution (other than SystemJS bundle format, which has less wide use) to ship a small number of bundles that contain the RxJSs code, while still supporting:

import { Observable } from 'rxjs/Observable';
import { of } from 'rxjs/observable/of';
import { map } from 'rxjs/operator/map';

(i.e., how things were through version 5)

But with the new, much smaller number of import targets, for example see this in the current README:

import { Observable, Subject, ReplaySubject, from, of, range } from 'rxjs';
import { map, filter, switchMap } from 'rxjs/operators';

It is now possible to ship a small number of UMD or FESM bundles, one for each of the handful of imports (rxjs, rxjs/operators, etc.).

This would take care of use cases for those of us with use cases that consume RxJS directly from something like a CDN, without having to create and publish such a bundle ourselves (as me and a number of others had done historically with SystemJS).

/cc pleading my case to @benlesh @IgorMinar etc ;-)

[steveblue]

+1 for not wanting to bundle rxjs just for my development environment.

[benlesh]

I'm trying to understand the use case and the work this would entail. Given the hoops we jump through to produce just _one_ UMD bundle, it sounds really awful to have to support this for every module, if that's what you want. So my initial reaction is :-1:

[steveblue]

If this is all that is required for a system.config.json then I am OK with the current state of the build. Do you see a problem with this @kylecordes ? This is a huge improvement from 5.5 IMHO.

    map: {
      'rxjs': 'lib:rxjs'
    },
    packages: {
      'rxjs/ajax': { main: 'index.js', defaultExtension: 'js' },
      'rxjs/operators': { main: 'index.js', defaultExtension: 'js' },
      'rxjs/testing': { main: 'index.js', defaultExtension: 'js' },
      'rxjs/websocket': { main: 'index.js', defaultExtension: 'js' },
      'rxjs': { main: 'index.js', defaultExtension: 'js' }
    }

[kylecordes]

@steveblue As I understand from that snippet, that looks like this type of configuration you would use if you are planning to load RxJS from its large number of individual JS files for each bit of functionality; whereas what I am looking for (and have been doing all along with a System bundle) is to load all of RxJS with a single HTTP request, or at most a small handful.

It seems in this thread that I might be the only person that cares about this though. Though I wonder - if few care about this for (for example) RxJS, why all the effort expended to load things like Angular as one or a small number of requests, rather than as numerous small individual files?

[steveblue]

@benlesh the only way I have found so far to bundle rxjs with closure compiler is when the rxjs entry points are bundled as FESM. I included an additional step in my build to do this, but it seems like a hack. It would probably be easy to implement as you just have to rollup the es2015 modules into FESM. Understood if UMD is hard. https://github.com/angular/closure-demo/issues/32

@kylecordes with 6.0.0 the number of entry points is way less than 5.5. I think it is just a handful of requests.

[kylecordes]

@steveblue Yes, the number of modules from which one imports is just a handful, hence this feature request. The work you did to build your FESMs, that's what I am hoping will happen in RxJS's build process. I think UMD output would be not much more complex than FESM output, though of course much less shakable.

I think these two use cases are worth a small amount of additional Rollup configuration/script?

• Easy compatibility with Closure Compiler.
• Use from a CDN with very few HTTP requests.

[mlc-mlapis]

@kylecordes ... with RxJS 6.0.0-tactical-rc.1 I tried to re-bundle to get an usable single RxJS file to load and work with SystemJS loader + Angular 6.0.0-rc.3.

I used this Gulp script:

gulp.task('rxjs', function() {
    var options = {
        normalize: true,
        runtime: false,
        sourceMaps: false,
        sourceMapContents: false,
        minify: true,
        mangle: true
    };
    var builder = new systemjsbuilder('./');
    builder.config({
        paths: {
            "n:*": "node_modules/*",
            "rxjs/*": "node_modules/rxjs/*.js"
        },
        map: {
            "rxjs": "n:rxjs"
        },
        packages: {
            "rxjs": {main: "index.js", defaultExtension: "js"}
        }
    });
    builder.bundle(
        'rxjs + ' +
        'rxjs/operators/index.js + ' +
        'rxjs/ajax/index.js + ' + 
        'rxjs/websocket/index.js + ' +
        'rxjs/testing/index.js',
        'assets/rxjs-bundle/rxjs.min.js',
        options
    );
});

and use with this config:

bundles: {
    "assets/rxjs-bundle/rxjs.min.js": [
        'rxjs',
        'rxjs/*',
        'rxjs/operators/*'
        'rxjs/ajax/*',
        'rxjs/websocket/*',
        'rxjs/testing/*'
    ]
},

Everything works as expected and the only problem to solve is the necessity to manually re-write:

rxjs/index -> rxjs
rxjs/operators/index -> rxjs/operators
rxjs/ajax/index -> rxjs/ajax
rxjs/websocket/index -> rxjs/websocket
rxjs/testing/index -> rxjs/testing

in all places in the final re-bundled rxjs.min.js.

Do you have any idea how to get the same without this manual re-write, just using some type of configuration for re-bundling?

[kylecordes]

@mlc-mlapis For the use case where I make a RxJS System bundle, I am currently (for another week or two...) working with "rxjs": "5.6.0-forward-compat.2", rather than 6, for compatibility with various other non-6 libraries. In case it helps though, the following bundle script does the job with that version:

const fs = require('fs');
const Builder = require("systemjs-builder");
const builder = new Builder('./');

builder.config({
  paths: {
    "rxjs/*": "rxjs/*.js"
  },
  packages: {
    "rxjs": {
      defaultExtension: "js"
    }
  },
  baseURL: "../node_modules"
});

// Forgive me, Internet. I need the output to work with TypeScript. As
// far as I can tell, there is currently no way at the point of
// consumption via SystemJS, to specify the esModule flag for modules
// loaded inside a System bundle.

builder.bundle('rxjs/index + rxjs/Rx', { sourceMaps: false }).then(output => {
  const code = output.source
    .split('"use strict";')
    .join('"use strict"; exports.__esModule = true;');
  fs.writeFileSync('../lib/rxjs.js', code);
});

  "bundles": {
    "lib:rxjs.js": [
      "rxjs",
      "rxjs/*",
      "rxjs/operator/*",
      "rxjs/operators/*",
      "rxjs/observable/*",
      "rxjs/scheduler/*",
      "rxjs/symbol/*",
      "rxjs/add/operator/*",
      "rxjs/add/observable/*",
      "rxjs/util/*"
    ]
  },

I suspect the bit of hackery in their to turn on __esModule is peripherally related to the rewriting you described. I feel like there is quite a bit of brainpower being spent unproductively in the current era around differences of opinion between various tool authors, about how modules should work. There will be a time when it all coalesces and just works without thinking about it, but unfortunately that time will be right _after_ we have all moved on to the next technology :-)

[mlc-mlapis]

@kylecordes ... yeap, thanks for your comment. I'll try probably something similar as you. It looks like this is the only option. The pleasing is, that the re-bundled RxJS 6 works without any problem. 😄

[aelbore]

@kylecordes @mlc-mlapis @steveblue

[email protected]
[email protected]

const Builder = require('systemjs-builder');
const promisify = require('util').promisify;
const fs = require('fs');

module.exports = done => {
  const options = {
    normalize: true,
    runtime: false,
    sourceMaps: true,
    sourceMapContents: false,
    minify: true, 
    mangle: false
  };
  const builder = new Builder('./');
  builder.config({
    paths: {
      'n:*': 'node_modules/*',
      'rxjs/*': 'node_modules/rxjs/*.js',
      "rxjs-compat/*": "node_modules/rxjs-compat/*.js",
      "rxjs/internal-compatibility": "node_modules/rxjs/internal-compatibility/index.js",
      "rxjs/testing": "node_modules/rxjs/testing/index.js",
      "rxjs/ajax": "node_modules/rxjs/ajax/index.js",
      "rxjs/operators": "node_modules/rxjs/operators/index.js",
      "rxjs/webSocket": "node_modules/rxjs/webSocket/index.js",
    },
    map: {
      'rxjs': 'n:rxjs',
      'rxjs-compat': 'n:rxjs-compat'
    },
    packages: {
      'rxjs': {
        main: 'index.js', 
        defaultExtension: 'js'
      },
      "rxjs-compat": {
        main: "index.js",
        defaultExtension: "js"
      }
    }
  });
  return builder.bundle('rxjs + rxjs/Rx', 'node_modules/.tmp/Rx.min.js', options)
    .then(output => {
      const writeFile = promisify(fs.writeFile);
      const code = output.source.replace(/rxjs\/index/gm, 'rxjs');
      return writeFile('node_modules/.tmp/Rx.min.js', 
        (options.sourceMaps) 
          ? code + `\n//# sourceMappingURL=Rx.min.js.map`
          : code);
    })
    .then(() => done())
    .catch(error => done(error));
};

My SystemJS Config

  "bundles": {
    "node_modules/.tmp/Rx.min.js": [ "rxjs", "rxjs/*" ]
  }

So far this works for me. :)

[steveblue]

I wrote a little node script that uses Rollup to bundle rxjs into FESM so Closure Compiler produces smaller bundles. It's kinda messy because I needed to edit the package.json of each rxjs package to make closure compiler happy. I'm used to doing this though now with ES2015 packages that don't have named exports (not rxjs). Closure Compiler can't handle import * from 'foo' so sometimes I need to rollup libraries into a FESM to overcome this limitation and edit the package.json so Closure Compiler knows how to interpret the module. It turns out Closure Compiler optimizes FESM way better than ESM. rxjs is ~10Kb smaller in my bundle because I built with FESM.

const path = require('path');
const fs = require('fs');
const spawn = require('child_process').spawn;
let editFile = (filePath) => {
    return new Promise((res) => {
        fs.readFile(filePath, 'utf-8', (error, stdout, stderr) => {
            let package = JSON.parse(stdout);
            package.es2015 = package.es2015.replace('_esm2015', '_fesm2015');
            console.log('editing ' + filePath);
            fs.writeFile(filePath, JSON.stringify(package), () => {
                res(filePath);
            })
        });
    });
};

let rollup = spawn('rollup', ['-c', 'rollup.rxjs.js'], { shell: true, stdio: 'inherit' });
rollup.on('exit', () => {
    console.log('rollup completed');
    Promise.all([editFile('node_modules/rxjs/package.json'),
    editFile('node_modules/rxjs/operators/package.json'),
    editFile('node_modules/rxjs/ajax/package.json'),
    editFile('node_modules/rxjs/testing/package.json'),
    editFile('node_modules/rxjs/websocket/package.json')]);
});

rollup.rxjs.js

export default [{
    input: 'node_modules/rxjs/_esm2015/index.js',
    output: {
        file: 'node_modules/rxjs/_fesm2015/index.js',
        format: 'es'
    }
},
{
    input: 'node_modules/rxjs/_esm2015/operators/index.js',
    output: {
        file: 'node_modules/rxjs/_fesm2015/operators/index.js',
        format: 'es'
    }
},
{
    input: 'node_modules/rxjs/_esm2015/ajax/index.js',
    output: {
        file: 'node_modules/rxjs/_fesm2015/ajax/index.js',
        format: 'es'
    }
},
{
    input: 'node_modules/rxjs/_esm2015/testing/index.js',
    output: {
        file: 'node_modules/rxjs/_fesm2015/testing/index.js',
        format: 'es'
    }
},
{
    input: 'node_modules/rxjs/_esm2015/websocket/index.js',
    output: {
        file: 'node_modules/rxjs/_fesm2015/websocket/index.js',
        format: 'es'
    }
}
];

[last-Programmer]

@aelbore How to make this work for development with systemjs without bundling? Thanks

[benlesh]

This was discussed ages ago, and we decided against it at the time. I'll close this, and if someone else comes across this need again, please file a new issue.