Loopback: Bundling isomorphic Loopback client with Webpack instead of Browserify

[Nutelac]

It's possible to bundle isomorphic Loopback client with Webpack instead of Browserify?

So far, I tried this:

NPM:
npm install webpack json-loader --save-dev

client/lbclient/lbclient.js

var loopback = require('loopback');
var boot = require('loopback-boot');

var client = module.exports = loopback();
boot(client);

client/lbclient/webpack.config.js:

module.exports = {
  entry: './lbclient.js',
  output: {
    path: __dirname,
    filename: 'browser.bundle.js'
  },
  resolve: {
    extensions: ['', '.js', '.json']
  },
  module: {
    loaders: [{
      test: /\.json$/,
      loader: 'json-loader'
    }]
  }
}

But i've got bunch of Module not found: Error: Cannot resolve module '[fs, net, tap, ...]' errors. I know it's because this modules are not available on client, but Browserify bundle does not have problem with that.

[BerkeleyTrue]

Have you tried https://www.npmjs.com/package/loopback-webpack-plugin?

[Nutelac]

I tried. Same problem.

[BerkeleyTrue]

Dang. I guess we need a first class plugin for loopback or add the functionality into loopback boot.

[mexpolk]

It would be also nice to consider Babel.js while implementing this feature (so that you can use ES6 on clients and servers). We use Webpack+Babel.js in our projects, so that would be awesome!

[acao]

I've had some problems using loopback && loopback-boot in a Babel context (no webpack), however it was also inside of an electron.js app. It seemed the mixin() function in loopback/lib/loopback.js was not providing express's methods although express was loading.

[me6iaton]

Same problem

[rob3c]

I can't get webpack to work with any loopback samples, either :-(

I see that loopback-boot has browserify-specific code, but I didn't trace through it much. What does loopback need to do differently in the client vs the server that it needs this code, and can it be more generic? Would it be easier to introduce webpack into a non-browserify sample and then set some LB flags or something to tell it it's in the browser?

[BerkeleyTrue]

@rob3c Because loopback relies heavily on JSON config files which are automatically imported at startup, these files need to be formatted for use with browserify/wepback which needs to know how to bundle them before hand.

Because they are automatically loaded at startup and not hard coded as requires or imports, Browserify or Webpack won't normally bundle them (this is true with any type of plugin system that automatically finds files for you at startup).

Loopback's Browserify component does just that, it runs the normal boot process but instead of caching the config files it writes them to a JS file (a pre-bundle) in such a way that when browserfied LoopBack tries to find these files on startup in the client Browerify knows where to look in the bundle.

[rob3c]

@BerkeleyTrue Thanks for the quick reply and especially the detailed info! I'm not sure I understand about the "hard coded" requirement with webpack to bundle them, though. Would you mind giving a specific example?

One significant difference between webpack and other bundlers is that webpack lets you "require anything". So that means any arbitrary asset you require() like JSON, SASS, CSS, HTML, fonts, etc. will get bundled (with or without transformation as configured) right along with your regular JS files thanks to webpack's powerful loader and plugin mechanisms. It's easy to have webpack literally inline them into a bundle along with your code in various ways. Or, as seems more relevant to your description, you can have it just include them in the dist folder alongside the bundles and return their URLs instead as the result of any require calls. Either way, by simply requiring them, they're guaranteed to be included in the output.

I bet you could easily skip any manual JSON bundling manipulation you're currently doing to make it browserify-friendly and just require them directly wherever you reference them.

Also, depending on how and where you do the require calls, you could leverage webpack's code splitting to optimize loading of 'chunks'. Then you can quickly load only the parts of your app that you need on-demand, rather than paying the cost of loading a single big bundle of everything up front when you may only be accessing a small part of the app's functionality in a given session. That would be particularly helpful, since the currently generated browserify bundle for just a simple Todo sample is almost 90k LOC unminified (yikes!), and webpack's code splitting + dead code removal could really trim that down.

[BerkeleyTrue]

I'm not sure I understand about the "hard coded" requirement with Webpack to bundle them

By hard coded I mean using require or import.

  var config = require('./config.');

One significant difference between webpack and other bundlers is that webpack lets you "require anything".

Webpack/browserify searches for these static statements and then resolves the module and includes it in the bundle. But loopback doesn't do this with all the config files mainly because they could have many different names (I'm including model.json files you create in this).

you're currently doing to make it browserify-friendly and just require them directly wherever you reference them.

Yes, you could but loopback can't know ahead of time what you are going to call your models. Hence why they look for them at runtime. loopback-boot is creating a special bundle that it then interprets when loaded on the client.

I bet you could easily skip any manual JSON bundling manipulation you're currently doing to make it browserify-friendly and just require them directly wherever you reference them.

I'm not bundling any JSON, loopback is require-ing them from disk at runtime.

I suggest you step through loopback-boots bundling procedure using Browserify to really understand what it is doing. You'll see that this isn't such an easy task to do with WebPack.

I suggest you start with https://github.com/strongloop/loopback-boot/blob/master/lib/compiler.js

depending on how and where you do the require calls, you could leverage webpack's code splitting to optimize loading of 'chunks'.

Loopback is open source. Could you form a PR to do this?

[rob3c]

Thanks for the additional detail. As you can tell, I don't know the LB internals yet, so I appreciate the tip on where to dive in!

I'll check out compiler.js next, but right now I've been looking at how the isomorphic demo's gulpfile invokes browserify to build the lbclient bundle. I stepped through the boot.compileToBrowserify() function and saw how it assembles 'instructions' for browserify that ultimately result in dynamic require calls for the model, mixin, component, etc. module source files. Along the way, the server-specific instructions that include express and such get filtered out before the client bundle is created and the modules exposed.

The first thing I'm going to try before getting carried away with anything else is to just remove that browserify dependency and recreate the essence of what the build:lb-client task is doing from within the webpack config.

Unfortunately, I'm racing the clock a bit on this as we're still evaluating LB vs some other options, and the team may decide to go with something else. Either way, I'm happy to share any webpack-related progress that seems like it might be useful.

[BerkeleyTrue]

@rob3c IMHO Loopback in the client is still needs a lot of work. It doesn't seem to garner the attention it deserves from the StrongLoop team. I have heard talks of a rebuild but have not seen any progress in that direction.

There are also better ways now to get data in the client. I'm currently evaluating Falcor to replace our REST api calls (We've been using Yahoo's Fetchr to create isomorphic api calls but I abhor their API). I abandoned LB in the client long ago after having to implement a lot of fixes on my own.

[rob3c]

@BerkeleyTrue Sharing your frustrating experience with LB on the client and the corresponding lack of attention and progress by the SL team is much appreciated, if a bit disheartening. I'm starting to get the feeling we'll be looking elsewhere...

I wasn't familiar with Fetchr, but it sounds like that may be a good thing lol. Falcor is certainly on our radar and one of the more interesting options. It's still pretty new (as an official release, that is), but I resonate with how it's structured.

[BerkeleyTrue]

Yes, Falcor is pretty new, but that's balance by the fact that Falcor is already used at Netflix. Fetchr will allow you to do REST in an isomorphic way but it definitely is lightweight and you will end up righting a lot of code that Falcor will take care of for you.

@rob3c I feel like the conversation has moved from the topic of this issue. You can ping me on Gitter if you want to chat more about it.

[Nutelac]

I think Relay or Falcor are currently best way to go.

[superkhau]

I will bump this up to major and see if if we have any plan in the works.

@bajtos @raymondfeng @ritch ^ This warrants some attention.

[bajtos]

+1 for supporting other bundlers like WebPack.

[malchak]

@superkhau Has there been any progress on this issue recently? Thanks!

[superkhau]

@malchak Not that I am aware of. Would you like to submit a patch?

[webup]

+1 for LB with Webpack and Babel~

[osenvosem]

+1 for webpack with babel

[jrhicks]

Using webpack's "externals" option I can build my front end with webpack and require the loopback client that is built by browserify. A working example is here: https://github.com/jrhicks/loopback-react-webpack

[zamb3zi]

I've been working on this for a few days and I now have a working method. It borrows ideas from the defunct loopback-webpack-plugin and it works fine with the latest webpack. To demonstrate it, I made a fork of the basic loopback example here loopback-webpack-example
It's pure webpack (no browserify needed) and can be used to create a single bundle file as in the example.

[dregre]

+1
I have essentially adopted jrhicks' solution but a Webpack integration would be preferred.

[kaktus42]

@bajtos @superkhau @raymondfeng @ritch

I will bump this up to major and see if if we have any plan in the works.

Something new about that?

[superkhau]

@kaktus42 I don't believe its anywhere near our current priorities, but PRs are always welcome. We can set aside some time to help review things, but won't be able to spend time coding up the features ourselves unfortunately ;) @bajtos can probably give a bit more insight here.

[bajtos]

Exactly as @superkhau wrote, isomorphic/universal support is not our priority for the foreseeable future.

[nsisodiya]

Loopback Sucks !! :(
I use webpack and unable to build client sdk !

[superkhau]

@nsisodiya We understand your frustrations. Would you please elaborate the problem so that we can see if we can work out a solution?

[nsisodiya]

@superkhau - Why you guys cannot give 100 lines of Pure JavaScript code which can work with webpack and can make a REST calls... to server..
Infact I wrote my own implementation and I am using it from long time but my implementation is not perfect.
Please Checkout - https://gist.github.com/nsisodiya/d3e286507dfc7579f143e7a027428284

[superkhau]

@nsisodiya Unfortunately, I'm not familiar enough with webpack to give you a good answer. @bajtos Do you want to chime in?

[bajtos]

Why you guys cannot give 100 lines of Pure JavaScript code which can work with webpack and can make a REST calls... to server..

@nsisodiya you are free to use whatever client API for making REST calls to the server, no need to stick to isomorphic/universal version of LoopBack.

Here are few options that should work in webpack:

• https://www.npmjs.com/package/request
• https://www.npmjs.com/package/superagent
• https://github.com/wcandillon/swagger-js-codegen

[nsisodiya]

@bajtos - Well, I am using superagent and request. That is not the answer I am looking for.
Please go through my gist, I do not know exact definition of API, example, for findById API, I do not know exact interface, I have coded most thing in my gist, but I still do not know this is 100% perfect or not..

[nsisodiya]

@bajtos - for making proper find API, I have to code like this...

  generateFind: function (APIMetaData) {
    return function (query) {
      var str = '';
      if (query !== undefined) {
        str = '?' + Methods.convertObjToQuery(query);
      }
      var url = Methods.getBaseUrl(APIMetaData) + '/api/' + APIMetaData.plural + str;
      return request.get(url).withCredentials().promise().then(function (res) {
        return res.body;
      });
    }
  },

convertObjToQuery: function (query) {
    return Object.keys(query).map(function (key) {
      return key + "=" + encodeURI(JSON.stringify(query[key]))
    }).join("&");
  },

This code, I wrote based on my observation in network tab. Your API documentation cannot help me in writing this kind code...

[bajtos]

@nsisodiya I understand your pain. Unfortunately I don't have any easy solution for you right now.

I do not know exact definition of API, example, for findById API, I do not know exact interface

You can use loopback-component-explorer to explore the REST API, the input parameters and outputs. This component is using loopback-swagger to generate Swagger description of application's API. It may be easier to learn about the API from the swagger spec. Also once you have Swagger, you can build a client using swagger-js-codegen I recommended earlier. That will still not give you MyModel.findById API , but at least you don't have to encode input parameters and parse response bodies yourself ((I think it generates a functions called MyModel_findById for findById).

If you are ok with using TypeScript, then you can try loopback-sdk-builder (see also https://github.com/strongloop/loopback-sdk-angular/issues/188)

You can also write your own client generator, using for example loopback-sdk-angular as the inspiration. BTW, this is the way how loopback-sdk-builder come to life too.

[Overdrivr]

Just to illustrate @bajtos point

EDIT: Err, no. It will work but not browser side, because CodeGens typescript output is for server-side.

Build script

build-client-sdk.js

var fs = require('fs');
var ts = require('typescript');
var CodeGen = require('swagger-js-codegen').CodeGen;

var file = './swagger-spec.json';
var libraryName = 'MyLib';

var swagger = JSON.parse(fs.readFileSync(file, 'UTF-8'));
var tsSourceCode = CodeGen.getTypescriptCode({ className: libraryName, swagger: swagger });
var jsSourceCode = ts.transpileModule(tsSourceCode, {}).outputText;
fs.writeFileSync(libraryName + '.ts', tsSourceCode, {flag: 'w+'});
fs.writeFileSync(libraryName + '.js', jsSourceCode, {flag: 'w+'});

Then run

lb export-api-def -o swagger-spec.json
node build-client-sdk.js

If you have a model Foo with a method findById, you can call it client-side using MyLib.Foo_findById.

[mohanasundarR]

is it possible to create bundle for loopback api server??

[bajtos]

I am afraid our focus is on LoopBack 4 and we don't have bandwidth to look into WebPack support for LB3.

[mohanasundarR]

Actually i created a exe file for lb3 and lb4. Anyway thanks for responding

for the issue.

REGARDS,

MOHANASUNDAR R,

PURSUING ELECTRONICS AND COMMUNICATION ENGINEERING,

INFO INSTITUTE OF ENGINEERING.
"YOU CAN AND YOU WILL---JUST BELIEVE"

[mohanasundarR]

Do we support webpack for lb4??

REGARDS,

MOHANASUNDAR R,

PURSUING ELECTRONICS AND COMMUNICATION ENGINEERING,

INFO INSTITUTE OF ENGINEERING.
"YOU CAN AND YOU WILL---JUST BELIEVE"

[bajtos]

Do we support webpack for lb4?

We had mixed experience with sharing code between server and client (browser), we didn't consider this feature for LB4. It's not implemented (or at least I am not aware of any such implementation) and not on our roadmap either.

Feel free to open a new issue in https://github.com/strongloop/loopback-next/issues/new where we can start with discussing the scenario at high level.

[mohanasundarR]

Sure i will open an issue to have a high level conversion.

REGARDS,

MOHANASUNDAR R,

PURSUING ELECTRONICS AND COMMUNICATION ENGINEERING,

INFO INSTITUTE OF ENGINEERING.
"YOU CAN AND YOU WILL---JUST BELIEVE"