Loopback: A better way for customizing built-in models and models from components

[bajtos]

Consider an application with a "User" model that has few extra properties or methods compared to the built-in User model.

At the moment, the recommended solution is to create a user model extending the built-in User model, which is rather silly.

What we would like to have instead

Support redefine:true as an alternative to base: "User". A redefined "User" model is defined in common/models/user.json and scaffolded by yo loopback. This will allow redefinition of common models in the server facet.

In the long term (loopback 3.x), we were discussing a mixin-based approach, where User would be just a thin wrapper mixing in login/logout and other features provided by the current User implementation.

---

Original issue description

Possible solutions:

1. Implement namespaces, move all built-in models to loopback namespace. Application models can live in the global namespace, i.e. User extends loopback.User.
2. Implement means allowing applications to modify the built-in models, so that they can use them directly.

See also https://github.com/strongloop/loopback-datasource-juggler/issues/119

A probably related problem: the canonical project layout has three facets - "common", "server", "client". In many cases it is needed to provide different implementation of model methods on the server. The current approach is to create a new "server" model extending the shared "common" model and make any changes in this subclass. While this keeps the design clean, it's also rather cumbersome.

[bajtos]

@altsang this issue is still present and we should address it. I am reopening the issue and also added an item to Roadmap brainstorming.

[clarkorz]

@bajtos How about this workaround: manually create a BaseUser model extending User model, just before app is created. Then create a User model extending BaseUser model via JSON file.

Detail:

// server/server.js
var loopback = require('loopback');
var boot = require('loopback-boot');

loopback.createModel('BaseUser', {}, { base: 'User' });

var app = module.exports = loopback();
....


// common/user.json
{
  "name": "User",
  "plural": "users",
  "base": "BaseUser",
  ...
}

[bajtos]

I don't think that solves the problem. As @raymondfeng wrote in strongloop/loopback-datasource-juggler#119, model names are unique and models must not be overriden/replaced.

[clarkorz]

@bajtos I realize that this issue is about to discuss a "better way". While I am just finding and leveraging workarounds for the moment. Sorry about this.

BTW: _create a user model extending the built-in User model_ This workaround requires use "user" as model name in all places, use "User" to retrieve model may return the built-in model in some case (loopback.findModel('User') !== loopback.findModel('user'), but app.models['user'] === app.models['User'], check https://github.com/strongloop/loopback/issues/464).

[fabien]

Here's an idea: given the new dynamic mixin functionality, it could be beneficial to implement some of the core models' functionality as mixins. In general, mixins are to be used when inheritance proves difficult.

For example, instead of:

var Member = modelBuilder.extend('User', { ... });

Do:

var Member = modelBuilder.extend('PersistedModel', { ... });
Member.mixin('User', { ... });
// and also:
var Client = modelBuilder.extend('PersistedModel', { ... });
Client.mixin('User', { ... });

Example mixin: user.js

module.exports = function(Model, options) {
   var fk = inflection.camelize(Model.modelName + '_id', true);
   Model.hasMany('AccessToken', { foreignKey: fk });
   ...
   Model.confirm = function (uid, token, redirect, fn) {
     ...;
   };
   ...
};

[frankcarey]

This is more than silly, it's a real pain when trying to use the passport stuff. Just to change the User model, I have to extend about 5 other models? Why can't we extend/modify the base model at least in code within models/user.js ? I tried that and it didn't seem to pick up the file when no user.json file was installed as well.

[clarkorz]

@frankcarey Are you using loopback.component-passport? It allows to setup a custom model as User model as long as it's a sub model of User.

And for extending built-in models, I'm currently using the second solution that @bajtos introduced. It's simple and works with loopback.component-passport, but you have to extend User model programmatically. You can create a boot script under _server/boot_ to do that. E.g.:

module.exports = function (app) {
  var User = app.models.User;
  User.defineProperty(...);
  User.validatesFormatOf(...);
  User.hasMany(...);
  User.myLogin = function () {...};
  ...
}

Note the script name, make sure it runs before any other boot scripts.

[frankcarey]

That seems to be working.. currently just trying to add a relationship of hasMany to User.. why so hard?

Note: to be explicit for others using this technique, I called the script above to /server/boot/0-user-override.js so that it runs before anything else.

UPDATE:
For those looking for the specific code:

module.exports = function (app) {
  var User = app.models.User;
  var SomeOther = app.models.SomeOther;

 // Example to add a relationship to User to some "Other" Model
 User.hasMany(SomeOther, {as: 'others', foreignKey: 'userId'});

 //console.log('override user');
};

[frankcarey]

@clarkorz THANK YOU! I'd been banging my head against that for a few days.. even started switching to sails instead because I couldn't progress.

[frankcarey]

It would be helpful to understand WHY we can't just extend the base User model while preserving the name? @clarkorz solution seems to be working well enough for me.

[bajtos]

It would be helpful to understand WHY we can't just extend the base User model while preserving the name?

My understanding is that loopback-datasource-juggler does not fully support extending existing models. @raymondfeng can you confirm?

If that's not the case, then it makes sense to provide a first-class support for extending models in both loopback and loopback-boot.

[raymondfeng]

We don’t allow you to create a sub model with the same name as the base. It will create a lot of confusions.

But the juggler does allow you to change the existing model. For loopback, we just need to have a way to express ‘redefine’ or ‘customize’ an existing model.

Thanks,

---

Raymond Feng
Co-Founder and Architect @ StrongLoop, Inc.

StrongLoop makes it easy to develop APIs in Node, plus get DevOps capabilities like monitoring, debugging and clustering.

On Sep 12, 2014, at 11:16 AM, Miroslav Bajtoš [email protected] wrote:

It would be helpful to understand WHY we can't just extend the base User model while preserving the name?

My understanding is that loopback-datasource-juggler does not fully support extending existing models. @raymondfeng can you confirm?

If that's not the case, then it makes sense to provide a first-class support for extending models in both loopback and loopback-boot.

—
Reply to this email directly or view it on GitHub.

[bajtos]

Here's an idea: given the new dynamic mixin functionality, it could be beneficial to implement some of the core models' functionality as mixins. In general, mixins are to be used when inheritance proves difficult.

I like this idea a lot. However, it is a breaking change, I'd like to come up with a solution that works in current loopback 2.x.

I have two approaches in my mind.

Option A

Customize a model at the definition time. Add a new option patch:true to model definition json. Such definition will trigger a special handling in loopback:

• no new model is created, but an existing model is modified (customized) instead
• the base class cannot be changed, or perhaps can be changed only to a model extending the original base class
• there may be more limitations necessary to preserve sane behaviour

Example:

// server/models/user.json
{
  "name": "User",
  "patch": true,
}

// server/models/user.js
module.exports = function(User) {
  // override methods, etc.
}

Option B

Patch the model at the configuration time. It is already possible to modify certain aspects of model relations via server/model-config.json, we need to add an option to specify a custom script file to execute.

// server/model-config.json
{
  "User": {
    "dataSource": "db",
    "runAfterAttach": "model-configs/user.js"
  }
}

// server/model-configs/user.js
module.exports = function(User, app) {
  // any code to be called after the model is attached
}

Dispute

Now that I wrote it down, I am strongly preferring the first option:

• Option A is more consistent, as all model-related scripts are run at the definition time, before the model is attached to an app and/or a datasource. Option B runs the code after the model was attached, which makes certain changes impossible (e.g. relation changes). This adds complexity that will be difficult to explain to new users.
• It seems to me that Option A will be easier to implement in tools like generator-loopback and StrongLoop Studio, since it adds just another option to the existing workflow for creating models. On the other hand, Option B adds a completely new workflow.

@ritch @raymondfeng et all watchers: What is your opinion? Can you come up with a third option? I'd like to get the feedback by the end of this week.

[ritch]

"patch": true,

This has my vote... its essentially a mixin instead of inheritance. Instead of a flag what about:

{"mix": "User", ...}

...or similar. The name is really not required when we know we are patching / mixing in anything defined in the model instead of extending.

[bajtos]

No no, this is different from mixins. There is a different issue/PR to support mixins in boot JSON files - see https://github.com/strongloop/loopback-boot/pull/33.

This is changing an existing model instead of creating a new one via inheritance. Possibly even _removing_ stuff, not only adding (mixing in) new stuff.

[raymondfeng]

I propose to use "redefine": true. It's the term used by XML schema. See http://www.w3schools.com/schema/el_redefine.asp. XML schema also has override which is designed to replace a model with the same name.

[ritch]

No no, this is different from mixins. There is a different issue/PR to support mixins in boot JSON files - see strongloop/loopback-boot#33.

You are missing the point. This is still essentially mixins. You say "customized" I'm saying "mixin". Maybe the mix property doesn't make sense because that is easily confused with something else. The point that we don't need a flag + name is still valid though.

This is changing an existing model instead of creating a new one via inheritance.

You will have a hard time convincing me that isn't exactly what a mixin is.

Thinking about this a bit more... I don't think we should have @fabien 's mixins and this... They are so similar that we should be able to do "patch": true style definitions with mixins.

Perhaps something like:

// common/mixins/my-mixin.js
module.exports = function(Mixin, options) {
  Mixin.foo = function() { console.log(options.bat); };
}

// common/mixins/my-mixin.json
{"bat": "baz", "properties": {"myProperty": {"type": "String"}}}

// server/model-config.json
{
  "User": {
    "dataSource": "db",
    "mixins": ["myMixin"]
  }
}

// somewhere in the app
User.foo(); // baz

// options are merged
User.properties.myProperty // => from my-mixin.json

[ritch]

Here's an idea: given the new dynamic mixin functionality, it could be beneficial to implement some of the core models' functionality as mixins.

This could be done as we move core models into components.

[bajtos]

I see, now the mixins idea makes more sense.

I am worried that if you all you want to is to add a new property to User model, you have to create a new mixin, it may feel like too much overhead. Or not.

Regardless of the underlying mechanism, I still believe models should be modified at "definition" time (i.e. via */models/my-model.json, not at the configuration time */model-config.json. One of the arguments is that there is no "configuration" shared between the client and the server. Your approach means that you have to keep client and server model config in sync to make sure you have the same set of mixins (properties, methods) on both sides.

[bajtos]

I was thinking that maybe we are going wrong direction here. What if we reworked loopback core to expose built-in models in the same way as custom models are exposed in apps now. I.e. move lib/models/* to common/models/* and split the definition to both json and js.

When we add namespaces to names as @ritch suggested elsewhere (e.g. loopback.common.User), then there is no need to change built-in models, everyone can simply subclass.

[raymondfeng]

It boils down to a list of questions:

1. How to define the built-in models (json + js or pure js)
2. How to make built-in models visible (loopback.common.User etc)
3. How to configure the official User model in the app
4. How to extend/redefine built-in models
5. How to make built-in models available to studio/generator

Maybe we should add an option to slc loopback so that we can generate submodels of built-in ones so that we can buy some time?

I think the component architecture can have potential good influence for the longer term solution

[ritch]

I was thinking that maybe we are going wrong direction here. What if we reworked loopback core to expose built-in models in the same way as custom models are exposed in apps now. I.e. move lib/models/* to common/models/* and split the definition to both json and js.

I like this approach... as long as we aren't duplicating code (specifically a model's definition: properties, ACLs, etc) from core (or components) into an app. I want to be able to upgrade components and core without having to re-add or manually update model definitions in my app.

[bajtos]

We had an online chat with @ritch and @raymondfeng to discuss the next steps.

The easiest solution is to re-introduce "user extending User" from loopback 1.x. The "user" model is defined in common/models/user.json and scaffolded by yo loopback. Benefits: integration into generators and Studio is for free. Cons: inconsistent naming (most models start with an upper-case letter).

A better solution is to implement redefine:true. A redefined "User" model is defined in common/models/user.json and scaffolded by yo loopback. Benefits: clean solution, allows redefinition of common models in the server facet. Cons: requires changes in generators and Studio. I am not sure how the UX of the GUI should look like - the effort estimate depends on that.

In the long term (loopback 3.x), we were discussing a mixin-based approach, where User would be just a thin wrapper mixing in login/logout and other features provided by the current User implementation.

@ritch @raymondfeng please add anything I missed out.

[frankcarey]

I would recommend moving the login/logout, email confirmation, pw reset,
etc to a new Auth model instead of using User.

Cheers,

Frank Carey

On Mon, Oct 6, 2014 at 12:44 PM, Miroslav Bajtoš [email protected]
wrote:

We had an online chat with @ritch https://github.com/ritch and
@raymondfeng https://github.com/raymondfeng to discuss the next steps.

The easiest solution is to re-introduce "user extending User" from
loopback 1.x. The "user" model is defined in common/models/user.json and
scaffolded by yo loopback. Benefits: integration into generators and
Studio is for free. Cons: inconsistent naming (most models start with an
upper-case letter).

A better solution is to implement redefine:true. A redefined "User" model
is defined in common/models/user.json and scaffolded by yo loopback.
Benefits: clean solution, allows redefinition of common models in the
server facet. Cons: requires changes in generators and Studio. I am not
sure how the UX of the GUI should look like - the effort estimate depends
on that.

In the long term (loopback 3.x), we were discussing a mixin-based
approach, where User would be just a thin wrapper mixing in login/logout
and other features provided by the current User implementation.

@ritch https://github.com/ritch @raymondfeng
https://github.com/raymondfeng please add anything I missed out.

—
Reply to this email directly or view it on GitHub
https://github.com/strongloop/loopback/issues/397#issuecomment-58047453.

[raymondfeng]

@frankcarey +1 to have a separate Authenticator model. But it should be separated from this issue.

[raymondfeng]

@bajtos Thank you for the summary. It captures the discussion well.

No matter which solution we decide, it needs to provide choices to:

1. Use the built-in model as-is
2. Extend/redefine the built-in model
3. Override the built-in model

Please note built-in models are not only for DRY (Don't repeat yourself), but also as the system models to support core functionality of LoopBack such as ACL. These system models have assumed properties and methods, which are typically defined as interface or mixin.

[pulkitsinghal]

For posterity, here is a gist with all the ways known to me for extending the built-in User model via boot script: https://gist.github.com/pulkitsinghal/43d4bae2467c686ec55b

I find it handy to use over the user and User model approach ... until the extend/redefine feature is implemented.

[bajtos]

Related: how to customize the Change model used for change-tracking. https://github.com/strongloop/loopback/issues/926

[uNki23]

hi there!

i'm facing the same problems: pain to customize the built in User model. Extending it by using my own 'user' model isn't working good with angular (injecting 'user' into the function doesn't work, 'User' does, but refers to the base model - see: https://stackoverflow.com/questions/29458355/loopback-angularjs-extending-user-model?answertab=active#tab-top) is there any best practice solution to customize the base User model? i'm doing in the boot-script way with code. thank you!

[pulkitsinghal]

@uNki23 - I switched to the nomenclature where I create a file named user-model.json which extends the base User and named it UserModel ... it prettifies everything for me in core and angular code ... a bit wordy though.

[alFReD-NSH]

I want to add a property to the built-in Role model. Any idea how to do this currently?

[uNki23]

correct me if i'm wrong, but it seems to work fine if you just "use" the property.

here is a sample function within an AngularJS UserController:

vm.addUser = function () {
User.create({
firstName: vm.firstName,
lastName: vm.lastName,
division: vm.division,
status: vm.status,
email: vm.email,
password: vm.password
})
.$promise
.then(function (user) {
console.log('Added User: ' + user.firstName + ' ' + user.lastName + '.');
});
};

works fine to me, although the first four properties are no properties of the base User model. I've not extended the built in User model in any way. the data is written to mongodb without any problem and i can access the properties by using User.firstName or User.division e.g.

[alFReD-NSH]

I'm trying to make it work in postgresql. It would work, if I built it in the schema myself but then I want loopback build the schemas.

[bajtos]

I want to add a property to the built-in Role model. Any idea how to do this currently?

You should be able to do it using the following workaround:

// server/server.js

var loopback = require('loopback');
// etc.

// ADD THIS LINE BEFORE CALLING boot()
loopback.Role.defineProperty('newPropertyName', { type: 'string' });

boot(app, __dirname);
// etc.

[mikesparr]

If anyone needs example of how to move the built-in user models to a datasource, I used the example-mysql repo exercise and added a few more automigrate functions to the script and changed DS for the built-in models. https://github.com/mikesparr/loopback-example-users-to-mysql Hope this is helpful for others.

[clarkbw]

Didn't notice this discussion previously, I made an initial pass at creating a User model purely from mixins in #1578. I agree with the above that defining everything in JS instead of the JSON does seem like a lot of overhead, if a mixin could provide a JSON definition file to be merged that might be an interesting way to clean up some of the code. You can see in my I created a 'properties' directory (email propery) because I assumed I'd be reusing those properties in higher level mixins

[mbeauchamp7]

Hi, has this been resolved?

[DragonTorus]

Hi! Is there any oficial, normal way(not workaround) to redefine base Model and its relations in loopback 2.27.0 & jugler 2.45.2? Thanks.

[JonnyBGod]

How is this feature coming up? Any timeframe for this to be implemented?

The lack of this feature is a big burden especially when you are crating a publicly available API where model names should make sense. Client is very different from User...

And using lower cased "user" breaks naming norms and integrations with third party libraries.

[jcolemorrison]

I was content to just leave it as user for now, despite the fact it's pretty goofy and causes confusion. But when just setting up something to autoupdate the base User model and the extended user model, both seem to automigrate each other. Therefore just wound up having to switch the user over to Client.

And @JonnyBGod is on point. Above anything else it creates a ridiculous goof in terms of naming and public facing APIs. As silly as it sounds, this one limitation is a constant pain point I tend to be annoyed with more than most other things in loopback.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been closed due to continued inactivity. Thank you for your understanding. If you believe this to be in error, please contact one of the code owners, listed in the CODEOWNERS file at the top-level of this repository.

[MichaelPlaxico]

This is still a relevant issue that causes serious problems in terms of API design and integration with third-party libraries. Is a fix being planned?

[bajtos]

This is still a relevant issue that causes serious problems in terms of API design and integration with third-party libraries. Is a fix being planned?

Not in the current code base. We are working on a new version of LoopBack -
see https://github.com/strongloop/loopback-next - where issues like this one should be easier to address.

[RishiMahendru]

Hi @bajtos is there a way to override persistedModel class methods and add our custom logic to used across all custom models

[bajtos]

@RishiMahendru sure. Create your own base model inheriting from PersistedModel, and then use this model as a base for all your application models.

This question is pretty off-topic here, please post in on StackOverflow if you need more information.