Mongoose: OverwriteModelError with mocha 'watch'

[floatingLomas]

I just updated from 3.4.0 to 3.5.1, and when 'watching' Mocha tests on a model, every time the file reloads, I get a OverwriteModelError from re-'require'ing - and I guess, redefining - the model.

There must be some value in making an 'overwrite' an error, but I'm going back to 3.4.0 for now because this is too much of a pain.

[aheckmann]

This is caused by a programming error. Calling either mongoose.model() or connection.model() with the same name and schema multiple times does not result in an error. Passing the name of a model that exists with a different schema produces the error. Overwriting models was never actually permitted.

If mocha watch re-requires your schema and calls mongoose.model(preExistingName, newSchema) then yes you'll get an error now. Previously in this scenario, mongoose wasn't compiling a new model at all, it would just swallow the programming error and return the old model, which is wrong behavior.

var schema = new Schema;
var A = mongoose.model('A', schema);
var B = mongoose.model('A', schema); // no error
var C = mongoose.model('A', new Schema); // previously unreported error

[floatingLomas]

Ok, that makes sense. Thanks!

[davisford]

@aheckmann -- i'm also hitting this, and not sure what i'm doing wrong. below is how i've got it set up. first clean test run is ok, if I edit + save a file, mocha triggers a re-run with --watch and I get

OverwriteModelError: Cannot overwrite "User" model once compiled.

user.js

var mongoose = require('mongoose'),
  Schema = mongoose.Schema,
  ObjectId = mongoose.SchemaTypes.ObjectId;

var userSchema = new Schema({
  name: { type: String }
}), 
  User;

// other virtual / static methods added to schema

User = mongoose.model('User', userSchema);
module.exports = User;

userSpec.js

var User = require('../models/user'),
  mongoose = require('mongoose');

mongoose.connection.on('error', function (err) {
  console.log(err);
}

mongoose.connection.on('open', function () {
  console.log('connected');
}

describe('User', function () {

  before(function (done) {
    mongoose.connect('mongodb://localhost/test');
    done();
  }

  after(function (done) {
    mongoose.disconnect(function () {
      console.log('disconnected');
      done();
    });
  });

  it('should test something', function (done) {
     // tests
     done();
  });
});

[aheckmann]

for mongoose tests we just create new connections each time and it works well.

var db = mongoose.createConnection()

https://github.com/LearnBoost/mongoose/blob/master/test/common.js#L74-L98

[davisford]

Posted follow up to google groups https://groups.google.com/d/topic/mongoose-orm/PXTjqqpaDFk/discussion

[j0ni]

I'm seeing this too. Here's a minimal example:

https://github.com/j0ni/mongoose-strangeness

Perhaps I'm missing the right way to define models...?

[aheckmann]

https://github.com/j0ni/mongoose-strangeness/blob/master/test.spec.js#L21

should be var Example = connection.model('Example', ExampleSchema)

[j0ni]

Thanks @aheckmann that works great.

[aheckmann]

it works b/c a new connection is created for each of your tests and the model is cached locally within the connection for each. the other way fails b/c models were getting compiled at the mongoose module level for each test, hence the conflicts.

[ragulka]

@aheckmann - I am getting this error when trying to test my models in a Locomotive app.

Basically, each test file boots the Locomotive app in the before() function. Booting the app creates a mongo db connection and loads in all my models - this is in my mongoose initializer file (runs once on app startup):

var mongoose = require("mongoose"),
    fs = require("fs");

module.exports = function() {

  // Connect to DB
  switch (this.env) {
    case 'development':
      mongoose.connect('mongodb://localhost/slipfeed');
      break;
    case 'test':
      mongoose.connect('mongodb://localhost/slipfeed');
      break;
    case 'production':
      mongoose.connect('mongodb://mongodb.example.com/prod');
      break;
  }

  // Load models  
  var app = this,
      modelPath = app.get('models');
  fs.readdirSync(modelPath).forEach(function(file) {
    var name = file.substr(0, file.lastIndexOf('.'));
    require(modelPath + '/' + name)(mongoose, app);
  });

}

In each of my model files, I am basically doing this:

module.exports = function(mongoose, app) {
  var Schema = mongoose.Schema;
  var User = new Schema(...);
  app.User = mongoose.model('User', User);
});

And in each of my model test files I am doing something like this:

var locomotive = require("locomotive"),
    app = new locomotive.Locomotive(),
    should = require("should");

describe( "User", function() {

  before( function (done) {
    app.boot( process.cwd(), 'test', function () {
      done();
    });
  });

  after( function (done) {
    mongoose.disconnect( function (err) {
      if (err) throw err;
      console.log('DISCONNECT')
      done();
    })
  });      

  ...tests go here

});

The app.boot() part just starts the server, it loads the configuration file and runs through intializers (which are just files that contain various pieces of code, for example starting db connections).

But after my first test file is finished, and mocha tries to load the next file, I will get the OverwriteModelError.

I understand that it probably has something to do with connections not being closed when loading the next test file, or perhaps I am initializing my models in a bad way.

Anyway, I tried adding mongoose.disconnect to my after() functions, but that did not help.

[aheckmann]

it has nothing to do with open or closed connections. you are attempting to redefine a model which already exists to a different schema.

var a = new Schema({ x: 'string' });
var b = new Schema({ x: 'string' });
mongoose.model('Thingy', a);
mongoose.model('Thingy', a); // ok, a === the registered schema instance
mongoose.model('Thingy', b); // error a !== the registered schema

[j0ni]

FWIW, I think some slightly less trivial example code would prevent this from coming up repeatedly. Perhaps a tiny express app, which defines a single model, initializes the connection in a way which is injectable, and a mocha integration test (in my case I'm using supertest, but whatever really) which injects the connection without triggering this problem.

If there is some app out there that you know of @aheckmann that we could peer at, and that you consider implements a canonical pattern, that would do the trick.

Here's what I have resorted to:

https://github.com/j0ni/beachenergy.ca/blob/master/datamodel/index.js

I don't like it, but it makes the problem go away. If I'm using a mongoose anti-pattern (seems likely) a canonical example would help.

[davisford]

Hi @aheckmann -- I'm 100% sure I am not creating a new model with a different schema. I made a skeleton / demo project to illustrate this issue: https://github.com/davisford/mongoose-test

Start mocha with make and the first time through, tests will pass. Now, edit models/user.js and save (e.g. CTRL/CMD + S), and mocha picks up the change with --watch, and we hit OverwriteModelError.

[pgraff]

Here is another option... I use nodemon. This ensures that the process is restarted each time the test runs:
$ nodemon --exec "mocha -R min" test

[OliverJAsh]

So what's the recommended solution to this? Create a new connection before each test? How else can we define a Mongoose model in our specs and not have it error whilst watching?

[pgraff]

I gave up trying to solve it with mocha alone. Instead what I do is that I run mocha via nodemon. This way, the node process is restarted and the problem is eliminated.

Petter Graff
(sent from my iPad, spelling mistakes likely :)
+1.512.784.3232
petter.[email protected]

On Feb 2, 2013, at 5:41 AM, Oliver Joseph Ash [email protected] wrote:

So what's the recommended solution to this? Create a new connection before each test? How else can we define a Mongoose model in our specs and not have it error whilst watching?

—
Reply to this email directly or view it on GitHub.

[OliverJAsh]

How do you run mocha via nodemon exactly? I tried this, but I'm probably far off:

nodemon /usr/local/share/npm/bin/mocha --reporter spec --ui bdd --watch ./server/test/*.js

[pgraff]

I use:
nodemon --exec "mocha -R min" test

Given the example you show below, it seems you use spec rather than min and have your tests in server/test... If so just change the parameters...

On Feb 2, 2013, at 7:00 AM, Oliver Joseph Ash [email protected] wrote:

How do you run mocha via nodemon exactly? I tried this, but I'm probably far off:

nodemon /usr/local/share/npm/bin/mocha --reporter spec --ui bdd --watch ./server/test/*.js
—
Reply to this email directly or view it on GitHub.

[OliverJAsh]

Nice one. Not quite as fast as --watch, but more importantly, it works.

I also noticed that when changes are made to the schema that the Mocha test requires, Mocha will not reload them as they are cached :(

On 2 Feb 2013, at 15:23, Petter Graff [email protected] wrote:

I use:
nodemon --exec "mocha -R min" test

Given the example you show below, it seems you use spec rather than min and have your tests in server/test... If so just change the parameters...

On Feb 2, 2013, at 7:00 AM, Oliver Joseph Ash [email protected] wrote:

How do you run mocha via nodemon exactly? I tried this, but I'm probably far off:

nodemon /usr/local/share/npm/bin/mocha --reporter spec --ui bdd --watch ./server/test/*.js
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub.

[yourcelf]

@j0ni's solution worked well for me. I ended up replacing all calls to "mongoose.model" with:

try {
    // Throws an error if "Name" hasn't been registered
    mongoose.model("Name")
} catch (e) {
    mongoose.model("Name", Schema)
}

I agree that it's not pretty to have to litter one's code with such calls. It'd be nice to have a shorthand for this that works for the most common case, which is that a module is simply being re-parsed at runtime with no change to the schema. Maybe call it something like a mongoose.getOrCreate(name, schema).

[coreycollins]

@yourcelf took your suggestion and worked for me. Thanks.

module.exports = getOrCreateModel('User', UserSchema);

[remicastaing]

There is another option: cleaning mongoose from all the models and schemas.
In my test code, I added:

 mongoose.models = {};
 mongoose.modelSchemas = {};

And It works fine, at least for me.

[sidwood]

@remicastaing Perfect, works for me. Thanks!

[plus-]

@remicastaing best workaround so far!

[leoasis]

@remicastaing Also worked for me, seems to be the best solution so far

[sodoku]

@remicastaing yes, looks like a working solution.

It still don't know what the actual problem is though.

[pdesantis]

@remicastaing Thanks, works for me as well!

[gastongarcia]

@remicastaing Working for me too. thanks.

[iandoe]

Sorry to scratch an old itch, but i am facing the same problem trying to run mocha -w with mongoose, tried every possible solution but now i am hitting the wall.

where should i put

 mongoose.models = {};
 mongoose.modelSchemas = {};

?

beforeEach, afterEach, after, before ?

Should i mongoose.createConnection() on beforeEach, close on afterEach ?

[remicastaing]

I put the

mongoose.models = {};
mongoose.modelSchemas = {};

right after

var mongoose = require('mongoose');

in a config.test.js file directly in the test folder (config.test.js the only *.test.js file in the test folder, other *.test.js files are in sub folders). As mocha go recursively ( --recursive option in mocha.opts) through the test folder, it starts with config.test.js.

[asci]

@remicastaing thanks! Work for me as well (jasmine)

[asci]

Just another case:
I had this issue, when I required model with different letters case in code and in test.
For example in spec:
var User = require('./models/user');
and in code
var User = require('./models/User');
Just keep in mind, there was no cached version for different typing

[mafiuss]

+1 to what @asci experienced. If you require the same model but with a different case in two different files, and then you require those in another, it will throw the 'Cannot overwrite' error

[juanpabloaj]

@iandoe I use mongoose.connect and put in the mocha test

after(function(done){
  mongoose.models = {};
  mongoose.modelSchemas = {};
  mongoose.connection.close();
  done();
});

[ArnaudRinquin]

:+1: :bow:

[joaom182]

:+1:

[vkarpov15]

If you want to create a model in beforeEach, create a new connection with var db = mongoose.createConnection(); and then db.model(...) as opposed to mongoose.model(...); in beforeEach. This behavior is by design rather than a bug.

[LiamKarlMitchell]

I have code that can be reloaded at runtime so I want to overwrite an existing model if I make a change to the schema or model functions during development.

Unless there is a good reason not to I don't see why I shouldn't be able to do something like this.

Example

delete mongoose.models['somemodel'];
var somemodel = db.mongoose.model('somemodel', someschema);

This works for me, I have been doing it that way for about a year.

Or maybe there is a way to change the schema and model functions without wiping an existing model/schema but I have not bothered to investigate further.

[hellboy81]

My be this solution can be helpful

@yourcelf has right.

[zirho]

:+1:

[ph3b]

I know this is old but if anyone stumbles upon this I solved it by making a database test utility file like this:

// DatabaseUtils.js
import mongoose from 'mongoose';
const MONGO_URI = 'mongodb://localhost:27017/collectionname'

export function tearDown(){
  mongoose.models = {};
  mongoose.modelSchemas = {};
  resetDb();
  if(mongoose.connection.readyState) mongoose.disconnect();
}

export function setUp(){
  if(!mongoose.connection.readyState) mongoose.connect(MONGO_URI);
}

export function resetDb(){
  if(mongoose.connection.db) mongoose.connection.db.dropDatabase();
}

Then from your test file you can use:

import { setUp, tearDown, resetDb } from './DatabaseUtils'

describe('Testing MongoDB models', function(){
  before(setUp)
  after(tearDown)
  afterEach(resetDb)

  it('Some test of a mongoose model', () =>  {
  // Test code here.
  }
})

[gunar]

This StackOverflow answer solves it like this:

let users
try { users = mongoose.model('users') }
catch (e) { users = mongoose.model('users', <UsersSchema...>) }

Edit: As pointed out by @DanielRamosAcosta, "The problem is that if your Schema changes, changes won't take efect."

[miqmago]

Thanks @remicastaing, I've lost 5 minutes until I reached your solution, should be more evident :)

[developius]

Thanks @juanpabloaj, that worked for me 🎉

[DanielRamosAcosta]

@gunar The problem is that if your Schema changes, changes won't take efect.

[fega]

@juanpabloaj Thanks bro!

[markotom]

I simply use, instead of create an ugly try/catch:

let users = mongoose.models.users || mongoose.model('users', <UsersSchema...>)

[tomyam1]

This discussion is really shameful
The actual authors of mongoose gave the correct solution (1 and 2) yet all the thumb ups goes to incorrect hacky solutions.
Really shameful

[fega]

@tomyam1 , maybe because those hacky solutions is solving our problems without any problems... :smiley:

[vkarpov15]

Sometimes a convenient solution works better than a technically correct one :)

[garethdown44]

@tomyam1 There is nothing shameful about collaborating to solve a problem and I am grateful to everyone who has posted up their own take on a solution here. Without going into detail - in my case, one of the 'incorrect hacky' suggestions has worked perfectly, and both of what you are calling 'correct' solutions did not.

Helping one another out is not shameful. Thanks everyone that has made a constructive post.

[tomyam1]

Well sirs (garethdown44, fega), then we disagree.
To avoid creating more spam, please use the emojis reactions.

To sum up this thread:

There are only three different solutions in this page:

1. Use cached model if it exists:

mongoose.models.users || mongoose.model('users', <UsersSchema...>)

or

try {
    // Throws an error if "Name" hasn't been registered
    mongoose.model("Name")
} catch (e) {
    mongoose.model("Name", Schema)
}

This is not a solution because it won't pick up changes in the models.

1. Clear the model cache

 mongoose.models = {};
 mongoose.modelSchemas = {};

Tried this solution and it didn't work, and who knows why?
It relies on modifying undocumented internal variables.
It probably did work when it was proposed in 2013, but it's 4 years later.

1. Save the model on the connection.
   This is the official solution that was proposed by the authors of mogoose.
   And yet it was almost completely overlooked. For me, this is shameful.

const Mongoose = require('mongoose');
const DB = Mongoose.createConnection(...);
const Model = DB.model("Name", schema);

[garethdown44]

@tomyam1 - Agree in principle with what you're saying. However to be fair that solution did come two years after the other (hacky) solution!!

In my case I chose the nodemon solution. When I realised what was happening and the way the test code was structured it would have required a big refactoring which I don't have time to do. I'm glad that the nodemon solution was proposed and that people +1'd it. That's all I'm saying.

[DanielRamosAcosta]

Sadly, Nodemon restarts the entire test suite. mocha --watch has much faster reload

[carchrae]

Sadly, Nodemon restarts the entire test suite, so mocha --watch has much faster reload

a slow reload and sane developer is tradeoff i'll make over slicing miliseconds (or seconds) off a test run.

also, as it hasn't been mentioned here, npm-watch is a tool to do this (it wraps nodemon) so you can define a simple npm script watch https://www.npmjs.com/package/npm-watch

[campbellgoe]

I had the same issue, so I check mongoose.modelNames() and determine if I compile the model or just retrieve the already compiled model, as mocha --watch causes this issue. So here is code:

mongoose.modelNames().indexOf("User") === -1 //if my model has not been compiled...
 ? mongoose.model("User", UserSchema) //...compile model and return it
 : mongoose.connection.model("User"); //else it is already compiled, so return this model

now you return this as a function (replace "User" with argument, and UserSchema with the argument for your Schema) for module.exports and when you require it you call this function.

https://stackoverflow.com/a/49806603/5674976

[maciekgrzybek]

@remicastaing works like a charm, thanks!

[thiagof]

On some special instantiations, it may be required to clear the models references

after((done) => {
  Object.keys(mongoose.models).forEach((k) => {
    delete mongoose.models[k];
  });
  Object.keys(mongoose.modelSchemas).forEach((k) => {
    delete mongoose.modelSchemas[k];
  });
  mongoose.connection.close();
  done();
});

[Crowbrammer]

mocha --watch does not rerequire anything outside of describe, i.e. won't redefine your Schemas.

The most efficient, elegant, and understandable way to test that makes the least number of connections to the db is to set up your connection, schemas, and models outside of your test suite.

This code is very DRY compared to my other example of setting up a model before each test (not necessary for me).

The following test suite works:

const expect = require("chai").expect;
const mongoose = require("mongoose"),
  UserSchema = require("../data/models/User");
const connection = mongoose.createConnection(
  process.env.MONGO_URL || "mongodb://127.0.0.1/test"
);
const User = connection.model("User", UserSchema);

describe("Database Testing", function() {
  it("MongoDB is working and repeatedly testable", function(done) {
    let user = User({
      username: "My user"
    });

    user
      .save()
      .then(doc => {
        console.log(doc); // This outputs the doc.
        expect(doc.username).to.equal("My user");
        done();
      })
      .catch(err => {
        console.error(err);
        expect(err).to.be.null;
        done();
      });
  });
});

From ../data/models/User.js

let mongoose = require("mongoose");

let UserSchema = new mongoose.Schema({
  username: String
});
module.exports = UserSchema; // 

I used to be in a state of confusion about how MongoDb made its connections, Schemas, and models. I didn't realize that you could define one model and use it many times (obv. but...)

I followed the documentation exactly and made a module that defined the schema and returned a model.

This meant that if I required it in a test, the Schema would be redefined many times, because it keeps rerequiring the module that defines the schema. And multiple definitions of the Schema are no bueno.

People's solutions (that work, albeit less efficiently) have you create a new connection before each test and often times a new model before each test.

But a simple understanding that:

• mocha --watch doesn't execute anything outside of describe more than once
• you can use the same connection, Schema, and model multiple times for different things.

solved this efficiently and elegantly.

Below is my less efficient, original way for getting this to work.

./test/db.spec.js

const expect = require("chai").expect;
const mongoose = require("mongoose"),

  // mocha --watch does not rerequire anything
  // outside of the test suite ("describe").
  // Define your Schemas and models separately, so
  // that you can define your Schema once above
  // your test suites, and your models many times
  // when testing. 
  UserSchema = require("../data/models/User");

describe("mongoose strangeness", function() {
  var connection = mongoose.createConnection(
    process.env.MONGO_URL || "mongodb://127.0.0.1/test"
  );

  // I tried the beforeEach and afterEach, but this wasn't needed:

  // beforeEach(function(done) {
  //   connection = mongoose.createConnection(
  //     process.env.MONGO_URL || "mongodb://127.0.0.1/test"
  //   );
  //   connection.once("open", function() {
  //     done();
  //   });
  // });

  // afterEach(function(done) {
  //   connection.close(function() {
  //     done();
  //   });
  // });

  it("MongoDB testable", function(done) {
    let User = connection.model("User", UserSchema);

    let user = User({
      username: "My user"
    });

    user
      .save()
      .then(doc => {
        console.log(doc); // This outputs the doc every time.
        expect(doc.username).to.equal("My user");
        done();
      })
      .catch(err => {
        console.error(err);
        expect(err).to.be.null;
        done();
      });
  });
});

Happy MongoDbing.

[verheyenkoen]

This code works for me:

if (mongoose.modelNames().includes('Model')) {
    mongoose.deleteModel('Model');
}

mongoose.model('Model', new mongoose.Schema({ ... }));

[vkarpov15]

@verheyenkoen is right. Here's the link to the deleteModel docs: https://mongoosejs.com/docs/api/connection.html#connection_Connection-deleteModel