Mongodb-memory-server: Unique-Indexes seems not working.

Created on 9 Nov 2018  ·  14Comments  ·  Source: nodkz/mongodb-memory-server

Hi,
i have this schema-index: 

profilesSchema.index(
    {
        type: 1,
        name: 1,
    },{
        unique: true,
    }
);

type and name is a combined index. This works live, but in the test i can add two documents with the same type and the same name.

I am using binary: {version: '4.0.3' } on both sides.

Are unique indexes are supported, yet?

discussion
Source
picture petershaw
👍3

Most helpful comment

I managed to work around this issue by calling:

await MyMongooseModel.ensureIndexes();

in the mocha before function of the test

picture eiabea on 19 Dec 2018
👍15 🎉9 🚀5 ❤4

All 14 comments

Publish your schema and create commands please

YuriGor picture YuriGor on 9 Nov 2018

My code:

const CompoundUnique = new mongoose.Schema({
  type:{type: String},
  name:{type: String},
});
CompoundUnique.index({
  type:1,
  name:1,
},{unique: true});
const CompoundU = mongoose.model('CompoundUnique',CompoundUnique);
let doc1 = await CompoundU.create({name:"bob",type:"human"});
let doc2 = await CompoundU.create({name:"bob",type:"human"});

Console:

MongoError: E11000 duplicate key error collection: slug.compounduniques index: type_1_name_1 dup key: { type: "human", name: "bob" }

Looks correct.

YuriGor picture YuriGor on 9 Nov 2018

I think that problem somewhere in mongoose. How i know in the latests versions need to call explicitly index creation. But maybe i'm wrong.

Anyway this package just downloads binaries from mongodb distro and starts mongo instances. It cannot affect on index creation.

PS. Try to use another engine (wiredtiger), maybe some problems with mongod engine

nodkz picture nodkz on 9 Nov 2018

I am running into the same issue and tried all engines:

  • mmapv1
    1) "before all" hook
(node:1953) UnhandledPromiseRejectionWarning: Error: undefined

Use debug option for more info: new MongoMemoryServer({ debug: true })
  • devnull
    doesn't even start
  • wiredTiger
    No unique constrains at all are working, getting duplicate entries every time

Everything worked fine as long as i only used one instance of mongodbMemoryServer, after creating a new test with it's own instance the issue occurred. The first test is working fine, but the second has the duplicate entries, so it also has something to do with the order of the tests.

eiabea picture eiabea on 19 Dec 2018

I think this is somehow a timing issue or missing await?

i disabled the auto start feature, because i think the auto start does not wait for the binary code

    mongoServer = new mongodbMemoryServer({
      autoStart: false,
      debug: true,
    });

i also disabled the "useCreateIndex" feature of the mongo client to get deprecation warnings

    await mongoose.connect(mongoUri, {
      useCreateIndex: false,
      useNewUrlParser: true,
      useFindAndModify: false,
    });

TL;DR:

  • autoStart is not waiting for the mongod to be in a running state?
  • start() returns before the indices are created?
  • creating a second instance leads to not creating the indices at all?

Here are the debug logs of my unit tests, i tried to mark some findings with "-->"

Track Controller Unit Tests

--> Before called

  Mongo[40535] Starting MongoDB instance with following options: {"port":40535,"dbName":"3922a4ed-dc44-4a79-ac59-99149a729659","uri":"mongodb://127.0.0.1:40535/3922a4ed-dc44-4a79-ac59-99149a729659","storageEngine":"ephemeralForTest","dbPath":"/tmp/mongo-mem-27846bcUyfF31M3p"} +0ms
  Mongo[40535] MongoBinary options: {"downloadDir":"/src/node_modules/.cache/mongodb-memory-server/mongodb-binaries","platform":"linux","arch":"x64","version":"latest"} +3ms
  Mongo[40535] MongoBinary: Mongod binary path: /src/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod +4ms
  Mongo[40535] MongoBinary: Download lock removed +33ms
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] MongoDB starting : pid=2795 port=40535 dbpath=/tmp/mongo-mem-27846bcUyfF31M3p 64-bit host=3b6995d53702
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] db version v4.1.6-206-g85df2fb7d7
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] git version: 85df2fb7d78178adcb59bda8a5d0d57df139e314
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] allocator: tcmalloc
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] modules: none
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] build environment:
  Mongo[40535]  +2ms
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten]     distarch: x86_64
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten]     target_arch: x86_64
  Mongo[40535] 2018-12-19T21:58:41.754+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 40535 }, security: { authorization: "disabled" }, storage: { dbPath: "/tmp/mongo-mem-27846bcUyfF31M3p", engine: "ephemeralForTest" } }
  Mongo[40535]  +1ms
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] 
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] ** NOTE: This is a development version (4.1.6-206-g85df2fb7d7) of MongoDB.
  Mongo[40535]  +34ms
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] **       Not recommended for production.
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] 
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] 
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
  Mongo[40535]  +1ms
  Mongo[40535] 2018-12-19T21:58:41.790+0000 I CONTROL  [initandlisten] 
  Mongo[40535] 2018-12-19T21:58:41.791+0000 I CONTROL  [initandlisten] ** NOTE: The ephemeralForTest storage engine is for testing only. 
  Mongo[40535] 2018-12-19T21:58:41.791+0000 I CONTROL  [initandlisten] **       Do not use in production.
  Mongo[40535] 2018-12-19T21:58:41.791+0000 I CONTROL  [initandlisten] 
  Mongo[40535]  +0ms
  Mongo[40535] 2018-12-19T21:58:41.791+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: d52f1ec4-395d-4298-8f5c-20a4735d7c09
  Mongo[40535]  +0ms
  Mongo[40535] 2018-12-19T21:58:41.792+0000 I SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
  Mongo[40535]  +1ms
  Mongo[40535] 2018-12-19T21:58:41.792+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 4.2
  Mongo[40535] 2018-12-19T21:58:41.792+0000 I SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
  Mongo[40535]  +0ms
  Mongo[40535] 2018-12-19T21:58:41.792+0000 I SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
  Mongo[40535]  +0ms
  Mongo[40535] 2018-12-19T21:58:41.793+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 61f0688c-8ee6-4163-a3b3-6d92da5bed86
  Mongo[40535] 2018-12-19T21:58:41.793+0000 I SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
  Mongo[40535] 2018-12-19T21:58:41.793+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/tmp/mongo-mem-27846bcUyfF31M3p/diagnostic.data'
  Mongo[40535]  +2ms
  Mongo[40535] 2018-12-19T21:58:41.794+0000 I NETWORK  [initandlisten] waiting for connections on port 40535
  Mongo[40535]  +1ms
  Mongo[40535] MongodbInstance: is ready! +1ms
  Mongo[40535] 2018-12-19T21:58:41.795+0000 I SHARDING [LogicalSessionCacheRefresh] Marking collection config.system.sessions as collection version: <unsharded>
  Mongo[40535] 2018-12-19T21:58:41.795+0000 I STORAGE  [LogicalSessionCacheRefresh] createCollection: config.system.sessions with generated UUID: 80c9bb10-f922-44e0-be88-8591ea2c54ca
  Mongo[40535] 2018-12-19T21:58:41.795+0000 I INDEX    [LogicalSessionCacheRefresh] build index on: config.system.sessions properties: { v: 2, key: { lastUse: 1 }, name: "lsidTTLIndex", ns: "config.system.sessions", expireAfterSeconds: 1800 }
  Mongo[40535] 2018-12-19T21:58:41.796+0000 I INDEX    [LogicalSessionCacheRefresh]      building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[40535] 2018-12-19T21:58:41.796+0000 I INDEX    [LogicalSessionCacheRefresh] build index collection scan done.  scanned 0 total records. 0 secs
  Mongo[40535]  +16ms
  Mongo[40535] 2018-12-19T21:58:41.812+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:36236 #1 (1 connection now open)
  Mongo[40535]  +7ms
  Mongo[40535] 2018-12-19T21:58:41.819+0000 I NETWORK  [conn1] received client metadata from 127.0.0.1:36236 conn1: { driver: { name: "nodejs", version: "3.1.10" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.9.144-1-MANJARO" }, platform: "Node.js v10.14.1, LE, mongodb-core: 3.1.9" }
  Mongo[40535]  +1ms

--> Deprecation warning of the mongo client (useCreateIndex: false)

(node:2784) DeprecationWarning: collection.ensureIndex is deprecated. Use createIndexes instead.

--> End of the mocha before call

Before DONE
    Create, find, delete

--> Indixes are getting created, but after starting the first test?

  Mongo[40535] 2018-12-19T21:58:41.840+0000 I SHARDING [conn1] Marking collection 3922a4ed-dc44-4a79-ac59-99149a729659.tracks as collection version: <unsharded>
  Mongo[40535] 2018-12-19T21:58:41.841+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:36238 #2 (2 connections now open)
  Mongo[40535]  +22ms
  Mongo[40535] 2018-12-19T21:58:41.849+0000 I SHARDING [conn2] Marking collection 3922a4ed-dc44-4a79-ac59-99149a729659.users as collection version: <unsharded>
  Mongo[40535] 2018-12-19T21:58:41.849+0000 I STORAGE  [conn1] createCollection: 3922a4ed-dc44-4a79-ac59-99149a729659.tracks with generated UUID: e624dcda-497d-4224-988f-62c5214efc49
  Mongo[40535]  +8ms
  Mongo[40535] 2018-12-19T21:58:41.852+0000 I INDEX    [conn1] build index on: 3922a4ed-dc44-4a79-ac59-99149a729659.tracks properties: { v: 2, unique: true, key: { guid: 1 }, name: "guid_1", ns: "3922a4ed-dc44-4a79-ac59-99149a729659.tracks", background: true }
  Mongo[40535] 2018-12-19T21:58:41.852+0000 I INDEX    [conn1]   building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn1] build index collection scan done.  scanned 0 total records. 0 secs
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn1] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn1] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I STORAGE  [conn2] createCollection: 3922a4ed-dc44-4a79-ac59-99149a729659.users with generated UUID: acc8af55-5683-4114-903c-f1c340f5a0a1
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] build index on: 3922a4ed-dc44-4a79-ac59-99149a729659.users properties: { v: 2, unique: true, key: { guid: 1 }, name: "guid_1", ns: "3922a4ed-dc44-4a79-ac59-99149a729659.users", background: true }
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2]   building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn1] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn1] index build resolved 0 duplicate key conflicts for unique index: guid_1
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] build index collection scan done.  scanned 0 total records. 0 secs
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] index build for guid_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.853+0000 I INDEX    [conn2] index build resolved 0 duplicate key conflicts for unique index: guid_1
  Mongo[40535]  +20ms
  Mongo[40535] 2018-12-19T21:58:41.870+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:36240 #3 (3 connections now open)
  Mongo[40535]  +2ms
      ✓ should create track
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] build index on: 3922a4ed-dc44-4a79-ac59-99149a729659.tracks properties: { v: 2, unique: true, key: { name: 1 }, name: "name_1", ns: "3922a4ed-dc44-4a79-ac59-99149a729659.tracks", background: true }
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1]   building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] build index collection scan done.  scanned 1 total records. 0 secs
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] index build for name_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] index build for name_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] index build for name_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.876+0000 I INDEX    [conn1] index build resolved 0 duplicate key conflicts for unique index: name_1
  Mongo[40535]  +8ms
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] build index on: 3922a4ed-dc44-4a79-ac59-99149a729659.users properties: { v: 2, unique: true, key: { email: 1 }, name: "email_1", ns: "3922a4ed-dc44-4a79-ac59-99149a729659.users", background: true }
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3]   building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] build index collection scan done.  scanned 0 total records. 0 secs
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] index build for email_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] index build for email_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] index build for email_1: drain applied 0 side writes. i: 0, d: 0, total: 0
  Mongo[40535] 2018-12-19T21:58:41.882+0000 I INDEX    [conn3] index build resolved 0 duplicate key conflicts for unique index: email_1
  Mongo[40535]  +4ms
      ✓ should fail to create track [duplicate]
      ✓ should find track by guid
      ✓ should fail to find track by guid [unknown guid]
      ✓ should find track by courseId
      ✓ should fail to find track by courseId [unknown courseId]

--> Test which is depending on unique constraint works!

      ✓ should all tracks
      ✓ should update track [courseId]
      ✓ should update track [name]
      ✓ should fail to update track [unknown guid]
      ✓ should delete track
      ✓ should fail to delete track [unknown guid]
  Mongo[40535] Shutdown MongoDB server on port 40535 with pid 2795 +48ms
  Mongo[40535] 2018-12-19T21:58:41.932+0000 I NETWORK  [conn1] end connection 127.0.0.1:36236 (2 connections now open)
  Mongo[40535] 2018-12-19T21:58:41.932+0000 I NETWORK  [conn2] end connection 127.0.0.1:36238 (1 connection now open)
  Mongo[40535] 2018-12-19T21:58:41.932+0000 I NETWORK  [conn3] end connection 127.0.0.1:36240 (0 connections now open)
  Mongo[40535]  +1ms
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I CONTROL  [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I NETWORK  [signalProcessingThread] removing socket file: /tmp/mongodb-40535.sock
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I FTDC     [signalProcessingThread] Shutting down full-time diagnostic data capture
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I CONTROL  [signalProcessingThread] now exiting
  Mongo[40535] 2018-12-19T21:58:41.933+0000 I CONTROL  [signalProcessingThread] shutting down with code:0
  Mongo[40535]  +1ms
  Mongo[40535] MongodbInstance: is failed: Mongod shutting down +0ms
  Mongo[40535] CLOSE: 0 +2ms
  Mongo[40535] Removing tmpDir /tmp/mongo-mem-27846bcUyfF31M3p +0ms

  User Controller Unit Tests
  Mongo[43355] Starting MongoDB instance with following options: {"port":43355,"dbName":"d6fd3054-21c2-4e1e-9f2e-74901205be73","uri":"mongodb://127.0.0.1:43355/d6fd3054-21c2-4e1e-9f2e-74901205be73","storageEngine":"ephemeralForTest","dbPath":"/tmp/mongo-mem-2784nbnV5osqM4mE"} +0ms
  Mongo[43355] MongoBinary options: {"downloadDir":"/src/node_modules/.cache/mongodb-memory-server/mongodb-binaries","platform":"linux","arch":"x64","version":"latest"} +1ms
  Mongo[43355] MongoBinary: found cached binary path for latest +0ms
  Mongo[43355] MongoBinary: Mongod binary path: /src/node_modules/.cache/mongodb-memory-server/mongodb-binaries/latest/mongod +0ms
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] MongoDB starting : pid=2817 port=43355 dbpath=/tmp/mongo-mem-2784nbnV5osqM4mE 64-bit host=3b6995d53702
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] db version v4.1.6-206-g85df2fb7d7
  Mongo[43355]  +26ms
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] git version: 85df2fb7d78178adcb59bda8a5d0d57df139e314
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] allocator: tcmalloc
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] modules: none
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten] build environment:
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten]     distarch: x86_64
  Mongo[43355] 2018-12-19T21:58:41.965+0000 I CONTROL  [initandlisten]     target_arch: x86_64
  Mongo[43355] 2018-12-19T21:58:41.966+0000 I CONTROL  [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 43355 }, security: { authorization: "disabled" }, storage: { dbPath: "/tmp/mongo-mem-2784nbnV5osqM4mE", engine: "ephemeralForTest" } }
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] 
  Mongo[43355]  +38ms
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] ** NOTE: This is a development version (4.1.6-206-g85df2fb7d7) of MongoDB.
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] **       Not recommended for production.
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] 
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] 
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] 
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] ** NOTE: The ephemeralForTest storage engine is for testing only. 
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] **       Do not use in production.
  Mongo[43355] 2018-12-19T21:58:42.004+0000 I CONTROL  [initandlisten] 
  Mongo[43355]  +1ms
  Mongo[43355] 2018-12-19T21:58:42.005+0000 I STORAGE  [initandlisten] createCollection: admin.system.version with provided UUID: 411b86d9-338d-41b3-882c-26455ef6c3c0
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.005+0000 I SHARDING [initandlisten] Marking collection admin.system.version as collection version: <unsharded>
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.005+0000 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 4.2
  Mongo[43355] 2018-12-19T21:58:42.005+0000 I SHARDING [initandlisten] Marking collection local.system.replset as collection version: <unsharded>
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.005+0000 I SHARDING [initandlisten] Marking collection admin.system.roles as collection version: <unsharded>
  Mongo[43355]  +1ms
  Mongo[43355] 2018-12-19T21:58:42.006+0000 I STORAGE  [initandlisten] createCollection: local.startup_log with generated UUID: 906264ef-2be1-418b-9190-a27b1d1a34ae
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.006+0000 I SHARDING [initandlisten] Marking collection local.startup_log as collection version: <unsharded>
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.006+0000 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/tmp/mongo-mem-2784nbnV5osqM4mE/diagnostic.data'
  Mongo[43355] 2018-12-19T21:58:42.007+0000 I NETWORK  [initandlisten] waiting for connections on port 43355
  Mongo[43355]  +1ms
  Mongo[43355] MongodbInstance: is ready! +0ms
  Mongo[43355] 2018-12-19T21:58:42.009+0000 I SHARDING [LogicalSessionCacheRefresh] Marking collection config.system.sessions as collection version: <unsharded>
  Mongo[43355] 2018-12-19T21:58:42.010+0000 I STORAGE  [LogicalSessionCacheRefresh] createCollection: config.system.sessions with generated UUID: 1d5fbc99-e500-47bc-9bb4-5f6efe1c6e68
  Mongo[43355]  +3ms
  Mongo[43355] 2018-12-19T21:58:42.010+0000 I INDEX    [LogicalSessionCacheRefresh] build index on: config.system.sessions properties: { v: 2, key: { lastUse: 1 }, name: "lsidTTLIndex", ns: "config.system.sessions", expireAfterSeconds: 1800 }
  Mongo[43355] 2018-12-19T21:58:42.010+0000 I INDEX    [LogicalSessionCacheRefresh]      building index using bulk method; build may temporarily use up to 500 megabytes of RAM
  Mongo[43355] 2018-12-19T21:58:42.010+0000 I INDEX    [LogicalSessionCacheRefresh] build index collection scan done.  scanned 0 total records. 0 secs
  Mongo[43355] 2018-12-19T21:58:42.011+0000 I NETWORK  [listener] connection accepted from 127.0.0.1:58666 #1 (1 connection now open)
  Mongo[43355]  +2ms
  Mongo[43355] 2018-12-19T21:58:42.012+0000 I NETWORK  [conn1] received client metadata from 127.0.0.1:58666 conn1: { driver: { name: "nodejs", version: "3.1.10" }, os: { type: "Linux", name: "linux", architecture: "x64", version: "4.9.144-1-MANJARO" }, platform: "Node.js v10.14.1, LE, mongodb-core: 3.1.9" }
  Mongo[43355]  +0ms

--> End of the mocha before call
--> No create index deprecation warning!

Before DONE
    UserController.checkPassword
      ✓ should check correct password
      ✓ should check incorrect password
    Create, find, delete
  Mongo[43355] 2018-12-19T21:58:42.023+0000 I SHARDING [conn1] Marking collection d6fd3054-21c2-4e1e-9f2e-74901205be73.users as collection version: <unsharded>
  Mongo[43355]  +11ms
  Mongo[43355] 2018-12-19T21:58:42.023+0000 I STORAGE  [conn1] createCollection: d6fd3054-21c2-4e1e-9f2e-74901205be73.users with generated UUID: cd60c0e2-80e5-4882-a3f8-1ae9011fa69c
  Mongo[43355]  +1ms
      ✓ should create user
      ✓ should fail to create user [duplicate]
      ✓ should find user by guid
      ✓ should fail to find user by guid [unknown guid]
      ✓ should find user by email
      ✓ should fail to find user by email [unknown email]

--> Test fails because 2 users are getting created with the same email address

      1) should all users
      ✓ should update user [name]
      ✓ should update user [role]
      ✓ should fail to update user [unknown guid]
      ✓ should delete user
      ✓ should fail to delete user [unknown guid]
  Mongo[43355] Shutdown MongoDB server on port 43355 with pid 2817 +50ms
  Mongo[43355] 2018-12-19T21:58:42.074+0000 I NETWORK  [conn1] end connection 127.0.0.1:58666 (0 connections now open)
  Mongo[43355]  +1ms
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I CONTROL  [signalProcessingThread] got signal 15 (Terminated), will terminate after current cmd ends
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I NETWORK  [signalProcessingThread] shutdown: going to close listening sockets...
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I NETWORK  [signalProcessingThread] removing socket file: /tmp/mongodb-43355.sock
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I FTDC     [signalProcessingThread] Shutting down full-time diagnostic data capture
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I STORAGE  [signalProcessingThread] shutdown: removing fs lock...
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I CONTROL  [signalProcessingThread] now exiting
  Mongo[43355]  +0ms
  Mongo[43355] 2018-12-19T21:58:42.075+0000 I CONTROL  [signalProcessingThread] shutting down with code:0
  Mongo[43355]  +0ms
  Mongo[43355] MongodbInstance: is failed: Mongod shutting down +0ms
  Mongo[43355] CLOSE: 0 +2ms
  Mongo[43355] Removing tmpDir /tmp/mongo-mem-2784nbnV5osqM4mE +0ms
eiabea picture eiabea on 19 Dec 2018

I managed to work around this issue by calling:

await MyMongooseModel.ensureIndexes();

in the mocha before function of the test

eiabea picture eiabea on 19 Dec 2018
👍15 🎉9 🚀5 ❤4

I encountered the same problem.

Solution from @eiabea with await MyMongooseModel.ensureIndexes(); works for me :) Thx!

But it's strange, I don't understand why for testing we need to put ensureIndexes()? Normally my application works fine, and uniqueness is checked correctly without ensuring it anyhow...

pciazynski picture pciazynski on 18 Feb 2019
👍2

Same problem here. When I run my application works fine but when I run my tests, this fails because mongoose cannot create index :(

And another question. I am populating my database to run my test. When I try to ensureIndexes() before populating an array of Items, it only inserts the first item in the array, so I have to execute it later. Like this:

// await Item.ensureIndexes() -- here the populate does not work!

await Promise.all(initialItems.map((item) => {
    Item.create(item)
}))

await Item.ensureIndexes() // Populate works fine!

Does anyone experience the same issue? 🤔

jesusvallez picture jesusvallez on 3 Apr 2019

@jesusvallez and all others: Maybe this information will help, even though it doesn't have to do directly with unique-indexes but indexes in general.

Info:

  • Setup uses ts-jest and @shelf/jest-mongodb
  • All packages are the latest

Test

const data ={ /* ... */ };

describe("Test Mongoose model billing", () => {
  it("create & save successfully", async done => {
    const entry = await new MyModel.(data).save();
   // ... rest but I'm not getting further than this anyway.
  });
});

Scenario 1 - all default in regards of ensureIndexes

      TypeError: Cannot read property 'Decimal128' of null

      at Object.<anonymous> (node_modules/mongoose/lib/types/decimal128.js:13:44)
      at Object.<anonymous> (node_modules/mongoose/lib/utils.js:7:17)
      at Object.<anonymous> (node_modules/mongoose/lib/statemachine.js:8:15)
      at Object.<anonymous> (node_modules/mongoose/lib/internal.js:7:22)

Scenario 2 - useCreateIndex as mongoose config

useCreateIndex: false,
// or
useCreateIndex: true,

Result both in output of scenario 1.

Scenario 2 - ensureIndexes as suggested by @pciazynski

Added right before the const entry ...-Part the following:

await MyModel.ensureIndexes();

which results than in

    : Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Timeout - Async callback was not invoked within the 5000ms timeout specified by jest.setTimeout.Error:

Note: The timeout duration doesn't matter here. Same result with 60000ms.

EDIT: Tried different engines as well.

bke-daniel picture bke-daniel on 14 Oct 2019

I can confirm this issue with "mongodb-memory-server": "^6.6.1" using mongodb-linux-x86_64-ubuntu1804-4.2.8.tgz.
I can also confirm that @eiabea's answer https://github.com/nodkz/mongodb-memory-server/issues/102#issuecomment-448774442 fixes the issue :+1: Thanks a lot!

This should be part of the documentation!

frederikheld picture frederikheld on 20 Jul 2020
👀1 👍1

I just tried to update the docs, but I'm not certain what to document exactly. Is this a specific issue in combination with Mongoose, or is it something that will happen with all MongoDB clients?

Can someone please describe the workaround and the reason why it is necessary in one or two concise sentences?

frederikheld picture frederikheld on 22 Jul 2020

from what i can tell, this isnt an mongodb or mongodb-memory-server issue, because all examples use mongoose, and for mongoose it is known to call either ensureIndexes or syncIndexes if the collection is already created (sometimes even if they are not created) - like in the comment from @eiabea

judging from this, i will close this issue

hasezoey picture hasezoey on 22 Jul 2020

I have understood that this is not a mongodb-memory-server issue. But it's something a lot of people will trip over when working with mongodb-memory-server. So IMHO it would be valuable to put a note into the docs that at least points to this discussion thread. Even better would be a short explanation why this is happening and how to fix it.

frederikheld picture frederikheld on 29 Jul 2020
👍1

Note that even after you call .createIndexes() or.ensureIndexes()`, you'll have to do it AGAIN if you ever drop a collection.

a613 picture a613 on 31 Aug 2020
👍1
Was this page helpful?
0 / 5 - 0 ratings

Related issues

hasezoey picture hasezoey  ·  4Comments
machadogj picture machadogj  ·  11Comments
lirbank picture lirbank  ·  5Comments
falkenhawk picture falkenhawk  ·  4Comments
yavorski picture yavorski  ·  6Comments