Sails: Sails v1.0 & sails-mongo: Can't create a document with custom id value

[sailsbot]

_From @Josebaseba on February 28, 2017 3:53_

sails: v1.0.0-27
sails-mongo: 1.0.0-4

I can't create a document with a custom id like: '58959f6f6e6d1b620acc7a00', it always gets overwritten by another "random" ID. I tried in all the possible ways but I couldn't.

Plus, if you have your models migration option set to 'alter' the id of all the objects keep changing, so if you have a model with an association to that other document you lose it.

It took me like an hour to realize about what was happening :)

_Copied from original issue: balderdashy/sails-mongo#456_

[sailsbot]

@Josebaseba Thanks for posting, we'll take a look as soon as possible. In the meantime, if you haven’t already, please carefully read the issue contribution guidelines and double-check for any missing information above. In particular, please ensure that this issue is about a stability or performance bug with a documented feature; and make sure you’ve included detailed instructions on how to reproduce the bug from a clean install. Finally, don’t forget to include the version of Node.js you tested with, as well as your version of Sails or Waterline, and of any relevant standalone adapters/generators/hooks.

Thank you!

[sailsbot]

_From @sgress454 on March 1, 2017 22:54_

Hi @Josebaseba -- we're aware of the issue and are working on it. We'll post back when there's a patch!

[sgress454]

This is fixed as of sails-mongo v1.0.0-5.

[algope]

Hello @sgress454, thanks for the update!

I've been testing it with version v1.0.0-5 and I'm still having the same issue when trying to insert a document with a custom id: AdapterError: Unexpected error from database adapter: Invalid primary key value provided for 'id'. Cannot interpret '600000' as a Mongo id.

[mikermcneil]

@algope There's a new flag you'll be able to use for this in Sails v1-- I know sgress454 is @looking at making sure it's in the docs now, so I'll let him cover the details, just wanted to give you the heads up in the mean time

[algope]

@mikermcneil Awesome, thank you for the quick reply.

[sgress454]

@algope This should be all set with the latest beta of Waterline and sails-mongo. The easiest way to get it in your project is:

npm cache clean
npm install sails-hook-orm@beta --force
npm install sails-mongo@beta --force

Then you should be able to set dontUseObjectIds: true in your model, and set up the id attribute however you want, e.g.:

js // api/models/User.js module.exports = { dontUseObjectIds: true, attributes: { id: { type: 'number', columnName: '_id' }, // <-- still need to set `columnName`! name: { type: 'string' }, ...etc... } }

Give that a whirl and let us know if it works for you!

[algope]

@sgress454 Confirmed! It works like a charm 😍
Thanks a lot for the work!

[SwiTool]

I don't know if it's linked but by using User.create(fields) i got the warning but it still creates the document. It looks like an ObjectID:

Warning: Records sent back from a database adapter should always have a valid property
that corresponds with the primary key attribute (`id`).  But in this result set,
after transforming columnNames back to attribute names for model `user`,
there is a record with a missing or invalid `id`.
Record:

{ id: 5c10d38c63e37b4b189e141a,
  createdAt: 1544606604697,
  updatedAt: 1544606604697,
  email: '[email protected]',
  emailPaypal: '',
  password: '$2a$10$xIL1WggFc523uIt8ogokA.JSRRAwrtl5ethg2tuzfIaslbVrL/9Wi',
  fullName: 'John Doe',
  isAdmin: false,
  isVerified: false,
  passwordResetToken: '',
  passwordResetTokenExpiresAt: 0,
  createdWithIp: '::ffff:127.0.0.1',
  credit: 0,
  lastSeenAt: 0 }

> You are seeing this warning because there are records in your database that don't
> match up with your models.  This is often the result of a model definition being
> changed without also migrating leftover data.  But it could also be because records
> were added or modified in your database from somewhere outside of Sails/Waterline
> (e.g. phpmyadmin, or another app).  In either case, to make this warning go away,
> you have a few options.  First of all, you could change your model definition so
> that it matches the existing records in your database.  Or you could update/destroy
> the old records in your database; either by hand, or using a migration script.
>
> (For example, to wipe all data, you might just use `migrate: drop`.)
>
> More rarely, this warning could mean there is a bug in the adapter itself.  If you
> believe that is the case, then please contact the maintainer of this adapter by opening
> an issue, or visit http://sailsjs.com/support for help.

sails version : ^1.1.0
sails-mongo version : ^1.0.1

My config/models.js :

module.exports.models = {
  schema: true,
  migrate: 'drop',
  dontUseObjectIds: true,
  attributes: {
    id: { type: 'number', columnName: '_id', autoIncrement: true, },
  }
   dataEncryptionKeys: {
    default: 'xxx'
  },
  cascadeOnDestroy: true
};

And my User.js :

module.exports = {
  dontUseObjectIds: true,
  attributes: {
    createdAt: { type: 'number', autoCreatedAt: true, },
    updatedAt: { type: 'number', autoUpdatedAt: true, },

    //....
  }
}

And when I want to create the user :

var newUserRecord = await User.create({
      email: newEmailAddress,
      emailPaypal: newEmailPaypal,
      password: await sails.helpers.passwords.hashPassword(inputs.password),
      fullName: inputs.fullName,
      createdWithIp: this.req.ip,
    })
    .intercept('E_UNIQUE', 'emailAlreadyInUse')
    .intercept({name: 'UsageError'}, 'invalid')
    .fetch();

Do you guys have an idea ?

[Josebaseba]

Hey @SwiTool, id: { type: 'string', columnName: '_id' } that's the proper configuration for the mongo adapter.

[SwiTool]

Hello @Josebaseba I got the same exact error message with string type.
It still looks like an ObjectID :

{ id: 5c112f310733356f4fe1e872,
  createdAt: 1544630065818,
  updatedAt: 1544630065818,
  email: '[email protected]',
  emailPaypal: '',
  password: '$2a$10$nDpJxfhNHToUDBDf/CNpo.15zimQzrJW7gym9cd.BB5iNAzoeIXju',
  fullName: 'John Doe',
  isAdmin: false,
  isVerified: false,
  passwordResetToken: '',
  passwordResetTokenExpiresAt: 0,
  createdWithIp: '::ffff:127.0.0.1',
  credit: 0,
  lastSeenAt: 0 }

[Josebaseba]

Don't use the dontUseObjectIds flag.

[SwiTool]

Yes but I need it to test the relations of my models and the logic integrations.
Without dontUseObjectIds I can't predict ObjectIDs so can't establish connections between models.

For logic integrations I don't need this, but still. if it's not possible i'll just comment every model test and won't load the fixtures that are associated to them.

[Josebaseba]

You can create a custom id without that flag:

var newUserRecord = await User.create({
      id: 'objectidstring',
      email: newEmailAddress,
      emailPaypal: newEmailPaypal,
      password: await sails.helpers.passwords.hashPassword(inputs.password),
      fullName: inputs.fullName,
      createdWithIp: this.req.ip,
    })
    .intercept('E_UNIQUE', 'emailAlreadyInUse')
    .intercept({name: 'UsageError'}, 'invalid')
    .fetch();

[SwiTool]

Sorry to bother you but I already tried and ended with :

  0 passing (3s)
  1 failing

  1) "before all" hook:
     AdapterError: Unexpected error from database adapter: Invalid primary key value provided for `id`.  Cannot interpret `1` as a Mongo id.
(Usually, this is the result of a bug in application logic.)
For more info on Mongo ids, see:
• https://docs.mongodb.com/manual/reference/bson-types/#objectid

The fixture loaded is this one :

[
  {
    "model":"user",
    "items":[
      {
        "id": "1",
        "email":"[email protected]",
        "fullName": "Foo Bar",
        "password": "123",
        "createdWithIp": "::1"
      },
      {
        "id": "2",
        "email":"[email protected]",
        "fullName": "Bar Foo",
        "password": "123",
        "createdWithIp": "::1"
      }
    ]
  }
]

[Josebaseba]

You need to use a real mongo id, something like this: "5af2d22aa7cec4083c4c4c4d"

[SwiTool]

Indeed. Thanks a lot :)

[mykoman]

You need to use a real mongo id, something like this: "5af2d22aa7cec4083c4c4c4d"

Could you expatiate what you mean by a real mongo id please? I am experiencing the same issue

[Josebaseba]

@mykoman sure, if you want to create a new document with a custom _id, it must be a valid mongoId.

You can't use a custom type _id, it must follow mongoId rules, but you can generate an _id like this:

https://docs.mongodb.com/manual/reference/method/ObjectId/#generate-a-new-objectid

To access that method from the sails object layer:

sails.getDatastore().driver.mongodb.ObjectID()