Sequelize: Error: Cannot find module 'pg-native'

[tornillo]

Exception occurs when trying to add two (or more) equal values in a unique field using findOrCreate.

Test code:

var arr = [0, 1];

var User = sequelize.define('User', {
    login: {
        type: Sequelize.STRING,
        unique: true
    }
});

User.sync({ force: true })
.then(function () {
    sequelize.transaction()
    .then(function (tx) {
        return Promise.map(arr, function () {
            return User.findOrCreate({
                where: {
                    login: 'test'
                },
                transaction: tx
            });
        })
        .then(function () {
            tx.commit();
        });
    });
});

SQL log:

Executing (default): DROP TABLE IF EXISTS "Users" CASCADE;
Executing (default): CREATE TABLE IF NOT EXISTS "Users" ("id"   SERIAL , "login" VARCHAR(255) UNIQUE, PRIMARY KEY ("id"));
Executing (default): SELECT i.relname AS name, ix.indisprimary AS primary, ix.indisunique AS unique, ix.indkey AS indkey, array_agg(a.attnum) as column_indexes, array_agg(a.attname) AS column_names, pg_get_indexdef(ix.indexrelid) AS definition FROM pg_class t, pg_class i, pg_index ix, pg_attribute a WHERE t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND a.attnum = ANY(ix.indkey) AND t.relkind = 'r' and t.relname = 'Users' GROUP BY i.relname, ix.indexrelid, ix.indisprimary, ix.indisunique, ix.indkey ORDER BY i.relname;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): START TRANSACTION;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SET autocommit = 1;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SAVEPOINT "6a28eaa0-9716-4db0-be66-967f8ec12058-savepoint-1";
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SAVEPOINT "6a28eaa0-9716-4db0-be66-967f8ec12058-savepoint-2";
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SELECT "id", "login" FROM "Users" AS "User" WHERE "User"."login" = 'test' LIMIT 1;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): SELECT "id", "login" FROM "Users" AS "User" WHERE "User"."login" = 'test' LIMIT 1;
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): CREATE OR REPLACE FUNCTION pg_temp.testfunc(OUT response "Users", OUT sequelize_caught_exception text) RETURNS RECORD AS $func_66f952b0ec354cdbb8923433f3248e4a$ BEGIN INSERT INTO "Users" ("id","login") VALUES (DEFAULT,'test') RETURNING * INTO response; EXCEPTION WHEN unique_violation THEN GET STACKED DIAGNOSTICS sequelize_caught_exception = PG_EXCEPTION_DETAIL; END $func_66f952b0ec354cdbb8923433f3248e4a$ LANGUAGE plpgsql; SELECT (testfunc.response).*, testfunc.sequelize_caught_exception FROM pg_temp.testfunc(); DROP FUNCTION IF EXISTS pg_temp.testfunc();
Executing (6a28eaa0-9716-4db0-be66-967f8ec12058): CREATE OR REPLACE FUNCTION pg_temp.testfunc(OUT response "Users", OUT sequelize_caught_exception text) RETURNS RECORD AS $func_2abdfe30c027469dac0b5fe594fb4dda$ BEGIN INSERT INTO "Users" ("id","login") VALUES (DEFAULT,'test') RETURNING * INTO response; EXCEPTION WHEN unique_violation THEN GET STACKED DIAGNOSTICS sequelize_caught_exception = PG_EXCEPTION_DETAIL; END $func_2abdfe30c027469dac0b5fe594fb4dda$ LANGUAGE plpgsql; SELECT (testfunc.response).*, testfunc.sequelize_caught_exception FROM pg_temp.testfunc(); DROP FUNCTION IF EXISTS pg_temp.testfunc();

Error stack:

Unhandled rejection Error: Cannot find module 'pg-native'
    at Function.Module._resolveFilename (module.js:336:15)
    at Function.Module._load (module.js:278:25)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/home/user/test/node_modules/pg/lib/native/index.js:1:76)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at null.<anonymous> (/home/user/test/node_modules/pg/lib/index.js:78:36)
    at /home/user/test/node_modules/lodash/index.js:3159:32
    at baseForOwn (/home/user/test/node_modules/lodash/index.js:2089:14)
    at baseClone (/home/user/test/node_modules/lodash/index.js:1799:39)
    at /home/user/test/node_modules/lodash/index.js:1800:23
From previous event:
    at Promise.catch (/home/user/test/node_modules/sequelize/lib/promise.js:21:17)
    at Object.<anonymous> (/home/user/test/node_modules/sequelize/lib/model.js:1199:15)
From previous event:
    at Promise.then (/home/user/test/node_modules/sequelize/lib/promise.js:21:17)
    at module.exports.Model.findOrCreate (/home/user/test/node_modules/sequelize/lib/model.js:1180:8)
    at /home/user/test/tests.js:30:16
From previous event:
    at /home/user/test/tests.js:29:18
    at processImmediate [as _immediateCallback] (timers.js:358:17)
From previous event:
    at Promise.then (/home/user/test/node_modules/sequelize/lib/promise.js:21:17)
    at Object.<anonymous> (/home/user/test/tests.js:26:2)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

Note: without the transaction is the same error

[tornillo]

The error occurs when calling optClone->lodash.cloneDeep from findOne and other similar methods.
The PG-module has a property 'native', which is a getter:

module.exports.__defineGetter__("native", function() {
  delete module.exports.native;
  module.exports.native = new PG(require('./native'));
  return module.exports.native;
});

During cloning this getter is called and throwing error.

Workarounds:

1: Before Sequelize initialization in application we can remove stupid getter:

var pg = require('pg');
delete pg.native;

It should be documented.

2: Delete getter when require PG in Sequelize connection-manager.js if config.native === false

3: Or we need to patch Units.clone for totally skipping property 'lib' cloning:

   cloneDeep: function(obj, fn) {
-    return lodash.cloneDeep(obj, function (elem) {
+    return lodash.cloneDeep(obj, function (elem, key) {
+      // Skip property 'lib' cloning (PG-native error workaround)
+      if (key === 'lib' && elem.hasOwnProperty('native')) {
+        return elem;
+      }
+

[janmeier]

Hmm, we've seen this problem before - optClone should be handling this problem already by not cloning transaction instances. Which version are you on?

[tornillo]

3.0.1 - latest
optClone clone all properties, pg.native is a getter.
Lodash cloneDeep iterates through all properties and gets a value of each -> getter called.

[tornillo]

...and only then transaction is skipped by checking:

elem instanceof Transaction

[janmeier]

Hmm, it seems we are trying to clone a sequelize instance - which has a reference to the pg instance... weird..

[tornillo]

Oh, I missed the important thing:
It occurs only when findOne called an the end of findOrCreate
https://github.com/sequelize/sequelize/blob/master/lib/model.js#L1207

[zooofeex]

I am having the same issue

[tornillo]

Commit https://github.com/sequelize/sequelize/commit/4f9b502e410614edfe863a7a7c3006ae4a495d1a solved the problem.
Bug is not reproduced in version 3.1.1
Thank you!

[mickhansen]

Great :)

[stoiacs]

In 3.3.2 I had the same issue, workaround by tornillo solved (delete pg.native;).

[jbrawnjr]

Still encountering the optClone issue in 3.21.0.

[benjaminhon]

i still have this error message occurring, but i have no idea why it came suddenly. At first it didn't have it then suddenly it came. Im using postgres, and its giving me and error that i have no mysql dialect in node modules as well as pg-native

[llonchj]

I'm experiencing same issue with 3.23.2 pg-native is required

[benjaminhon]

I realised that it was because i "new Sequelize()" in browserify. I removed it and it was ok.

[lrojas94]

may I ask @benjaminhon, what do you mean by removed the new Sequelize()? how do you init now?

[benjaminhon]

@lrojas94, i mean that i removed new Sequelize() from running on the client browser

[lrojas94]

I see.. I still have the issue though, and I'm not currently using it on the front-end at all... Did you manage to solve it @llonchj ??

[mz2]

Happening for me too with a many-to-many query of the following kind:

Version.findAll({
        include:[{
            model:Feed, 
            through:{
                attributes:[],
                where: {
                    feedIdentifier:appID
                }
            }
        }]

(With version 3.23.3)

[TechnicalRhino]

same in version 4.37.6. Please reopen this bug.

[TwoAbove]

Can confirm, happening with v3.30.4

[nassan]

Confirming, happening with v4.29.2

[bludnic]

In my case, I fixed this way:

Changed from this:
const sequelize = new Sequelize(process.env.DATABASE_URL, { native: true, ssl: true })

To this:
const sequelize = new Sequelize(process.env.DATABASE_URL, { dialect: 'postgres', dialectOptions: { ssl: true } })

[dancrumb]

It should be noted that this is not purely a code issue. It depends on the environment.

I have the same code running in my local environment and my staging environment.

I see this issue only in my staging environment:

May be relevant:

staging - Aurora PostgreSQL - PostgreSQL 9.6.3 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.9.3, 64-bit
local - PostgreSQL - PostgreSQL 10.2 (Debian 10.2-1.pgdg90+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 6.3.0-18) 6.3.0 20170516, 64-bit

[rzetterberg]

I have this problem in 4.37.10 too.

I've found the exact line of code which triggers this error me, it's when I'm comparing a non-persistent Sequelize instance with a regular object using: expect(myObject).to.deep.equal(mySequelizeInstance);, where expect is defined as const { expect } = require('chai');.

I guess the problem arises when Chai's deep-eql tries to inspect the native getter, or something similar.
I get the same problem if I use lodash's cloneDeep on a Sequelize instance.

So if you have code that inspects/clones a Sequelize instance you could be faced with this problem. For me the workaround was just to make sure that I got a plain object using the instance's method toJSON before doing any kind of inspection or cloning.

[Jolg42]

Hi there!

It just happened to me as well but I found why ;-)

So first the delete pg.native; didn't work for me.

What happened in my case is that I have winston as a logger and I was doing something like logger.debug({'fontFromDb': fontFromDb}).
The logger tries to log everything and call this getter that shouldn't be called and 💥

So make sure you don't have anything like a clone or a logger on an instance.
If you need to clone or log do it with a raw: true query that returns a JSON or turn the instance to JSON like this: fontFromDb.toJSON()

[Allain55]

I have this problem in 5.8.6 with NodeJS 12.2 and pg 7.11
When native is set to true, I get a "Cannot find module 'pg-native'" error message

[rusekr]

As I've seen pg-native wont compile with node 12 because of libpq wont compile https://github.com/brianc/node-libpq/pull/67

[cyrilchapon]

I'm facing this error inside my unit tests myself.

What happens is I'm testing "the return of a function" with sinon.js like so :

      sinon.assert.calledWith(
        underlyingFunction,
        sinon.match.instanceOf(MySequelizeModel)
          .and(sinon.match.has('theProp', expectedValue))
      )

When the test fails (passed sequelize instance doesn't have "theProp" equal to expectedValue);

• sinon complains and logs the sequelize instance (assuming console.log) to help me compare the expected value with actual value
• I guess the console.log (calling an underlying toString() call ?) calls "_the getter_" and causes the error

EDIT: in that special case, a .toJSON() would be ok, but I don't have the choice to call it before logging (sinon.js does it for me)

[abstructs]

I get this error when using chai "deep equals" on a model instance, using .toJSON() seemed to fix the issue

[papb]

From the looks of it, the workaround by tornillo should work for everyone.

const pg = require('pg');
delete pg.native;

[rusekr]

From the looks of it, the workaround by tornillo should work for everyone.

const pg = require('pg');
delete pg.native;

Fixed this by not installing pg-native module and sequelize config:

var db = new Sequelize(..., {
  "dialect": "postgres",
  "native": false
});

Made some research about performance drawbacks and found nothing.

[cyrilchapon]

delete pg.native doesn't work for me, neither does the native: false inside the options

[papb]

@cyrilchapon Are you sure you are applying this before anything else runs, i.e., in the first line of your entry point?

[cyrilchapon]

@papb pretty sure, but not confident.
This is in a unit-test environement.

I'll double check

EDIT : well, I confirm yes.

I'm running something like mocha test.bootstrap.js a-unit-test.js

and set up

const pg = require('pg')
delete pg.native

at the very first line of both test.bootstrap.js and a-unit-test.js

[cyrilchapon]

Just a word. This issue is 4,5 years ago.
I think one could expect a little more than an obscure an not-always-working workaround to fix an issue on a ORM, IMHO. Just to say

[cyrilchapon]

Related : https://github.com/sequelize/sequelize/issues/3101

[papb]

@cyrilchapon I agree! Hence this issue is still open.

It's unfortunate that the workaround didn't work for you... If you can investigate further it would be great! Thanks!!

[AlariCode]

So after reading pg module code, I found that problem with bundling is in index.js file:

if (typeof process.env.NODE_PG_FORCE_NATIVE !== 'undefined') {
  module.exports = new PG(require('./native'))
} else {
  module.exports = new PG(Client)

  // lazy require native module...the native module may not have installed
  module.exports.__defineGetter__('native', function () {
    delete module.exports.native
    var native = null
    try {
      native = new PG(require('./native'))
    } catch (err) {
      if (err.code !== 'MODULE_NOT_FOUND') {
        throw err
      }
      /* eslint-disable no-console */
      console.error(err.message)
      /* eslint-enable no-console */
    }
    module.exports.native = native
    return native
  })

It requires ./native file which is depending on pg-native. So to pack it with webpack we need to ignore this import. To do it just use webpack.IgnorePlugin:

new webpack.IgnorePlugin(/.\/native/),

After this addition, everything works fine.

[1valdis]

This bug blocks our production project; we use sequelize v4.38.0. And it's 4.5 years old! delete pg.native doesn't work. Can this be fixed already?

[mickhansen]

@1valdis Do you have code that does deep clone on sequelize objects? Like a logger? The issue is usually not with Sequelize, if you find it to be with Sequelize, you're more than welcome to submit a PR to fix it.

[1valdis]

@mickhansen we can't reproduce it reliably. It may or may not happen during one of requests. It may or may not happen after we update or downgrade packages. There's no situation which I can share and say "see, under this specific circumstances it triggers"...

[papb]

As AlariCode said:

So after reading pg module code, I found that problem with bundling is in index.js file

This is a problem with pg, not Sequelize, and it's being worked on very recently: see node-postgres#1894 and node-postgres#1992.

[1valdis]

@papb under pull request you referenced they turned this around on sequelize. Hmm...

[papb]

@1valdis Hmmm, thanks for pinging, I will look into it later

[Volshebniqq]

For me, this error occurred when I tried to put Sequelize instance inside of a Winston logger.

logger.info('getSegmentData', { lastAttempt, previousAttempt });

Fix:

logger.info('getSegmentData', { 
    lastAttempt: lastAttempt ? lastAttempt.get({ plain: true }) : null,
    previousAttempt: previousAttempt ? previousAttempt.get({ plain: true }) : null
});

[papb]

@Volshebniqq thanks for posting, I've seen other people have similar problems with the winston logger, perhaps it is the one trying to deep clone stuff? Does anyone know if there is a related issue in the winston repository?

[jamiesoncj]

Just to add that I see this issue when trying to use the Winston logger too - for example as shown below. Haven't found an issue in the Winston repo yet.

const transports = [
  new winston.transports.Console({
    json: true,
    colorize: true,
    handleExceptions: true,
    humanReadableUnhandledException: true,
  }),
];

const logger = winston.createLogger({
  level: 'debug',
  transports,
});

const sequelize = new Sequelize(config.databaseUrl, {
    logging: logger.debug.bind(logger),
    dialect: 'postgres',
    pool: {
      max: 5,
      min: 0,
      idle: 10000,
    },
    protocol: 'postgres',
    dialectOptions: {
      ssl: true,
    },
  });

[Eoksni]

@jamiesoncj I have the same issue as you and managed to workaround that by changing

    logging: logger.debug.bind(logger),

to

    logging: sql => logger.debug(sql),

No idea why it works, kinda makes no sense.

[jamiesoncj]

@Eoksni thank you for the tip - this works for me!

[Eoksni]

Strange, I was having this error exactly with 3.2.1 winston.

[cyrilchapon]

Please,
This issue is about sequelize models / transactions / whatever-objects messing up with whatever-logging strategy.

Maybe we should try not to polute this thread with workarounds for every-loggers-on-earth.
At the end of the day, console.log(sequelizeModel) just messes; the issue is about fixing that.

[anistark]

This issue still persists. Anybody fixed it yet?

[papb]

@anistark I think you answered your own question :sweat_smile: hopefully it will be fixed as a side-effect of #11832

[anistark]

@papb 😅 What I meant was workarounds that might have worked for others successfully.
Anyway, hope this gets merged soon. 🤞 Thanks for the swift response.

[1valdis]

pg@8 has been released, which contains a (hopefully) fix for this. Is it safe to upgrade though?... I just ran npm i -s pg@8 and my very basic routes seem to be working fine...

[1valdis]

@papb I've been using pg@8 with Sequelize 5 for a month, having Winston as logger, and made a production app on the combo without ever encountering cannot find module 'pg-native'. Can you confirm that? In fact, nothing has broken in Sequelize 5 with pg@8 (and pg changelog suggested there are only very subtle breaking changes), so I think my upgrade was totally worth it.

[type]

I had this issue with a "beforeUpdate" and "beforeCreate" hook that were (sometimes) setting my query to [Op.notIn]: [123, null] or [Op.not]: null

Fixed the [Op.not] as follows:

 const conflictCount = await sequelize.models.SuggestedTime.count({
            where: {
              // this spread syntax fixes the query by not adding anything to the where unless there is a ParentSuggestedTimeId
              ...(suggestedTime.ParentSuggestedTimeId && {
                id: {
                  [Op.not]: suggestedTime.ParentSuggestedTimeId
                },
              }),
            }
          });

The [Op.notIn] can be fixed with a similar technique or running e.g. _.compact on the array of (potentially-null-containing) values

where: {
 id: {
         [Op.notIn]: _.compact([suggestedTime.id, suggestedTime.ParentSuggestedTimeId])
      },
}

[superrandres]

It happens to me, when making the query that is linked in the image. Basically I am relating two models of different schemas with different Sequelize instances. But when I make the query from line 13 (hasMany) there are no errors, the errors occur when executing the query from line 24 (belongsTo), both queries work but the last one generates the following:

auto-rest_1 | Cannot find module 'pg-native' auto-rest_1 | Require stack: auto-rest_1 | - /app/general-persistence/node_modules/pg/lib/native/client.js auto-rest_1 | - /app/general-persistence/node_modules/pg/lib/native/index.js auto-rest_1 | - /app/general-persistence/node_modules/pg/lib/index.js auto-rest_1 | - /app/general-persistence/node_modules/sequelize/lib/dialects/abstract/connection-manager.js auto-rest_1 | - /app/general-persistence/node_modules/sequelize/lib/dialects/postgres/connection-manager.js auto-rest_1 | - /app/general-persistence/node_modules/sequelize/lib/dialects/postgres/index.js auto-rest_1 | - /app/general-persistence/node_modules/sequelize/lib/sequelize.js auto-rest_1 | - /app/general-persistence/node_modules/sequelize/index.js auto-rest_1 | - /app/general-persistence/index.js auto-rest_1 | - /app/logic/front/auth/clientes.js auto-rest_1 | - /app/auto-rest/cookiesManagment.js auto-rest_1 | - /app/auto-rest/index.js
I also have an error that I have not yet checked in MongoDb when storing hook afterFind information that is not given in other queries.
Error: cyclic dependency detected

[1valdis]

@superrandres do you use [email protected] or above?

[superrandres]

Hello @1valdis thanks for answering. I indicate the versions that I am using:
"pg": "^7.14.0",
"pg-hstore": "^2.3.3",
"sequelize": "^5.22.3"
The MongoDB error has nothing to do with this.

[1valdis]

@superrandres you have [email protected]. Try using pg@8 or above (npm i -s pg@8) and see if the error is still there. I indicated a few post ago that upgrading pg from 7 to 8 solved the issue for me.

[superrandres]

@1valdis This solved the problem, thank you very much.

[jeromeSH26]

Hi,
still got this error by just testing the connection (using native=true) in the connection options. Have to turn it to false.

The workaround to delete the pg.native property is failing as this is a readonly property (in typescript)

"pg": "^8.3.3",
"pg-hstore": "^2.3.3",
"sequelize": "^6.3.5",

const dbConnection: Sequelize = new Sequelize(
        DB_PARAMS.DB_DATABASE,
        DB_PARAMS.DB_USER,
        DB_PARAMS.DB_PWD,
        {
            host: DB_PARAMS.DB_HOST,
            dialect: DB_PARAMS.DB_DIALECT.toLowerCase() as Dialect,
            logging: DB_PARAMS.DB_LOG,
            minifyAliases: true,
            native: true,
            pool: {
                min: DB_PARAMS.DB_MINPOOL,
                max: DB_PARAMS.DB_MAXPOOL,
            },
            port: DB_PARAMS.DB_PORT,
            quoteIdentifiers: true,
            typeValidation: true,
        }
    );

[sshaginyan]

Same issue as @jeromeSH26

[soryy708]

If the failure happens at build time (e.g. when running webpack), and you don't want to use pg-native, you can add it to webpack's externals s.t. it doesn't complain if it can't find the package.

[ps2goat]

@jeromeSH26 @sshaginyan

This should work in typescript, though I haven't tested.

delete (<any>pg).native;

[ps2goat]

I have confirmed that the line above works in Typescript. I had similar issues with TypeORM, not Sequelize. It appears to be an issue with the pg npm package, which is odd. I've seen it work on other machines...

UPDATE: This fix worked with an older version of Node (10.22.1), but not 14..14.0

[papb]

Since version 8, the .native getter is no longer enumerable, therefore this problem can be avoided by upgrading to pg 8. For this reason I will close this issue now. If anyone still experiences this problem with pg 8 please let me know and I will reopen this.

However, in principle, it is Sequelize's fault to deep clone these things to begin with. I've opened #13168 to track this.