Mongodb-memory-server: Postinstall Failure `linux/mongodb-linux-x86_64-debian81-latest.tgz`

[Moggers]

Not sure if this is upstream or mongodb themselves, but the above resolved binary is throwing:

<Error>
<Code>AccessDenied</Code>
<Message>Access Denied</Message>
<RequestId>3BB9EE1261D433D5</RequestId>
<HostId>
z8fk3EzYBQAo83J9pdvWOmxLqiROkYc116+r8jpAOWlAQ7yBYz1jlFPlxHt+1OESUeB2cMy1AAU=
</HostId>
</Error>

This happens when not specifying a version - as a workaround specifying 3.4.19-rc1 seems to work.

[nkhlmn]

My travis builds recently started failing as well. Related?

> [email protected] postinstall /home/travis/build/nikhilkamineni/codex/node_modules/mongodb-memory-server
> node ./postinstall.js
mongodb-memory-server: checking MongoDB binaries cache...
failed to download/install MongoDB binaries. The error:
Error: MongoBinaryDownload: md5 check is failed
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: `node ./postinstall.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

[dankelleher]

@nikhilkamineni yes it's the same issue - the download server is reporting access denied when downloading the binary or looking up the md5.

[PacoDu]

Same error here when using npm install --save-dev mongodb-memory-server

node: v11.6.0
npm: 6.5.0

$ npm install mongodb-memory-server --save-dev

> [email protected] postinstall /home/*******/node_modules/mongodb-memory-server
> node ./postinstall.js

mongodb-memory-server: checking MongoDB binaries cache...
failed to download/install MongoDB binaries. The error:
Error: MongoBinaryDownload: md5 check is failed
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] postinstall: `node ./postinstall.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

[rocknrolla777]

workaround: you can use "MONGOMS_DISABLE_POSTINSTALL=1" variable.

MONGOMS_DISABLE_POSTINSTALL=1 npm install

[hpgmiskin]

I am having the same issue. I am able to get past the post install script using MONGOMS_DISABLE_POSTINSTALL=true yarn install which @rocknrolla777 mentions. However when running tests the binary attempts to download again and fails with the same md5 check is failed.

The output to getos() command is { os: 'linux', dist: 'Debian', release: '8.11' }

[hpgmiskin]

I have managed to install with the following command (thanks to @Moggers for suggesting to specifically define the version of mongodb)

MONGOMS_VERSION=4.0.0 yarn add mongodb-memory-server --dev

[rocknrolla777]

@hpgmiskin Is MongoDB version specified in tests?

  binary: {
    version: '4.0.5',
  },

[hpgmiskin]

@rocknrolla777 I have not got the binary version specified in tests. Thank for pointing out 👍

[chemitaxis]

Hi @hpgmiskin and @rocknrolla777, Do I need to be the same version in MONGOMS_VERSION and the binary version option? Thanks.

[hpgmiskin]

Hi @chemitaxis you can either install with the MONGOMS_DISABLE_POSTINSTALL=true flag and set the mongo version in the test script using binary version. Alternatively you can install and run with the MONGOMS_VERSION=4.0.3. I would recommend what @rocknrolla777 has done which is disable post install and then set the binary version in your test script.

[rocknrolla777]

Hi @chemitaxis, I absolutely agree with @hpgmiskin. Just disable post install and set binary version. If MongoDB version is not declared in your test script and you install node modules with a specific version (not latest), then your test script installs the latest version anyway. It means binaries will be downloaded twice.

[chemitaxis]

Ok, thanks ;)

[nodkz]

It looks like that some problems happen in MongoDB distro with binaries and md5 files.

So bumping a new MAJOR version with breaking changes:

• DISABLE md5 check by default (need to use MONGOMS_MD5_CHECK if you want to enable this check)
• DO NOT halt packages installation if cannot be downloaded binaries

[nodkz]

Current problem with latest packages for UBUNTU.

For example:
http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1804-latest.tgz
https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-latest.tgz
http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-latest.tgz
http://downloads.mongodb.org/linux/mongodb-linux-x86_64-ubuntu1404-v4.0-latest.tgz

Somebody can ping MongoDB team?

[rocknrolla777]

The same for DEBIAN -
http://downloads.mongodb.org/linux/mongodb-linux-x86_64-debian81-latest.tgz

[nodkz]

Maybe affected other OS and versions.

See all version binaries, some links are broken:
https://www.mongodb.org/dl/linux/x86_64-ubuntu1404
https://www.mongodb.org/dl/linux/x86_64-ubuntu1804
https://www.mongodb.org/dl/osx/x86_64

For now as workaround you need to find working binary manually. Eg.

MONGOMS_VERSION=4.0.0 yarn add mongodb-memory-server --dev

And use this version in your tests:

new MongoMemoryServer({
  binary: {
    version: '4.0.0', // or you use process.env.MONGOMS_VERSION = '4.0.0'
  },
});

Or use process.env.MONGOMS_VERSION = '4.0.0' instead of binary.version.

Or use dotenv

[nodkz]

TO ALL
Please Retweet and Like this tweet https://twitter.com/nodkz/status/1085575353442144258
MongoDB's Jira is too slow for fixing this issue and should be used other channels.

[chemitaxis]

Done ✅

Sent from my iPhone

On 16 Jan 2019, at 17:33, Pavel Chertorogov [e-mail] wrote:

TO ALL
Please Retweet and Like this tweet https://twitter.com/nodkz/status/1085575353442144258
MongoDB's Jira is too slow for fixing this issue and should be used other channels.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[nodkz]

Currently fixing this issue via

refactor: add LATEST_VERSION constant in code for global changing latest version on some value.
Change latest version on 4.0.3 😂 cause latest version on download return 403 http-status code. But 4.0.3 version downloads normally 🙃

Due to broken latest packages I cannot run all tests on CI and publish a new version. Damn it!

[nodkz]

Just published new MAJOR 3.0.0. version for fixing this issue.

Please check it and confirm that all works as expected in your codebases.
Except that you are working on 4.0.3 version instead of latest.

[rocknrolla777]

@nodkz thanks for quick issue resolving! 👍

[nodkz]

TO ALL
Please Retweet and Like this tweet
https://twitter.com/nodkz/status/1085575353442144258

MongoDB's Jira is too slow for fixing this issue and should be used other channels.

[mhassan1]

I passed this tweet to some friends at MongoDB. Let's see what they say.

[rvillane]

I just tried v3.0.0 but I'm getting the following error, any ideas?

20137 verbose stack Error: [email protected] postinstall: `node ./postinstall.js`
20137 verbose stack Exit status 1
20137 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
20137 verbose stack     at EventEmitter.emit (events.js:159:13)
20137 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
20137 verbose stack     at ChildProcess.emit (events.js:159:13)
20137 verbose stack     at maybeClose (internal/child_process.js:943:16)
20137 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)
20138 verbose pkgid [email protected]
20139 verbose cwd /Users/spdhw/git/direct-order-transformer
20140 verbose Darwin 17.7.0
20141 verbose argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
20142 verbose node v9.3.0
20143 verbose npm  v6.5.0
20144 error code ELIFECYCLE
20145 error errno 1
20146 error [email protected] postinstall: `node ./postinstall.js`
20146 error Exit status 1
20147 error Failed at the [email protected] postinstall script.
20147 error This is probably not a problem with npm. There is likely additional logging output above.
20148 verbose exit [ 1, true ]

[rvillane]

updating to node 10 fixes it

[mattlord]

Thank you for reporting the issue to us! We've been looking into this today and now have a fix in place. The next -latest build links should be working correctly and in the meantime we're trying to manually correct the ones already in place.

I also want to note that the -latest builds are ~ nightly builds from a given GitHub branch -- where no version means master. So linux/mongodb-linux-x86_64-debian81-latest.tgz would in fact be a ~ nightly build from the master branch, which is the 4.1 development branch today (not GA). It seems that perhaps you were assuming that -latest was a static link to the latest GA release (4.0.5 as of today)? It's reasonable given the build name, but -latest is just a build from the branch at the time -- it's not an official release. I'll look into renaming these to something more clear, e.g. "-nightly" (even though they're generated on events vs a schedule) or instead leaving those as-is and creating and maintaining new static links to get the latest GA release and/or the latest GA release for a specific major version, so for example as of right now:

linux/mongodb-linux-x86_64-debian81-stable.tgz   --> linux/mongodb-linux-x86_64-debian81-4.0.5.tgz
linux/mongodb-linux-x86_64-debian81-36stable.tgz --> linux/mongodb-linux-x86_64-debian81-3.6.9.tgz

That way you have a static link that can be used to get the latest official GA release, which is I think what you really wanted here.

Thanks again! Please let me know if you have any followup questions or issues.

[nodkz]

@mattlord thanks for fast response and clarification about latest builds. I thought that they work like latest tag in docker naming or like in cdn js lib delivery.

Anyway I will be glad to any naming for GA latest packages. Hope that -36stable.tgz is typo and you will use existed mask for package naming like it did for current latest names - -v3.6-latest. Please let me know when it will be implemented. Tnx.

And yep I also have the old issue abou similar binaries for Alpine linux - https://jira.mongodb.org/browse/SERVER-36790 Where can be download executable binary for Alpine, like it did eg. for Ubuntu. Of course if it's possible

For now this package works for most OS and CI with zero configuration and no need in preinstalled mongod on the system. Also it may spin up in parallel different versions and greatly works for parallel tests (when simultaneously worked several test processes with own mongod in-memory instance).

Thanks.

PS. In Twitter we trust! 😉

[nodkz]

@zhongjixiuxing can you open a new issue and provide small repro repo. I'll try to resolve it till the end of week. Tnx.

[zhongjixiuxing]

@zhongjixiuxing can you open a new issue and provide small repro repo. I'll try to resolve it till the end of week. Tnx.

@nodkz I'm so sorry, it's usage problem, I didn't look at the new document. I remove this my comment, thanks for your reply.

[hasezoey]

@nodkz could this be closed because it got fixed?

[hasezoey]

i just found export const LATEST_VERSION: string = '4.0.3'; and noticed the provided links by @nodkz are not denied anymore - should we revert to latest again?

[nodkz]

@hasezoey thanks for pointing 🙏 this fixed in https://github.com/nodkz/mongodb-memory-server/commit/5bfc9c004177f3142f6e809aef8e5b06e7e379fb

[nodkz]

@hasezoey reverted back due to CI build problems https://circleci.com/gh/nodkz/mongodb-memory-server/462 Have no time to figure this out. (PS: Shame to me to commit directly to the master 😬Rush is a bad thing. I bring trash to commit history. 😳)

[hasezoey]

@nodkz i just tested locally, and get a lot of STDOUT: {"t":{"$date":"2020-02-28T18:24:38.796+0100"},"s":"I", "c":"REPL_HB", "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Heartbeat to 127.0.0.1:38639 failed after 2 retries, response status: HostUnreachable: Error connecting to 127.0.0.1:38639 :: caused by :: Connection refused"}}

i have looked into this more, and it seems that it is because of

  MongoMS:MongoInstance Mongo[35459]: STDOUT: {"t":{"$date":"2020-02-28T19:17:34.544+0100"},"s":"F", "c":"-",       "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Invalid access at address: 0x20"}}
  MongoMS:MongoInstance  +0ms
  MongoMS:MongoInstance Mongo[35459]: STDOUT: {"t":{"$date":"2020-02-28T19:17:34.544+0100"},"s":"F", "c":"-",       "id":0,"ctx":"ReplCoord-0","msg":"{}","attr":{"message":"Got signal: 11 (Segmentation fault)."}}

but couldnt find a fix or bug report for this, and i dont know how mongodb's jira works
PS: latest downloads an dev version nowdays ... (on my system v4.3.3-635-gabcbc1e)