Mongodb-memory-server: Fails to run on Debian Buster (CURL_OPENSSL_3 not found)

[grantcarthew]

Debian Buster, the latest release of Debian GNU/Linux was released on 4th of July 2019.

I upgraded my developer virtual machine to Buster and the mongodb-memory-server hangs.

To test it out I ran this script:

const { MongoMemoryServer } = require('mongodb-memory-server')

async function main () {
  try {
    console.log('Start')
    const mongod = new MongoMemoryServer()
    console.log('Server created...')
    const uri = await mongod.getConnectionString()
    console.log('1')
    const port = await mongod.getPort()
    console.log('2')
    const dbPath = await mongod.getDbPath()
    console.log('3')
    const dbName = await mongod.getDbName()
    console.log('4')

    // some code
    //   ... where you may use `uri` for as a connection string for mongodb or mongoose

    // you may check instance status, after you got `uri` it must be `true`
    console.log('Getting Instance Info...')
    console.dir(mongod.getInstanceInfo())
    mongod.getInstanceInfo() // return Object with instance data

    // you may stop mongod manually
    await mongod.stop()

    // when mongod killed, it's running status should be `false`
    console.dir(mongod.getInstanceInfo())
    console.log('End')
  } catch (err) {
    console.error(err)
  }
}

main()

When running this script the console output displays:

Start
Server created...

After a few seconds the console prompt returns with no error output.

Am I missing something here?

[nodkz]

Please try MongoMemoryServer({ debug: true }) and provide its input here.

[grantcarthew]

Hi @nodkz,

Thanks for the hard work and response.

Here is the result:

Autostarting MongoDB instance...
Called MongoMemoryServer.start() method:
Server created...
Called MongoMemoryServer.ensureInstance() method:
Starting MongoDB instance with following options: {"port":45613,"dbName":"a74753cd-cec0-4662-8795-c9acc837acac","uri":"mongodb://127.0.0.1:45613/a74753cd-cec0-4662-8795-c9acc837acac","storageEngine":"ephemeralForTest","dbPath":"/tmp/mongo-mem-5467Cw2IQ3JGR3sO","tmpDir":{"name":"/tmp/mongo-mem-5467Cw2IQ3JGR3sO"}}
MongoBinary options: {"downloadDir":"/fm/fm-api/node_modules/.cache/mongodb-memory-server/mongodb-binaries","platform":"linux","arch":"x64","version":"4.0.3"}
MongoBinary: Mongod binary path: /fm/fm-api/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.3/mongod
Mongo[45613]: Called MongoInstance._launchKiller(parent: 5467, child: 5478):
MongoBinary: Download lock removed
Mongo[45613]: STDERR: /fm/fm-api/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.3/mongod: /usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by /fm/fm-api/node_modules/.cache/mongodb-memory-server/mongodb-binaries/4.0.3/mongod)

Mongo[45613]: CLOSE: 1
Mongo[45613]: [MongoKiller]: exit - [0,null]

Looks like CURL_OPENSSL_3 is the issue.

sudo apt install libcurl3
[sudo] password for grant:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package libcurl3 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  libcurl4

E: Package 'libcurl3' has no installation candidate

The libcurl4 package is already installed. Must be one of the dependencies upgraded with the Buster release.

No solution yet. Working on it.

[grantcarthew]

No MongoDB on Buster yet.

Follow the progress and vote here: https://jira.mongodb.org/browse/SERVER-37768

[grantcarthew]

Sorry to dump this in your lap @nodkz but I am rolling back my VM to the snapshot I took prior to the upgrade. I imagine we need to wait until there is a release of MongoDB on Buster before this will be rectified. I'll keep an eye on this issue.

Thanks again for you hard work. This is a great project.

[grantcarthew]

Update: https://jira.mongodb.org/browse/SERVER-37768

Issue is status is closed

Resolution is fixed

[grantcarthew]

Tested this just now. It is still an issue:

• mongodb-memory-server v5.2.3
• Downloaded mongod v4.0.3

[grantcarthew]

Hi @nodkz

There is still an issue but maybe not with this package.

I just upgraded my dev VM to Debian 10 and tried it again with no luck.

The postinstall.js script builds this download link https://fastdl.mongodb.org/linux/mongodb-linux-x86_64-debian10-4.0.3.tgz which does not exist.

Looking at the MongoDB downloads for Linux it looks like there is no debian10-4.0.3 download yet.

[nodkz]

@grantcarthew thanks for reporting!
@msanguineti I've just reverted your pull request #229

Not all MongoDB binaries will be rebuilt under debian10. And for now, we should create a better mechanism for version controlling. Eg. if mongo below v4.2.0 then use debian9 as max version, if upper 4.2.0 then Debian 10 is available.

For now, we have just 4 builds debian10:

Anyway, right now I have no bandwidth to resolve this issue myself. Sorry.

Pull Request welcome ;)

[msanguineti]

I did notice it, unfortunately.

You do have a 4.0.3 version hardcoded as 'latest' version. I understand that this is because the true 'latest' version is also a problem.

A better solution would be in case of a download problem to fetch a list of valid versions from mongodb.org by os and architecture and present it to the user (as part of the error message)

I might give it a shot.

[msanguineti]

@grantcarthew I don't know for you, but for me this is a major problem. Since debian 10 is now officially the 'stable' release and not having something like mongodb-memory-server supporting it is a bit of bummer. So, I will pin to version 5.2.7 and just pass in the config:

{
  "binary": {
    "version": "latest"
  }
}

@nodkz this is obviously not a trivial fix. But the '4.0.3' hard-coded version trick might come back biting your hand. It would be preferable to leave my patch in (since Debian 10 is the stable version) and let it fail with a meaningful message such as: default version x.y.z not found. Please specify a version in the config

[grantcarthew]

Life saver! Thanks @msanguineti

I've got it working now.

For anyone else looking for a temporary fix, I used my package.json file with the following config:

{
  "config": {
    "mongodbMemoryServer": {
      "version": "latest"
    }
  },
  "devDependencies": {
    "mongodb-memory-server":  "5.2.7"
  }
}

[scottpatrickwright]

As as a side note I burned way more time with this issue than necessary due to the lack of an error being thrown on a failed call to await the result of getConnectionString().

If this is trivial - with gratitude and respect - is there reasoning around not just throwing an error which could be caught and logged? I am having a hard time imagining a scenario where I would want MongoDB Memory Server to silently swallow a complete failure to start up.

[grantcarthew]

Hey @scottpatrickwright, check this out: https://github.com/nodkz/mongodb-memory-server/commit/2ba4ae5

I think that covers what you are talking about.

[scottpatrickwright]

@grantcarthew thanks for the reply. It looks like it. I am assuming then that instanceFailed() triggers throwing an error rather than just logging to the console? This is so that it can either be handled or if it is unhandled it will surface the precise issue in an error that bubbles up and brings down the process.

FWIW my situation would be an example of the issue around not throwing the error. All my tests break and it's not clear why. I manage to pin point the issue by only running a small subset of tests that still produced the error - then more or less line by line logging to find out where the flow broke down. Even then it wasn't clear as I wasn't aware of the debug=true option (my fault of course) so my code just sat there awaiting a promise until my tests timed out and I still had very little to work with in terms of diagnosing the actual issue.

[scottpatrickwright]

Also from what I can see from a quick skim this method instanceFailed() does not actually throw an error. Although this line appears to:

https://github.com/nodkz/mongodb-memory-server/blob/04a0a8c6533c791cd2704d5c037eca4f6ae74622/packages/mongodb-memory-server-core/src/util/MongoBinaryDownload.ts#L86

[hasezoey]

If this is trivial - with gratitude and respect - is there reasoning around not just throwing an error which could be caught and logged?

the problem is that the structure currently dosnt allow such logging
structure:

1. getArchiveNameLinux gets called
2. getArchiveNameLinux calls getLinuxOSVersionString to get the os string
3. getLinuxOSVersionString calls getDebianVersionString which gives something like debian10 back
4. getArchiveNameLinux appends the wanted version (by default 4.0.3)

and the current structure dosnt allow it because there is no checking if the version is even valid (because it is just the string assembler) and no os-specific code is "allowed" in there

so like @nodkz said, it would require version-string generation (and probably download) to be rewritten

[hasezoey]

because this issue gone off-topic, i will close it
-> made an issue with a proposal on how to fix some things: #258

PS: current fix is to overwrite the version to be used (default: 4.0.3, change it to: 4.2)

[hane-smitter]

I got the same problem of (CURL_OPENSSL_3 not found) on my linux machine. But i managed a workaround by trying to install a different version of the debian by choosing a different debian distribution on the mongodb download page. I hope this one gonna help somebody!