Sails: using sails-mongo adaptor makes server unresponsive

[LouAdrien]

.Node version: 11.0.0
Sails version _(sails)_: 1.2.3
ORM hook version _(sails-hook-orm)_: 2.1.1
Sockets hook version _(sails-hook-sockets)_: 2.0.0
Organics hook version _(sails-hook-organics)_: 0.16.1
Grunt hook version _(sails-hook-grunt)_: 4.0.1
Uploads hook version _(sails-hook-uploads)_: not present
DB adapter & version _(e.g. [email protected])_: [email protected]
Skipper adapter & version _(e.g. [email protected])_: [email protected]

---

Using the default web app template,
Setting datastore to use sails-mongo adapter with the following setting :

url: 'mongodb://localhost/dbname'

Will result in the server very quickly become unresponsive (but unrelatedly to any direct action).
The adapter can connect to the db (using blueprints to create models will successfully insert entries to db collections for example).
KILL signature as to be sent to the process (cannot kill it with ctrl+c)

Disable the adapter fixes the issue instantly.

[sailsbot]

@LouAdrien Thanks for posting! We'll take a look as soon as possible.

In the mean time, there are a few ways you can help speed things along:

• look for a workaround. _(Even if it's just temporary, sharing your solution can save someone else a lot of time and effort.)_
• tell us why this issue is important to you and your team. What are you trying to accomplish? _(Submissions with a little bit of human context tend to be easier to understand and faster to resolve.)_
• make sure you've provided clear instructions on how to reproduce the bug from a clean install.
• double-check that you've provided all of the requested version and dependency information. _(Some of this info might seem irrelevant at first, like which database adapter you're using, but we ask that you include it anyway. Oftentimes an issue is caused by a confluence of unexpected factors, and it can save everybody a ton of time to know all the details up front.)_
• read the code of conduct.
• if appropriate, ask your business to sponsor your issue. _(Open source is our passion, and our core maintainers volunteer many of their nights and weekends working on Sails. But you only get so many nights and weekends in life, and stuff gets done a lot faster when you can work on it during normal daylight hours.)_
• let us know if you are using a 3rd party plugin; whether that's a database adapter, a non-standard view engine, or any other dependency maintained by someone other than our core team. _(Besides the name of the 3rd party package, it helps to include the exact version you're using. If you're unsure, check out this list of all the core packages we maintain.)_
  

  ---

Please remember: never post in a public forum if you believe you've found a genuine security vulnerability. Instead, disclose it responsibly.

For help with questions about Sails, click here.

[johnabrams7]

@LouAdrien Thanks for bringing this to attention. Just to check, have you reviewed the Using MongoDB with Sails docs to rule out any missing config? Any additional config to your project beyond the MongoDB adapter?

[LouAdrien]

Hello, yes I have indeed reviewed the docs you mentioned.

To be more sure of what was happening, I can reproduce this issue with a new, "empty" project (I chose empty when generating the project with sails new) with only two modifications :
1) a policy '*':true to allow all routes
2) a single model "Company" containing one member field type string
3) switch to sails-mongo adaptor and follow the directions listed here

After reloading the homepage and the /company page listing the models, the server will become unresponsive.

Any ideas about where this could come from? Which other info would you need?
Regards,

[LouAdrien]

@johnabrams7 can you reproduce this issue on your side? Should I report this on the mongo adapter repo? I thought as this happen while following some of the basic steps of the getting started guides, this should be handled as a main repo issue.

Please tell me how I can help solve this, but I would really need a fix to make sure we can keep using sails in our project...

[johnabrams7]

@LouAdrien I'm going to test this out on mongo - recently upgraded to macOS Catalina and went through the curious new hurdles to get it going again lol (Great article for Installing MongoDB on macOS _Catalina_ here).

In the meantime, yes, sharing a mongo adapter repo can still be helpful to capture it in this exact state for the issue. Appreciate all the info on this so far 👍

[LouAdrien]

Ok great, FYI In case it has any impact, I am using docker on my computer as the mongodb instance for the mongo adapter (btw, you should try it, install a working mongo container is a 10 seconds process with docker)

[johnabrams7]

@LouAdrien I wasn't able to reproduce the mongo server timeout using your steps (thanks for that).

Test notes:

• Tested with a system (homebrew mongo-community) and docker install of mongo on OSX 10.15.2 Catalina, Sails 1.2.3 (new empty project), Mongo 4.0.4, and [email protected].
• Specially used mongodb://localhost/dbname url and dbname created db in mongo for consistency. - Set policy '*':true.
• Created the company model with member: { type: 'string' }.
• Navigated to the homepage and /company models page without the local server immediately nor eventually timing out. Let it run for a bit checking the OSX Activity Monitor hardware utilization and mongo server status for any heavy cpu use, memory leaks, infinite loops, errors, etc - didn't see any.
• Ctrl+C was able to shutdown both mongo server instances.

For further environment context:
What OS and version of docker, mongo, & OS are you running this on? All local or virtual machines/cloud/etc?
Also making a repo on your end for this could still help to make sure everything is being generated correctly and matching in the code as intended.

Appreciate all the help so far. 👍

[LouAdrien]

Hello, in order to try and help even more, I have created a video, fully demonstrating everything I do, and the issue eventually happening at the end. The video start right after I did sails new test-project.

https://www.youtube.com/watch?v=THIVsieVBLs
You'll see that my mongodb client hangs also when sails does. But I have checked, it's not a mongodb issue, when I restart the sailsjs process, it can interact with mongodb without issue, just waiting for the hang to re-appear.

You will find a repository with the exact code added here (including node_module folder). https://github.com/LouAdrien/reproduce-sails-issue

As to your questions :
OS : MacOS Mojave 10.14.5, all normal local laptop setup.
mongodb 4.2.2 ( but I think i could reproduce with previous versions ) on docker
Created with the following docker-compose instructions :
```mongodb:
image: mongo:4.2
container_name: wedo-mongo
ports:
- "27017:27017"
volumes:
- "$PWD/dockerdata/mongo:/data/db"

docker info bellow 

Client: Docker Engine - Community
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:22:34 2019
OS/Arch: darwin/amd64
Experimental: false

Server: Docker Engine - Community
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.12)
Go version: go1.12.12
Git commit: 633a0ea
Built: Wed Nov 13 07:29:19 2019
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.2.10
GitCommit: b34a5c8af56e510852c35414db4c1f4fa6172339
runc:
Version: 1.0.0-rc8+dev
GitCommit: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
docker-init:
Version: 0.18.0
```

[LouAdrien]

Hello, @johnabrams7 did the video and extended information help you investigate the issue?

[LouAdrien]

@johnabrams7 hello, I have some news on this : this issue happens also with the postgres adaptor. Basically, it happens as soon as I setup any adaptor in the datastore config.
This is now a huge issue for our project as I continued to develop it based on the assumption that I could switch to another adapter if necessary. This is not the case anymore and we cannot use sails at all :(

Sidenote : did you try use exactly the same nodejs version? I dont see it listed in your post. Is there some exact requirements for the version of nodejs used?

[johnabrams7]

Hey @LouAdrien, thanks a ton for the video - yes that has helped much further! I definitely see the unresponsive server behaviour. It looks like it begins after creating a member record in the Company model but that may also be a coincidence. In my reproduction steps, I'd only left this model with a member field type string but hadn't actually created a member record.

I'm thinking this might be related to sails-mongo compatibility as it doesn’t _officially_ support mongodb 4 (not fully compatible at-least / more experimental) but there’s a PR in the works to upgrade the drivers to support newer versions. At the moment, sails-mongo officially supports version mongo 3.4, but we’re about to release a patch to expand official support to mongo 3.6 as well.

As for Postgres, what version of that db are you using? Is it the same unresponsive issue after creating a record?

Ah, I actually used Node.js version 12.14.0 - I will do another test to reproduce this with the same Node.js version you're using (11.0.0) along with Postman to create the db record in mongo versions 4.2.2, 3.6, and 3.4 to see how each version responds. Does this issue occur without Studio 3T running? Curious to give that a try!

[LouAdrien]

Hello @johnabrams7, the issue is not related to any action, it will happen eventually even without interacting with the app.

Also, it will happen with the postgres adaptor too. Not related to sails-mongo.

Lastly, I have switched to node 12.14.0, and it seems the issue is not happening anymore. This might be related to node v11 and I was unlucky to use this one. However I still think you should test with node 11.0.0 and try to reproduce, or at least give a warning on the official website of which node version are supported.

Regards,

[johnabrams7]

@LouAdrien Ah good to know the unresponsiveness eventually occurs regardless of any action.

Awesome, glad to hear Node 12.14.0 hasn't reproduced the issue! Yes, it's possible this could be specific to Node v11 - I'll attempt to reproduce the issue with that version and update docs as necessary. Appreciate the additional testing!

UPDATE:
I was able to reproduce this issue with node v11 and Studio 3T running but wasn't able to get the node timeout again on several other long wait tests w/o Studio 3T running (I'm wondering if Studio 3T might play a role). Node definitely started hogging most of the CPU - stuck at 90%+ utilization and sails refused to Ctrl+C shutdown w/o killing the process:

I'll continue further tests in attempt to reproduce the intermittent issue in different circumstances / configurations.

[johnabrams7]

@LouAdrien Ok, I was able to reproduce this issue again after a few hours of working on another project with node v11, sails, and mysql - no mongo, sails-mongo, or Studio 3T running. It appears to be a Node v11 specific timeout issue that's intermittent / unpredictable. This session had a few db queries, but I was mostly generating and developing elsewhere - node process became unresponsive after about 2 hour. I haven't experienced this issue with node v8, v10 & 12 before - just v11 so far. The best workaround seems to be using a different version of node besides v11 - preferably 12.x LTS. I'll update the team and look for similar cases. Feel free to keep us updated if other versions are having the same issue. Hope this helps 👍