Habitica: MongoDB 4.0

[paglias]

Things to keep in mind when upgrading:

• [ ] maxTimeMs
• [ ] apply transactions where needed
  
  
  
  • [ ] transactions are only available on replica sets
  
  
• [ ] review all issues and see what could be fixed after transactions are available

[lemoness]

I've updated the labels on this ticket.

[TheHollidayInn]

@paglias Didn't we update this?

[paglias]

Not yet, also v4 is out and mongoose already supports transactions http://mongoosejs.com/docs/transactions.html#with-mongoose-documents-and-save-

[paglias]

We're running on MongoDB 3.4 now

[justinr1234]

@paglias what's the status of this? MongoDB Atlas is force upgrading everyone to use Mongo 3.6 as of January 2020. What work needs to be done to get this updated?

[paglias]

@justinhhorner thanks for the heads up! We're aware of the deadline although we don't use mongodb atlas and plan to upgrade to 3.6 before it. The work on this repo has already been done and we only have to upgrade some support libraries used in a different repository

[paglias]

The only breaking change was the removal of pushAll but we don't use it anymore

[paglias]

Upgraded to 3.6

[paglias]

Reopened for version 4

[paglias]

Increasing the priority here as mongo 4 transactions are needed to fix several data consistency bugs

[benkelaar]

If the cause of https://github.com/HabitRPG/habitica/issues/8500 and https://github.com/HabitRPG/habitica/issues/7645 is indeed unsync over multiple documents upgrading to MongoDB 4 and using a transaction would fix that as well.

@paglias Is there anything I can do to help make this happen? According to this matrix you should be able to use a MongoDB version 4 server without upgrading the client. That might be a good first step to take.

[paglias]

@benkelaar in terms of client driver we should be ok as the current one used by mongoose indeed supports 4.0 (and 4.2 as well since it uses 3.5.7 https://github.com/Automattic/mongoose/blob/master/package.json#L24) but we should make sure the codebase doesn't use anything that got deprecated in 4.0 https://docs.mongodb.com/manual/release-notes/4.0-compatibility/ and ideally also 4.2 https://docs.mongodb.com/manual/release-notes/4.2-compatibility/

It's in my list of things to do but quite at the bottom of it so if you want to give it a look that would be great! In terms of database settings we already use Wired Storage and the auth mechanism is up to date so those shouldn't be a problem, we just have to make sure the queries don't use something that got deprecated

[benkelaar]

@paglias Am I right in concluding that the only dropped things described in those pages that can be used in queries and mongo scripts are the following?

For 4.0:

1. $isolated operator
2. db.createCollection() with autoIndexId set to false

For 4.2:

1. db.collection.group()
2. eval command
3. cursor.maxScan()
4. geoNear command
5. $type: 0

[paglias]

@benkelaar sorry for the delay! yeah it looks like those are the only things that changed and on the server side should be good with upgrading to v4. I'm going to bump this up on my list and hope to get to it by the end of June

[paglias]

The next steps are outlined here #12334