Cht-core: Optimize the `fix-user-db-security` migration for databases with many users

Created on 14 Feb 2019  Â·  3Comments  Â·  Source: medic/cht-core

It currently doesn't do any batching, and generates N parallel PUT requests to CouchDB, where N === the user count.

This is almost certainly A Bad Ideaâ„¢ for larger databases.

We should at least not make those PUTs all concurrent.

This should be backported to 3.2.x.

API 1 - High Bug Upgrading

Most helpful comment

The good news is, Stefan has already fixed this here: https://github.com/medic/medic/commit/2763c05c31fd53f87f8f48b65f887eca8ddb406e

I've backported it to 3.3.x here: https://github.com/medic/medic/commit/f534da5d2f05eac1b827d229c17124d889e3752b

I think this should be released in 3.3.0 instead of making a new 3.2.2 because any new release on 3.2.x will require RT for a few days and delay 3.3.0 too much. So putting this in 3.3.0 and getting it out ASAP makes more sense to me. If there's some delay on getting 3.3.0 out we can revisit this.

All 3 comments

The good news is, Stefan has already fixed this here: https://github.com/medic/medic/commit/2763c05c31fd53f87f8f48b65f887eca8ddb406e

I've backported it to 3.3.x here: https://github.com/medic/medic/commit/f534da5d2f05eac1b827d229c17124d889e3752b

I think this should be released in 3.3.0 instead of making a new 3.2.2 because any new release on 3.2.x will require RT for a few days and delay 3.3.0 too much. So putting this in 3.3.0 and getting it out ASAP makes more sense to me. If there's some delay on getting 3.3.0 out we can revisit this.

AT: Run the "fix-user-db-security" migration on a database with a large number of users and make sure it passes and that each request to set user security in the CouchDB log is made one at a time and not all as quickly as possible.

Thanks to @dianabarsan script I was able to add a bunch of users and see the Couchdb crash and then not crash after upgrading the gamma instance properly.

Was this page helpful?
0 / 5 - 0 ratings