Mongoose: Concurrency Control

[duncangroenewald]

Mongoose does not currently appear to support any form of concurrency control. So if two processes are concurrently performing updates then the last update overwrites any earlier updates. Perhaps someone could include timestamped updates in the feature list. Include the timestamp in the query predicate and fail the update if no update is performed.

[vkarpov15]

See comments on https://github.com/Automattic/mongoose/issues/4004#issuecomment-312529620 . We have support for plugins that do concurrency control, and are looking for community members willing to contribute plugins. Would you be willing to help?

[duncangroenewald]

Hi, sorry I won’t be able to help develop a solution but am more than happy to help test anything that gets developed. All we need is a simple timestamp based optimistic concurrency control solution.

Regards

On 6 Jul 2017, at 5:09 am, Valeri Karpov [e-mail] wrote:

See comments on #4004 (comment) https://github.com/Automattic/mongoose/issues/4004#issuecomment-312529620 . We have support for plugins that do concurrency control, and are looking for community members willing to contribute plugins. Would you be willing to help?

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/Automattic/mongoose/issues/5424#issuecomment-313197970, or mute the thread https://github.com/notifications/unsubscribe-auth/ANKu3CJsKx5La6Xtb193AAixkAtj9kLbks5sK992gaJpZM4OK_xt.

[marcospgp]

This is still a big issue for Mongoose

[vkarpov15]

@marcospgp there's a good plugin out there for this. Adding support for OCC is a one-liner with this plugin.

[marcospgp]

I already knew about that plugin, but it didn't seem up to our standards -
have a look at this issue
https://github.com/eoin-obrien/mongoose-update-if-current/issues/5

On Wed, Mar 6, 2019, 22:26 Valeri Karpov [e-mail] wrote:

Closed #5424 https://github.com/Automattic/mongoose/issues/5424.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Automattic/mongoose/issues/5424#event-2185584785, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADTc_byvev9YHUb-eFKFnYmmU2swDqigks5vUEChgaJpZM4OK_xt
.

[vkarpov15]

@marcospgp why does it not seem up to your standards? Would it be helpful if we made this an officially supported plugin?

[marcospgp]

No, I believe using the version field is not a good idea when we can use updatedAt.

I would add an option called "optimisticallyConcurrentSave" that checks the updatedAt field, if present, on every save and if it is not the same as the document that is being attempted to be pushed to the database, throw a (descriptive) error.

It would help to add a guide to the docs that describes concurrency with mongoose or something along those lines.

[duncangroenewald]

Not sure if you will see this response of mine given I am just responding via email - but for what it’s worth I would agree that:

1). Concurrency control should be a 1st class citizen (i.e. not a plug in but a built in feature) - such as the option ‘optimisticallyConcurrentSave’ suggested below. And I am inferring from the discussion that Marcos also thinks it should be a standard function rather than a plug in. Almost all systems I work on have some form of concurrency control to guarantee transactional integrity. ‘Mostly we don’t inadvertently overwrite any data’ is not good enough.

2).Use if timestamps for concurrency control is pretty common - so consistency with this would reduce unnecessary confusion/learning. However Version might be able to be used, but it is not my understanding that every record update will result in a Version increment automatically - I seem to recall you still have to increment yourself. Whereas updatedAt, if present should always be updated by the server whenever any field on the object is changed - it is also important to ensure that the timestamp that gets used is one provided by the server and not the client and that timezones etc are not messed up. I seem to recall running into some issues with this at one point but I could be mistaken.

On 11 Mar 2019, at 11:54 am, Marcos Pereira [e-mail] wrote:

No, I believe using the version field is not a good idea when we can use updatedAt.

I would add an option called "optimisticallyConcurrentSave" that checks the updatedAt field, if present, on every save and if it is not the same as the document that is being attempted to be pushed to the database, throw a (descriptive) error.

It would help to add a guide to the docs that describes concurrency with mongoose or something along those lines.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub https://github.com/Automattic/mongoose/issues/5424#issuecomment-471372237, or mute the thread https://github.com/notifications/unsubscribe-auth/ANKu3Pa3kJ2WrTa5q90AkcloIEQpBPyaks5vVakpgaJpZM4OK_xt.

[lonix1]

Being part of mongoose would be ideal, but an official plugin would be better than nothing (and that plugin linked above is a good start).

I realise many are scratching their heads and asking "what's all this fuss about concurrency... it doesn't matter to me"? But like so many others have commented here and elsewhere on StackOverflow and many blog posts over many years: how can any serious dbms or dbms driver not have controllable concurrency options?

We need this desperately. Any large system deals with this problem, it's not an edge case. It needs to be officially supported (not an unofficial plugin) to give peace of mind that our systems won't blow up one day because of API changes.

And thank you @vkarpov15 for being open to this discussion.

[vkarpov15]

We'll look into adding more official support for this. For now, it will be a plugin, but whether we ship it with Mongoose or make it an officially supported plugin is open to debate.

I'd love to read some of the blog posts and stackoverflow discussions you mentioned @lonix1 , any links?

[lonix1]

I didn't save them :smile: but they're there. There are all sorts of workarounds for concurrency control, and if memory serves, most of them link to the work done by that guy who made the plugin above, because that's the only widely used code so far. unless you want to roll your own, but I guess most people would be scared to do that for something so critical. and get it wrong

[eoin-obrien]

I'm rather late to the party here, but I'm the author of the OCC plugin mentioned above mongoose-update-if-current.

I've just added the option to use a timestamp-based strategy (i.e. with updatedAt) instead of a version-based strategy. It's something I've been meaning to get around to for a while, and it should be live on npm by the end of the week. Any additional feedback and/or criticism that could help improve would be appreciated as well.

@vkarpov15 If I can be of any assistance to you regarding OCC in Mongoose, I'd be more than happy to help out!