Parse-server: Transaction/Commit Support

[mjgerace]

Feature Request

For a while now, parse has lacked transaction/commit support. Due to upcoming Postgres support, transactions should be very easy to implement, and provides a much-needed way to secure many api requests in a row with adequate rollback.

No workarounds presented thus far have been nearly as efficient as ACID transactions. I think that if Parse wants to consider itself a top-tier open source package, it needs to have this feature for database integrity.

[joy4eg]

+1 (#2013)

[Beingbook]

Is there no alternative?

[vitaly-t]

Transactions are already used in PostgresStorageAdapter.js, not sure how sufficient though.

[pewh]

Hey, what is the current status? Is there any example how to implement transaction/commit?

[vitaly-t]

@pewh As per my earlier message:

Transactions are already used in PostgresStorageAdapter.js, not sure how sufficient though.

Method .tx (transaction) is used in 6 places there. Not sure how sufficient it is in general, as it requires to understand how the whole thing works, which is a lot.

[trylovetom]

I think mongodb need it, too

[pewh]

.tx is coming from pg-promise (https://github.com/parse-community/parse-server/blob/master/src/Adapters/Storage/Postgres/PostgresClient.js#L17). I guess we can make Parse support transaction/commit like this:

1. Convert Parse.query into pg-promise instance
2. Use batch like this https://github.com/parse-community/parse-server/blob/master/src/Adapters/Storage/Postgres/PostgresStorageAdapter.js#L546-L549

P.s: I'm newbie, so i don't know if it possible or not

[trylovetom]

@pewh I think we need any query or update function support transaction/commit.

[trylovetom]

@pewh Let's do it ?

[manjukas]

Is it possible to have transactions with mongo, as mongodb doesn't really support multi document transactions?

Optimistic locking would be nice for Object class. Just need a integer type attribute, which gets incremented each time the object is saved. And save will fail if someone else has modified since the last read. This can be consistently implemented across Mongo and Postgres.

[flovilmart]

If you need a transactional RDBMS, you should not use MongoDB in the first place, and if you really need transactions, baking it into parse-server may prove painful and inefficient (i.e., you should probably not use parse-server if you intend to have a highly transactional system, I would personally not use it for that).

[heresmyinfo]

@flovilmart We have an application that is currently based on Parse, but needs the reliability of transactional support. When you say "painful and inefficient", are you referring to high-performance or difficulty in implementation? We're not as concerned with performance as reliability. If it's just performance you are referring to, how much effort do you think it would take to implement transactional logic now that we have switched to Postgres (which is working great)? Do you have anyone you can refer us to?

[charleskoehl]

We also have an application based on Parse. Recently, it became necessary to use transactions to support a ledger of user loyalty rewards points. We are considering these options:

1. Switch to Firebase and use its new database, "Firestore," (in beta; has documents organized into collections and supports multi-document transactions and batch writes), and "Cloud Functions" (in beta; a cross between AWS Lambda and Parse Cloud Functions), which appears to duplicate many of the features of Parse
2. Ask the ParseServer maintainers to support transactions using Fawn or similar.

@flovilmart: If someone were to create a PR to add transaction support using Fawn, would you consider merging it, or is that a feature that you don't want to support period?

[flovilmart]

I’m always cautious about integrating 3rd party frameworks.

While this is a great idea, stuffing more features into the main repository is perhaps something that will be costly in the long run.

If we support transactions for mongo, then we’ll need the same level of support for Postgres.

I’m curious of what the REST interface would look like, where it would live, wether we support cross class transactions, wether we force id based identification of the affected documents etc...

I’m not against evaluation, as I believe there’s some value in it.

If the implementation is just a wrapper upon Fawn, then an adapter design is perhaps more suited.

A priori, i’m Not against it as it’s a quite requested feature, i’d Be willing to help with the design of the feature, and discuss the interfaces, and help with SDK client implementations.

Is it something that would be only available for the masterKey at first only exposed as a REST endpoint and possible to the JS SDK?

What do you think?

Is it something you’d willing to invest some time in?

[charleskoehl]

Thanks for your thoughts.

I think an adapter pattern would be best to maintain independence from Fawn or any other specific implementation.

I think it should support cross-class transactions. It should be available only for the masterKey at first since I think a significant portion of transactions would modify data related to multiple users.

As I'm currently the sole developer (tech co-founder) of a startup with a quickly-approaching MVP deadline, I would only be able to invest time in the feature starting in 6 months or so.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[oallouch]

Ok to close this for now, even if it would be great to have this feature.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[gtzinos]

https://docs.mongodb.com/master/core/transactions/

[mjgerace]

Not to start this up again, but I thing given the fact that mongo is implementing transactions and that postgres support is fully fleshed should further the argument for transactions. Pretty much any and all applications that want to take themselves seriously and promote data integrity will utilize such a feature. It is crippling parse that there is no current support for this.

[acinader]

I think you mean: 'To start this up again'. ;).

Do you have some ideas on how we might implement? What SDK do you use primarily?

[acinader]

@flovilmart lays out some good questions for a starting point:

https://github.com/parse-community/parse-server/issues/2159#issuecomment-358489154

[mjgerace]

@acinader

To begin to answer these questions, let me give my (somewhat) uneducated thoughts.

It's no longer required to use any third-party package or integration. MongoDB supports transactions as of 4.0 (I checked, it looks like we are already version compliant for this feature set). Postgres obviously has supported such transactions for a while.

As for what the rest interface would look like, I don't know yet. I think that it would be prudent to begin with support on the expressjs side of things, building out the javascript client, as that is likely where most of your users reside. Specific design details have not been well thought out by me - I made this post over 3 years ago, and have not had a look at how parse-server is architected in recent days.

I will take a look and work on understanding where this could fit in, but no longer are third-party packages necessary. Feature parity between the two DBs is doable without any additional integration.

Another way of phrasing what I am saying, to be clear, is only implementing transaction support on the business-logic side of things. Obviously there is little to no secure use-case for transactions from the client perspective.

[davimacedo]

The JS SDK has a Parse.Object.saveAll() method that uses the Parse Server's batch endpoint. Maybe including support to a transactional saveAll() process would be a good starting point. What do you guys think?

[gtzinos]

Excellent idea @davimacedo

[davimacedo]

@gtzinos @mjgerace is this something that you guys would be willed to tackle?

[yuki-takeichi]

Another good starting point is supporting an atomic update operation including addRelation or RemoveRelation operators for REST API. A relation field has an independent MongoDB collection where each relation is stored separately. REST API can specify an update for an object itself and its relation in the same call as if its updates are atomic. Because these update queries are not in a transaction, there are possible inconsistent states a query that has addRelation or RemoveRelation operators poses when the operation fails in the middle.

Supporting atomic saveAll() is also a good idea, but I'm not sure that saveAll() ensures to call /batch API because of possible future changes. We might have to revise the spec for saveAll(), and I think this leads to a discussion for client SDK transaction interfaces. (For example, bringing a concept of Session or Transaction to Parse Server and its SDKs.)