Cockroach: kv: implement Upgrade lock strength for unreplicated locks

Created on 29 May 2020 · 7Comments · Source: cockroachdb/cockroach

The lockTable currently only supports the exclusive lock strength, but was designed to support varying degrees of lock strengths. Specifically, we'd eventually like to add support for Shared and Upgrade locks as well.

https://github.com/cockroachdb/cockroach/blob/9e1666be01bd45358741c776bcc7137b56c7ce87/pkg/kv/kvserver/concurrency/lock/locking.proto#L40-L50

We currently have no need for Shared locks, but Upgrade locks would be immediately useful for unreplicated locks acquired during the initial row-fetch of an UPDATE statement (implicit SFU) or by a SELECT ... FOR UPDATE statement (explicit SFU).

https://github.com/cockroachdb/cockroach/blob/9e1666be01bd45358741c776bcc7137b56c7ce87/pkg/kv/kvserver/concurrency/lock/locking.proto#L90-L130

The major benefit of this form of lock is that it does not conflict with non-locking reads. This avoids blocking these reads, at the expense of undermining some of the protection against transactions retries that is provided by SFU. This is because these non-locking reads are allowed to slip underneath Upgrade locks and bump the timestamp cache, which may force the UPDATE to have to push its timestamp when it returns to write an intent. This is probably ok, and will certainly be once we also support Shared locks and SELECT ... FOR SHARE so that users can have full control over row-level locking.

This should improve throughput on YCSB-B (95% reads, 5% updates). It's unclear how it will affect YCSB-A (50% reads, 50% updates).

A-kv-transactions C-enhancement C-performance

Source

nvanbenschoten

👍1

Most helpful comment

hi. need this for Keycloak working and may be Liquibase

z0mb1ek on 15 Sep 2020

👍5

All 7 comments

hi. need this for Keycloak working and may be Liquibase

z0mb1ek on 15 Sep 2020

👍5

@nvanbenschoten do you have thoughts on the likelihood for this in 21.1. and then as an add-on, what are the chances of getting ranged locks?

ajwerner on 22 Oct 2020

I'd defer that to @sumeerbhola. This change doesn't seem particularly involved, though we wouldn't want to make the change until some of the larger lock table changes land in this release. Ranged locks are a much larger ask, but are being considered in the design of https://github.com/cockroachdb/cockroach/pull/55664.

nvanbenschoten on 22 Oct 2020

@ajwerner Replicated range locks have performance implications, since one can't use storage engine bloom filters when looking for conflicting range locks in the lock table. I was speculating on https://github.com/cockroachdb/cockroach/pull/55664 that we could make the leaseholder maintain an in-memory summary to prevent a performance drop, though a new leaseholder will need to fallback to those reads, until it initializes its in-memory state by reading the range locks from storage (which is not ideal).
Can you make do with unreplicated range locks?

sumeerbhola on 23 Oct 2020

Unreplicated range locks would be great. I’ll take what I can get. Replicated locks would enable us to avoid refreshing reads but unreplicated locks are a lot better than nothing.

ajwerner on 23 Oct 2020