Kotlinx.coroutines: Reentrant lock

[ilyawaisman]

Is reentrant lock for coroutines on the roadmap?

I've seen a discussion of it where a problem of coroutine identity was raised (https://github.com/Kotlin/kotlinx.coroutines/issues/965). But couldn't a Job in the coroutine context be that identity?

[ilyawaisman]

Sample implementation could be like this:

import kotlinx.coroutines.Job
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlin.coroutines.coroutineContext

class CoroutineReentrantLock {
    private val mutex = Mutex()
    @Volatile
    private var myJob: Job? = null

    val isLocked
        get() = myJob != null

    suspend fun <R> withLock(action: suspend () -> R): R {
        val currentJob = checkNotNull(coroutineContext[Job]) {
            "Not found Job in current coroutine context, this is not supported"
        }
        if (myJob === currentJob) {
            // myJob is not null only under mutex
            return action()
        } else {
            // if myJob is null take mutex immediately otherwise wait until it becomes free
            mutex.withLock {
                check(myJob == null) { "myJob = $myJob, currentJob = $currentJob" }
                myJob = currentJob
                try {
                    return action()
                } finally {
                    myJob = null
                }
            }
        }
    }
}

[elizarov]

Coroutines don't have a stable identity and using Job as one is not correct since many innocent primitives like withContext and coroutineScope do introduce a new job without conceptually launching a new coroutine. There are some possible workarounds, but, conceptually, coroutines (unlike threads) are too ephemeral. Any attempt to build a reentrant lock is bound to be fragile in practice. So no --- reentrant lock is not on a roadmap.

[danielesegato]

@elizarov I think a reentrant mutex would be possible by allowing entering the lock if a parent scope/context (ancestor) is holding the lock.

Isn't this feasible?

[fvasco]

I think a reentrant mutex would be possible by allowing entering the lock if a parent scope/context (ancestor) is holding the lock

How many child scopes should enter in the same reentrant mutex?

[danielesegato]

I don't think it makes sense to have such a a limit.
It's either a child scope or isn't.

The developer is in control of that scope and can write code accordingly.

or maybe I misunderstood your question?

Structured concurrency should allow to release the lock only when all is completed I believe.

[fvasco]

I cannot figure a common use case for it that cannot be solved with:

mutex.withLock {
  coroutineScope {
    // launch child coroutines
  }
}

[danielesegato]

The use case is this:

suspend fun doSomeOperation() {
  mutex.withLock {
    // do stuff
    // call another function
    doSomeOtherOperation()
    // do some other stuff
  }
}

suspend fun doSomeOtherOperation() {
  mutex.withLock {
    // do something
  }
}

I know the content of doSomeOtherOperation() can be exported as an internal function without the lock and reused but this is possible with synchronize keyword without using coroutines because it's a reentrant lock. Instead the code above creates a deadlock cause mutex is not a reentrant semaphore.

Sometimes you don't wanna split the logic like that for readability or your code is more complex than this and the mutex gets in the way.

EDIT: to clarify
an API like that should have the 2 methods excluding each other when called from externally but when calling doSomeOperation() inside that mutex it should be able to call doSomeOtherOperation() without being locked at the mutex.

[fvasco]

Hi, @danielesegato,
you should consider that synchronize isn't a good basis for comparison because synchronized is not re-entrant with children threads.
With synchronized you should use a Fork+Join tool with an external synchronization, as I previously suggested.

Instead, your last example look similar to #1382 where you need to encapsulate functions that access to a resource (inside a Mutex).

[vklidu]

There is already a nice example of solution from Roman Elizarov here https://elizarov.medium.com/phantom-of-the-coroutine-afc63b03a131

suspend fun <T> Mutex.withReentrantLock(block: suspend () -> T): T {
  val key = ReentrantMutexContextKey(this)
  // call block directly when this mutex is already locked in the context
  if (coroutineContext[key] != null) return block()
  // otherwise add it to the context and lock the mutex
  return withContext(ReentrantMutexContextElement(key)) {
    withLock { block() }
  }
}

class ReentrantMutexContextElement(
  override val key: ReentrantMutexContextKey
) : CoroutineContext.Element

data class ReentrantMutexContextKey(
  val mutex: Mutex
) : CoroutineContext.Key<ReentrantMutexContextElement>

[ZakTaccardi]

@elizarov is this still not feasible given your example linked here:

https://github.com/Kotlin/kotlinx.coroutines/issues/1686#issuecomment-777357672

If it's still not feasible, what makes the mutex example you provided unreliable?

[elizarov]

It just has too much hassle. Given the rare need it is hard to substantiate adding it.