Kotlinx.coroutines: Potential issues with CoroutineScope and runBlocking semantics

[Dico200]

runBlocking is different from other coroutine builders because it is not an extension of CoroutineScope.
If there is an outer scope, programmer might mean to carry over elements from its context. It does not do this unless you explicitly pass the context as a parameter.

Example 1

Here's an example where it leads to confusion:

class App : CoroutineScope {
    override val coroutineContext = Job() + Dispatchers.Main
    fun dispose() = coroutineContext.cancel()

    fun specialFunction() {
        runBlocking { 
            launch {
                // programmer intends to wait for this coroutine, which should run on Main thread, to complete.
                // Uses runBlocking instead of suspend function.
                // But launch does not use outer scope, because runBlocking doesn't do so and overrides it with its own.
            }.join()
        }
    }
}

launch call in specialFunction must be replaced with [email protected] to fix the issue.

Idiomatic code would be to use suspend function,
and if that's not permissible, the programmer should probably do this:

fun specialFunction() {
    val job = launch {
        // coroutine runs in main thread as expected
    }

    runBlocking {
        job.join()
    }
}

Example 2

Assume the programmer has a good reason to carry over the outer context into the runBlocking call.
For example:

class App : CoroutineScope {
    override val coroutineContext = Job() + newSingleThreadContext("MyMainThread")
    fun dispose() = coroutineContext.cancel()

    fun specialFunction() {
        launch { 
            runBlocking(coroutineContext) {
                println("Hello from runBlocking")
            }
        }
    }
}

suspend fun main() {
    with(App()) {
        specialFunction()
        coroutineContext[Job]!!.join()
        dispose()
    }
}

Here, the programmer passed outer coroutine context as a parameter.
Now, the runBlocking coroutine will be cancelled when parent is cancelled.
However, this code will never return because there is a deadlock.
From the runBlocking documentation:

• When [CoroutineDispatcher] is explicitly specified in the [context], then the new coroutine runs in the context of
• the specified dispatcher while the current thread is blocked.

So what if the current thread is the same as the dispatcher's single thread? You get a deadlock.

I just wanted to share some difficulties I encountered around runBlocking and CoroutineScope. Sure, there are ways around these issues, but I think it is worth considering small changes to the behaviour of runBlocking here.

I think it might make sense to have a version of runBlocking that is an extension of CoroutineScope. That way, cases where there is an outer scope, we can have the logic for carrying over its context implemented inside the runBlocking function. The new version will be prioritized by the compiler when there is a CoroutineScope, and its context elements are copied as the programmer would expect. I wonder what your thoughts are. I struggle to find great use cases, but I think this edge case is relatively difficult to maneauver as a programmer at the moment.

[elizarov]

Can you please clarify what are the actual use-cases for the examples you've given? Why there might be a need to use runBlocking in the otherwise non-blocking code in the same class that implements CoroutineScope?

The reason that runBlocking is not an extension of CoroutineScope is precisely that it is used to bridge the code that does not use coroutines at all with coroutines code.

[Dico200]

While a concrete use case is not present, I'm saying with example 1 that it can be difficult to get runBlocking usage right in a place where there is an outer CoroutineScope, which is then hidden (unless you use this@ClassName syntax). That can be confusing for programmers that don't know its precise use case and that have never run into the problem.
They might want to join with a job they launch, synchronously (from a non-suspend function). Example 1 shows a straight forward way of doing that with runBlocking.

If you think that's a total non-issue, that's okay. I wrote the issue for it to be considered and thought about before the api is final.

[elizarov]

I think that it, at least, deserved some kind of an inspection. Maybe we shall warn on any runBlocking usage inside classes that implement CoroutineContext, but we'll need more use-cases to study what is going on.

[dave08]

Does this issue also apply to sendBlocking on RendezVousChannels? If the receiver is not ready to receive() and the caller is being cancelled, the sendBlockings runBlocking will maybe still be running and not get cancelled either...?

[ds84182]

Ran into this today. I wanted to cancel the CoroutineContext's job synchronously before exiting the process, so I wrote runBlocking { job.cancelAndJoin() }. The runBlocking created a job under the job I was trying to cancel, which caused a deadlock.

[qwwdfsad]

caller is being cancelled, the sendBlockings runBlocking will maybe still be running and not get cancelled either...?

Cancellation is cooperative. sendBlocking is regular non-suspend function, so you can't cancel it at all.

I wrote runBlocking { job.cancelAndJoin() }. The runBlocking created a job under the job I was trying to cancel, which caused a deadlock.

Could you please clarify how did you achieve this? Because runBlocking does not creates job under the current one.

[qwwdfsad]

As Roman correctly mentioned, runBlocking is not an extension on the scope for a reason and it never will be.

runBlocking is a very fragile mechanism which is frequently misused. Maybe it is good idea to have an inspection for runBlocking calls if scope receiver is present. But before doing so we should ensure that it won't produce too much false-positives for people who use it properly (e.g. in Android onDestroy methods to join the job)

[ZakTaccardi]

@qwwdfsad I would like to know more details on how this can safely be used. My particular use case is that I need a CoroutineScope[Job]!!.invokeOnCompletion { .. } to be executed before onPause(..) completes.

Is there a way to run a piece of code synchronously once the CoroutineScope's Job is cancelled?

var resumedScope: CoroutineScope? = null
override fun onResume() {
  resumedScope = CoroutineScope(Dispatchers.Main)

  // launch some coroutines with the scope

  val completionHandler : CompletionHandler = // ..
  resumedScope.coroutineContext[Job]!!.invokeOnCompletion(completionHandler)
}

override fun onPause() {
  resumedScope.cancel()
  // how can I ensure that the completionHandler is invoked before `onPause()` completes?
}

[Dico200]

@ZakTaccardi

resumedScope?.let { 
    runBlocking {
        it[Job]!!.cancelAndJoin()
    }
}

EDIT: This may not work since onPause probably gets called on Main thread too. In that case this code will hang and there's no way to do it, unless you can invoke pending tasks from the Android event loop within the runBlocking call.

[Dico200]

@ZakTaccardi

resumedScope?.let { 
    runBlocking {
        it[Job]!!.cancelAndJoin()
    }
}

EDIT: This may not work since onPause probably gets called on Main thread too. In that case this code will hang and there's no way to do it, unless you can invoke pending tasks from the Android event loop within the runBlocking call.

Actually, you can try this:

resumedScope?.let { s -> 
       s[Job]!!.cancel()
       runBlocking {
           withContext(Dispatchers.Main) {
               // This MIGHT not deadlock because the dispatchers use the same thread? 
               // If Main dispatcher uses fair ordering, it probably won't work.
               while (s.isAlive) {
                   yield() // invoke Android scheduler?
               }
            }
        }
    }
}

NO IDEA if this works.

EDIT 2, use UNDISPATCHED to work around fair dispatcher.

resumedScope?.let { s -> 
       s[Job]!!.cancel()
       runBlocking {
           launch(Dispatchers.Main, start = CoroutineStart.UNDISPATCHED) {
               while (s.isAlive) {
                   yield() // invoke Android scheduler?
               }
           }.join()

           // And if that works but it's not reliable...
          while (s.isAlive) {
              launch(Dispatchers.Main, start = CoroutineStart.UNDISPATCHED) {
                  yield() // invoke Android scheduler?
              }
          }
        }
    }
}

If that works... cool I guess...

[elizarov]

I would like to know more details on how this can safely be used. My particular use case is that I need a CoroutineScope[Job]!!.invokeOnCompletion { .. } to be executed before onPause(..) completes.

cancel is guaranteed to synchronously execute all invokeOnCompletion handlers, so in your code examples there is a guaranteed that it executed before onPause returns. These handlers are explicitly designed to be synchronous:
https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-job/invoke-on-completion.html

Does it help? Can we close this issue?

[Dico200]

I realize that I was an idiot. Completion handlers are not dispatched of course.

Does it help? Can we close this issue?

@elizarov As far as I'm concerned, the thing you replied to is not directly related to the issue. You said you might consider an IDE inspection to check for correct use of runBlocking, but it might be complex to filter false positives there. I am happy to close it here, and you can reopen it if you want to pursue that idea. Sorry for leaving it open all this time :)

[ZakTaccardi]

@elizarov I think part of the issue I was having was that the job I was registering a CompletionHandler for was the child job of a CoroutineScope, not the parent scope's job (the one that has .cancel() called on it in onPause(). Maybe that caused the cancellation to be dispached? I'm not sure how structured concurrency cancellation plays out for child jobs that get cancelled