Coveragepy: Conflicting concurrency settings: ['multiprocessing', 'thread', 'greenlet', 'eventlet', 'gevent']

[lxiaopei]

Describe the bug
A clear and concise description of the bug.

In my application, I have used all of these ['multiprocessing', 'thread', 'greenlet', 'eventlet', 'gevent']

I set the below in my .coveragerc file:
[run]
branch = True
parallel = True
concurrency =
multiprocessing
greenlet
thread
gevent
eventlet

But when run, I got:

Conflicting concurrency settings: ['multiprocessing', 'thread', 'greenlet', 'eventlet', 'gevent']

Then I changed to just set concurrency=gevent or thread(the default), I still got no coverage data, the same as this issue:
https://github.com/nedbat/coveragepy/issues/667

root@sc2-hs3-nsbu-ncp-dhcp--26-107:/usr/local/bin# coverage debug sys
-- sys -------------------------------------------------------
version: 5.2
coverage: /usr/local/lib/python3.5/dist-packages/coverage/__init__.py
tracer: -none-
CTracer: available
plugins.file_tracers: -none-
plugins.configurers: -none-
plugins.context_switchers: -none-
configs_attempted: .coveragerc
setup.cfg
tox.ini
pyproject.toml
configs_read: -none-
config_file: None
config_contents: -none-
data_file: -none-
python: 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609]
platform: Linux-4.4.0-87-generic-x86_64-with-Ubuntu-16.04-xenial
implementation: CPython
executable: /usr/bin/python3
def_encoding: utf-8
fs_encoding: ascii
pid: 1536
cwd: /usr/local/bin
path: /usr/local/bin
/usr/lib/python35.zip
/usr/lib/python3.5
/usr/lib/python3.5/plat-x86_64-linux-gnu
/usr/lib/python3.5/lib-dynload
/usr/local/lib/python3.5/dist-packages
/usr/lib/python3/dist-packages
environment: -none-
command_line: /usr/local/bin/coverage debug sys
sqlite3_version: 2.6.0
sqlite3_sqlite_version: 3.11.0
sqlite3_temp_store: 0
sqlite3_compile_options: ENABLE_COLUMN_METADATA
ENABLE_DBSTAT_VTAB
ENABLE_FTS3
ENABLE_FTS3_PARENTHESIS
ENABLE_JSON1
ENABLE_LOAD_EXTENSION
ENABLE_RTREE
ENABLE_UNLOCK_NOTIFY
ENABLE_UPDATE_DELETE_LIMIT
HAVE_ISNAN
LIKE_DOESNT_MATCH_BLOBS
MAX_SCHEMA_RETRY=25
OMIT_LOOKASIDE
SECURE_DELETE
SOUNDEX
SYSTEM_MALLOC
TEMP_STORE=1
THREADSAFE=1

To Reproduce
How can we reproduce the problem? Please be specific. Don't just link to a failing CI job. Answer the questions below:

1. What version of Python are you using?
2. What version of coverage.py are you using? The output of coverage debug sys is helpful.
3. What versions of what packages do you have installed? The output of pip freeze is helpful.
4. What code are you running? Give us a specific commit of a specific repo that we can check out.
5. What commands did you run?

Expected behavior
A clear and concise description of what you expected to happen.

Additional context
Add any other context about the problem here.

[nedbat]

Can you give me a reproducible case to use? Or say more about how you are using all of the concurrency models? I'm not sure coverage will be able to measure everything if you are using many concurrency techniques.

[lxiaopei]

@nedbat thanks for the quick reply.
Below is a sample of my code, tailor it for confidential info, I used privileged module from oslo_privsep,

from oslo_privsep import priv_context
def priv_context_factory(lock_name=None, priv_ctx_name=None, pyfile=None):
_errors = []
if priv_ctx_name is None:
_errors.append('priv_ctx_name')
if pyfile is None:
_errors.append('pyfile')
if _errors:
exception_msg = 'Missing arguments: %s' % ', '.join(_errors)
raise Exception(exception_msg)
priv_context.init(root_helper=PRIVSEP_ROOT_HELPER)
return PrivContext(pyfile, lock_name=lock_name,
pypath=__name__ + '.' + priv_ctx_name,
capabilities=PRIVSEP_CAPS)

proxy_pri = priv_context_factory(priv_ctx_name='proxy_pri',
pyfile='kube_proxy.main')

import eventlet # noqa
import sys # noqa

eventlet.monkey_patch() # noqa

import queue
import socket
import ssl
import threading
import time

import gevent
import gevent.lock
import gevent.queue

from proxy.common import privilege

def link_exception(greenthread, func):
pass

def replace_gevent_modules():
gevent.lock.Semaphore = threading.Semaphore
gevent.queue = queue.Queue
gevent.sleep = time.sleep
gevent.socket = socket
gevent.spawn = eventlet.spawn
gevent.ssl = ssl
eventlet.greenthread.GreenThread.link_exception = link_exception

@privilege.proxy_pri.entrypoint
def main():
# Need to replace gevent modules after switch to
# the privileged child process
replace_gevent_modules()
from kube.common import utils
utils.set_logger_level(None, 'DEBUG' if cfg.CONF.debug else 'INFO')
from kube.proxy import proxy
proxy.proxy_run()

[lxiaopei]

@nedbat I still got the problem. Any update for this issue?

[frnkvieira]

I had a similar problem because I would like to use greenlet and thread at the same time.
SQLAlchemy upcoming version (1.4) is using greenlets under the hood and adds support to asyncio. If you combine the popular FastAPI framework which uses threads for sync routes then you get the problem, threads and greenlets running in the same application. Luckily, I can avoid using sync routes and went full blow async but that's not the case for many.

[daviskirk]

As described in #1082, I think that there will be a lot of users that will have this problem once sqlalchemy 1.4 comes out of beta. There will be many projects that can't go "full async" as it will take the community years (if ever) to have the same quality of tools on both async and sync side of things. Obviously web dev (starlette/fastapi) are one large case (which you might even be able to trick by running sync tests+coverage and then async tests+coverage and combine the two), but even in otherwise "fully async" code there's the typical "run in threadpool executor" pattern, which is especially tricky because it will almost always involve BOTH threads and greenlets (if sqlalchemy is being used) in the same function.

Perhaps we can even get sqlalchemy to have a "run in sync mode" flag that would at least make running coverage easier? Don't know if that's a good trade off though.