Bentoml: cli throw RecursionError when using --enable-microbatch

[co42]

Describe the bug
When I use the cli with serve-gunicorn --enable-microbatch I got:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/Users/cregal/ovh/serving-backends/venv/lib/python3.8/site-packages/gunicorn/config.py", line 55, in __getattr__
    if name not in self.settings:
  File "/Users/cregal/ovh/serving-backends/venv/lib/python3.8/site-packages/gunicorn/config.py", line 55, in __getattr__
    if name not in self.settings:
  File "/Users/cregal/ovh/serving-backends/venv/lib/python3.8/site-packages/gunicorn/config.py", line 55, in __getattr__
    if name not in self.settings:
  [Previous line repeated 993 more times]
RecursionError: maximum recursion depth exceeded

To Reproduce
use the option --enable-microbatch with the cli:
bentoml serve-gunicorn --enable-microbatch myservice

Environment:

• MacOS 10.15.6
• Python 3.8.2, BentoML-0.8.5, gunicorn 20.0.4

[yubozhao]

@co42 Thank you for reporting. I can reproduce it locally and I am looking into this.

[bojiang]

This is a bug under extreme conditions, which needs to be met at the same time:

1. Use multiprocessing to start the gunicorn application.
2. Python >= 3.8
3. The platform is MacOS

Reference:

1. https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
2. https://github.com/benoitc/gunicorn/blob/20.0.4/gunicorn/config.py#L54 (which actually caused the RecursionError)

[bojiang]

More details:
For MacOS, before Python3.8, the multiprocessing module starts new processes in fork mode by default, and starts new processes in spawn mode by default since 3.8.
The latter will use pickle to pass objects between processes.

And because of the use of __getattr__ in https://github.com/benoitc/gunicorn/blob/20.0.4/gunicorn/config.py#L54, the magic methods __getstate__ and __setstate__ required by pickle to reconstruct the Config object was covered, Which in turn caused the RecursionError.