Protobuf: ImportError with Python 3.9

[jcugat]

What version of protobuf and what language are you using?
Version: 3.13.0
Language: Python

What operating system (Linux, Windows, ...) and version?
Docker container with Debian GNU/Linux 10 (buster)

What runtime / compiler are you using (e.g., python version or gcc version)
CPython 3.9.0

What did you do?
Python 3.8 works as expected:

$ docker run -it python:3.8 bash
root@9368d8cf9ab0:/# pip install protobuf
Collecting protobuf
  Downloading protobuf-3.13.0-cp38-cp38-manylinux1_x86_64.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 524 kB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from protobuf) (50.3.2)
Collecting six>=1.9
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, protobuf
Successfully installed protobuf-3.13.0 six-1.15.0
root@9368d8cf9ab0:/# PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="cpp" python -c 'from google.protobuf import descriptor'
root@9368d8cf9ab0:/# echo $?
0
root@9368d8cf9ab0:/# exit

But Python 3.9 fails the same import:

$ docker run -it python:3.9 bash
root@5966444c1878:/# pip install protobuf
Collecting protobuf
  Downloading protobuf-3.13.0-py2.py3-none-any.whl (438 kB)
     |████████████████████████████████| 438 kB 522 kB/s
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (from protobuf) (50.3.2)
Collecting six>=1.9
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: six, protobuf
Successfully installed protobuf-3.13.0 six-1.15.0
root@5966444c1878:/# PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="cpp" python -c 'from google.protobuf import descriptor'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 48, in <module>
    from google.protobuf.pyext import _message
ImportError: cannot import name '_message' from 'google.protobuf.pyext' (/usr/local/lib/python3.9/site-packages/google/protobuf/pyext/__init__.py)
root@5966444c1878:/# echo $?
1
root@5966444c1878:/# python -V
Python 3.9.0
root@5966444c1878:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

What did you expect to see
Import should work just like with previous versions.

What did you see instead?
An import error.

[alexprengere]

I confirm this issue on Python 3.9. For the record, a similar issue happened for Python 3.8.

Python 3.9 was released a month ago (after months of beta). For many projects relying on protobuf, this is a blocker to upgrade Python, since only the Python implementation works with PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION="python", but is typically 10 times slower than the cpp implementation.

[MichaelAquilina]

We're experiencing a similar issue where a missing protobuf wheel for python 3.9 is blocking our upgrade.

[busunkim96]

Hi @anandolee,

I work on the Google Cloud Client Libraries. We're starting to add 3.9 tests and noticed 3.9 protobuf wheels are not yet published. Is there a timeline of when you expect 3.9 wheels to published?

I'd be happy to open something like https://github.com/protocolbuffers/protobuf/pull/7009 for 3.9 if that's all that is needed.

Thanks!

[MichaelAquilina]

Hi there :wave: I just was wondering if there was an estimate of if/when this issue will be fixed? I'm happy to contribute to help move this along if necessary