Cryptography: Got "sorry, but this version only supports 100 named groups" while installing

[chitoge]

Hello,

When I tried to install cryptography globally with pip install cryptography on Ubuntu, I got the following error:

Collecting cryptography
  Downloading cryptography-1.5.2.tar.gz (400kB)
    100% |████████████████████████████████| 409kB 764kB/s 
Requirement already up-to-date: idna>=2.0 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: pyasn1>=0.1.8 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: six>=1.4.1 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: setuptools>=11.3 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: enum34 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: ipaddress in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: cffi>=1.4.1 in /usr/local/lib/python2.7/dist-packages (from cryptography)
Requirement already up-to-date: pycparser in /usr/local/lib/python2.7/dist-packages (from cffi>=1.4.1->cryptography)
Installing collected packages: cryptography
  Running setup.py install for cryptography ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mKAWjL/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-0njxcc-record/install-record.txt --single-version-externally-managed --compile:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-mKAWjL/cryptography/setup.py", line 334, in <module>
        **keywords_with_side_effects(sys.argv)
      File "/usr/lib/python2.7/distutils/core.py", line 111, in setup
        _setup_distribution = dist = klass(attrs)
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 318, in __init__
        _Distribution.__init__(self, attrs)
      File "/usr/lib/python2.7/distutils/dist.py", line 287, in __init__
        self.finalize_options()
      File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 376, in finalize_options
        ep.load()(self, ep.name, value)
      File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 181, in cffi_modules
        add_cffi_module(dist, cffi_module)
      File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 48, in add_cffi_module
        execfile(build_file_name, mod_vars)
      File "/usr/local/lib/python2.7/dist-packages/cffi/setuptools_ext.py", line 24, in execfile
        exec(code, glob, glob)
      File "src/_cffi_src/build_openssl.py", line 81, in <module>
        extra_link_args=extra_link_args(compiler_type()),
      File "/tmp/pip-build-mKAWjL/cryptography/src/_cffi_src/utils.py", line 61, in build_ffi_for_binding
        extra_link_args=extra_link_args,
      File "/tmp/pip-build-mKAWjL/cryptography/src/_cffi_src/utils.py", line 70, in build_ffi
        ffi.cdef(cdef_source)
      File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 105, in cdef
        self._cdef(csource, override=override, packed=packed)
      File "/usr/local/lib/python2.7/dist-packages/cffi/api.py", line 119, in _cdef
        self._parser.parse(csource, override=override, **options)
      File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 299, in parse
        self._internal_parse(csource)
      File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 304, in _internal_parse
        ast, macros, csource = self._parse(csource)
      File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 260, in _parse
        ast = _get_parser().parse(csource)
      File "/usr/local/lib/python2.7/dist-packages/cffi/cparser.py", line 40, in _get_parser
        _parser_cache = pycparser.CParser()
      File "/usr/local/lib/python2.7/dist-packages/pycparser/c_parser.py", line 87, in __init__
        outputdir=taboutputdir)
      File "/usr/local/lib/python2.7/dist-packages/pycparser/c_lexer.py", line 66, in build
        self.lexer = lex.lex(object=self, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/pycparser/ply/lex.py", line 911, in lex
        lexobj.readtab(lextab, ldict)
      File "/usr/local/lib/python2.7/dist-packages/pycparser/ply/lex.py", line 233, in readtab
        titem.append((re.compile(pat, lextab._lexreflags | re.VERBOSE), _names_to_funcs(func_name, fdict)))
      File "/usr/lib/python2.7/re.py", line 194, in compile
        return _compile(pattern, flags)
      File "/usr/lib/python2.7/re.py", line 249, in _compile
        p = sre_compile.compile(pattern, flags)
      File "/usr/lib/python2.7/sre_compile.py", line 583, in compile
        "sorry, but this version only supports 100 named groups"
    AssertionError: sorry, but this version only supports 100 named groups

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-mKAWjL/cryptography/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-0njxcc-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-mKAWjL/cryptography/

Seems like there are other users having this bug recently as well: https://community.letsencrypt.org/t/certbot-auto-fails-while-setting-up-virtual-environment-complains-about-package-hashes/20529

I'm using a fresh installation of Ubuntu 16.04, with Python 2.7.12 and pip 8.1.2.

Thanks.

[reaperhulk]

This is due to https://github.com/eliben/pycparser/issues/148

[jeffrey4l]

OpenStack Kolla failing with the same error

http://logs.openstack.org/60/379960/9/check/gate-kolla-dsvm-build-centos-binary-centos-7-nv/ab2993c/console.html#_2016-10-03_01_21_03_809098

[reaperhulk]

As a temporary workaround (please do not use this in perpetuity, you will regret it) you should be able to do pip install cryptography --no-binary :all: and that will ignore the bad pycparser wheel. It will also ignore any other wheels, which is not what you really want, but it will resolve the issue until the offending pycparser wheel is fixed.

[anantzoid]

@reaperhulk Still getting AssertionError: sorry, but this version only supports 100 named groups on running this command.

[jeffrey4l]

install --no-binary pycparser is enough

[eka]

Same problem here.

[lukashes]

pip install cryptography --no-binary :all:

Does not work for me (--no-binary pycparse too). May be exists hack to fix it in developer machine?

[BTheunissen]

As with @lukashes neither pip install cryptography --no-binary :all: or --no-binary pycparser are working for myself unfortunately. Newly installed Ubuntu 16.04.

[eka]

@BTheunissen did you try pip install cryptography --no-binary pycparser ?

[BTheunissen]

@eka Apologies, I did initially but I messed up my original response, now edited.

[koleror]

Same problem here

[zetaz]

Worked after pycparser reinstall.

[eka]

For those with problems with --no-binary, that options is on latest versions of pip, try to upgrade pip pip install -U pip

[pporada-gl]

Thanks for the work around! We appreciate it over here.

[xavierhardy]

I also started having this problem this morning.

[xavierhardy]

What about forcing the pyparser requirement to a known and stable version?

[pporada-gl]

@xavierhardy
For us, we ran this temporary fix thanks to the others in this thread.

pip install git+https://github.com/eliben/pycparser@release_v2.14
pip install cryptography --no-binary pycparser

[reaperhulk]

We don't directly depend on pycparser so we can't pin it.

[xavierhardy]

@pporada-gl, yes, I've seen it, I'm just suggesting a cleaner and longer-term workaround. If you add in your project dependencies (before cryptography or the dependency that needs it)

pycparser==2.13

That fixes it too. Up to you to choose which workaround you prefer.

Or using the shell

pip install pycparser==2.13

[Ayrx]

Pinning the version does nothing because nothing prevents someone from uploading a bad wheel with the same version number.

[xavierhardy]

It's the version right before, the broken wheel is 2.14, but I get your point.

[alex]

Upstream has resolved this issue. If you have a local wheel cached you may need to delete I think.

[gmhurley]

It looks like pycparser v2.15 was released a couple hours ago and I started receiving this same error again. The workaround for me was to install pycparser==2.14.

[alex]

https://github.com/eliben/pycparser/issues/151 is the correct upstream bug to follow for resolution of this issue.

[chiefy]

Thanks @xavierhardy @alex - pinning version fixed issue for me. FWIW - was seeing it happen during upgrade of Ansible from 2.0 -> 2.2-devel

[reaperhulk]

pycparser 2.16 has been released and resolves this issue (again)