pip doesn't work in FIPS mode

[Pajinek]

• Pip version:
  pip 7.1.0 from /usr/lib/python2.6/site-packages (python 2.6)

• Python version:
  Python 2.6.6

• Operating System:
  redhat-release-server-6Server-6.8.0.5.el6.x86_64

Description:

I can't install any package by pip on systems in FIPS mode (Federal Information Processing Standard ). Problem is using hashlib and md5.

>>> hashlib.new("md5").digest_size
*** ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

I received following traceback.

> /usr/lib/python2.6/site-packages/pip/download.py(490)_check_hash()
-> if download_hash.digest_size != hashlib.new(link.hash_name).digest_size:
(Pdb) hashlib.new(link.hash_name)
*** ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

>> pip install --upgrade pip
Exception:
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/pip/basecommand.py", line 223, in main
    status = self.run(options, args)
  File "/usr/lib/python2.6/site-packages/pip/commands/install.py", line 290, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python2.6/site-packages/pip/req/req_set.py", line 334, in prepare_files
    functools.partial(self._prepare_file, finder))
  File "/usr/lib/python2.6/site-packages/pip/req/req_set.py", line 321, in _walk_req_to_install
    more_reqs = handler(req_to_install)
  File "/usr/lib/python2.6/site-packages/pip/req/req_set.py", line 491, in _prepare_file
    session=self.session)
  File "/usr/lib/python2.6/site-packages/pip/download.py", line 826, in unpack_url
    session,
  File "/usr/lib/python2.6/site-packages/pip/download.py", line 674, in unpack_http_url
    from_path, content_type = _download_http_url(link, session, temp_dir)
  File "/usr/lib/python2.6/site-packages/pip/download.py", line 887, in _download_http_url
    _download_url(resp, link, content_file)
  File "/usr/lib/python2.6/site-packages/pip/download.py", line 627, in _download_url
    _check_hash(download_hash, link)
  File "/usr/lib/python2.6/site-packages/pip/download.py", line 490, in _check_hash
    if download_hash.digest_size != hashlib.new(link.hash_name).digest_size:
AttributeError: 'NoneType' object has no attribute 'digest_size'

[RonnyPfannschmidt]

both pip and your python are massively outdated,
please check with an upstream version and note that the fips modification looks like something that should be handled by redhat's pip package, not pip upstream itself

[Pajinek]

Hi, thank you for your answer. I tried run new pip on CentOS7 and the result is identical.

pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7) - Python 2.7.5

Unknown hash name: md5

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 299, in run
    requirement_set.prepare_files(finder)
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 810, in unpack_url
    hashes=hashes
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 649, in unpack_http_url
    hashes)
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 871, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/usr/lib/python2.7/site-packages/pip/download.py", line 595, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/usr/lib/python2.7/site-packages/pip/utils/hashes.py", line 44, in check_against_chunks
    raise InstallationError('Unknown hash name: %s' % hash_name)
InstallationError: Unknown hash name: md5

[dstufft]

Current PyPI relies on MD5 hashes which is not FIPS compatible. You can switch over to using https://pypi.org/simple/ which uses sha256 hashes and thus should be FIPS compatible.

[flowersw]

Sorry, @dstufft , how can you switch over to using https://pypi.org/simple/ ? Is it pip configuration? Can it be done from the command-line? Thanks

Nevermind. I figured it out: By specifiying the -i option. Thanks

[RonnyPfannschmidt]

@Pajinek i believe its a RHEL Issue that the downstream version of pip uses a index that will break it in fips mode, i believe it can be elevated by configuring the default index as @dstufft mentioned

[lukehinds]

Wanted to mention, MD5 is not secure if pip is using this for integrity verification of packages. Its prone to collision attacks and apps should move to using sha2 or gpg (if they can - although I understand migration can be a challenge for already computed hashes).

That is why FIPS is blocking MD5, as its a security hazard.

[gshennessy]

I'm having the same problem. Can someone give an example using the -I option with https://pypi.org/simple please?

[flowersw]

@gshennessy , I think you can just run something like this:

pip install -i https://pypi.org/simple/ requests , replacing requests with your desired package.

[gshennessy]

The problem still seems present, although it displays differently.

[root@lin2v56ad-03 ~]# pip install -i https://pypi.org/simple/ pyfits
Collecting pyfits
Downloading https://files.pythonhosted.org/packages/45/98/d6d25932e6a82fa8456d38ab307bfb8945a1e1dd4e896730555e3b61cfc5/pyfits-3.4.tar.gz (1.7MB)
100% |████████████████████████████████| 1.7MB 220kB/s
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-4YavCt/pyfits/setup.py", line 14, in
zip_safe=False
File "/usr/lib64/python2.7/distutils/core.py", line 112, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 265, in __init__
self.fetch_build_eggs(attrs.pop('setup_requires'))
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 289, in fetch_build_eggs
parse_requirements(requires), installer=self.fetch_build_egg
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 618, in resolve
dist = best[req.key] = env.best_match(req, self, installer)
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 862, in best_match
return self.obtain(req, installer) # try and download/install
File "/usr/lib/python2.7/site-packages/pkg_resources.py", line 874, in obtain
return installer(requirement)
File "/usr/lib/python2.7/site-packages/setuptools/dist.py", line 339, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 611, in easy_install
self.local_index
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 598, in fetch_distribution
return dist.clone(location=self.download(dist.location, tmpdir))
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 515, in download
found = self._download_url(scheme.group(1), spec, tmpdir)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 762, in _download_url
return self._attempt_download(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 768, in _attempt_download
headers = self._download_to(url, filename)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 661, in _download_to
checker = HashChecker.from_url(url)
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 247, in from_url
return cls(**match.groupdict())
File "/usr/lib/python2.7/site-packages/setuptools/package_index.py", line 235, in __init__
self.hash = hashlib.new(hash_name)
File "/usr/lib64/python2.7/hashlib.py", line 111, in __hash_new
return _hashlib.new(name, string, usedforsecurity)
ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips

----------------------------------------

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-4YavCt/pyfits/
[root@lin2v56ad-03 ~]#

[dstufft]

This is really an issue with PyPI not with pip, and since Warehouse solves this already I'm going to just close this issue. If you want regular PyPI to drop the use of MD5 prior to the switch to Warehouse, please start a thread on distutils-sig.

[shaunbrady]

@dstufft Do we have a time frame on when pypi.org will be considered not pre-production? Need to justify the use to the same DoD players that don't want to use MD5. Let me know if there is a better place to ask. Thanks!

[pradyunsg]

While I don't know about the time-frame, "launch" is basically tracked in https://github.com/pypa/warehouse/milestone/1.

[shaunbrady]

@pradyunsg Exactly what we're after. Thanks! Will watch.

[dstufft]

@shaunbrady FWIW the only difference between today and when it will be production ready is:

1. UI fixes to clean up the UI from demo/test data and make sure it works across platforms etc.
2. A productionized deployment that can sustain the full traffic of switching everyone to using Warehouse.

For API use, (1) doesn't matter. For (2) the issue is basically our current provider's AUP limits the amount of monthly bandwidth we can use on Warehouse, which is > the amount of bandwidth the full traffic load of PyPI uses. For individual users to switch over there shouldn't be any issue (and indeed our upload traffic is low enough compared to our read traffic that Warehouse is the production instance for uploads now).

So that warning is really mostly a "please excuse any UI weirdness" at this point in time.

[shaunbrady]

@dstufft Sounds good! Thanks for the effort!