Pipenv: MemoryError: needs a '--no-cache-dir' install option

[octoberclub]

Fails with a Memory Error when using pipenv install on a machine without large memory - e.g. a minimal docker image, (although it appears to have adequate memory for most development - 800Mb / 1Gb available, and the same package works ok on most other environments) - see stack trace below

It appears to be a problem with pip itself.
Excessive memory use when caching large packages #2984
Stack overflow: memory-error-while-using-pip-install-matplotlib

The recommended approach, pip install --no-cache-dir can be used as a workaround to allow this to run without reading the whole package into memory.

Is it possible to supply that argument using pipenv? Are there any other ways to fix it?

Pipfile

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[packages]
mapnik = "*"

MemoryError

# pipenv install
Creating a virtualenv for this project...
⠋New python executable in /root/.local/share/virtualenvs/opt-zvmYt2-H/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /root/.local/share/virtualenvs/opt-zvmYt2-H
No package provided, installing all dependencies.
Pipfile found at /opt/Pipfile. Considering this to be the project home.
Installing dependencies from Pipfile.lock...
An error occured while installing! 7/30 - 00:00:34
Collecting mapnik==0.1
  Downloading mapnik-0.1-cp27-none-any.whl (37.4MB)

Exception:
Traceback (most recent call last):
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/commands/install.py", line 335, in run
    wb.build(autobuilding=True)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/wheel.py", line 749, in build
    self.requirement_set.prepare_files(self.finder)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/req/req_set.py", line 620, in _prepare_file
    session=self.session, hashes=hashes)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 821, in unpack_url
    hashes=hashes
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 659, in unpack_http_url
    hashes)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 882, in _download_http_url
    _download_url(resp, link, content_file, hashes)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 603, in _download_url
    hashes.check_against_chunks(downloaded_chunks)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/utils/hashes.py", line 46, in check_against_chunks
    for chunk in chunks:
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 571, in written_chunks
    for chunk in chunks:
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/utils/ui.py", line 139, in iter
    for x in it:
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/download.py", line 560, in resp_read
    decode_content=False):
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 357, in stream
    data = self.read(amt=amt, decode_content=decode_content)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 314, in read
    data = self._fp.read(amt)
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 63, in read
    self._close()
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/filewrapper.py", line 50, in _close
    self.__callback(self.__buf.getvalue())
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/controller.py", line 275, in cache_response
    self.serializer.dumps(request, response, body=body),
  File "/root/.local/share/virtualenvs/opt-zvmYt2-H/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/serialize.py", line 87, in dumps
    ).encode("utf8"),
MemoryError

[kennethreitz42]

If we were going to do this, we'd make it default for everyone, or use an environment variable, which i'm mildly opposed to doing.

[kennethreitz42]

open an issue with pip, to see if they'll honor an environment variable for this.

[nateprewitt]

Just as a note, I believe pip will support setting this in pip.conf. I'd look into that as a possible option.

[erinxocon]

Just to add on to @nateprewitt I do believe pip.conf supports this setting.

[dfpoon]

As a workaround, I type the following in the terminal instead of using the pip.conf file:

$> export PIP_NO_CACHE_DIR=false

It will work. Hope it will help those who read this thread.

reference: https://pip.pypa.io/en/stable/user_guide/#configuration