Pipenv: Error installing new venv from existing Pipfile; ValueError: Duplicate keys!

Created on 7 Mar 2019  Â·  2Comments  Â·  Source: pypa/pipenv

Issue description

I'm trying to install an existing Pipfile from a shared repo on my local.

I can successfully create the virtual environment, however when it starts to install my packages I get an error regarding duplicate keys

ValueError: Duplicate keys!

tomlkit.exceptions.KeyAlreadyPresent: Key "autopep8" already exists.

Which seems odd because there are no packages installed in this fresh environment.

Expected result

When I run $ pipenv install --three I would expect the virtualenv to be created and the packages specified in the Pipfile to be installed.

Actual result

# ~/path/to/project master*
$ pipenv install --three

Creating a virtualenv for this project…
Pipfile: ~/path/to/project/Pipfile
Using /usr/local/bin/python3 (3.7.1) to create virtualenv…
â ¦ Creating virtual environment...Using base prefix '/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7'
New python executable in /Users/me/.local/share/virtualenvs/data-sci-docs-c6tNPoIQ-/shims/python/bin/python3.7
Also creating executable in /Users/me/.local/share/virtualenvs/data-sci-docs-c6tNPoIQ-/shims/python/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/local/bin/python3

âś” Successfully created virtual environment! 
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 527, in _parse_pipfile
    return tomlkit.parse(contents)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/tomlkit/api.py", line 49, in parse
    return Parser(string).parse()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/tomlkit/parser.py", line 146, in parse
    key, value = self._parse_table()
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/tomlkit/parser.py", line 965, in _parse_table
    values.append(_key, item)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/tomlkit/container.py", line 139, in append
    raise KeyAlreadyPresent(key)
tomlkit.exceptions.KeyAlreadyPresent: Key "autopep8" already exists.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/toml/decoder.py", line 456, in loads
    multibackslash)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/toml/decoder.py", line 728, in load_line
    raise ValueError("Duplicate keys!")
ValueError: Duplicate keys!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/pipenv/cli/command.py", line 254, in install
    editable_packages=state.installstate.editables,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 1741, in do_install
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 574, in ensure_project
    pypi_mirror=pypi_mirror,
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 506, in ensure_virtualenv
    python=python, site_packages=site_packages, pypi_mirror=pypi_mirror
  File "/usr/local/lib/python3.7/site-packages/pipenv/core.py", line 946, in do_create_virtualenv
    sources = project.pipfile_sources
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 850, in pipfile_sources
    if "source" not in self.parsed_pipfile:
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 509, in parsed_pipfile
    parsed = self._parse_pipfile(contents)
  File "/usr/local/lib/python3.7/site-packages/pipenv/project.py", line 531, in _parse_pipfile
    return toml.loads(contents)
  File "/usr/local/lib/python3.7/site-packages/pipenv/vendor/toml/decoder.py", line 458, in loads
    raise TomlDecodeError(str(err), original, pos)
toml.decoder.TomlDecodeError: Duplicate keys! (line 22 column 1 char 335)

Steps to replicate

I put the commands I used above, and here is my Pipfile

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pandas = "*"
numpy = "*"
matplotlib = "*"
sklearn = "*"
psycopg2 = "*"
jupyter = "*"
pystan = "*"
fbprophet = "*"
scipy = "*"
ipykernel = "*"
autopep8 = "*"
jupyter-contrib-nbextensions = "*"
jupyter-nbextensions-configurator = "*"
autopep8 = "*"

[requires]
python_version = "3.7"


$ pipenv --support | pbcopy

$ pipenv --support

Pipenv version: '2018.11.26'

Pipenv location: '/usr/local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/local/opt/python/bin/python3.7'

Python installations found:

  • 3.7.1: /usr/local/bin/python3
  • 3.7.1: /usr/local/bin/python3.7m
  • 2.7.10: /usr/bin/python
  • 2.7.10: /usr/bin/pythonw
  • 2.7.10: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.1',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '18.2.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST '
                     '2018; root:xnu-4903.241.1~1/RELEASE_X86_64',
 'python_full_version': '3.7.1',
 'python_version': '3.7',
 'sys_platform': 'darwin'}

System environment variables:

  • TMPDIR
  • __CF_USER_TEXT_ENCODING
  • HOME
  • SHELL
  • Apple_PubSub_Socket_Render
  • SSH_AUTH_SOCK
  • PATH
  • LOGNAME
  • XPC_SERVICE_NAME
  • USER
  • SECURITYSESSIONID
  • XPC_FLAGS
  • GOOGLE_API_CREDS
  • LANGUAGE_SHEET_ID
  • LC_CTYPE
  • LESS
  • LSCOLORS
  • OLDPWD
  • PAGER
  • PIPENV_PYTHON
  • PROMPT_EOL_MARK
  • PWD
  • PYENV_SHELL
  • REDSHIFT_DBNAME
  • REDSHIFT_HOST
  • REDSHIFT_PASS
  • REDSHIFT_PORT
  • REDSHIFT_USER
  • SHLVL
  • ZSH
  • TERM_PROGRAM
  • TERM_PROGRAM_VERSION
  • LANG
  • TERM
  • VIRTUAL_ENV_DISABLE_PROMPT
  • _
  • PIP_DISABLE_PIP_VERSION_CHECK
  • PYTHONDONTWRITEBYTECODE
  • PIP_SHIMS_BASE_MODULE
  • PIP_PYTHON_PATH
  • PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

  • PIPENV_PYTHON: /shims/python

Debug–specific environment variables:

  • PATH: /Users/me/.pyenv/shims:/Users/me/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/me/.pyenv/shims:/Users/me/google-cloud-sdk/bin
  • SHELL: /bin/zsh
  • LANG: en_US.UTF-8
  • PWD: /Users/me/path/to/project

Contents of Pipfile ('/Users/me/path/to/project/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pandas = "*"
numpy = "*"
matplotlib = "*"
sklearn = "*"
psycopg2 = "*"
jupyter = "*"
pystan = "*"
fbprophet = "*"
scipy = "*"
ipykernel = "*"
autopep8 = "*"
jupyter-contrib-nbextensions = "*"
jupyter-nbextensions-configurator = "*"
autopep8 = "*"

[requires]
python_version = "3.7"

Contents of Pipfile.lock ('/Users/me/path/to/project/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "7e7ef69da7248742e869378f8421880cf8f0017f96d94d086813baa518a65489"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.7"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

picture robertdefilippi

Most helpful comment

Just remove the duplicate entry as instructed, and all things are fixed.

picture frostming on 7 Mar 2019
👍2 😄1

All 2 comments

Just remove the duplicate entry as instructed, and all things are fixed.

frostming picture frostming on 7 Mar 2019
👍2 😄1

Shame on me ... I can't believe I missed that. Thanks!

robertdefilippi picture robertdefilippi on 7 Mar 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

FooBarQuaxx picture FooBarQuaxx  Â·  3Comments
marc-fez picture marc-fez  Â·  3Comments
AkiraSama picture AkiraSama  Â·  3Comments
johnjiang picture johnjiang  Â·  3Comments
Californian picture Californian  Â·  3Comments