Pipenv: April (then May) 2020 Release Tracking Issue

[techalchemy]

This is an internal tracking issue which will tie in the related issues which will be tackled/still need to be updated for the purposes of cutting the release. It's been a long time coming (see https://github.com/pypa/pipenv/issues/4058#issuecomment-565550646 and https://github.com/pypa/pipenv/issues/3742#issuecomment-492100711 for some past comments on that) and there's a tentative goal of getting a pre-release out in March 2020.

(Edited by @brainwane to say: fixing some brokenness in the continuous integration setup is delaying this release 2020.04.1a1 till -- new estimate -- 21 April 2020.)

(Edited by @brainwane to say: the prerelease 2020.4.1b1 is now out, as of 29 April, and Dan aims to get the next release out in about a week.)

(Edited by @brainwane in conversation with Dan on 5 March 2020 and then throughout March & April)

• [x] Other Items in the March 2020 release milestone:
  
  
  
  • [x] #3613 Pipenv installs wrong packages (ignoring package lock)
  
  
  • [x] #4137 Update README and Read the Docs project to point to docs at pipenv.pypa.io
  
  
• [x] [Fix pip-shims test failures](https://github.com/sarugaku/pip-shims/issues/58) (merge the bugfix/56 branch)
• [x] requirementslib fixes:
  
  
  
  • [x] sarugaku/requirementslib#204 requirementslib issue AST parsing of setup.py fails on non-addition type binOps
  
  
  • [x] sarugaku/requirementslib#205 AST parsing fails on non-utf8 setup.py files
  
  
  • [x] sarugaku/requirementslib/issues/214 We shouldn't install upgraded versions of test dependencies in CI
  
  
  • [x] sarugaku/requirementslib#219 calculator Add requirement calculator for artifacts in PyPI
  
  
  • [x] Review requirements-lib issues for other pipenv release blockers
  
  
• [x] Make new releases of related libraries:
  
  
  
  • [x] [requirementslib](https://pypi.org/project/requirementslib/)
  
  
  • [x] [pip-shims](https://pypi.org/project/pip-shims/)
  
  
  • [x] [pythonfinder](https://pypi.org/project/pythonfinder/)
  
  
• [x] Finish, approve and merge #4169 to address missing dependencies/imports
  
  
  
  • [x] Fix news entry/entries
  
  
  • [x] Fix broken CI configuration
  
  
  • [x] Get tests to pass
  
  
• [x] Fix #4188 by either getting pyup API key working again or choosing and implementing an alternative
• [x] Get GitHub tests to pass on master (skipping the test mentioned in #4201)
• [x] Improve release automation to allow generation of prereleases
  
  
  
  • [x] Generate changelog in draft mode
  
  
  • [x] Don't autotag & push tag to repo
  
  
• [x] Update and run through "Test locally" and "Uploading the release" for packaging/release checklist
• [x] Package and upload pipenv prerelease version 2020.04.1a1 to PyPI

That's what Dan aims to do by 21 April 2020. Then:

• [x] Make another new release of requirementslib that incorporates a fix to https://github.com/sarugaku/requirementslib/issues/216
  
  
  
  • [x] Relock requirementslib dependencies
  
  
• [x] Publicize pre-release and ask for a week of testing
  
  
  
  • [x] Especially testing on Windows against PEP 517 backends and involving virtualenvs
  
  
• [x] After ~1 week (if no showstopper bugs), publish new release

How others can help:

• replying to new users' questions in pipenv's GitHub issues
• help develop a "Pipenv workflows to test" list, along these lines

[JBKahn]

This seems like a good place to ask about when the next release might be, what the blockers are and if there is anything someone can do to help?

[adawalli]

My team is anxious for the next update, specifically to address #3298. Are there major blockers still left?

[AlJohri]

linking https://github.com/pypa/pipenv/issues/3742

[brainwane]

I noticed https://github.com/pypa/packaging.python.org/issues/701 today and donated about 90 minutes of time helping @techalchemy get some more clarity on what's blocking him from making the new release (IRC conversation)). https://github.com/pypa/pipenv/issues/3369#issue-389160510 now has a release checklist. @techalchemy could use help with those release blockers, in case @JBKahn or anyone else wants to help out.

[Froskekongen]

@techalchemy: When looking at the issues linked here, a lot of them are fixed. I think it would be nice to check the boxes for the issues that have been fixed, so that people can see that there is progress by just looking at the first post in this thread (:

[amhrasmussen]

@brainwane, @techalchemy, excuse my interference and potential ignorance, but may I suggest taking #2227 and/or #3520 out of this release? My impression is that getting more or less anything released is essential and beneficial. #2227 looks like a new feature while #3520 has a workaround described (with no objection from reporter).

Similarly, is it strictly needed to "make new releases of related libraries" before getting next pipenv out? While looking into #3613, I found the current master branch to just work...

I'm not suggesting these issues are not important, just that they should not block the improvements already in master from getting to the people.

[shamoons]

March is almost over

[fridex]

It looks like https://github.com/pypa/pipenv/issues/3520 is fixed in the current master, https://github.com/pypa/pipenv/issues/2227 does not look like a critical fix for an update.

Is there anything blocking where the community can be helpful? I'm happy to give you my hand. Otherwise, I see just release updates and docs updates.

[brainwane]

@Froskekongen @amhrasmussen @fridex thanks for the pointers! I donated some more time yesterday and @techalchemy and I worked through a few more of the relevant items on the list.

When looking at the issues linked here, a lot of them are fixed. I think it would be nice to check the boxes for the issues that have been fixed, so that people can see that there is progress by just looking at the first post in this thread (:

Thanks! Several more are checked now. :-)

It looks like #3520 is fixed in the current master

Could you please say that in a comment on #3520? Thanks.

2227 does not look like a critical fix for an update.

@brainwane, @techalchemy, excuse my interference and potential ignorance, but may I suggest taking #2227 and/or #3520 out of this release? My impression is that getting more or less anything released is essential and beneficial. #2227 looks like a new feature while #3520 has a workaround described (with no objection from reporter).

Thanks. Dan agreed with you and we removed #2227 as a blocker on this release. If you could help with #3520 by confirming that the workaround works, and commenting there, that would be great.

Similarly, is it strictly needed to "make new releases of related libraries" before getting next pipenv out? While looking into #3613, I found the current master branch to just work...

I'm not suggesting these issues are not important, just that they should not block the improvements already in master from getting to the people.

As I understand it, those libraries are important for pipenv functionality, as well as properly testing pipenv to ensure the release works on the supported OS/environment combinations. I could be wrong but Dan said "tbh most of the work happens in the ancillary libraries these days".

Is there anything blocking where the community can be helpful? I'm happy to give you my hand. Otherwise, I see just release updates and docs updates.

Please help by replying to new users' questions in pipenv's GitHub issues; that way Dan doesn't have to worry about replying to those, and can concentrate on this release.

Getting closer to a release!

[brainwane]

New update email from @techalchemy on distutils-sig (mirrored on the pypa-dev list). Includes a few ways you can help.

[brainwane]

@AEHamrick I saw your comment https://github.com/pypa/pipenv/issues/3742#issuecomment-498312920 -- I took a quick look at #3382 and #3757, and it sounds like those are already fixed in master. Is that right? If they are not yet fixed in master, please say so in a comment here, so we can consider adding them to the release milestone. Thanks!

[AEHamrick]

@brainwane Hey, I appreciate you checking in. Candidly, I started using Poetry quite a while ago due to the lack of activity here, but if I can find time I'll set up a test env and see if the situations I was hitting those bugs seem to be ok now.

[brainwane]

I totally understand, @AEHamrick -- thanks.

I donated some more time yesterday to help @techalchemy move further. Dan made new releases of pythonfinder and requirementslib, and got further on merging the vendor-update branch to address missing dependencies/imports. He's still working on that today ("had some tests fail but not too many- 17 / 230 or so" as of earlier today).

After the Pipenv maintainers publish a pre-release, some manual testing from users will help them feel confident about pushing the canonical release. I suggest: you can help develop a "Pipenv workflows to test" list, along these lines.

Also, anyone who's following this issue, consider retweeting this tweet about the current status and how people can help.

[brainwane]

vistir is actually already up to date; removed that from the release TODO.

And @techalchemy has started a Pipenv release process instruction guide -- if there are steps that should be in there and are not, please comment here or add some bullet points in that document.

[brainwane]

Dan's closer to being able to merge the feature/vendor-update branch:

1 failed, 231 passed, 21 skipped in 442.72 seconds

[brainwane]

Dan got the tests to pass locally, and has now made pull request #4169 and is ensuring that the tests pass; right now they are running very slowly and sometimes failing. So he's working on that right now -- specifically, trying to get Azure syntax to work. Depending on how much he can speed up some of those tests, and whether they flake out, and whether he has to chase intermittent failures, he may be able to make a pre-release today! Or it might be a few days from now.

People have asked how they can help. In the next few days or weeks, @techalchemy may need particular help testing the new pre-release on Windows. Please use the :eyes: emoji reaction to this comment if you're willing to help test on Windows.

And if you know a lot about Azure pipelines templates, consider coming into the IRC channel and helping out there today and/or tomorrow?

[brainwane]

See https://github.com/pypa/pipenv/pull/4169#issuecomment-607348834 for an update on PR progress.

[brainwane]

Dan's continuing to make progress on getting the tests to pass (as the comments on #4169 note, the tests started failing a few months ago, so he's dealing not just with this one PR but with backlogged technical debt here). I changed the issue title to reflect reality.

[brainwane]

See https://github.com/pypa/pipenv/pull/4169#issuecomment-610017500 for a happy update on PR progress.

[brainwane]

If you can help figure out Windows test failures on Azure, please take a look at #4169 and comment.

How others can help:

• replying to new users' questions in pipenv's GitHub issues
• helping develop a "Pipenv workflows to test" list, along these lines
• retweeting this tweet about the current status and how people can help
• improving our release process -- @techalchemy has started a Pipenv release process instruction guide -- if there are steps that should be in there and are not, please comment in #3369 or add some bullet points in that document
• volunteering to test on Windows

[brainwane]

Thanks to the folks who contributed Windows and Azure testing expertise in #4169!

Dan has just merged #4169 and is determining whether any other pull requests also need to be merged before he can make a new pre-release. Once @techalchemy does make that pre-release, he'll be asking for help testing it.

There's a separate issue, #4130, to work out longer-term plans on improving the roadmap, and contributor and maintainer processes.

[brainwane]

The new issue #4188 urgently needs a fix, since it is breaking existing installations, so Dan's working on that first. I've updated the checklist at the beginning of this issue accordingly.

[brainwane]

@techalchemy does not intend to look further through the existing issues to seek out release blockers -- after publishing the prerelease and recruiting testers, he'll be sifting through issues to see whether any of them should block the new release.

Now that he and other pipenv developers have solved the Windows testing problem (in #4169 and other PRs) and Dan's in the midst of addressing #4188 (thanks to the PyUp folks for a call yesterday), he'd like to have a prerelease of pipenv out this weekend or Monday. The version number is tentatively slated to be 2020.04.1a1. We'll then be recruiting testers for about a week of manual testing.

[bneijt]

I've not been able to fix it yet, but you have at least one test that is not parallel safe. The tests run with pytest -n 3 in .azure-pipelines/steps/run-tests-windows.yml but this will cause random failures like the the cache clear test test_pipenv_clear in tests\integration\test_cli.py.

Probably all cli tests share cache paths and should either not be run in parallel or have each PipEnvInstance use a separate root/home path.

See also two run of the same code:
https://github.com/bneijt/pipenv/runs/596610863?check_suite_focus=true
https://github.com/bneijt/pipenv/runs/596726284?check_suite_focus=true

Any one of the following might help, but I'm new to the project so I'm not sure what to do yet:

• Set up a PR for scheduled builds instead of only on pull, this will give more insight into if builds fail without changes
• Disable parallel tests for the cli suite
• Make sure PipenvInstance sets up a unique cache path for each instance.
• Create an issue and leave it for another time

Please advise.

[ttamg]

Just a quick note to say thanks to all of you supporting the pipenv release. It is much appreciated. pipenv should be the go to tool, particularly for Python newbies in my view as it abstracts away a lot of the virtualenv hassles that a newbie should not have to bother with at the start. Great stuff!

[techalchemy]

Hey all just checking in -- hit a minor snag with a breakage on a dependent release and a few more azure issues. All green as of tonight so I'll be prereleasing tomorrow :) Thanks for your patience!

[techalchemy]

As another brief update, I have published a build to https://test.pypi.org for the moment for anyone eager to get their hands on a copy -- it can be installed via pip install --index-url=https://test.pypi.org pipenv. I have not yet validated anything about this build, but needed to publish something in order to claim ownership of the name on test PyPI :)

The final step here will be porting over a github action to do a series of releases, first onto the test instance, then handling the actual release. This is how I am managing releases across most of my other projects now, and it should make the process a bit smoother since it removes me as a single point of failure. Thanks again for your patience and let me know if you run into any issues

[mgmarino]

Thanks @techalchemy! FYI, this worked for me:

pip install -i https://test.pypi.org/simple/ pipenv

after doing a pip uninstall pipenv

[mungojam]

Is there a requirement to have a certain version of pip installed? I thought that it used a vendored one within pipenv itself, but I get this error:

File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvcore.py", line 862, in do_install_dependencies
normal_deps, procs, failed_deps_queue, requirements_dir, install_kwargs
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvcore.py", line 776, in batch_install
use_pep517=use_pep517,
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvcore.py", line 1484, in pip_install
no_deps=no_deps, require_hashes=not ignore_hashes
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvcore.py", line 1341, in get_pip_args
if project.environment.pip_version >= parse_version("19.0"):
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvproject.py", line 343, in environment
self._environment.extend_dists(pipenv_dist)
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvenvironment.py", line 102, in extend_dists
extras = self.resolve_dist(dist, self.base_working_set)
File "C:UsersaaAppDataRoamingPythonPython36site-packagespipenvenvironment.py", line 97, in resolve_dist
dist = working_set.find(req)
File "C:UsersaaAppDataRoamingPythonPython36site-packagespkg_resources__init__.py", line 644, in find
raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (pip 9.0.1 (c:programdataanaconda3libsite-packages), Requirement.parse('pip>=18.0')**)

We are running python 3.6.0, and I'm reluctant to update our global version of pip because that has previously meant that we couldn't run pip install blah anymore and had to do python -m pip install blah.

[JeanFred]

Thanks @techalchemy for the hard work! Looking forward to a release

it can be installed via pip install --index-url=https://test.pypi.org pipenv.

I quickly gave this a try in a virtualenv, and got the following:

> pip install -i https://test.pypi.org/simple/ pipenv
Looking in indexes: https://test.pypi.org/simple/
Collecting pipenv
  Downloading https://test-files.pythonhosted.org/packages/d7/0d/aa8ee85ece59f7ee27282a6a39ec1d061560d76b27a5c40a4b478473a5dd/pipenv-2020.4.1a1-py2.py3-none-any.whl (3.8 MB)
     |████████████████████████████████| 3.8 MB 1.7 MB/s 
Requirement already satisfied: pip>=18.0 in ./.venc/lib/python3.6/site-packages (from pipenv) (20.0.2)
Requirement already satisfied: setuptools>=36.2.1 in ./.venc/lib/python3.6/site-packages (from pipenv) (46.1.3)
Collecting certifi
  Downloading https://test-files.pythonhosted.org/packages/0e/67/0383b38ee00ffa9f93348998d0b474c1f5aa4a4ff562957294f1477a47b5/certifi-2016.8.8-py2.py3-none-any.whl (384 kB)
     |████████████████████████████████| 384 kB 4.1 MB/s 
ERROR: Could not find a version that satisfies the requirement virtualenv-clone>=0.2.5 (from pipenv) (from versions: none)
ERROR: No matching distribution found for virtualenv-clone>=0.2.5 (from pipenv)

I assume this is because virtualenv-clone is not availble on test.pypi.org ; installing virtualenv-clone and virtualenv manually in the venv did the trick for me.

Sharing this in case this is helpful to others :)

[mgmarino]

@JeanFred try:

pip uninstall pipenv

and then rerun the install

[sim0nx]

I just updated my current pipenv:

pip install -i https://test.pypi.org/simple/ -U pipenv

Works like a charm! Thank you @techalchemy !

[ncoghlan]

@bneijt Filing a new issue for the test flakiness you found would be a good idea, as otherwise I fear that info will be lost in the pre-release testing feedback here.

[techalchemy]

apologies for the lack of follow up on this @JeanFred you can fix this temporarily with pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple --pre pipenv

If you can wait a bit, I will be pushing a prerelease to PyPI shortly

[nunoachenriques]

apologies for the lack of follow up on this @JeanFred you can fix this temporarily with pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple --pre pipenv

If you can wait a bit, I will be pushing a prerelease to PyPI shortly

Hi @techalchemy , thanks for your effort! Was I too quick?

# pip install --index-url https://test.pypi.org/simple --extra-index-url https://pypi.org/simple --pre pipenv
Looking in indexes: https://test.pypi.org/simple, https://pypi.org/simple
Collecting pipenv
  Downloading https://test-files.pythonhosted.org/packages/d7/0d/aa8ee85ece59f7ee27282a6a39ec1d061560d76b27a5c40a4b478473a5dd/pipenv-2020.4.1a1-py2.py3-none-any.whl (3.8MB)
    100% |████████████████████████████████| 3.8MB 368kB/s 
Requirement already satisfied: setuptools>=36.2.1 in /usr/lib/python3/dist-packages (from pipenv) (40.8.0)
Collecting virtualenv-clone>=0.2.5 (from pipenv)
Could not install packages due to an EnvironmentError: 404 Client Error: Not Found for url: https://test.pypi.org/simple/virtualenv-clone/

[techalchemy]

Greetings all, I went ahead and pushed a pre-release to regular pypi so you can avoid the complexity of test pypi -- feel free to try out pip install --upgrade --pre pipenv and you should see 2020.4.1b1 install

I'll send out more detail tomorrow when I've had some sleep but would encourage anyone who wishes to test this out a bit to consult or build upon the manual workflows listed in this document

Thanks again for your patience!

[Flimm]

This is exciting! Thank you to everyone who is working on this.

[hroncok]

I haven't read all this yet, but I've seen an alpha release and than immediately a beta. In Fedora, we will test the prerelease, but we need couple days and the speedy beta after the alpha surprised me a bit.

[techalchemy]

@hroncok the alpha release on main pypi was largely to validate the release automation change, the beta release will be up for a week at least. Feel free to ping me on IRC if you need more time or have any additional challenges with repackaging

[hroncok]

Feel free to ping me on IRC if you need more time or have any additional challenges with repackaging

Will do if needed. Thank you.

EDIT: I've unsubscribed here, because I suspect a lot of "the new release works for me" user reports here.

[nunoachenriques]

@techalchemy thank you! Already tested in a project and it's doing just fine!

# pip uninstall -y virtualenv virtualenv-clone pipenv
Uninstalling virtualenv-20.0.18:
  Successfully uninstalled virtualenv-20.0.18
Uninstalling virtualenv-clone-0.5.4:
  Successfully uninstalled virtualenv-clone-0.5.4
Uninstalling pipenv-2020.4.1a1:
  Successfully uninstalled pipenv-2020.4.1a1
# pip install --pre pipenv
Collecting pipenv
  Downloading https://files.pythonhosted.org/packages/c6/ae/de435f1da25056a181cbc3241d747b521f0caf8db0e5ca19400b9f120e7d/pipenv-2020.4.1b1-py2.py3-none-any.whl (3.8MB)
    100% |████████████████████████████████| 3.8MB 391kB/s 
Collecting virtualenv-clone>=0.2.5 (from pipenv)
  Using cached https://files.pythonhosted.org/packages/83/b8/cd931487d250565392c39409117436d910232c8a3ac09ea2fb62a6c47bff/virtualenv_clone-0.5.4-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=36.2.1 in /usr/lib/python3/dist-packages (from pipenv) (40.8.0)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from pipenv) (2018.8.24)
Requirement already satisfied: pip>=18.0 in /usr/lib/python3/dist-packages (from pipenv) (18.1)
Collecting virtualenv (from pipenv)
  Using cached https://files.pythonhosted.org/packages/23/1a/d55f5fb904564365c3d364fd41701a74ad497379f04fd548cccc3f227d98/virtualenv-20.0.18-py2.py3-none-any.whl
Requirement already satisfied: filelock<4,>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from virtualenv->pipenv) (3.0.12)
Requirement already satisfied: distlib<1,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from virtualenv->pipenv) (0.3.0)
Requirement already satisfied: importlib-metadata<2,>=0.12; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from virtualenv->pipenv) (1.6.0)
Requirement already satisfied: six<2,>=1.9.0 in /usr/lib/python3/dist-packages (from virtualenv->pipenv) (1.12.0)
Requirement already satisfied: appdirs<2,>=1.4.3 in /usr/local/lib/python3.7/dist-packages (from virtualenv->pipenv) (1.4.3)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata<2,>=0.12; python_version < "3.8"->virtualenv->pipenv) (3.1.0)
Installing collected packages: virtualenv-clone, virtualenv, pipenv
Successfully installed pipenv-2020.4.1b1 virtualenv-20.0.18 virtualenv-clone-0.5.4
# pipenv --version
pipenv, version 2020.4.1b1 

[mhindery]

I'm having an issue with importing a requirements file. This is with python 3.8.2, and the latest pipenv from above:

Mathieus-MacBook-Pro: mhindery$ pipenv install -r requirements.txt 
Creating a virtualenv for this project…
Pipfile: /<...>/Pipfile
Using /<...>/.pyenv/versions/3.8.2/bin/python3.8 (3.8.2) to create virtualenv…
⠧ Creating virtual environment...created virtual environment CPython3.8.2.final.0-64 in 444ms
  creator CPython3Posix(dest=/<...>-7y_bNLv7, clear=False, global=False)
  seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/<...>/virtualenv/seed-app-data/v1.0.1)
  activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator

✔ Successfully created virtual environment! 
Virtualenv location: /<...>-7y_bNLv7
Requirements file provided! Importing into Pipfile…
Traceback (most recent call last):
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/project.py", line 789, in write_toml
    formatted_data = tomlkit.dumps(data).rstrip()
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/tomlkit/api.py", line 42, in dumps
    return data.as_string()
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/tomlkit/container.py", line 343, in as_string
    s += self._render_aot(k, v)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/tomlkit/container.py", line 412, in _render_aot
    cur += self._render_aot_table(table, prefix=_key)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/tomlkit/container.py", line 421, in _render_aot_table
    if not table.is_super_table():
AttributeError: 'dict' object has no attribute 'is_super_table'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mhindery/.pyenv/versions/3.8.2/bin/pipenv", line 10, in <module>
    sys.exit(cli())
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/cli/command.py", line 231, in install
    retcode = do_install(
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/core.py", line 1982, in do_install
    import_requirements(r=project.path_to(requirements), dev=dev)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/core.py", line 230, in import_requirements
    project.add_index_to_pipfile(index, verify_ssl=trusted)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/project.py", line 990, in add_index_to_pipfile
    self.write_toml(p)
  File "/Users/mhindery/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pipenv/project.py", line 793, in write_toml
    document[section] = tomlkit.container.Table()
TypeError: __init__() missing 3 required positional arguments: 'value', 'trivia', and 'is_aot_element'
Mathieus-MacBook-Pro: mhindery$ pipenv --version
pipenv, version 2020.4.1b1
Mathieus-MacBook-Pro: mhindery$ python --version
Python 3.8.2
Mathieus-MacBook-Pro:usermanagement mhindery$ pyenv --version
pyenv 1.2.18
Mathieus-MacBook-Pro: mhindery$ 

[frostming]

@mhindery A quick fix is landed in #4219

[yhoiseth]

I discovered a seemingly non-critical error which I reported in the separate #4220.

[remidebette]

Hi,
It is nice to see the release taking form.

I wanted to give a try to the pre release version on Mac Os and got most locks and installs of my projects' libraries working, except the 2 mentioned here:
https://github.com/pypa/pipenv/issues/4227

[brainwane]

@techalchemy has announced the prerelease on Python's Discourse forum with details including:

• target for release (Thursday, May 7th)
• what's new (highlights & full changelog)
• what Pipenv needs help testing
• other changes in the project: process and communications changes, release cadence & financial support
• thank-yous

[brainwane]

Windows testing: Hi to the folks who volunteered earlier! Please test Pipenv and file bugs between now and May 6th, if you can.

@command-tab
@mungojam
@snakeice
@nicholasAlert
@madsmtm
@Mause
@gmsantos
@davekeanexero
@lucidyan
@bratao
@sudopriestmx
@amhrasmussen

[gmsantos]

Windows testing: Hi to the folks who volunteered earlier! Please test Pipenv and file bugs between now and May 6th, if you can.

I ran the commands in this link and pipenv update --dry-run results in an exception #4229

It also happened with other projects and on Linux/Ubuntu as well. pipenv update works as expected.

[brainwane]

I haven't checked with @techalchemy but I just quickly looked through this repo and the four Sarugaku repositories (such as requirementslib) to see how this beta is going.

Changes:

4232 came up and was fixed by #4236.

4217 came up and is fixed by #4241 (which also addresses #3396).

4219 and #4230 are now merged in.

Still open:

4231 is still open and Dan needs help reproducing it on Mac OS X and/or Linux -- "For setup.py's which have setup_requires like cython, pipenv does not seem to be honoring it while locking the Pipfile".

[mgmarino]

@brainwane FYI: sarugaku/requirementslib#229 is a fix to the regression described in sarugaku/requirementslib#228 which is part of this beta.

[Systemcluster]

@brainwane I'd think #4218 would be a blocker as well? It's a regression and completely breaks both existing and new projects with certain dependencies.

[techalchemy]

To provide an additional update here, I will try to have this release out tomorrow (I realize this is a day later than planned, but a few of the issues were relatively tricky to track down -- huge thanks to those of you who were able to test, provide feedback, and help provide insight into some of the subtle issues going on.

[techalchemy]

I've posted a more thorough postmortem on the release blocking issue here -- this is now waiting on builds to finish but as it's now 1am on Saturday morning I will most likely refrain from releasing until Monday to avoid breaking anything while nobody is around to respond / react.

Thanks again to everyone who has provided valuable debugging information to help nail down some of these issues, it has made the process much smoother.

[Immortalin]

@techalchemy ?

[liammonahan]

@Immortalin Even if Dan said to potentially expect something on Monday, your comment is not constructive. Please check yourself.

[liammonahan]

Dan, we all appreciate your hard work. I think most of us realize that we need to find more ways to be helpful to you.

[gentooboontoo]

4251 could be a showstopper (pipenv install --outdated failure).

[washeck]

Do you plan to release another beta version? I am using version 2020.4.1b1 and I encountered an error in locking a project with psycopg2. I see there were fixes such as #4231 so I'd rather check it using the latest version of pipenv with all bugfixes rather than reporting something already fixed.

[techalchemy]

Here's a release update. Tl;dr: We ran into unexpected issues updating dependent libraries that pipenv vendors, and that caused a delay. There'll be a new prerelease sometime in the next few days.

Details:
Late last week, I was merging what should have been a simple fix in vistir to address #4195. I was making this change as part of what should have been some quick revendoring in preparation to release; vistir is one of the libraries that provides some of the cross-platform/Python 2/3 compatibility support for Pipenv. However, CI failed across the board for all kinds of strange reasons, and this has been a bit time-consuming to figure out and address properly. I couldn't just revert the fix and release anyway, because without this fix, there was a risk of breaking cross-platform and compatibility in Pipenv.

Plan:

• ~Release the libraries (https://github.com/sarugaku/vistir / https://github.com/sarugaku/requirementslib)~
• ~Re-vendor updated libraries in Pipenv~
• ~Merge remaining low-risk, ready-to-merge PRs in Pipenv~
• ~Review Pipenv issues for release blockers~
• ~Pre?Release: probably make a fresh prerelease of Pipenv prior to a canonical release~

Additional notes

• Failures were not related to the change, but may be related to changes in how Azure (CI environment) creates Python instances
• In one case this may have exposed a significant potential bug in how output is encoded and decoded on Windows, while the other case relates to representing paths to the latest version of MacOS
• The functionality is tested thoroughly by property-based tests, so we can be confident that the failures are real issues and the tests would capture ongoing issues
• Thanks to the other library and Pipenv maintainers, and testers and other users, for helping me track down stuff

[r-richmond]

First off let me say thank you so much for the work you are putting into this release and keeping this project going. Pipenv has saved me multiple times over.

All that said and at the risk of going slightly off topic. Given the latest hiccup in the release process

Late last week, I was merging what should have been a simple fix in vistir to address #4195. I was making this change as part of what should have been some quick revendoring in preparation to release; vistir is one of the libraries that provides some of the cross-platform/Python 2/3 compatibility support for Pipenv. However, CI failed across the board for all kinds of strange reasons, and this has been a bit time-consuming to figure out and address properly. I couldn't just revert the fix and release anyway, because without this fix, there was a risk of breaking cross-platform and compatibility in Pipenv.

Have you considered dropping python 2 support (it is EOL) and removing its requirements (i.e. vistir and maybe others)?

I assume long term python2 support will be dropped but if dropping it now simplifies the release process, reduces the complexity of the project, and makes it easier to cut this release perhaps it should be done now rather than later?

p.s. thanks again for your hard work

edit: as flimm suggested I opened another issue #4261 for discussion on dropping python 2 to try and keep the conversation here on 2020's first release

[Flimm]

Let's keep conversation about dropping support for Python 2 in a separate GitHub issue, as I have a feeling it could get noisy.

[GPHemsley]

@r-richmond vistir is used in the Python 3 branch of logic, too, so I don't think removing support for Python 2 would make this release _easier_.

[techalchemy]

So I have gone ahead and cut another pre-release of pipenv (2020.4.1b2 -- the release will be tagged with the date it comes out so don't worry to much about the naming).

I think this release captures a majority, if not all of the changes I'm hoping to include. I did merge one significant change since the last pre-release which should avoid re-launching processes to attempt to pip install already-satisfied dependencies, so please report any issues as I plan to release for real on Wednesday of next week.

Thanks again to everyone who helped test, provided feedback, and helped get fixes merged!

[brainwane]

I just spoke with @techalchemy . He is checking on #4263 and #3592 to ensure they are not blockers. He also said

i only saw one issue about vendored import paths ...
revendoring finished up and didn't resolve the import path issue, so i'll likely just accept the PR on the issue & generate a patch for it for now

I presume this is #4267 but I'm not sure.

Once those are taken care of, I believe he aims to release today.

[techalchemy]

^ Release is up, thank you to everyone who helped with testing!

[haizaar]

Wow! Absolutely amazing! Thank you so much for bringing pipenv back to life!

On Thu, 28 May 2020, 15:17 Dan Ryan, [e-mail] wrote:

^ Release is up, thank you to everyone who helped with testing!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pipenv/issues/3369#issuecomment-635107609, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAOGWNSCAEOBRJ5S4WZHADRTXXWHANCNFSM4GJK7LZA
.

[sim0nx]

^ Release is up, thank you to everyone who helped with testing!

Very much appreciated! :+1: :partying_face:

[brainwane]

Per https://pypi.org/project/pipenv/#history , Pipenv 2020.5.28 is now out, so I'm closing this issue and thus the release milestone.

Thanks to @techalchemy - and thanks to Canonical for letting him work on this during some of his day job time.

In my opinion: If you use Python for your job, and want better and more frequent releases of this and other Python packaging/distribution/installation tools, ask your employer to chip in with a sponsorship. If it has a bigger budget, the Packaging Working Group can pay contractors to consistently work on these projects and maintain them well.

And if you'd like to help out by volunteering, read this announcement and follow issue #4130 on improving Pipenv's roadmap and contributor-maintainer processes, which is probably where more of that discussion will happen.

Thanks to everyone who contributed to this release -- users, testers, signal-boosters, reviewers, patch authors, and the people who said nice things in comments here or on mailing lists and social media!

[brainwane]

(I wrote a blog post about what it took to break the bottleneck and get this release out, and about what you can do to replicate this for other projects -- either yourself or through my firm, Changeset Consulting.)

[haizaar]

Sumana, thank you for sharing that story. And even more for putting your
time and energy into pipenv revival. As you mentioned, the unlocked value
is enormous.

On Fri, 11 Sep 2020 at 21:05, Sumana Harihareswara [e-mail]
wrote:

(I wrote a blog post https://www.harihareswara.net/sumana/2020/09/08/0
about what it took to break the bottleneck and get this release out, and
about what you can do to replicate this for other projects -- either
yourself or through my firm, Changeset Consulting https://changeset.nyc/
.)

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pipenv/issues/3369#issuecomment-691029882, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAAOGWNIKWJKNTMUH5PNUZ3SFIAAZANCNFSM4GJK7LZA
.

--
Zaar

[g1augusto]

Hi everyone,

does this address the WSL issues seen on #3488 ?

[uranusjr]

The issue you mentioned cannot be addressed in pipenv. You need to configure your environment as described in the issue yourself.