Pyenv: add python env to PyCharm 2016.3 failed

[Feiox]

Operating path: preferences > Project > Project Interpreter > add Local
then choice my py env (e.g. /usr/local/var/pyenv/versions/some/bin/python) > open
At this time, Pycharm automatically selects the symbolic link source directory for py bin
(python ->/usr/local/var/pyenv/versions/3.5.2/bin/python)

so, the environment is wrong.

now, I can use the implementation of cp command to solve, but this is certainly not the best way.
cp /usr/local/var/pyenv/versions/3.5.2/bin/python /usr/local/var/pyenv/versions/some/bin/python

Only I encountered this problem?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[fubarhouse]

I got around this problem last week with the use of vitualenvs, and now I fully recommend them!

See https://github.com/yyuu/pyenv-virtualenv for more information.

[hung-phan]

Same issue even with pyenv-virtualenv

[fubarhouse]

Try setting a global version in PyCharm via virtualenv, and remove the .idea folder from the project, and invalidate caches and restart.

Beyond this I can't help.

[hung-phan]

Tried, still doesn't work. I think the problem is:

https://github.com/yyuu/pyenv-virtualenv/issues/194

[DanyC97]

please be aware that if you are going to delete the .idea folder as suggested by @fubarhouse , following the restart you won't have any projects attached/ opened as before.

Also as mentioned by @hung-phan it won't solve your issue...unfortunately

[DanyC97]

done more tests and it seems the issue described above is happening only on python 3.x not on python 2.x .. see below (is indeed related to the symlink vs binary file as mentioned by OP)

i have 2 virtualenvs: dani based on 2.x and playground based on 3.x

# ll /usr/local/opt/pyenv/versions/3.6.0/envs/playground/bin/ |grep python
lrwxr-xr-x  1 user1  admin     6B 16 Jan 18:07 python3 -> python
lrwxr-xr-x  1 user1  admin    46B 16 Jan 18:07 python -> /usr/local/opt/pyenv/versions/3.6.0/bin/python
# ll /usr/local/opt/pyenv/versions/2.7.13/envs/dani/bin/ |grep python
-rwxr-xr-x  1 user1  admin   1.9M 16 Jan 22:55 python2.7
lrwxr-xr-x  1 user1  admin     9B 16 Jan 22:55 python2 -> python2.7
lrwxr-xr-x  1 user1  admin     9B 16 Jan 22:55 python -> python2.7
-rwxr-xr-x  1 user1  admin   2.3K 16 Jan 22:56 python-config

And i can't confirm that the 3.x python installed by pyenv does contain all the binaries as expected so suspect this is purely a pyenv-virtualenv issue

[fubarhouse]

Screenshots of the PyCharm Python selection preferences panel would also be useful to help you.

[yyuu]

On 3.3+, pyenv-virtualenv will use python -m venv if virtualenv is not available. It might be the cause of different behavior between 2.x and 3.x.

[adobeDan]

The issue doesn't have to do with whether or not virtualenv is available. It has to do with the fact that virtualenvs created by pyenv for python 3.x do not contain hard links to the python binary, they contain symlinks. PyCharm, sadly, goes through symlinks to the python interpreter that contains them. I've submitted a PyCharm issue about this. It would be nice if there were a way to get pyenv to use hardlinks rather than symlinks in virtualenvs.

[DanyC97]

@yyuu fyi i do have virtualenv installed so then i'd expect the same behavior between 2.x and 3.x

[piotr-dobrogost]

PyCharm, sadly, goes through symlinks to the python interpreter that contains them.

This is regression in PyCharm 2016.3 – https://youtrack.jetbrains.com/issue/PY-21787 – and has nothing to do with pyenv.

[DanyC97]

@piotr-dobrogost i think we have 2 issues here:

• one caused by PyCharm as you mentioned
• the other one caused by pyenv, mainly around the inconsistency between 2.x & 3.x where even if virtualenv is installed could be that still python -m venv is used. Will play more to see if i can figure out...

[DanyC97]

i found the issue and pushed a PR, tested and PyCharm is a happy bunny now.
In case someone is interested, see below how

• pyenv-virtualenv is checking for the presence of virtualenv file
• if not present then it check for venv

+ [pyenv-virtualenv:141] '[' -x /usr/local/opt/pyenv/versions/3.6.0/bin/virtualenv ']'^M
+ [pyenv-virtualenv:144] pyenv-exec python -m venv --help^M
+ [pyenv-virtualenv:145] HAS_M_VENV=1^M
+ [pyenv-virtualenv:149] '[' -n '' ']'^M
+ [pyenv-virtualenv:152] '[' -n 1 ']'^M
+ [pyenv-virtualenv:152] '[' -z '' ']'^M
+ [pyenv-virtualenv:152] '[' -z '' ']'^M

and then it run the below command

``` [pyenv-virtualenv:520] cd /usr/local/opt/pyenv/cache^M

• [pyenv-virtualenv:521] '[' -n '' ']'^M
• [pyenv-virtualenv:524] '[' -n 1 ']'^M
• [pyenv-virtualenv:525] pyenv-exec python -m venv /usr/local/opt/pyenv/versions/3.6.0/envs/pyenv-test2^M
• [pyenv-exec:20] '[' python = --complete ']'^M
  ++ [pyenv-exec:24] pyenv-version-name^M
• [pyenv-version-name:6] '[' -z 3.6.0 ']'^M
  ```

but the way python -m venv /usr/local/opt/pyenv/versions/3.6.0/envs/pyenv-test2 is run that means by default will symlink. In my PR i've added --copies to no longer symlink but copy the binaries

[blueyed]

I do not get what the issue is from your last comment.
Please elaborate.

[DanyC97]

@blueyed i've updated my previous comment with more info, hth

[arischow]

add '--copies'

pyenv virtualenv <specific python version> <environment name> --copies

[iodbh]

For the record: i had the same issue, using --copies did solve it

[fr0der1c]

Same issue. Seems it is caused by masOS native file chooser.

The good news is: PyCharm 2017.1 started using their own file chooser instead of macOS native chooser again, and this bug has been fixed.
For older version of PyCharm, as a workaround, disable Help | Find Action | "registry" | "ide.mac.file.chooser.native".

[mikenerone]

I am at a loss to explain this (and perhaps I should open a new issue), but I am experiencing this behavior with IDEA 2017.2 CE + the Python plugin on MacOS, except neither --copies nor the fact that IDEA is using its own file chooser seems to fix it. I've tried adding virtualenvs as Python SDKs (i.e. "interpreter" in PyCharm) under the following scenarios:

• Created with /opt/local/bin/pyvenv (this is a MacPorts Python 3.5.4): Works
• Created with pyenv virtualenv 2.7.14: Works
• Created with ~/.pyenv/versions/2.7.14/virtualenv: Works
• Created with pyenv virtualenv 3.5.2: Fails (site-packages missing from path)
• Created with ~/.pyenv/versions/3.5.2/pyvenv: Fails (site-packages missing from path)
• Created with pyenv virtualenv 3.5.4: Fails (site-packages missing from path)
• Created with ~/.pyenv/versions/3.5.4/pyvenv: Fails (site-packages missing from path)
• Created with pyenv virtualenv 3.6.1: Fails (site-packages missing from path)
• Created with ~/.pyenv/versions/3.6.1/pyvenv: Fails (site-packages missing from path)

Side note: adding from the original, non-virtualenv location works in all cases.

So the problem seems to occur only if the "source" Python used to create the virtualenv is a pyenv-installed Python 3.x, regardless of whether it's created with pyenv virtualenv or pyvenv. For the life of me, I haven't been able to figure out what's going on. E.g. Manually examining sys.path from the command line, it looks right in all cases. I'm stumped.

[fubarhouse]

@mikelikesbikes,

Have you seen this -> https://youtrack.jetbrains.com/issue/PY-21787

Also, would it be an option to flag this with JetBrains?

[mikenerone]

@fubarhouse I'm assuming you meant me...yes, I had seen that referenced further up. As I mentioned, I'm seeing this behavior even _with_ passing the --copies parameter when creating the virtualenv. On the other hand, I backed up to IDEA 2017.1, and I now see it there as well. I've also tried regressing pyenv back to commits as far back as last November, and it seems that nothing fixes it. So it doesn't seem like something recently introduced by _either_ tool, and I've probably got something going on at my end that I just haven't put my finger on yet.

[chrahunt]

I will close this since the underlying issue was fixed in PyCharm 2017.1. @mikenerone if you still experience the issue you mentioned with the latest pyenv and PyCharm, please do not hesitate to create an issue with details and we will investigate.