Esp-idf: Why is esp-idf installation pushing it self as default pip/python? (IDFGH-1640)

[nardev]

I have same issue with platformio and now i see same thing after manually installing esp-idf.
It suggests putting it's bin directory into a global path an than you realize that the python virtual environment is used as primary pip/python.

Am i missing something or they simply don't think that they are going to destroy ppls environments?

[igrr]

@nardev can you please mention which ESP-IDF version you are using (as suggested in the issue template) and which documentation page has this issue? Thank you.

[nardev]

@nardev can you please mention which ESP-IDF version you are using (as suggested in the issue template) and which documentation page has this issue? Thank you.

Thank you for the reply.

I used latest version and followed this guidelines here: https://docs.espressif.com/projects/esp-idf/en/latest/get-started/

[igrr]

Thanks! Can you point more exactly where it "suggests putting it's bin directory into a global path"? The current approach should be (on Linux/Mac):

• install prerequisites: CMake, ninja, Python, git (one time operation)
• run install.sh — this downloads IDF-specific tools and puts them into IDF_TOOLS_PATH (default is ~/.espressif). These tools are not added to PATH.
• in the shell where you intend to use ESP-IDF, run . /path/to/idf/export.sh. This will update PATH environment variable in this shell only, adding the required tools and the python environment. This doesn't affect any other shell in the system — only processes launched from that specific shell window will see the extra PATH variable entries.

Does this make sense? Any issues with this approach?

[nardev]

Thank you very much for your effort.

As soon as i try this i'll report.

Btw, i saw the suggestion that .....esp/esp-idf/export.sh should be sourced so that is what puts idf so high and esp-idf pip/python overrides my default system pip/python.

[igrr]

Btw, i saw the suggestion that .....esp/esp-idf/export.sh should be sourced so that is what puts idf so high and esp-idf pip/python overrides my default system pip/python.

That's right, but this is something we need to do to use an ESP-IDF-specific Python environment. The expectation here is that if you want to use some other python environment (such as the system one), you can do this from a different terminal.

[nardev]

Could be great to override it a bit with some aliases or it's another complication.

[igrr]

Do you mean adding an alias for export.sh, or something else?

[Alvin1Zhang]

@nardev Could you help share if any updates for this issue? Thanks.

[Alvin1Zhang]

@nardev Could you help share if any updates for this issue? Thanks.

[neilpanchal]

I am having the same issues as @nardev.

On macOS, the system python (still on 2.7) points to /usr/bin/python by running which python. It is ill-advised (!!!) to change this as described here.

The next best thing is to install python using brew install python which symlinks the installation in /usr/local/Celler/ to /usr/local/bin/python3. Then, just run brew doctor which will instruct the user if /usr/local/bin is not in front of /usr/bin.

Great! So, we have python symlinked to /usr/local/Cellar/ which is the python version from brew.

When running install.sh, we are still polluting everyone's macOS with esp-idf dependencies - this python is python2.7 located at /usr/bin and it still doesn't use /usr/local/bin/python.

So, it would be great if ESP-IDF would provide the user to set a specific binary to python as an env variable. For e.g., setting ESP_IDF_PYTHON_BINARY=/usr/local/bin/python would tell the install.sh script to use brewed python, not macOS system binary (and thus polluting global installation).

[neilpanchal]

@igrr Thoughts on using Pipenv as a package manager for ESP-IDF?

[igrr]

When running install.sh, we are still polluting everyone's macOS with esp-idf dependencies

Can you please elaborate why? Install.sh creates a private virtual environment under $IDF_TOOLS_PATH/python_env/... and pip installs the packages there. If somehow the packages got installed into the global Python environment, this is a bug — please provide more details so we can troubleshoot.

So, it would be great if ESP-IDF would provide the user to set a specific binary to python

install.sh uses the Python binary which is available on the PATH, i.e. the one you get if you run which python. So if your default Python is the homebrew one, it is the one which will be used as the base interpreter to create a virtualenv.

[Alvin1Zhang]

@neilpanchal Would you please help share more details or updates? Thanks.

[chengyuhui]

You can also automate this step, making ESP-IDF tools available in every terminal, by adding this line to your .profile or .bash_profile script.

It is probably better to note in the docs that this may pollute the user's python environment globally.

[igrr]

@chengyuhui absolutely, but where did you find this sentence? I don't see it in this discussion.

[chengyuhui]

@igrr It's here: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/get-started/#id5

[igrr]

Aaah, so that's where this is coming from. I wrote this line in the docs, and now I keep wondering why people put export.sh into their profile scripts!

Thanks a lot for the pointer, will fix ASAP.