Poetry: 1.0.10 -> 1.1 regression: EnvCommandError: 'No module named setuptools' on Python 3.7

[gitpushdashf]

This may because it's installing a nested package. Python 3.8 doesn't have this issue but 3.7 does. I don't have a reduced test case yet. I'm hoping the traceback is helpful and someone knows what's going on from it. Python 3.7 and 3.8 are fine with Poetry 1.0.10, everything else being identical. Python 3.8 with Poetry 1.1.2 is fine as well, just not Python 3.7 with Poetry 1.1.2.

 EnvCommandError
  Command ['/usr/local/bin/python', '-m', 'pip', 'install', '--no-deps', '-U', '/builds/[redacted]'] errored with the following return code 2, and output: 
  Looking in indexes: https://[redacted]
  Processing ./packages/[redacted]
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
  ERROR: Exception:
  Traceback (most recent call last):
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
      status = self.run(options, args)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
      return func(self, options, args)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
      reqs, check_supported_wheels=not options.target_dir
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
      discovered_reqs.extend(self._resolve_one(requirement_set, req))
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
      abstract_dist = self._get_abstract_dist_for(req_to_install)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
      abstract_dist = self.preparer.prepare_linked_requirement(req)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 483, in prepare_linked_requirement
      req, self.req_tracker, self.finder, self.build_isolation,
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
      abstract_dist.prepare_distribution_metadata(finder, build_isolation)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
      self._setup_isolation(finder)
    File "/usr/local/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
      reqs = backend.get_requires_for_build_wheel()
    File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
      'config_settings': config_settings
    File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
      raise BackendUnavailable(data.get('traceback', ''))
  pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
    File "/usr/local/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend
      obj = import_module(mod_path)
    File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/masonry/api.py", line 1, in <module>
      from poetry.core.masonry.api import build_sdist
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/core/masonry/__init__.py", line 10, in <module>
      from .builder import Builder
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/core/masonry/builder.py", line 6, in <module>
      from .builders.sdist import SdistBuilder
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in <module>
      from .wheel import WheelBuilder
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/wheel.py", line 18, in <module>
      from packaging.tags import sys_tags
    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
      import distutils.util
    File "/usr/local/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 83, in create_module
      return importlib.import_module('._distutils', 'setuptools')
    File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
  ModuleNotFoundError: No module named 'setuptools'

[abn]

@gitpushdashf looks like this is an issue with the version of pip installed with your 3.7 installation. Looks like you are installing without a virtual environment. If this is the case, first try after perorming /usr/local/bin/python -m pip install -U pip.

Command ['/usr/local/bin/python', '-m', 'pip', 'install', '--no-deps', '-U', '/builds/[redacted]'] errored with the following return code 2, and output: 

Another possibility is, if you are using an Ubuntu distro, the distutils version installed might be interfering with your install.

[gitpushdashf]

Thanks for replying!

This is using the python:3.7 and python3.8 Docker images which I believe are Debian based. With that image should I be doing python3 -m pip install or is pip3 install okay?

[abn]

@gitpushdashf fwiw; I cannot reproduce this in a container, any chance you can provide some steps to reproduce this? Typically the containers should have python pointing to the tag version and pip pointing to the corresponding pip version. The use of python3.7 -m pip is more done for safety and portability.

$ podman run --rm -i --entrypoint bash python:3.7 -c 'pip --version'
pip 20.2.3 from /usr/local/lib/python3.7/site-packages/pip (python 3.7)

[abn]

Also this indicates you are using the old build-system configuration in your dependency project.

    File "/tmp/pip-build-env-l4t_85ci/overlay/lib/python3.7/site-packages/poetry/masonry/api.py", line 1, in <module>
      from poetry.core.masonry.api import build_sdist

I would recommend you try moving to the standalone build backend.

[jonapich]

I am also getting this error and I can reproduce it.

https://github.com/jonapich/test-poetry-setuptools

Clone that repo, and try "poetry install" in the test-functools folder, you will get the same stacktrace as in the bug report.

This seems pretty serious 🤔

I'm on 1.1.3, Windows, using powershell. I installed using the get-poetry bootstrap.

I have confirmed that this bug appears when going from 1.0.10 -> 1.1 and it was also working in 1.0.9

Also, I never used python 3.7; only 3.6 and 3.8.

[guhcampos]

I can confirm this is a problem and affects anything that ultimately still depends on calling python setup.py, such as:

• installing a poetry powered package with pip install -e
• using a local package in pyproject.toml like @jonapich demonstrates with test-testing = { path = "../test-testing/" }

I can confirm the problem with Python 3.6, 3.7 and 3.8 alike.

And this is really hard to debug because even if you literally copy and paste the command it's trying to run, it just works, in my case for a sample package:

[EnvCommandError]
Command ['/home/gustavo/.cache/pypoetry/virtualenvs/package-94uOVQQp-py3.8/bin/pip', 'install', '--no-deps', '-U', '-e', '/home/gustavo/my-package/package/../packb'] errored with the following return code 1, and output: 
Looking in indexes: https://pypi.org/simple, Obtaining file:///home/gustavo/my-package/packb
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'done'
Installing collected packages: packb
  Running setup.py develop for packb
    ERROR: Command errored out with exit status 1:
     command: /home/gustavo/.cache/pypoetry/virtualenvs/package-94uOVQQp-py3.8/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gustavo/my-package/packb/setup.py'"'"'; __file__='"'"'/home/gustavo/my-package/packb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /home/gustavo/my-package/packb/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/gustavo/.cache/pypoetry/virtualenvs/package-94uOVQQp-py3.8/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gustavo/my-package/packb/setup.py'"'"'; __file__='"'"'/home/gustavo/my-package/packb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

But if I simply copy and paste the command below, it runs ok:

○ → /home/gustavo/.cache/pypoetry/virtualenvs/package-94uOVQQp-py3.8/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/gustavo/my-package/packb/setup.py'"'"'; __file__='"'"'/home/gustavo/my-package/packb/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
running develop
running egg_info
creating packb.egg-info
writing packb.egg-info/PKG-INFO
writing dependency_links to packb.egg-info/dependency_links.txt
writing top-level names to packb.egg-info/top_level.txt
writing manifest file 'packb.egg-info/SOURCES.txt'
reading manifest file 'packb.egg-info/SOURCES.txt'
writing manifest file 'packb.egg-info/SOURCES.txt'
running build_ext
Creating /home/gustavo/.cache/pypoetry/virtualenvs/package-94uOVQQp-py3.8/lib/python3.8/site-packages/packb.egg-link (link to .)
Adding packb 0.0.0 to easy-install.pth file

Installed /home/gustavo/my-package/package

To make this "work" I had to install bogus setup.py files to my packages, such as:

from setuptools import setup
setup(name="packb")

Yet it still fails when running from within poetry.

[abn]

@jonapich based on your description I have attempted to recreate the scenario (minus the windows environment). Let me know what needs to be changed to reproduce the issue.

podman run --rm -i --entrypoint bash python:3.8 <<EOF
set -ex
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - --version 1.0.10
source ~/.poetry/env
poetry --version
git clone https://github.com/jonapich/test-poetry-setuptools.git
pushd test-poetry-setuptools/test-functools/
poetry lock
poetry install
poetry self update
poetry install
rm -rf \$(poetry env info -p)
poetry --version
poetry install
rm -rf \$(poetry env info -p)
git checkout poetry.lock
poetry install
EOF

Console Output

+ curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py
+ python - --version 1.0.10
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

$HOME/.poetry/bin

This path will then be added to your `PATH` environment variable by
modifying the profile file located at:

$HOME/.profile

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing version: 1.0.10
  - Downloading poetry-1.0.10-linux.tar.gz (29.66MB)

Poetry (1.0.10) is installed now. Great!

To get started you need Poetry's bin directory ($HOME/.poetry/bin) in your `PATH`
environment variable. Next time you log in this will be done
automatically.

To configure your current shell run `source $HOME/.poetry/env`

+ source /root/.poetry/env
++ export PATH=/root/.poetry/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ PATH=/root/.poetry/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ poetry --version
Poetry version 1.0.10
+ git clone https://github.com/jonapich/test-poetry-setuptools.git
Cloning into 'test-poetry-setuptools'...
+ pushd test-poetry-setuptools/test-functools/
/test-poetry-setuptools/test-functools /
+ poetry lock
Creating virtualenv test-functools-HE92XDqH-py3.8 in /root/.cache/pypoetry/virtualenvs
The lock file might not be compatible with the current version of Poetry.
Upgrade Poetry to ensure the lock file is read properly or, alternatively, regenerate the lock file with the `poetry lock` command.
Updating dependencies
Resolving dependencies...

Writing lock file
+ poetry install
Installing dependencies from lock file


Package operations: 15 installs, 0 updates, 0 removals

  - Installing pyparsing (2.4.7)
  - Installing six (1.15.0)
  - Installing attrs (20.2.0)
  - Installing iniconfig (1.1.1)
  - Installing packaging (20.4)
  - Installing pluggy (0.13.1)
  - Installing py (1.9.0)
  - Installing toml (0.10.1)
  - Installing mypy-extensions (0.4.3)
  - Installing pytest (6.1.1)
  - Installing typed-ast (1.4.1)
  - Installing typing-extensions (3.7.4.3)
  - Installing inflection (0.5.1)
  - Installing mypy (0.790)
  - Installing test-testing (0.1.4 ../test-testing)
  - Installing test-functools (0.1.4)
+ poetry self update
Updating to 1.1.3
 - Downloading poetry-1.1.3-linux.tar.gz 0%
 - Downloading poetry-1.1.3-linux.tar.gz 10%
 - Downloading poetry-1.1.3-linux.tar.gz 20%
 - Downloading poetry-1.1.3-linux.tar.gz 30%
 - Downloading poetry-1.1.3-linux.tar.gz 40%
 - Downloading poetry-1.1.3-linux.tar.gz 50%
 - Downloading poetry-1.1.3-linux.tar.gz 60%
 - Downloading poetry-1.1.3-linux.tar.gz 70%
 - Downloading poetry-1.1.3-linux.tar.gz 80%
 - Downloading poetry-1.1.3-linux.tar.gz 90%
 - Downloading poetry-1.1.3-linux.tar.gz 100%

Poetry (1.1.3) is installed now. Great!
+ poetry install
Installing dependencies from lock file

No dependencies to install or update

Installing the current project: test-functools (0.1.4)
++ poetry env info -p
+ rm -rf /root/.cache/pypoetry/virtualenvs/test-functools-HE92XDqH-py3.8
+ poetry --version
Poetry version 1.1.3
+ poetry install
Creating virtualenv test-functools-HE92XDqH-py3.8 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 15 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing attrs (20.2.0)
  • Installing iniconfig (1.1.1)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing toml (0.10.1)
  • Installing mypy-extensions (0.4.3)
  • Installing pytest (6.1.1)
  • Installing typed-ast (1.4.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing inflection (0.5.1)
  • Installing mypy (0.790)
  • Installing test-testing (0.1.4 /test-poetry-setuptools/test-testing)

Installing the current project: test-functools (0.1.4)
++ poetry env info -p
+ rm -rf /root/.cache/pypoetry/virtualenvs/test-functools-HE92XDqH-py3.8
+ git checkout poetry.lock
+ poetry install
Creating virtualenv test-functools-HE92XDqH-py3.8 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 15 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing attrs (20.2.0)
  • Installing iniconfig (1.1.1)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing toml (0.10.1)
  • Installing mypy-extensions (0.4.3)
  • Installing pytest (6.1.1)
  • Installing typed-ast (1.4.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing inflection (0.5.1)
  • Installing mypy (0.790)
  • Installing test-testing (0.1.4 /test-poetry-setuptools/test-testing)

Installing the current project: test-functools (0.1.4)

[jonapich]

I was able to make it work on a clean windows 2019 server as well... 🤔 But you need py3.8 or below, because there's no wheels for typed_ast in 3.9 and you'll need to install the vs c++ build tools.

I'll have to investigate what's different on my system.

[jonapich]

So far, I can still reproduce the issue when going through poetry, but not when going through pip. My issue matches the OP's behavior:

Console Output

❭❭  test-functools  10:04  (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | python - --uninstall
# We are sorry to see you go!

This will uninstall Poetry.

It will remove the `poetry` command from Poetry's bin directory, located at:

%USERPROFILE%\.poetry\bin

This will also remove Poetry from your system's PATH.

❭❭  test-functools  10:05  poetry
poetry: The term 'poetry' is not recognized as a name of a cmdlet, function, script file, or executable program.
Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

Suggestion [4,General]: The most similar commands are: where, popd, copy, gerr, clear, sort, history, set, etsn, Setup.


❭❭  test-functools ✗  10:05  (Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py -UseBasicParsing).Content | py -3.8
Retrieving Poetry metadata

# Welcome to Poetry!

This will download and install the latest version of Poetry,
a dependency and package manager for Python.

It will add the `poetry` command to Poetry's bin directory, located at:

%USERPROFILE%\.poetry\bin

This path will then be added to your `PATH` environment variable by
modifying the `HKEY_CURRENT_USER/Environment/PATH` registry key.

You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.

Installing version: 1.1.3
  - Downloading poetry-1.1.3-win32.tar.gz (38.81MB)

Poetry (1.1.3) is installed now. Great!

To get started you need Poetry's bin directory (%USERPROFILE%\.poetry\bin) in your `PATH`
environment variable. Future applications will automatically have the
correct environment, but you may need to restart your current shell.


❭❭  test-functools  10:06  poetry --version
Poetry version 1.1.3


❭❭  test-functools ✗  10:06  py -3.8 -c "import sys; print(sys.executable)"
C:\Users\jonapich\AppData\Local\Programs\Python\Python38\python.exe


❭❭  test-functools ✗  10:07  poetry env use C:\Users\jonapich\AppData\Local\Programs\Python\Python38\python.exe
Creating virtualenv test-functools-44VBZfUY-py3.8 in C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs
Using virtualenv: C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8


❭❭  test-functools  10:07  poetry install
Installing dependencies from lock file

Package operations: 17 installs, 0 updates, 0 removals

  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing atomicwrites (1.4.0)
  • Installing attrs (20.2.0)
  • Installing colorama (0.4.4)
  • Installing iniconfig (1.1.1)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing toml (0.10.1)
  • Installing mypy-extensions (0.4.3)
  • Installing pytest (6.1.1)
  • Installing typed-ast (1.4.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing inflection (0.5.1)
  • Installing test-testing (0.1.4 C:/Users/jonapich/Documents/workspace/test-poetry/test-testing)
  • Installing mypy (0.790)

  EnvCommandError

  Command C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\Scripts\pip.exe install --no-deps -U C:/Users/jonapich/Documents/workspace/test-poetry/test-testing errored with the following return code 2, and output:
  Processing c:\users\jonapich\documents\workspace\test-poetry\test-testing
    Installing build dependencies: started
    Installing build dependencies: finished with status 'done'
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
  ERROR: Exception:
  Traceback (most recent call last):
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\cli\base_command.py", line 216, in _main
      status = self.run(options, args)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\cli\req_command.py", line 182, in wrapper
      return func(self, options, args)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\commands\install.py", line 324, in run
      requirement_set = resolver.resolve(
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 183, in resolve
      discovered_reqs.extend(self._resolve_one(requirement_set, req))
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 388, in _resolve_one
      abstract_dist = self._get_abstract_dist_for(req_to_install)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 340, in _get_abstract_dist_for
      abstract_dist = self.preparer.prepare_linked_requirement(req)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\operations\prepare.py", line 482, in prepare_linked_requirement
      abstract_dist = _get_prepared_distribution(
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\operations\prepare.py", line 91, in _get_prepared_distribution
      abstract_dist.prepare_distribution_metadata(finder, build_isolation)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\distributions\sdist.py", line 38, in prepare_distribution_metadata
      self._setup_isolation(finder)
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\distributions\sdist.py", line 96, in _setup_isolation
      reqs = backend.get_requires_for_build_wheel()
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 160, in get_requires_for_build_wheel
      return self._call_hook('get_requires_for_build_wheel', {
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 265, in _call_hook
      raise BackendUnavailable(data.get('traceback', ''))
  pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 86, in _build_backend
      obj = import_module(mod_path)
    File "C:\Users\jonapich\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
    File "<frozen importlib._bootstrap>", line 991, in _find_and_load
    File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
    File "<frozen importlib._bootstrap>", line 991, in _find_and_load
    File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 783, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-2_yuojjf\overlay\Lib\site-packages\poetry\core\masonry\__init__.py", line 10, in <module>
      from .builder import Builder
    File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-2_yuojjf\overlay\Lib\site-packages\poetry\core\masonry\builder.py", line 6, in <module>
      from .builders.sdist import SdistBuilder
    File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-2_yuojjf\overlay\Lib\site-packages\poetry\core\masonry\builders\__init__.py", line 2, in <module>
      from .wheel import WheelBuilder
    File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-2_yuojjf\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 18, in <module>
      from packaging.tags import sys_tags
    File "C:/Users/jonapich/AppData/Local/Temp/pip-build-env-2_yuojjf/overlay/Lib/site-packages/poetry/core/_vendor\packaging\tags.py", line 7, in <module>
      import distutils.util
    File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\_distutils_hack\__init__.py", line 83, in create_module
      return importlib.import_module('setuptools._distutils')
    File "C:\Users\jonapich\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
  ModuleNotFoundError: No module named 'setuptools'

  WARNING: You are using pip version 20.2.2; however, version 20.2.4 is available.
  You should consider upgrading via the 'C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\Scripts\python.exe -m pip install --upgrade pip' command.


  at ~\.poetry\lib\poetry\utils\env.py:948 in _run
       944│                 output = subprocess.check_output(
       945│                     cmd, stderr=subprocess.STDOUT, **kwargs
       946│                 )
       947│         except CalledProcessError as e:
    →  948│             raise EnvCommandError(e, input=input_)
       949│
       950│         return decode(output)
       951│
       952│     def execute(self, bin, *args, **kwargs):


❭❭  test-functools  10:07  C:\Users\jonapich\AppData\Local\Programs\Python\Python38\python.exe -m venv py38
❭❭  test-functools ✗  10:08  cd .\py38\Scripts\


❭❭  Scripts ✗  10:08  ./pip.exe install --no-deps -U C:/Users/jonapich/Documents/workspace/test-poetry/test-testing
Processing c:\users\jonapich\documents\workspace\test-poetry\test-testing
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: test-testing
  Building wheel for test-testing (PEP 517) ... done
  Created wheel for test-testing: filename=test_testing-0.1.4-py3-none-any.whl size=1057 sha256=0dc5e08a26902022cb80200b444cb79ce44410e91677bccdf7170e2174120e60
  Stored in directory: c:\users\jonapich\appdata\local\pip\cache\wheels\9f\5a\af\cb6a6cf29a333225d43d6c533921e8a09291dc81b67555a9cd
Successfully built test-testing
Installing collected packages: test-testing
Successfully installed test-testing-0.1.4
WARNING: You are using pip version 20.2.1; however, version 20.2.4 is available.
You should consider upgrading via the 'c:\users\jonapich\documents\workspace\test-poetry\test-functools\py38\scripts\python.exe -m pip install --upgrade pip' command.


❭❭  Scripts  10:08  cd C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\Scripts
❭❭  Scripts  10:08  ./pip.exe install --no-deps -U C:/Users/jonapich/Documents/workspace/test-poetry/test-testing
Processing c:\users\jonapich\documents\workspace\test-poetry\test-testing
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\cli\base_command.py", line 216, in _main
    status = self.run(options, args)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\cli\req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\commands\install.py", line 324, in run
    requirement_set = resolver.resolve(
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\resolution\legacy\resolver.py", line 340, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\operations\prepare.py", line 482, in prepare_linked_requirement
    abstract_dist = _get_prepared_distribution(
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\operations\prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\distributions\sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_internal\distributions\sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 160, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\pip\_vendor\pep517\_in_process.py", line 86, in _build_backend
    obj = import_module(mod_path)
  File "C:\Users\jonapich\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-1hbdbs59\overlay\Lib\site-packages\poetry\core\masonry\__init__.py", line 10, in <module>
    from .builder import Builder
  File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-1hbdbs59\overlay\Lib\site-packages\poetry\core\masonry\builder.py", line 6, in <module>
    from .builders.sdist import SdistBuilder
  File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-1hbdbs59\overlay\Lib\site-packages\poetry\core\masonry\builders\__init__.py", line 2, in <module>
    from .wheel import WheelBuilder
  File "C:\Users\jonapich\AppData\Local\Temp\pip-build-env-1hbdbs59\overlay\Lib\site-packages\poetry\core\masonry\builders\wheel.py", line 18, in <module>
    from packaging.tags import sys_tags
  File "C:/Users/jonapich/AppData/Local/Temp/pip-build-env-1hbdbs59/overlay/Lib/site-packages/poetry/core/_vendor\packaging\tags.py", line 7, in <module>
    import distutils.util
  File "C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\lib\site-packages\_distutils_hack\__init__.py", line 83, in create_module
    return importlib.import_module('setuptools._distutils')
  File "C:\Users\jonapich\AppData\Local\Programs\Python\Python38\lib\importlib\__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'

WARNING: You are using pip version 20.2.2; however, version 20.2.4 is available.
You should consider upgrading via the 'C:\Users\jonapich\AppData\Local\pypoetry\Cache\virtualenvs\test-functools-44VBZfUY-py3.8\Scripts\python.exe -m pip install --upgrade pip' command.

[jonapich]

I went through the trouble of removing everything python and poetry off my system. I then reinstalled py3.8 using the latest installer and then reinstalled poetry using the Invoke-Webrequest method from the docs.

The behavior is still the same. Some observations:

• Through poetry run python -m pip list I can see setuptools is installed in the project's virtual environment.
• If I launch poetry run python I can import distutils.util

This is where poetry breaks it for me even though python/pip can handle it:

• If I launch poetry shell and then proceed to pip install ../test-testing/ it will fail.
• If I create my own environment python -m venv virtualenv and then call virtualenv/Scripts/pip install ../test-testing/ it works.

Additional note: The dependencies in test-testing have no effect on the issue. I tried to remove them all but I'm still unable to install it if poetry is involved with it.

[jonapich]

I wasn't able to find the cause of the problem. After a fresh install of Windows 10, I am no longer experiencing the issue and I can install the test repo that I created for this issue.

[gitpushdashf]

Did something change between 1.1.3 and 1.1.4 that may have fixed this?

Python 3.7 Docker container:

root@c55cbfe70290:~/test-poetry-setuptools/test-functools# poetry install
Creating virtualenv test-functools-vuIxD4Qa-py3.7 in /root/.cache/pypoetry/virtualenvs
Installing dependencies from lock file

Package operations: 17 installs, 0 updates, 0 removals

  • Installing zipp (3.3.1)
  • Installing importlib-metadata (2.0.0)
  • Installing pyparsing (2.4.7)
  • Installing six (1.15.0)
  • Installing attrs (20.2.0)
  • Installing iniconfig (1.1.1)
  • Installing packaging (20.4)
  • Installing pluggy (0.13.1)
  • Installing py (1.9.0)
  • Installing toml (0.10.1)
  • Installing mypy-extensions (0.4.3)
  • Installing pytest (6.1.1)
  • Installing typed-ast (1.4.1)
  • Installing typing-extensions (3.7.4.3)
  • Installing inflection (0.5.1)
  • Installing mypy (0.790)
  • Installing test-testing (0.1.4 /root/test-poetry-setuptools/test-testing)

Installing the current project: test-functools (0.1.4)

[jonapich]

The repo reproduced the issue on my system only. Then I had to reformat for a non related reason, and I could not reproduce the issue anymore.

Sorry I cannot be of any more help. This issue is resolved as far as I'm concerned. Is your original issue still happening? Did you try to repo in a container or VM?

[gitpushdashf]

I can't reproduce this now. I'll close it out unless someone else runs into it on 1.1.4.

[kbakk]

@gitpushdashf Would you mind reopening this issue - I'm seeing this on 1.1.4 (and 1.1.3).

Here's a minimal project where I'm able to reproduce the ModuleNotFoundError: No module named 'setuptools'.

reproduce-3153.zip

$ poetry install
Creating virtualenv reproduce-issue-QXZ_ZScA-py3.7 in /Users/kristofferb/Library/Caches/pypoetry/virtualenvs
Installing dependencies from lock file

Installing the current project: reproduce-issue (0.1.0)

$ poetry env info                                                                             

Virtualenv
Python:         3.7.6
Implementation: CPython
Path:           /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7
Valid:          True

System
Platform: darwin
OS:       posix
Python:   /Users/kristofferb/.pyenv/versions/3.7.6

$ poetry run pip install -e .
Obtaining file:///Users/kristofferb/Code/temp/gist-reproduce-3153
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Installing collected packages: reproduce-issue
  Attempting uninstall: reproduce-issue
    Found existing installation: reproduce-issue 0.1.0
    Uninstalling reproduce-issue-0.1.0:
      Successfully uninstalled reproduce-issue-0.1.0
  Running setup.py develop for reproduce-issue
    ERROR: Command errored out with exit status 1:
     command: /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/kristofferb/Code/temp/gist-reproduce-3153/setup.py'"'"'; __file__='"'"'/Users/kristofferb/Code/temp/gist-reproduce-3153/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
         cwd: /Users/kristofferb/Code/temp/gist-reproduce-3153/
    Complete output (3 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    ModuleNotFoundError: No module named 'setuptools'
    ----------------------------------------
  Rolling back uninstall of reproduce-issue
  Moving to /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7/lib/python3.7/site-packages/reproduce_issue-0.1.0.dist-info/
   from /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7/lib/python3.7/site-packages/~eproduce_issue-0.1.0.dist-info
  Moving to /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7/lib/python3.7/site-packages/reproduce_issue.pth
   from /private/var/folders/3j/l__gnf4n6t73fpcg8nk3tqhc0000gn/T/pip-uninstall-cz50uscc/reproduce_issue.pth
ERROR: Command errored out with exit status 1: /Users/kristofferb/Library/Caches/pypoetry/virtualenvs/reproduce-issue-QXZ_ZScA-py3.7/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/Users/kristofferb/Code/temp/gist-reproduce-3153/setup.py'"'"'; __file__='"'"'/Users/kristofferb/Code/temp/gist-reproduce-3153/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.

$ poetry --version           
Poetry version 1.1.4

I'm on MacOS 10.15.7 fwiw.

[finswimmer]

I don't understand completely what's going on here. But the root cause seems to be, that setuptools started to vendor distutils in a way, that whenever a module imports distutils it gets redirected to the vendored version of setuptools. And that seems not work in any case. See https://github.com/pypa/distutils/issues/17

So the problem is not, that setuptools is not installed. The problem is, that the version of setuptools that comes with the isolated build environment doesn't work as expected. A workaround is to run export SETUPTOOLS_USE_DISTUTILS=stdlib before a pip install.

So at it stands now, it looks to me a setuptools problem and not a poetry problem.

[kbakk]

In https://github.com/python-poetry/poetry/issues/1135#issuecomment-719902456 @NeilGirdhar pointed out that one would need to change pyproject.toml to include setuptools:

[build-system]
-requires = ["poetry_core>=1.0"]
+requires = ["setuptools", "poetry_core>=1.0"]
build-backend = "poetry.core.masonry.api"

With this added in my above example (reproduce-3153.zip) it now builds:

poetry run pip install -e .
Obtaining file:///Users/kristofferb/Code/temp/gist-reproduce-3153
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Installing collected packages: reproduce-issue
  Attempting uninstall: reproduce-issue
    Found existing installation: reproduce-issue 0.1.0
    Uninstalling reproduce-issue-0.1.0:
      Successfully uninstalled reproduce-issue-0.1.0
  Running setup.py develop for reproduce-issue
Successfully installed reproduce-issue

And indeed, as others have mentioned in https://github.com/python-poetry/poetry/issues/34, this is not a poetry issue. But it seems like many would like poetry to have a workaround.

[jonapich]

When I had black as a dependency I also had to add setuptools-scm and wheel manually because they had that in their build requires.Sent from my Galaxy
-------- Original message --------From: Kristoffer Bakkejord [e-mail] Date: 2020-11-14 06:57 (GMT-05:00) To: python-poetry/poetry [e-mail] Cc: Jonathan Piché [e-mail], Mention [e-mail] Subject: Re: [python-poetry/poetry] 1.0.10 -> 1.1 regression: EnvCommandError: 'No module named setuptools' on Python 3.7 (#3153)
In #1135 (comment) @NeilGirdhar pointed out that one would need to change pyproject.toml to include setuptools:
[build-system]
-requires = ["poetry_core>=1.0"]
+requires = ["setuptools", "poetry_core>=1.0"]
build-backend = "poetry.core.masonry.api"
With this added in my above example (reproduce-3153.zip) it now builds:
poetry run pip install -e .
Obtaining file:///Users/kristofferb/Code/temp/gist-reproduce-3153
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Installing collected packages: reproduce-issue
Attempting uninstall: reproduce-issue
Found existing installation: reproduce-issue 0.1.0
Uninstalling reproduce-issue-0.1.0:
Successfully uninstalled reproduce-issue-0.1.0
Running setup.py develop for reproduce-issue
Successfully installed reproduce-issue
And indeed, as others have mentioned in #34, this is not a poetry issue. But it seems like many would like poetry to have a workaround.

—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.

[hyliu1989]

And indeed, as others have mentioned in #34, this is not a poetry issue. But it seems like many would like poetry to have a workaround.

Yes, a poetry workaround would be perfect. I can also reproduce this error in the setting of https://github.com/python-poetry/poetry/issues/2015#issuecomment-727519651. I got that it is caused by build isolation but I am afraid that keeping things to the minimal will end up having a lot of boilerplate text in pyproject.toml.