Poetry: Some tests fail because of wrong rights on /foo
- [x] I am on the latest Poetry version.
- [x] I have searched the issues of this repo and believe that this is not a duplicate.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption).
- OS version and name: openSUSE/Tumbleweed (Linux)
- Poetry version: 1.0.0b8
Issue
Plenty of tests fail because of the traceback similar to this one (from ):
[ 88s] ____________________________ test_add_no_constraint ____________________________
[ 88s]
[ 88s] app = <tests.console.conftest.Application object at 0x7fd532dff4a8>
[ 88s] repo = <tests.console.conftest.Repository object at 0x7fd532b587b8>
[ 88s] installer = <poetry.installation.noop_installer.NoopInstaller object at 0x7fd532bc1470>
[ 88s]
[ 88s] def test_add_no_constraint(app, repo, installer):
[ 88s] command = app.find("add")
[ 88s] tester = CommandTester(command)
[ 88s]
[ 88s] repo.add_package(get_package("cachy", "0.1.0"))
[ 88s] repo.add_package(get_package("cachy", "0.2.0"))
[ 88s]
[ 88s] > tester.execute("cachy")
[ 88s]
[ 88s] tests/console/commands/test_add.py:19:
[ 88s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 88s] /usr/lib/python3.7/site-packages/cleo/testers/command_tester.py:61: in execute
[ 88s] self._status_code = command.run(args, self._io)
[ 88s] /usr/lib/python3.7/site-packages/clikit/api/command/command.py:116: in run
[ 88s] return self.handle(self.parse(args), io)
[ 88s] /usr/lib/python3.7/site-packages/clikit/api/command/command.py:120: in handle
[ 88s] status_code = self._do_handle(args, io)
[ 88s] /usr/lib/python3.7/site-packages/clikit/api/command/command.py:163: in _do_handle
[ 88s] self._dispatcher.dispatch(PRE_HANDLE, event)
[ 88s] /usr/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:22: in dispatch
[ 88s] self._do_dispatch(listeners, event_name, event)
[ 88s] /usr/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:89: in _do_dispatch
[ 88s] listener(event, event_name, self)
[ 88s] poetry/console/config/application_config.py:86: in set_env
[ 88s] env = env_manager.create_venv(io)
[ 88s] poetry/utils/env.py:587: in create_venv
[ 88s] self.build_venv(str(venv), executable=executable)
[ 88s] poetry/utils/env.py:647: in build_venv
[ 88s] build(path)
[ 88s] /usr/lib64/python3.7/venv/__init__.py:60: in create
[ 88s] context = self.ensure_directories(env_dir)
[ 88s] /usr/lib64/python3.7/venv/__init__.py:107: in ensure_directories
[ 88s] create_if_needed(env_dir)
[ 88s] /usr/lib64/python3.7/venv/__init__.py:96: in create_if_needed
[ 88s] os.makedirs(d)
[ 88s] /usr/lib64/python3.7/os.py:211: in makedirs
[ 88s] makedirs(head, exist_ok=exist_ok)
[ 88s] /usr/lib64/python3.7/os.py:211: in makedirs
[ 88s] makedirs(head, exist_ok=exist_ok)
[ 88s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 88s]
[ 88s] name = '/foo', mode = 511, exist_ok = False
[ 88s]
[ 88s] def makedirs(name, mode=0o777, exist_ok=False):
[ 88s] """makedirs(name [, mode=0o777][, exist_ok=False])
[ 88s]
[ 88s] Super-mkdir; create a leaf directory and all intermediate ones. Works like
[ 88s] mkdir, except that any intermediate path segment (not just the rightmost)
[ 88s] will be created if it does not exist. If the target directory already
[ 88s] exists, raise an OSError if exist_ok is False. Otherwise no exception is
[ 88s] raised. This is recursive.
[ 88s]
[ 88s] """
[ 88s] head, tail = path.split(name)
[ 88s] if not tail:
[ 88s] head, tail = path.split(head)
[ 88s] if head and tail and not path.exists(head):
[ 88s] try:
[ 88s] makedirs(head, exist_ok=exist_ok)
[ 88s] except FileExistsError:
[ 88s] # Defeats race condition when another thread created the path
[ 88s] pass
[ 88s] cdir = curdir
[ 88s] if isinstance(tail, bytes):
[ 88s] cdir = bytes(curdir, 'ASCII')
[ 88s] if tail == cdir: # xxx/newdir/. exists if xxx/newdir exists
[ 88s] return
[ 88s] try:
[ 88s] > mkdir(name, mode)
[ 88s] E PermissionError: [Errno 13] Permission denied: '/foo'
[ 88s]
[ 88s] /usr/lib64/python3.7/os.py:221: PermissionError
mcepl
All 52 comments
Hello @mcepl ,
can you please add some more context to your issue? What are you trying to do? What is your expected behavior?
Thanks a lot.
fin swimmer
finswimmer
on 29 Nov 2019
Trying to build package of poetry for openSUSE and trying to run the test suite (as with any other Python package). There is complete build log with all information to the last really excruciating details attached to the previous comment.
If I may add one question not specifically related to this issue (unfortunately, I haven’t found any IRC, Gitter, matrix, etc. channel, email list related to poetry; I tried IRC channel #pypa on Freenode, but I’ve got no response). I haven’t found any documentation on how to build poetry without using poetry, which I would need badly for avoid cyclic dependency and reproducible bulids (we do the same with pip and setuptools, they are build without using themselves). However, when I try using plain wheel and pip on building and installing poetry, I get this:
[ 3s] + python3 -mpip wheel --no-deps --use-pep517 --no-build-isolation --progress-bar off --verbose .
[ 3s] Created temporary directory: /tmp/pip-ephem-wheel-cache-ntxqa7ma
[ 3s] Created temporary directory: /tmp/pip-req-tracker-f9fvuz2u
[ 3s] Created requirements tracker '/tmp/pip-req-tracker-f9fvuz2u'
[ 3s] Created temporary directory: /tmp/pip-wheel-i5pjraet
[ 3s] Processing /home/abuild/rpmbuild/BUILD/poetry-1.0.0b8
[ 3s] Created temporary directory: /tmp/pip-req-build-_uifhzz9
[ 3s] Added file:///home/abuild/rpmbuild/BUILD/poetry-1.0.0b8 to build tracker '/tmp/pip-req-tracker-f9fvuz2u'
[ 3s] Created temporary directory: /tmp/pip-modern-metadata-mxkh5z0c
[ 3s] Preparing wheel metadata: started
[ 3s] Running command /usr/bin/python3 /usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpd6rg47lo
[ 3s] running dist_info
[ 3s] creating /tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info
[ 3s] writing /tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/PKG-INFO
[ 3s] writing dependency_links to /tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/dependency_links.txt
[ 3s] writing top-level names to /tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/top_level.txt
[ 3s] writing manifest file '/tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/SOURCES.txt'
[ 3s] reading manifest file '/tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/SOURCES.txt'
[ 3s] writing manifest file '/tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.egg-info/SOURCES.txt'
[ 3s] creating '/tmp/pip-modern-metadata-mxkh5z0c/UNKNOWN.dist-info'
[ 3s] adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
[ 3s] Preparing wheel metadata: finished with status 'done'
[ 3s] Source in /tmp/pip-req-build-_uifhzz9 has version 0.0.0, which satisfies requirement UNKNOWN==0.0.0 from file:///home/abuild/rpmbuild/BUILD/poetry-1.0.0b8
[ 3s] Removed UNKNOWN==0.0.0 from file:///home/abuild/rpmbuild/BUILD/poetry-1.0.0b8 from build tracker '/tmp/pip-req-tracker-f9fvuz2u'
[ 3s] Building wheels for collected packages: UNKNOWN
[ 3s] Created temporary directory: /tmp/pip-wheel-akh74kp9
[ 3s] Destination directory: /tmp/pip-wheel-akh74kp9
[ 3s] Building wheel for UNKNOWN (PEP 517): started
[ 3s] Running command /usr/bin/python3 /usr/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmp6fyrk0i6
[ 4s] running bdist_wheel
[ 4s] running build
[ 4s] installing to build/bdist.linux-x86_64/wheel
[ 4s] running install
[ 4s] running install_egg_info
[ 4s] running egg_info
[ 4s] creating UNKNOWN.egg-info
[ 4s] writing UNKNOWN.egg-info/PKG-INFO
[ 4s] writing dependency_links to UNKNOWN.egg-info/dependency_links.txt
[ 4s] writing top-level names to UNKNOWN.egg-info/top_level.txt
[ 4s] writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
[ 4s] reading manifest file 'UNKNOWN.egg-info/SOURCES.txt'
[ 4s] writing manifest file 'UNKNOWN.egg-info/SOURCES.txt'
[ 4s] Copying UNKNOWN.egg-info to build/bdist.linux-x86_64/wheel/UNKNOWN-0.0.0-py3.7.egg-info
[ 4s] running install_scripts
[ 4s] adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
[ 4s] creating build/bdist.linux-x86_64/wheel/UNKNOWN-0.0.0.dist-info/WHEEL
[ 4s] creating '/tmp/pip-wheel-akh74kp9/tmpbq4r2qst/UNKNOWN-0.0.0-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
[ 4s] adding 'UNKNOWN-0.0.0.dist-info/LICENSE'
[ 4s] adding 'UNKNOWN-0.0.0.dist-info/METADATA'
[ 4s] adding 'UNKNOWN-0.0.0.dist-info/WHEEL'
[ 4s] adding 'UNKNOWN-0.0.0.dist-info/top_level.txt'
[ 4s] adding 'UNKNOWN-0.0.0.dist-info/RECORD'
[ 4s] removing build/bdist.linux-x86_64/wheel
[ 4s] Building wheel for UNKNOWN (PEP 517): finished with status 'done'
[ 4s] Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=1792 sha256=709970268deee4e1050d890f9bb5c779fc612cfc002ed04d56f3b4d76c375d35
[ 4s] Stored in directory: /home/abuild/rpmbuild/BUILD/poetry-1.0.0b8
[ 4s] Successfully built UNKNOWN
[ 4s] Cleaning up...
[ 4s] Removing source in /tmp/pip-req-build-_uifhzz9
[ 4s] Removed build tracker '/tmp/pip-req-tracker-f9fvuz2u'
[ 4s] 1 location(s) to search for versions of pip:
[ 4s] * https://pypi.org/simple/pip/
[ 4s] Getting page https://pypi.org/simple/pip/
[ 4s] Found index url https://pypi.org/simple
[ 4s] Getting credentials from keyring for https://pypi.org/simple
[ 4s] Getting credentials from keyring for pypi.org
[ 4s] Looking up "https://pypi.org/simple/pip/" in the cache
[ 4s] Request header has "max_age" as 0, cache bypassed
[ 4s] Starting new HTTPS connection (1): pypi.org:443
[ 4s] Could not fetch URL https://pypi.org/simple/pip/: connection error: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fa338971630>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')) - skipping
[ 4s] Given no hashes to check 0 links for project 'pip': discarding no candidates
Obviously UNKNOWN-0.0.0-py3-none-any.whl is not what I want. And yes, the build has to happen in the environment which doesn’t have access to the Internet. And yes, I am using the latest wheel distribution available (0.33.6). What I do wrong?
mcepl
on 29 Nov 2019
@mcepl
Arch Linux now packages poetry by generating the setup.py for poetry and all its dependencies using https://github.com/dephell/dephell
@jayvdb has packaged dephell for OpenSUSE if I remember correctly.
eli-schwartz
on 1 Dec 2019
If I may add one question not specifically related to this issue (unfortunately, I haven’t found any IRC, Gitter, matrix, etc. channel, email list related to poetry; I tried IRC channel #pypa on Freenode, but I’ve got no response).
@mcepl: There is a discordapp server. See here: https://github.com/sdispater/poetry/issues/969#issuecomment-503907025
finswimmer
on 1 Dec 2019
I almost got all tests working without a venv using
sed -i 's:"/foo":"/tmp/foo":' tests/conftest.py
sed -i 's:/foo:/tmp/foo:' tests/console/commands/test_config.py tests/config/test_config.py
sed -i 's:/foo/virtualenv:/tmp/foo/virtualenv:' tests/utils/test_env.py
mkdir ~/.bin
ln -s /usr/bin/python3 ~/.bin/python
export PATH=$PATH:~/.bin
There were still some failures, but only a few.
jayvdb
on 18 Dec 2019
@eli-schwartz , ya, correct, and I have been doing the poetry packaging for a while now. Thanks for raising https://github.com/dephell/dephell/issues/330 about the current annoyance with that approach, but it is minor and easy to workaround.
Creating a testenv works around the /foo test failures, obviously.
I still have one failure, test_default_with_excluded_data, not sure about it.
jayvdb
on 18 Dec 2019
jayvdb
on 18 Dec 2019
@jayvdb I've ignored that test by running the testsuite using:
# only works inside git repositories
pytest -k 'not test_default_with_excluded_data'
Because the unittest in question is exercising poetry's ability to autogenerate exclusions based on a .gitignore, so it assumes that it's being run from a git checkout of poetry in order to successfully mock poetry.vcs.git.Git.get_ignored_files.
eli-schwartz
on 18 Dec 2019
Thanks. I did the same without knowing precisely the problem. It looked more like a test logic failure than runtime code problem. Happy to have someone one step ahead of me ;-)
jayvdb
on 18 Dec 2019
# only works inside git repositories pytest -k 'not test_default_with_excluded_data'
For reference: This is no longer necessary with 1.1.1. The test moved to poetry-core. git init helps there.
bnavigator
on 5 Oct 2020
Why is this bug report closed just because some unrelated bug mentioned 6 comments in happens to be resolved?
The error seems to have moved around a bit, but I still cannot package poetry due to:
__________________ test_builder_should_execute_build_scripts ___________________
extended_without_setup_poetry = <poetry.poetry.Poetry object at 0x7fa15ce21790>
def test_builder_should_execute_build_scripts(extended_without_setup_poetry):
env = MockEnv(path=Path("/foo"))
builder = EditableBuilder(extended_without_setup_poetry, env, NullIO())
> builder.build()
tests/masonry/builders/test_editable_builder.py:214:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
poetry/masonry/builders/editable.py:57: in build
added_files += self._add_scripts()
poetry/masonry/builders/editable.py:150: in _add_scripts
with script_file.open("w", encoding="utf-8") as f:
/usr/lib/python3.8/pathlib.py:1221: in open
return io.open(self, mode, buffering, encoding, errors, newline,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = PosixPath('/usr/bin/foo'), name = '/usr/bin/foo', flags = 524865
mode = 438
def _opener(self, name, flags, mode=0o666):
# A stub for the opener argument to built-in open()
> return self._accessor.open(self, flags, mode)
E PermissionError: [Errno 13] Permission denied: '/usr/bin/foo'
/usr/lib/python3.8/pathlib.py:1077: PermissionError
+1 for the report of the bug message (we also skip this one on openSUSE) but -1 for the complaint. Nobody told them about the different PermissionError.
bnavigator
on 20 Oct 2020
+1 for the report of the bug message (we also skip this one on openSUSE) but -1 for the complaint. Nobody told them about the _different_ PermissionError.
I am a bit lost: do we have opened ticket for this here?
mcepl
on 20 Oct 2020
I am a bit lost: do we have opened ticket for this here?
No we just referenced this one here. Bad practise on our side, nothing to chastice the python-poetry developers
%check
# A venv is necessary gh#python-poetry/poetry#1645#issuecomment-566872684
python3 -m venv testenv
source testenv/bin/activate
# needs connection to pypi.org
donttest="test_lock_no_update"
# tests downloading
donttest+=" or test_execute_executes_a_batch_of_operations"
# tries to install /usr/bin/foo
donttest+=" or test_builder_should_execute_build_scripts"
%pytest -k "not ($donttest)"
There are a bunch of errors cropping up in various places, but they all have to do with the exact same problem -- the testsuite keeps on trying to do work, in the directory /foo, or otherwise write to /.
After modifying the testsuite to properly report errors in some deeply nested thing, I found the cause of another /foo error that was dropping an assert with no information, but I'm on mobile right now so I don't have the exact details.
It seems reasonable that they're all the same underlying issue, represented by this ticket.
EDIT: posted the resulting log below, it was /usr/bin not /foo.
eli-schwartz
on 20 Oct 2020
I have added some cleanup for the tests suites so that they work better for the use cases being described above. I am assuming (since it is not entirely clear), that the environments are your typical distro packaging environments. #3255
@bnavigator this should allow you to re-enable most of those tests. Please let me know if additional tests are still failining. Particularly, if any tests still require pypi access or if data is written to non temporary directories.
Regarding the use of dephell, I am quite curious as to why this is being used for building a PEP 517 package. Sounds like it introduces more problems than what it solves.
I'd also urge that the coversation be kept constructive.
abn
on 20 Oct 2020
I have added some cleanup for the tests suites so that they work better for the use cases being described above. I am assuming (since it is not entirely clear), that the environments are your typical distro packaging environments. #3255
@bnavigator this should allow you to re-enable most of those tests. Please let me know if additional tests are still failining. Particularly, if any tests still require pypi access or if data is written to non temporary directories.
Thanks. Will test and report back.
Regarding the use of dephell, I am quite curious as to why this is being used for building a PEP 517 package. Sounds like it introduces more problems than what it solves.
+1 from me. dephell brings its own dependency hell (https://github.com/dephell/dephell/issues/347) and obviously fails too often on reading compliant specification files. Historically, it was necessary to create a setup.py and use the standard setuptools way of installing. Now we can also use %pyproject_wheel macro, which uses pip as PEP517 frontend.
I'd also urge that the coversation be kept constructive.
+1
bnavigator
on 20 Oct 2020
Meh, #3255 does not apply cleanly onto 1.1.3 and after rebasing, two tests still fail:
poetry-pr3255-testcleanup-1.1.3.patch.txt
[ 80s] =================================== FAILURES ===================================
[ 80s] _________________ test_execute_executes_a_batch_of_operations __________________
[ 80s]
[ 80s] config = <poetry.config.config.Config object at 0x7fcb5999ff10>
[ 80s] pool = <poetry.repositories.pool.Pool object at 0x7fcb4a7f85e0>
[ 80s] io = <clikit.io.buffered_io.BufferedIO object at 0x7fcb48dfa580>
[ 80s] tmp_dir = '/tmp/poetry_16euh8xy', mock_file_downloads = None
[ 80s]
[ 80s] def test_execute_executes_a_batch_of_operations(
[ 80s] config, pool, io, tmp_dir, mock_file_downloads
[ 80s] ):
[ 80s] config = Config()
[ 80s] config.merge({"cache-dir": tmp_dir})
[ 80s]
[ 80s] env = MockEnv(path=Path(tmp_dir))
[ 80s] executor = Executor(env, pool, config, io)
[ 80s]
[ 80s] file_package = Package(
[ 80s] "demo",
[ 80s] "0.1.0",
[ 80s] source_type="file",
[ 80s] source_url=Path(__file__)
[ 80s] .parent.parent.joinpath(
[ 80s] "fixtures/distributions/demo-0.1.0-py2.py3-none-any.whl"
[ 80s] )
[ 80s] .resolve()
[ 80s] .as_posix(),
[ 80s] )
[ 80s]
[ 80s] directory_package = Package(
[ 80s] "simple-project",
[ 80s] "1.2.3",
[ 80s] source_type="directory",
[ 80s] source_url=Path(__file__)
[ 80s] .parent.parent.joinpath("fixtures/simple_project")
[ 80s] .resolve()
[ 80s] .as_posix(),
[ 80s] )
[ 80s]
[ 80s] git_package = Package(
[ 80s] "demo",
[ 80s] "0.1.0",
[ 80s] source_type="git",
[ 80s] source_reference="master",
[ 80s] source_url="https://github.com/demo/demo.git",
[ 80s] )
[ 80s]
[ 80s] > assert 0 == executor.execute(
[ 80s] [
[ 80s] Install(Package("pytest", "3.5.2")),
[ 80s] Uninstall(Package("attrs", "17.4.0")),
[ 80s] Update(Package("requests", "2.18.3"), Package("requests", "2.18.4")),
[ 80s] Uninstall(Package("clikit", "0.2.3")).skip("Not currently installed"),
[ 80s] Install(file_package),
[ 80s] Install(directory_package),
[ 80s] Install(git_package),
[ 80s] ]
[ 80s] )
[ 80s] E assert 0 == 1
[ 80s] E +0
[ 80s] E -1
[ 80s]
[ 80s] tests/installation/test_executor.py:98: AssertionError
[ 80s] ------------------------------ Captured log call -------------------------------
[ 80s] DEBUG urllib3.connectionpool:connectionpool.py:939 Starting new HTTPS connection (1): files.pythonhosted.org:443
[ 80s] DEBUG httpretty.core:core.py:432 setsockopt(6, 1, 1) failed
[ 80s] DEBUG urllib3.connectionpool:connectionpool.py:433 https://files.pythonhosted.org:443 "GET /packages/ed/96/271c93f75212c06e2a7ec3e2fa8a9c90acee0a4838dc05bf379ea09aae31/pytest-3.5.0-py2.py3-none-any.whl HTTP/1.1" 200 1116
[ 80s] DEBUG urllib3.connectionpool:connectionpool.py:271 Resetting dropped connection: files.pythonhosted.org
[ 80s] DEBUG httpretty.core:core.py:432 setsockopt(6, 1, 1) failed
[ 80s] DEBUG urllib3.connectionpool:connectionpool.py:433 https://files.pythonhosted.org:443 "GET /packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl HTTP/1.1" 200 1116
[ 80s] __________________ test_builder_should_execute_build_scripts ___________________
[ 80s]
[ 80s] extended_without_setup_poetry = <poetry.poetry.Poetry object at 0x7fcb581b8b50>
[ 80s] tmp_dir = '/tmp/poetry_2as2wnyx'
[ 80s]
[ 80s] def test_builder_should_execute_build_scripts(extended_without_setup_poetry, tmp_dir):
[ 80s] env = MockEnv(path=Path(tmp_dir) / "foo")
[ 80s] builder = EditableBuilder(extended_without_setup_poetry, env, NullIO())
[ 80s]
[ 80s] > builder.build()
[ 80s]
[ 80s] tests/masonry/builders/test_editable_builder.py:214:
[ 80s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 80s] poetry/masonry/builders/editable.py:57: in build
[ 80s] added_files += self._add_scripts()
[ 80s] poetry/masonry/builders/editable.py:150: in _add_scripts
[ 80s] with script_file.open("w", encoding="utf-8") as f:
[ 80s] /usr/lib64/python3.8/pathlib.py:1218: in open
[ 80s] return io.open(self, mode, buffering, encoding, errors, newline,
[ 80s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 80s]
[ 80s] self = PosixPath('/usr/bin/foo'), name = '/usr/bin/foo', flags = 524865
[ 80s] mode = 438
[ 80s]
[ 80s] def _opener(self, name, flags, mode=0o666):
[ 80s] # A stub for the opener argument to built-in open()
[ 80s] > return self._accessor.open(self, flags, mode)
[ 80s] E PermissionError: [Errno 13] Permission denied: '/usr/bin/foo'
[ 80s]
[ 80s] /usr/lib64/python3.8/pathlib.py:1074: PermissionError
[ 80s] =============================== warnings summary ===============================
[ 80s] tests/helpers.py:139
[ 80s] /home/abuild/rpmbuild/BUILD/poetry-1.1.3/tests/helpers.py:139: PytestCollectionWarning: cannot collect test class 'TestApplication' because it has a __init__ constructor (from: tests/console/commands/test_init.py)
[ 80s] class TestApplication(Application):
[ 80s]
[ 80s] -- Docs: https://docs.pytest.org/en/stable/warnings.html
[ 80s] =========================== short test summary info ============================
[ 80s] FAILED tests/installation/test_executor.py::test_execute_executes_a_batch_of_operations
[ 80s] FAILED tests/masonry/builders/test_editable_builder.py::test_builder_should_execute_build_scripts
[ 80s] ======== 2 failed, 616 passed, 4 skipped, 1 warning in 70.68s (0:01:10) ========
Edit: I see you force pushed the PR after my first rebase. Same result with second rebase
poetry-pr3255-testcleanup-1.1.3-2.patch.txt
bnavigator
on 20 Oct 2020
BTW, the venv is still necessary because many tests call python directly instead of sys.executable
bnavigator
on 20 Oct 2020
Directly building from 43cd07c has the same effect.
bnavigator
on 20 Oct 2020
BTW, the venv is still necessary because many tests call
pythondirectly instead ofsys.executable
The use of "python" is intentional in most cases as commands need to be executed by a different interpretor than that which poetry is running under.
As for the failing test, I have pushed a fix for the test_builder_should_execute_build_scripts failures. I am not entirely sure what is causing your failire for test_execute_executes_a_batch_of_operations. I tried locally without a network and it did not fail.
abn
on 20 Oct 2020
The use of "python" is intentional in most cases as commands need to be executed by a different interpretor than that which poetry is running under.
Do you mean a different version or just a different process? If the latter, you still should use sys.executable because you cannot assume that python is in the PATH. openSUSE Tumbleweed only has python3*. python is Python 2 where installed.
bnavigator
on 20 Oct 2020
With e1ba4f8 the test still fails, because you still want to write into the system root:
[ 78s] __________________ test_builder_should_execute_build_scripts ___________________
[ 78s]
[ 78s] extended_without_setup_poetry = <poetry.poetry.Poetry object at 0x7fb02a479880>
[ 78s] tmp_dir = '/tmp/poetry_1tz_zqxq'
[ 78s]
[ 78s] def test_builder_should_execute_build_scripts(extended_without_setup_poetry, tmp_dir):
[ 78s] env = MockEnv(path=Path(tmp_dir) / "foo")
[ 78s] builder = EditableBuilder(extended_without_setup_poetry, env, NullIO())
[ 78s]
[ 78s] > builder.build()
[ 78s]
[ 78s] tests/masonry/builders/test_editable_builder.py:226:
[ 78s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 78s]
[ 78s] self = <poetry.masonry.builders.editable.EditableBuilder object at 0x7fb02a4e4a60>
[ 78s]
[ 78s] def build(self):
[ 78s] self._debug(
[ 78s] " - Building package <c1>{}</c1> in <info>editable</info> mode".format(
[ 78s] self._package.name
[ 78s] )
[ 78s] )
[ 78s]
[ 78s] if self._package.build_script:
[ 78s] if self._package.build_should_generate_setup():
[ 78s] self._debug(
[ 78s] " - <warning>Falling back on using a <b>setup.py</b></warning>"
[ 78s] )
[ 78s]
[ 78s] return self._setup_build()
[ 78s]
[ 78s] self._run_build_script(self._package.build_script)
[ 78s]
[ 78s] added_files = []
[ 78s] added_files += self._add_pth()
[ 78s] added_files += self._add_scripts()
[ 78s] > self._add_dist_info(added_files)
[ 78s]
[ 78s] poetry/masonry/builders/editable.py:58:
[ 78s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 78s]
[ 78s] self = <poetry.masonry.builders.editable.EditableBuilder object at 0x7fb02a4e4a60>
[ 78s] added_files = [PosixPath('/home/abuild/.local/lib/python3.8/site-packages/extended_project.pth')]
[ 78s]
[ 78s] def _add_dist_info(self, added_files):
[ 78s] from poetry.core.masonry.builders.wheel import WheelBuilder
[ 78s]
[ 78s] added_files = added_files[:]
[ 78s]
[ 78s] builder = WheelBuilder(self._poetry)
[ 78s] dist_info = self._env.site_packages.joinpath(builder.dist_info)
[ 78s]
[ 78s] self._debug(
[ 78s] " - Adding the <c2>{}</c2> directory to <b>{}</b>".format(
[ 78s] dist_info.name, self._env.site_packages
[ 78s] )
[ 78s] )
[ 78s]
[ 78s] if dist_info.exists():
[ 78s] shutil.rmtree(str(dist_info))
[ 78s]
[ 78s] > dist_info.mkdir()
[ 78s]
[ 78s] poetry/masonry/builders/editable.py:201:
[ 78s] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[ 78s]
[ 78s] self = PosixPath('/usr/lib/python3.8/site-packages/extended_project-1.2.3.dist-info')
[ 78s] mode = 511, parents = False, exist_ok = False
[ 78s]
[ 78s] def mkdir(self, mode=0o777, parents=False, exist_ok=False):
[ 78s] """
[ 78s] Create a new directory at this given path.
[ 78s] """
[ 78s] if self._closed:
[ 78s] self._raise_closed()
[ 78s] try:
[ 78s] > self._accessor.mkdir(self, mode)
[ 78s] E PermissionError: [Errno 13] Permission denied: '/usr/lib/python3.8/site-packages/extended_project-1.2.3.dist-info'
[ 78s]
[ 78s] /usr/lib64/python3.8/pathlib.py:1284: PermissionError
______________________________________________________________ test_execute_executes_a_batch_of_operations ______________________________________________________________
config = <poetry.config.config.Config object at 0x7f06204f3640>, pool = <poetry.repositories.pool.Pool object at 0x7f0620596880>
io = <clikit.io.buffered_io.BufferedIO object at 0x7f0620599520>, tmp_dir = '/tmp/poetry_0fc5d49f', mock_file_downloads = None
def test_execute_executes_a_batch_of_operations(
config, pool, io, tmp_dir, mock_file_downloads
):
config = Config()
config.merge({"cache-dir": tmp_dir})
env = MockEnv(path=Path(tmp_dir))
executor = Executor(env, pool, config, io)
file_package = Package(
"demo",
"0.1.0",
source_type="file",
source_url=Path(__file__)
.parent.parent.joinpath(
"fixtures/distributions/demo-0.1.0-py2.py3-none-any.whl"
)
.resolve()
.as_posix(),
)
directory_package = Package(
"simple-project",
"1.2.3",
source_type="directory",
source_url=Path(__file__)
.parent.parent.joinpath("fixtures/simple_project")
.resolve()
.as_posix(),
)
git_package = Package(
"demo",
"0.1.0",
source_type="git",
source_reference="master",
source_url="https://github.com/demo/demo.git",
)
ret = executor.execute(
[
Install(Package("pytest", "3.5.2")),
Uninstall(Package("attrs", "17.4.0")),
Update(Package("requests", "2.18.3"), Package("requests", "2.18.4")),
Uninstall(Package("clikit", "0.2.3")).skip("Not currently installed"),
Install(file_package),
Install(directory_package),
Install(git_package),
]
)
expected = """
Package operations: 4 installs, 1 update, 1 removal
• Installing pytest (3.5.2)
• Removing attrs (17.4.0)
• Updating requests (2.18.3 -> 2.18.4)
• Installing demo (0.1.0 {})
• Installing simple-project (1.2.3 {})
• Installing demo (0.1.0 master)
""".format(
file_package.source_url, directory_package.source_url
)
expected = set(expected.splitlines())
output = set(io.fetch_output().splitlines())
> assert expected == output
E assert {'',\n ' • Installing demo (0.1.0 '\n '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/distributions/demo-0.1.0-py2.py3-none-any.whl)',\n ' • Installing demo (0.1.0 master)',\n ' • Installing pytest (3.5.2)',\n ' • Installing simple-project (1.2.3 '\n '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/simple_project)',\n ' • Removing attrs (17.4.0)',\n ' • Updating requests (2.18.3 -> 2.18.4)',\n 'Package operations: 4 installs, 1 update, 1 removal'} == {'',\n ' 1073│ raise ValueError("I/O operation on closed path")',\n ' 1074│ ',\n ' 1075│ def _opener(self, name, flags, mode=0o666):',\n ' 1076│ # A stub for the opener argument to built-in open()',\n ' 1078│ ',\n ' 1079│ def _raw_open(self, flags, mode=0o777):',\n ' 1080│ ',\n ' 1081│ Open the file pointed by this path and return a file '\n 'descriptor,',\n ' → 1077│ return self._accessor.open(self, flags, mode)',\n ' PermissionError',\n " [Errno 13] Permission denied: '/usr/bin/baz'",\n ' at /usr/lib/python3.8/pathlib.py:1077 in _opener',\n ' • Installing demo (0.1.0 '\n '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/distributions/demo-0.1.0-py2.py3-none-any.whl)',\n ' • Installing demo (0.1.0 master)',\n ' • Installing pytest (3.5.2)',\n ' • Installing simple-project (1.2.3 '\n '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/simple_project)',\n ' • Removing attrs (17.4.0)',\n ' • Updating requests (2.18.3 -> 2.18.4)',\n 'Package operations: 4 installs, 1 update, 1 removal'}
E Extra items in the right set:
E ' PermissionError'
E ' 1076│ # A stub for the opener argument to built-in open()'
E ' → 1077│ return self._accessor.open(self, flags, mode)'
E ' at /usr/lib/python3.8/pathlib.py:1077 in _opener'
E ' 1073│ raise ValueError("I/O operation on closed path")'
E ' 1079│ def _raw_open(self, flags, mode=0o777):'
E ' 1075│ def _opener(self, name, flags, mode=0o666):'
E " [Errno 13] Permission denied: '/usr/bin/baz'"
E ' 1080│ '
E ' 1074│ '
E ' 1078│ '
E ' 1081│ Open the file pointed by this path and return a file descriptor,'
E Full diff:
E {
E '',
E - ' 1073│ raise ValueError("I/O operation on closed path")',
E - ' 1074│ ',
E - ' 1075│ def _opener(self, name, flags, mode=0o666):',
E - ' 1076│ # A stub for the opener argument to built-in open()',
E - ' 1078│ ',
E - ' 1079│ def _raw_open(self, flags, mode=0o777):',
E - ' 1080│ ',
E - ' 1081│ Open the file pointed by this path and return a file '
E - 'descriptor,',
E - ' → 1077│ return self._accessor.open(self, flags, mode)',
E - ' PermissionError',
E - " [Errno 13] Permission denied: '/usr/bin/baz'",
E - ' at /usr/lib/python3.8/pathlib.py:1077 in _opener',
E ' • Installing demo (0.1.0 '
E '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/distributions/demo-0.1.0-py2.py3-none-any.whl)',
E ' • Installing demo (0.1.0 master)',
E ' • Installing pytest (3.5.2)',
E ' • Installing simple-project (1.2.3 '
E '/build/python-poetry/src/poetry-1.1.2/tests/fixtures/simple_project)',
E ' • Removing attrs (17.4.0)',
E ' • Updating requests (2.18.3 -> 2.18.4)',
E 'Package operations: 4 installs, 1 update, 1 removal',
E }
tests/installation/test_executor.py:126: AssertionError
I discovered the cause of this error when modifying this: https://github.com/python-poetry/poetry/blob/68d6939f2e9cac1178fc5ffbcc3bac7a9db43fce/tests/installation/test_executor.py#L98-L126
so the assert 0 == executor.execute no longer masked the output which is tested by: assert expected == output
eli-schwartz
on 20 Oct 2020
Okay, so the failed connection attempt debug log is misleading. Another permission error in system root. Good catch, Eli!
bnavigator
on 20 Oct 2020
For all those tests the assertion of the returncode should be last.
bnavigator
on 20 Oct 2020
@bnavigator from your logs it seems that your are running as an unprivileged user, with system interpretor. For that scenario, there are a few bugs in 1.1.z. See #3107.
Do you mean a different version or just a different process? If the latter, you still should use
sys.executablebecause you cannot assume thatpythonis in the PATH. openSUSE Tumbleweed only haspython3*.pythonis Python 2 where installed.
Different version when dealing with virtual environments. When dealing with system environment, we still use sys.executable and sys.prefix This is also whats essentially causing your woes, as I am assuming you are running as an unprivileged user.
This works for me without a virtual environment.
FROM python:3.8
RUN curl -sL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
COPY . /opt/poetry
WORKDIR /opt/poetry
RUN ~/.poetry/bin/poetry config virtualenvs.create false
RUN ~/.poetry/bin/poetry install
RUN pytest tests/
Yeah, that was what I did -- saved it as ret = then added assert 0 == ret all the way at the end (which nevertheless failed beforehand).
eli-schwartz
on 20 Oct 2020
Running tests with system privileges is no option. Neither for rpmbuild (openSUSE, Fedora, ...) nor for Eli's Archlinux.
bnavigator
on 20 Oct 2020
For that scenario, there are a few bugs in 1.1.z
The same tests fail on master (1.2.0a0) + #3255 (= e1ba4f8 )
bnavigator
on 20 Oct 2020
@bnavigator that fix (#3107) is not in master yet.
Running _tests_ with system privileges is no option. Neither for rpmbuild (openSUSE, Fedora, ...) nor for Eli's Archlinux.
Not recommending that. What I am saying is you need to run your tests in a virtual environment. We do not really support running the test suite under the system interpretor.
I have updated the PR again with a few more fixes.
abn
on 20 Oct 2020
Question: does poetry's CI test runner get run as root, or does it get run via a virtualenv as a non-privileged user as you just suggested?
eli-schwartz
on 20 Oct 2020
Poetry implicitly manages the virtual environment.
abn
on 20 Oct 2020
But, does it actually test if the suite can run successfully as an unprivileged user?
Or does it only work because e.g. github workflows happen to run as root?
eli-schwartz
on 20 Oct 2020
@eli-schwartz the workflow does not explcitly drop down to an unprivileged user, but that is out of scope for the project. Most developers do not run the tests suites as the root user anyway.
Note that the issue that you are facing is effectively because the test suite is verifying that it can "install" to the site of the python interpretor that is running the test suite, as it is designed to do at the moment.
abn
on 21 Oct 2020
because the test suite is verifying that it can "install" to the
siteof the python interpretor that is running the test suite,
That's the info we needed! :tada:
This works:
# a virtualenv is necessary gh#python-poetry/poetry#1645
virtualenv testenv
source testenv/bin/activate
# poetry as it will be packaged
PYTHONPATH="/home/abuild/rpmbuild/BUILDROOT/python-poetry-1.2.0~a0+pr3255-0.x86_64/usr/lib/python3.8/site-packages"
# use system site-packages, we can't get packages into the virtualenv by downloading
export PYTHONPATH+=":/usr/lib/python3.8/site-packages:/usr/lib64/python3.8/site-packages"
export PYTHONDONTWRITEBYTECODE=1
# pytest needs to be called from the virtualenv python interpreter
python -m pytest -v tests
deactivate
622 passed, 4 skipped, 1 warning in 68.44s (0:01:08)
With the above, #3255 only has an effect on the test_lock_no_update. The other tests don't need #3255.
bnavigator
on 21 Oct 2020
OK, tried to run tests like this:
python -m venv --system-site-packages --without-pip poetrytests
./poetrytests/bin/python -m pytest
I'm down to fewer failures, but it still seems to be trying to create /foo/virtualenvs/simple-project-CDKM9e_d-py3.8:
=================================== FAILURES ===================================
_______ test_export_exports_requirements_txt_file_locks_if_no_lock_file ________
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
action = _StoreAction(option_strings=[], dest='dest', nargs=None, const=None, default=None, type=<bound method Creator.validate...al_ref.builtin.cpython.cpython3.CPython3Posix'>>, choices=None, help='directory to create virtualenv at', metavar=None)
arg_string = '/foo/virtualenvs/simple-project-CDKM9e_d-py3.8'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
> result = type_func(arg_string)
/usr/lib/python3.8/argparse.py:2422:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'virtualenv.create.via_global_ref.builtin.cpython.cpython3.CPython3Posix'>
raw_value = '/foo/virtualenvs/simple-project-CDKM9e_d-py3.8'
@classmethod
def validate_dest(cls, raw_value):
"""No path separator in the path, valid chars and must be write-able"""
def non_write_able(dest, value):
common = Path(*os.path.commonprefix([value.parts, dest.parts]))
raise ArgumentTypeError(
"the destination {} is not write-able at {}".format(dest.relative_to(common), common),
)
# the file system must be able to encode
# note in newer CPython this is always utf-8 https://www.python.org/dev/peps/pep-0529/
encoding = sys.getfilesystemencoding()
refused = OrderedDict()
kwargs = {"errors": "ignore"} if encoding != "mbcs" else {}
for char in ensure_text(raw_value):
try:
trip = char.encode(encoding, **kwargs).decode(encoding)
if trip == char:
continue
raise ValueError(trip)
except ValueError:
refused[char] = None
if refused:
raise ArgumentTypeError(
"the file system codec ({}) cannot handle characters {!r} within {!r}".format(
encoding, "".join(refused.keys()), raw_value,
),
)
if os.pathsep in raw_value:
raise ArgumentTypeError(
"destination {!r} must not contain the path separator ({}) as this would break "
"the activation scripts".format(raw_value, os.pathsep),
)
value = Path(raw_value)
if value.exists() and value.is_file():
raise ArgumentTypeError("the destination {} already exists and is a file".format(value))
if (3, 3) <= sys.version_info <= (3, 6):
# pre 3.6 resolve is always strict, aka must exists, sidestep by using os.path operation
dest = Path(os.path.realpath(raw_value))
else:
dest = Path(os.path.abspath(str(value))).resolve() # on Windows absolute does not imply resolve so use both
value = dest
while dest:
if dest.exists():
if os.access(ensure_text(str(dest)), os.W_OK):
break
else:
> non_write_able(dest, value)
/usr/lib/python3.8/site-packages/virtualenv/create/creator.py:146:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
dest = PosixPath('/')
value = PosixPath('/foo/virtualenvs/simple-project-CDKM9e_d-py3.8')
def non_write_able(dest, value):
common = Path(*os.path.commonprefix([value.parts, dest.parts]))
> raise ArgumentTypeError(
"the destination {} is not write-able at {}".format(dest.relative_to(common), common),
)
E argparse.ArgumentTypeError: the destination . is not write-able at /
/usr/lib/python3.8/site-packages/virtualenv/create/creator.py:103: ArgumentTypeError
During handling of the above exception, another exception occurred:
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
args = ['--no-download', '--no-periodic-update', '--python', '/build/python-poetry/src/poetry-1.1.2/poetrytests/bin/python', '/foo/virtualenvs/simple-project-CDKM9e_d-py3.8']
namespace = VirtualEnvOptions(with_traceback=False, verbose=2, quiet=0, app_data=/build/.local/share/virtualenv, reset_app_data=Fa...undle, no_pip=False, no_setuptools=False, no_wheel=False, no_periodic_update=True, symlink_app_data=False, prompt=None)
def parse_known_args(self, args=None, namespace=None):
if args is None:
# args default to the system args
args = _sys.argv[1:]
else:
# make sure that args are mutable
args = list(args)
# default Namespace built from parser defaults
if namespace is None:
namespace = Namespace()
# add any action defaults that aren't present
for action in self._actions:
if action.dest is not SUPPRESS:
if not hasattr(namespace, action.dest):
if action.default is not SUPPRESS:
setattr(namespace, action.dest, action.default)
# add any parser defaults that aren't present
for dest in self._defaults:
if not hasattr(namespace, dest):
setattr(namespace, dest, self._defaults[dest])
# parse the arguments and exit if there are any errors
try:
> namespace, args = self._parse_known_args(args, namespace)
/usr/lib/python3.8/argparse.py:1800:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
arg_strings = ['--no-download', '--no-periodic-update', '--python', '/build/python-poetry/src/poetry-1.1.2/poetrytests/bin/python', '/foo/virtualenvs/simple-project-CDKM9e_d-py3.8']
namespace = VirtualEnvOptions(with_traceback=False, verbose=2, quiet=0, app_data=/build/.local/share/virtualenv, reset_app_data=Fa...undle, no_pip=False, no_setuptools=False, no_wheel=False, no_periodic_update=True, symlink_app_data=False, prompt=None)
def _parse_known_args(self, arg_strings, namespace):
# replace arg strings that are file references
if self.fromfile_prefix_chars is not None:
arg_strings = self._read_args_from_files(arg_strings)
# map all mutually exclusive arguments to the other arguments
# they can't occur with
action_conflicts = {}
for mutex_group in self._mutually_exclusive_groups:
group_actions = mutex_group._group_actions
for i, mutex_action in enumerate(mutex_group._group_actions):
conflicts = action_conflicts.setdefault(mutex_action, [])
conflicts.extend(group_actions[:i])
conflicts.extend(group_actions[i + 1:])
# find all option indices, and determine the arg_string_pattern
# which has an 'O' if there is an option at an index,
# an 'A' if there is an argument, or a '-' if there is a '--'
option_string_indices = {}
arg_string_pattern_parts = []
arg_strings_iter = iter(arg_strings)
for i, arg_string in enumerate(arg_strings_iter):
# all args after -- are non-options
if arg_string == '--':
arg_string_pattern_parts.append('-')
for arg_string in arg_strings_iter:
arg_string_pattern_parts.append('A')
# otherwise, add the arg to the arg strings
# and note the index if it was an option
else:
option_tuple = self._parse_optional(arg_string)
if option_tuple is None:
pattern = 'A'
else:
option_string_indices[i] = option_tuple
pattern = 'O'
arg_string_pattern_parts.append(pattern)
# join the pieces together to form the pattern
arg_strings_pattern = ''.join(arg_string_pattern_parts)
# converts arg strings to the appropriate and then takes the action
seen_actions = set()
seen_non_default_actions = set()
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
argument_values = self._get_values(action, argument_strings)
# error if this argument is not allowed with other previously
# seen arguments, assuming that actions that use the default
# value don't really count as "present"
if argument_values is not action.default:
seen_non_default_actions.add(action)
for conflict_action in action_conflicts.get(action, []):
if conflict_action in seen_non_default_actions:
msg = _('not allowed with argument %s')
action_name = _get_action_name(conflict_action)
raise ArgumentError(action, msg % action_name)
# take the action if we didn't receive a SUPPRESS value
# (e.g. from a default)
if argument_values is not SUPPRESS:
action(self, namespace, argument_values, option_string)
# function to convert arg_strings into an optional action
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
action, option_string, explicit_arg = option_tuple
# identify additional optionals in the same arg string
# (e.g. -xyz is the same as -x -y -z if no args are required)
match_argument = self._match_argument
action_tuples = []
while True:
# if we found no optional action, skip it
if action is None:
extras.append(arg_strings[start_index])
return start_index + 1
# if there is an explicit argument, try to match the
# optional's string arguments to only this
if explicit_arg is not None:
arg_count = match_argument(action, 'A')
# if the action is a single-dash option and takes no
# arguments, try to parse more single-dash options out
# of the tail of the option string
chars = self.prefix_chars
if arg_count == 0 and option_string[1] not in chars:
action_tuples.append((action, [], option_string))
char = option_string[0]
option_string = char + explicit_arg[0]
new_explicit_arg = explicit_arg[1:] or None
optionals_map = self._option_string_actions
if option_string in optionals_map:
action = optionals_map[option_string]
explicit_arg = new_explicit_arg
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if the action expect exactly one argument, we've
# successfully matched the option; exit the loop
elif arg_count == 1:
stop = start_index + 1
args = [explicit_arg]
action_tuples.append((action, args, option_string))
break
# error if a double-dash option did not use the
# explicit argument
else:
msg = _('ignored explicit argument %r')
raise ArgumentError(action, msg % explicit_arg)
# if there is no explicit argument, try to match the
# optional's string arguments with the following strings
# if successful, exit the loop
else:
start = start_index + 1
selected_patterns = arg_strings_pattern[start:]
arg_count = match_argument(action, selected_patterns)
stop = start + arg_count
args = arg_strings[start:stop]
action_tuples.append((action, args, option_string))
break
# add the Optional to the list and return the index at which
# the Optional's string args stopped
assert action_tuples
for action, args, option_string in action_tuples:
take_action(action, args, option_string)
return stop
# the list of Positionals left to be parsed; this is modified
# by consume_positionals()
positionals = self._get_positional_actions()
# function to convert arg_strings into positional actions
def consume_positionals(start_index):
# match as many Positionals as possible
match_partial = self._match_arguments_partial
selected_pattern = arg_strings_pattern[start_index:]
arg_counts = match_partial(positionals, selected_pattern)
# slice off the appropriate arg strings for each Positional
# and add the Positional and its args to the list
for action, arg_count in zip(positionals, arg_counts):
args = arg_strings[start_index: start_index + arg_count]
start_index += arg_count
take_action(action, args)
# slice off the Positionals that we just parsed and return the
# index at which the Positionals' string args stopped
positionals[:] = positionals[len(arg_counts):]
return start_index
# consume Positionals and Optionals alternately, until we have
# passed the last option string
extras = []
start_index = 0
if option_string_indices:
max_option_string_index = max(option_string_indices)
else:
max_option_string_index = -1
while start_index <= max_option_string_index:
# consume any Positionals preceding the next option
next_option_string_index = min([
index
for index in option_string_indices
if index >= start_index])
if start_index != next_option_string_index:
positionals_end_index = consume_positionals(start_index)
# only try to parse the next optional if we didn't consume
# the option string during the positionals parsing
if positionals_end_index > start_index:
start_index = positionals_end_index
continue
else:
start_index = positionals_end_index
# if we consumed all the positionals we could and we're not
# at the index of an option string, there were extra arguments
if start_index not in option_string_indices:
strings = arg_strings[start_index:next_option_string_index]
extras.extend(strings)
start_index = next_option_string_index
# consume the next optional and any arguments for it
start_index = consume_optional(start_index)
# consume any positionals following the last Optional
> stop_index = consume_positionals(start_index)
/usr/lib/python3.8/argparse.py:2009:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
start_index = 5
def consume_positionals(start_index):
# match as many Positionals as possible
match_partial = self._match_arguments_partial
selected_pattern = arg_strings_pattern[start_index:]
arg_counts = match_partial(positionals, selected_pattern)
# slice off the appropriate arg strings for each Positional
# and add the Positional and its args to the list
for action, arg_count in zip(positionals, arg_counts):
args = arg_strings[start_index: start_index + arg_count]
start_index += arg_count
> take_action(action, args)
/usr/lib/python3.8/argparse.py:1965:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
action = _StoreAction(option_strings=[], dest='dest', nargs=None, const=None, default=None, type=<bound method Creator.validate...al_ref.builtin.cpython.cpython3.CPython3Posix'>>, choices=None, help='directory to create virtualenv at', metavar=None)
argument_strings = ['/foo/virtualenvs/simple-project-CDKM9e_d-py3.8']
option_string = None
def take_action(action, argument_strings, option_string=None):
seen_actions.add(action)
> argument_values = self._get_values(action, argument_strings)
/usr/lib/python3.8/argparse.py:1858:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
action = _StoreAction(option_strings=[], dest='dest', nargs=None, const=None, default=None, type=<bound method Creator.validate...al_ref.builtin.cpython.cpython3.CPython3Posix'>>, choices=None, help='directory to create virtualenv at', metavar=None)
arg_strings = ['/foo/virtualenvs/simple-project-CDKM9e_d-py3.8']
def _get_values(self, action, arg_strings):
# for everything but PARSER, REMAINDER args, strip out first '--'
if action.nargs not in [PARSER, REMAINDER]:
try:
arg_strings.remove('--')
except ValueError:
pass
# optional argument produces a default when not present
if not arg_strings and action.nargs == OPTIONAL:
if action.option_strings:
value = action.const
else:
value = action.default
if isinstance(value, str):
value = self._get_value(action, value)
self._check_value(action, value)
# when nargs='*' on a positional, if there were no command-line
# args, use the default if it is anything other than None
elif (not arg_strings and action.nargs == ZERO_OR_MORE and
not action.option_strings):
if action.default is not None:
value = action.default
else:
value = arg_strings
self._check_value(action, value)
# single argument or optional argument produces a single value
elif len(arg_strings) == 1 and action.nargs in [None, OPTIONAL]:
arg_string, = arg_strings
> value = self._get_value(action, arg_string)
/usr/lib/python3.8/argparse.py:2389:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
action = _StoreAction(option_strings=[], dest='dest', nargs=None, const=None, default=None, type=<bound method Creator.validate...al_ref.builtin.cpython.cpython3.CPython3Posix'>>, choices=None, help='directory to create virtualenv at', metavar=None)
arg_string = '/foo/virtualenvs/simple-project-CDKM9e_d-py3.8'
def _get_value(self, action, arg_string):
type_func = self._registry_get('type', action.type, action.type)
if not callable(type_func):
msg = _('%r is not callable')
raise ArgumentError(action, msg % type_func)
# convert the value to the appropriate type
try:
result = type_func(arg_string)
# ArgumentTypeErrors indicate errors
except ArgumentTypeError:
name = getattr(action.type, '__name__', repr(action.type))
msg = str(_sys.exc_info()[1])
> raise ArgumentError(action, msg)
E argparse.ArgumentError: argument dest: the destination . is not write-able at /
/usr/lib/python3.8/argparse.py:2428: ArgumentError
During handling of the above exception, another exception occurred:
tester = <cleo.testers.command_tester.CommandTester object at 0x7fce6d726c10>
poetry = <poetry.poetry.Poetry object at 0x7fce6d720700>
def test_export_exports_requirements_txt_file_locks_if_no_lock_file(tester, poetry):
assert not poetry.locker.lock.exists()
> _export_requirements(tester, poetry)
tests/console/commands/test_export.py:79:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/console/commands/test_export.py:60: in _export_requirements
tester.execute("--format requirements.txt --output requirements.txt")
/usr/lib/python3.8/site-packages/cleo/testers/command_tester.py:61: in execute
self._status_code = command.run(args, self._io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:116: in run
return self.handle(self.parse(args), io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:120: in handle
status_code = self._do_handle(args, io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:171: in _do_handle
return getattr(handler, handler_method)(args, io, self)
/usr/lib/python3.8/site-packages/cleo/commands/command.py:92: in wrap_handle
return self.handle()
poetry/console/commands/export.py:53: in handle
self.call("lock", options)
/usr/lib/python3.8/site-packages/cleo/commands/command.py:110: in call
return command.run(args, self.io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:116: in run
return self.handle(self.parse(args), io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:120: in handle
status_code = self._do_handle(args, io)
/usr/lib/python3.8/site-packages/clikit/api/command/command.py:163: in _do_handle
self._dispatcher.dispatch(PRE_HANDLE, event)
/usr/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:22: in dispatch
self._do_dispatch(listeners, event_name, event)
/usr/lib/python3.8/site-packages/clikit/api/event/event_dispatcher.py:89: in _do_dispatch
listener(event, event_name, self)
poetry/console/config/application_config.py:119: in set_env
env = env_manager.create_venv(io)
poetry/utils/env.py:645: in create_venv
self.build_venv(venv, executable=executable)
poetry/utils/env.py:686: in build_venv
return virtualenv.cli_run(
/usr/lib/python3.8/site-packages/virtualenv/run/__init__.py:26: in cli_run
of_session = session_via_cli(args, options, setup_logging)
/usr/lib/python3.8/site-packages/virtualenv/run/__init__.py:43: in session_via_cli
options = parser.parse_args(args)
/usr/lib/python3.8/argparse.py:1768: in parse_args
args, argv = self.parse_known_args(args, namespace)
/usr/lib/python3.8/site-packages/virtualenv/config/cli/parser.py:104: in parse_known_args
return super(VirtualEnvConfigParser, self).parse_known_args(args, namespace=namespace)
/usr/lib/python3.8/argparse.py:1807: in parse_known_args
self.error(str(err))
/usr/lib/python3.8/argparse.py:2521: in error
self.exit(2, _('%(prog)s: error: %(message)s\n') % args)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = VirtualEnvConfigParser(prog='virtualenv', usage=None, description=None, formatter_class=<class 'virtualenv.config.cli.parser.HelpFormatter'>, conflict_handler='error', add_help=False)
status = 2
message = 'virtualenv: error: argument dest: the destination . is not write-able at /\n'
def exit(self, status=0, message=None):
if message:
self._print_message(message, _sys.stderr)
> _sys.exit(status)
E SystemExit: 2
/usr/lib/python3.8/argparse.py:2508: SystemExit
----------------------------- Captured stderr call -----------------------------
usage: virtualenv [--version] [--with-traceback] [-v | -q] [--app-data APP_DATA] [--reset-app-data] [--upgrade-embed-wheels] [--discovery {builtin}] [-p py] [--creator {builtin,cpython3-posix,venv}] [--seeder {app-data,pip}] [--no-seed]
[--activators comma_sep_list] [--clear] [--system-site-packages] [--symlinks | --copies] [--no-download | --download] [--extra-search-dir d [d ...]] [--pip version] [--setuptools version] [--wheel version] [--no-pip]
[--no-setuptools] [--no-wheel] [--no-periodic-update] [--symlink-app-data] [--prompt prompt] [-h]
dest
virtualenv: error: argument dest: the destination . is not write-able at /
=============================== warnings summary ===============================
tests/helpers.py:139
/build/python-poetry/src/poetry-1.1.2/tests/helpers.py:139: PytestCollectionWarning: cannot collect test class 'TestApplication' because it has a __init__ constructor (from: tests/console/commands/test_init.py)
class TestApplication(Application):
-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/console/commands/test_export.py::test_export_exports_requirements_txt_file_locks_if_no_lock_file
======= 1 failed, 609 passed, 4 skipped, 1 warning in 113.99s (0:01:53) ========
Did you forget to activate?
bnavigator
on 21 Oct 2020
Why would I need to activate, a function that simply adds the venv python executable to the $PATH, if I explicitly invoke /path/to/venv/python?
eli-schwartz
on 21 Oct 2020
Note: I've gotten this in 1.1.2 and 1.1.3, but I did not try any betas or git master...
eli-schwartz
on 21 Oct 2020
Not recommending that. What I am saying is you need to run your tests in a virtual environment. We do not really support running the test suite under the system interpretor.
From the package maintainers' point of view running the test suite using the system interpreter (and not in the virtual environment) is exactly what you want to do. This way you can test whether the software behaves correctly under the exact Python version that is currently packaged with all needed dependencies (that often have some downstream patches).
Eli by using arguments like --system-site-packages and --without-pip is effectively trying to create a virtual environment that behaves like there was none.
mpolanski
on 21 Oct 2020
I don't know, maybe because the quirky poetry test suite internals still call the python from the PATH?
bnavigator
on 21 Oct 2020
Well, I tried rerunning the test with:
(
source ./poetrytests/bin/activate
python -m pytest
)
And it had no effect. 1.1.3 still fails in the same place.
eli-schwartz
on 21 Oct 2020
From the package maintainers' point of view running the test suite using the system interpreter (and not in the virtual environment) is exactly what you want to do. This way you can test whether the software behaves correctly under the exact Python version that is currently packaged with all needed dependencies (that often have some downstream patches).
The project test suite is not necessarily developed with distro packaging requirements in mind, but rather with the intent to test the project's functional requirements. What this means is that certain assumptions about how it is run might have crept in that might not work within the a distro's package build environment. The test suite, for example, tries to assess that the system interpreter (which to poetry is the interpretor that starts poetry) site can be modified and a script can be installed in it's prefix (side-effect of another test case that installs a package as a pre-requisite). However, note that this is a functional test, and the failure is expected when the test suite is run with a user that does not have the privilege to modify the site - as will the real world scenario.
I am not suggesting that this cannot be improved. However, we do not, at this time, have the resources/bandwidth to ensure that the test suite itself works outside the current recommended development environment. We would definitely appreciate pull requests to make the situation better as these build environments are not readily accessible for us. Things might also get better with #3107 and #3255.
abn
on 22 Oct 2020
- on the 1.1.3 release, even running poetry's tests from inside a virtualenv in which the python command and
site.getsitepackages()andsysconfig.get_paths()["purelib"]are all writable, still fails with:virtualenv: error: argument dest: the destination . is not write-able at / - poetry's recommended install method uses the system interpreter which it cannot write to, so this assumption should be tested by the testsuite too, i.e. test if users can get-poetry.py (or
pip install --user poetrytoo, frankly) and still be able to use poetry as expected... This is not really just about distros.
It sounds like #3107 would be a critically important improvement.
eli-schwartz
on 22 Oct 2020
- on the 1.1.3 release, even running poetry's tests from inside a virtualenv in which the python command and
site.getsitepackages()andsysconfig.get_paths()["purelib"]are all writable, still fails with:virtualenv: error: argument dest: the destination . is not write-able at /
Honestly, I do not think we can help you here because this is a build environment issue. Our CI works within virtual environments and will work just fine, and even development environments that run as unprivileged users work just fine both under poetry managed venv and tox managed venvs.
Here is a setup with an unprivileged user using system interpreter under alpine that works, maybe it will help.
podman run --rm -i --entrypoint sh python:3.8-alpine <<EOF
set -xe
apk --quiet add build-base libffi-dev openssl-dev git # dependencies requried for cryptography
pip install -q poetry # install poetry to system
adduser -D developer
su - developer
set -xe
git clone https://github.com/python-poetry/poetry.git
cd poetry
poetry config virtualenvs.in-project true
which poetry
poetry install # use system poetry to create venv and install required dependencies
source .venv/bin/activate # manual activation to demonstrate
which python
pytest -q tests/
EOF
- poetry's recommended install method uses the system interpreter which it cannot write to, so this assumption should be tested by the testsuite too, i.e. test if users can get-poetry.py (or
pip install --user poetrytoo, frankly) and still be able to use poetry as expected... This is not really just about distros.
The get-poetry.py installation does not install to system site. It simply extracts the relevant archive into ~/.poetry and adds ~/.poetry/bin to PATH. The dependencies are vendored. The system interpreter is merely used as a run time. Poetry itself does not require the site to be writeable. The case where it needs to be is if a user, using the system interpretor decides to run poetry install or such on a project with poetry config virtualenvs.create false configured. This is where #3107 is required, and iff the user is an unprivielged user. This scenario is not necesarily common. Cases where the poetry config virtualenvs.create false is used are in containers or ci environmenrs where the user is already root, hence why it does not show up as an issue often, and even in cases where it does it is because it is used incorrectly. It is important to note that, this is not because poetry's functionality is broken.
It sounds like #3107 would be a critically important improvement.
The change proposed in #3107 is to add pip install --user like behaviour for 1.1. That is a new feature. Further, if you are using distro packages for dependencies you would not even encounter this issue as this has to do with how poetry deals with system site when installing editable packagees (ie. the project under development) - this is not the case as I imagine the package (poetry) would already have been installed prior to testing preferrably using a wheel or sdist.
abn
on 23 Oct 2020
Here is an example where #3255 and #3107 (added some additional fixes to support these cases) are applied, without a virtualenvironment and test run by an unprivileged user.
Using alpine as the example since I am guessing that is the one with build failures still based on above comments.
podman run --rm -i --entrypoint sh python:3.8-alpine <<EOF
set -xe
apk --quiet add build-base libffi-dev openssl-dev git
pip install -q 'pytest>=5.4.3,<5.5' 'pytest-mock>=1.9,<2' pytest-cov pytest-sugar httpretty
cd /opt
git clone https://github.com/python-poetry/poetry.git
cd poetry
git fetch origin pull/3255/head:3255
git checkout 3255
wget https://patch-diff.githubusercontent.com/raw/python-poetry/poetry/pull/3107.diff
git apply 3107.diff
pip install -q .
adduser -D developer
chown -R developer:developer /opt/poetry
su - developer
cd /opt/poetry
pytest tests/
EOF
FYI:
The case where it needs to be is if a user, using the system interpretor decides to run poetry install or such on a project with poetry config virtualenvs.create false configured. This is where #3107 is required, and iff the user is an unprivielged user. This scenario is not necesarily common.
Here is the scenario for rpmbuild on the openSUSE build service. I imagine all RPM based distributions (Fedora and family) are similar. Also Debian and Archlinux in their build and packaging tools.
- All requirements have to be installed before the build
%build,%installand%checksections run as unprivileged userabuild- The working directory is
/home/abuild/rpmbuild/BUILD/<name-version-arch>/<pkg-archiveroot>/ %installinstalls everything into/home/abuild/rpmbuild/BUILDROOOT/<name-version-arch>/, The desired system site-packages has to be created under this tree.- no network connection
- root only for installing the RPM package into the system
So if a project decides to require full poetry for end-user installation, it would be great for poetry to support
- [ ] run as unprivileged user without a virtual environment
- [ ] command line option to specify target installation root
- [ ] command line option to not try installing or updating any requirements, just fail if they are not in system site-packages
PIP as PEP517 frontend with poetry-core backend can deal with this using the "build wheel and install it into BUILDROOT approach": poetry-core-obs_log.txt
The dephell approach of converting pyproject.toml to setup.py is a workaround, but it brings its own problems.
I know that this defeats the intention of poetry.
bnavigator
on 23 Oct 2020
@bnavigatorI think with the above patches your build should work without internet connectivity as well as without a virtual environment.
So if a project decides to require full poetry for end-user installation, it would be great for poetry to support
I do not understand this bit. If I understand correct, if python-package-a depends on python-package-b, b is installed systemwide as a seperate package when a gets installed. So, I fail to understand the need to install to a specific site.
I feel that there is some misunderstanding here on what poetry is intended for. For cases where you are building a package for an application or library, I believe you must follow PEP 517 build by using a PEP 517 frontend. The only reason you will ever need poetry to install a package as an end-user, is if you are running an application with pinned dependencies (poetry.lock). This is not something I would recommend when it comes to packaging for distros.
- run as unprivileged user without a virtual environment
Already possible with the above PRs.
- command line option to specify target installation root
This is something that will arrive with bundling (but not exactly as you expect it to). That said you shouldn't have to use it because poetry build produces the wheel required, and this is what should be used using existing mechanisms to generate your build root.
- command line option to not try installing or updating any requirements, just fail if they are not in system site-packages
Can't you just not use the poetry install command? If all requirements are already installed you do not need poetry at all, a pip install src/ should suffice.
abn
on 23 Oct 2020
@bnavigatorI think with the above patches your build should work without internet connectivity as well as without a virtual environment.
As poetry currently builds, installs and tests fine (with https://github.com/python-poetry/poetry/issues/1645#issuecomment-713179414), I see no reason to merge the patches before you make a new release.
So if a project decides to require full poetry for end-user installation, it would be great for poetry to support
I do not understand this bit. If I understand correct, if
python-package-adepends onpython-package-b,bis installed systemwide as a seperate package whenagets installed. So, I fail to understand the need to install to a specific site.
Only the python-package-a needs to be installed into the rpmbuild BUILDROOT during packaging. Everything else must be provided by the system.
I feel that there is some misunderstanding here on what poetry is intended for. For cases where you are building a package for an application or library, I believe you must follow PEP 517 build by using a PEP 517 frontend. The only reason you will ever need
poetryto install a package as an end-user, is if you are running an application with pinned dependencies (poetry.lock). This is not something I would recommend when it comes to packaging for distros.
Can't you just not use the
poetry installcommand? If all requirements are already installed you do not needpoetryat all, apip install src/should suffice.
Exactly. So any project deploying their sdist package to PyPI should never depend on poetry install to install for end-users. (Side note: Sometimes we have to use the GitHub source archive instead of the PyPI sdist, because they do not package the test files. Another layer of complication in case only the sdist creation makes it independent of poetry)
bnavigator
on 23 Oct 2020
@bnavigator that fix (#3107) is not in master yet.
It is now in the release of 1.1.4, right? Unfortunately, the 1.1.4 tagged source code continues to fail the test mentioned above:
I'm down to fewer failures, but it still seems to be trying to create
/foo/virtualenvs/simple-project-CDKM9e_d-py3.8:=================================== FAILURES =================================== _______ test_export_exports_requirements_txt_file_locks_if_no_lock_file ________
The only change is in the random identifier, which is now /foo/virtualenvs/simple-project-heHRHPuF-py3.8...
eli-schwartz
on 13 Nov 2020
Related issues
ulope
·
3Comments
ambv
·
3Comments
mozartilize
·
3Comments
ghost
·
3Comments
probablykasper
·
3Comments
Most helpful comment
From the package maintainers' point of view running the test suite using the system interpreter (and not in the virtual environment) is exactly what you want to do. This way you can test whether the software behaves correctly under the exact Python version that is currently packaged with all needed dependencies (that often have some downstream patches).
Eli by using arguments like
--system-site-packagesand--without-pipis effectively trying to create a virtual environment that behaves like there was none.