Poetry: Poetry install no longer removes dev dependencies

[bertjwregeer]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [ ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MacOS
• Poetry version: 0.12.2

Issue

\
There is currently no way to go from a develop poetry install to one that has --no-dev.

In previous versions, if you ran poetry install --no-dev it would remove all dev dependencies from the current virtual environment.

[sdispater]

Yes, I can definitely reproduce.

Was it working in 0.11.5? If yes, it's definitely a change made in the resolver that's causing this issue.

[bertjwregeer]

This was working on the 0.11 series at one point, I remember using it.

[sdispater]

Commit 76801fd should fix the issue. I will try to make a bugfix release as soon as possible.

[sdispater]

This is fixed in the latest release.

[goosemo]

This behavior seems like a bug, especially in how --no-dev is documented.

--no-dev: Do not install dev dependencies

This doesn't state that poetry will uninstall anything. Can this be addressed with docs and perhaps a flag or configuration to not have this behavior happen?

My use case for example is to not have poetry uninstall every python package I have installed inside a docker container, or a developer machine, when it is configured to not use virtualenvs.

[prokher]

@goosemo As for me it is looks like a bug as well. Consider the following use-case.

In my Dockerfile I install 2 Python packages with poetry install --no-dev. The first one requires a package A, while the second one has this package A in it in the [tool.poetry.dev-dependencies] section of pyproject.toml. As a result, the second poetry install --no-dev invocation removes the package A from the container and the first package breaks.

Probably poetry install indeed shall behave like this by default (e.g. for the backward compatibility), but there must be a command line argument to not remove anything already installed.

@sdispater I am not sure if this is OK to comment on the closed issues in your repository, please tell if it is necessary to create a new one.

[laike9m]

+1 on previous comments. Having no way to turn off this behavior of removing packages is not ideal, and I believe I have a legit use case to prove it.

I'm using Poetry for dependency management and Tox for testing.

The depdencies I have are like this:

[tool.poetry.dependencies]
python = "^3.7.0"
pygments = "^2.6.1"
crayons = "^0.3.0"
deepdiff = "^5.0.0"
grpcio = "^1.30.0"
protobuf = "^3.12.2"
shortuuid = "^1.0.1"
attrs = "^19.3.0"
get-port = "^0.0.5"
jsonpickle = "^1.4.1"
typing-extensions = "^3.7.4"

[tool.poetry.dev-dependencies]
tox = "^3.14"
tox-pyenv = "^1.1"
pytest-xdist = "^1.31"
pytest = "^5.4.3"
pyhamcrest = "^2.0.2"
dephell = "^0.8.3"
bytecode = "^0.11.0"
grpcio-tools = "^1.30.0"

Not every dev dependency is needed in tests, for example grpcio-tools, is not needed. So here's my tox.ini file:

[tox]
isolated_build = true
envlist = py37,py38

[testenv]
commands =
    pip install poetry
    poetry config virtualenvs.create false
    poetry install --no-dev
    pip install pytest pytest-xdist pyhamcrest

    py.test -s -vv {posargs} {toxinidir}/test/ --assert=plain

Because I don't want to install every dev dependency, I first poetry install --no-dev, then manually install needed dev dependencies, including pytest, pytest-xdist, pyhamcrest.

The result is, every time test is run, Poetry removes all dev dependencies because of the default behavior. They are then installed by pip. The uninstalling process made tests very slow. If there's a way to turn it off, those packages can be cached and tests can run a lot faster.

[jdahm]

@laike9m I have this same problem and worked around it for now by exporting the set of non-dev dependencies to a requirements.txt file format, and installing those with pip. My project's noxfile.py now start with:

import tempfile
from contextlib import contextmanager

import nox

@contextmanager
def requirements_from_poetry(session, dev=False):
    """Return a requirements.txt file with dependencies from Poetry."""
    with tempfile.NamedTemporaryFile() as tmpfile:
        args = [
            "poetry",
            "export",
            "--format=requirements.txt",
            f"--output={tmpfile.name}",
        ]
        if dev:
            args += ["--dev"]

        session.run(*args, external=True)
        yield tmpfile.name


def install_with_constraints(session, *args, **kwargs):
    """Install packages constrained by Poetry's lock file."""
    with requirements_from_poetry(session, dev=True) as requirements:
        session.install(f"--constraint={requirements}", *args, **kwargs)


@nox.session(python="3.8")
def tests(session):
    """Run the test suite."""
    args = session.posargs or ["--cov", "-m", "not e2e"]
    install_with_constraints(
        session, "coverage[toml]", "pytest", "pytest-cov",
    )
    with requirements_from_poetry(session) as requirements:
        session.install("-r", requirements)
    session.install(".")
    session.run("pytest", *args)

[laike9m]

@jdahm Thanks for showing the workaround. I still have faith in Poetry that this will eventually be supported. @sdispater any thoughts?