Poetry: Poetry remove should not need -d argument

[caleb15]

• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] I have searched the documentation and believe that my question is not covered.

Feature Request

I was very surprised to find out I need to specify -d to remove dev dependencies. Unless there's a reason I'm unaware of, removing a package should remove it regardless of whether it is a dev dependency or regular dependency.

Implementing this would bring poetry in line with other package managers like npm or pip that allow you to remove packages regardless of their status.

[mathieucaroff]

I think the stakes here are mostly about the principle of least surprise.

Allowing to uninstall packages without the --dev argument would reduce the surprise for developers coming from npm, pip and yarn, however, poetry should keep the --dev flag for retro-compatibility. But this means we have a useless --dev flag for remove, which is a surprise. -- Note however, that none of the surprises are dangerous, in the sense that none of them "defies expectations".

In my opinion, the interface of poetry should stay the same. However, when a user tries to uninstall a dev dependency without --dev, poetry should tell them the correct command line to remove the dependency.

Currently:

$ poetry remove autopep8 yapf

[ValueError]
Package autopep8 not found

It'd be nice to have a hint:

$ poetry remove autopep8 yapf

[ValueError]
Package autopep8 not found

autopep8 is installed as a developer dependency.
Use `poetry remove --dev autopep8` to remove it.

[caleb15]

But this means we have a useless --dev flag for remove, which is a surprise

A useless flag is less surprising than a required flag. A required flag means people have to use it, which means more surprises. A useless flag is only a surprise if looking through the documentation. After a while of it being deprecated it can be removed.

It's also easier for the developer if they don't have to specify --dev.

[caleb15]

Actually let me restate, I don't feel like I said that clearly. Following the principle of least suprise, it would be better to not require the --dev flag.

Requiring the --dev flag: People are very surprised. Regardless of whether you have a message telling them what to do, there's still the initial surprise and a different behavior that they have to remember.

Not requiring: People are only slightly surprised when looking through the documentation

[mathieucaroff]

Your least-surprise quantification argument actually convinced me.

I wonder if we should aim to remove the --dev flag from the CLI, or keep it to reduce friction.

After pondering a bit, I think we should aim to remove it.

Side note:
I just checked what how yarn remove did it and I figured yarn (1.22.0) simply ignores all unknown flags (>~<).

[bhaveshpraveen]

I'd like to work on this issue.

[abn]

@bhaveshpraveen were you still thinking of working on this?

[bhaveshpraveen]

@abn I did not receive any response so I didn't pick it up then. If it's alright, I'd like to pick this up now. I'll be able to work on this over the weekend.

[silverwind]

Here's my thoughts. I see two options to resolve:

1. Ignore the --dev argument on remove like other package managers and remove the dependency from both types. This is my preference because when removing a package, I do not want to think about where it was defined. If the same package is defined in both, remove it in both.

2. Provide a more helpful warning or a prompt when a package is in another category then derived from the --dev argument. Not my preference, but better then the current "not found" error.

[abn]

@bhaveshpraveen sure go for it; as discussed here and elsewhere, the expected solution is similar to (2) as described by @silverwind.

1. If --dev is provided, only attempt to remove development dependency.
2. If --dev is not specified and
   a. dependency does not exist in in main, but exists in dev raise a confirmation prompt. Note that the prompt must only be a warning if non-interactive mode is used. (@finswimmer we could also just remove this without a prompt since there is no ambiguity here, unsure about when it exists in both - if removing both might upset existing use cases)
   b. dependency exists in main, only remove from main.

The aspect to remember is that a dependency can existin in both dev and main categories at the same tiem. A user must be able to operate on these individually - this means remove main and remove dev independently if both are present.

[mathieucaroff]

I like @silverwind's Ignore solution, (1. Ignore the --dev argument on remove), because I believe that the scenario of editing a command and not taking the time to remove the --dev option is the most common.

I do not see the point of installing a package both as a dependency and a dev-dependencies. All dependencies are also dev-dependencies since a developer needs the dependencies to test the code. As such, I believe the support for the use case of having a package installed both as a dependency and a dev-dependency ("installed as both") should be extremely limited. Actually, it is quite unfortunate that poetry supports adding a package both as dependency and as dev-dependency (does it?) ("adding as both").

[silverwind]

Actually, it is quite unfortunate that poetry supports adding a package both as dependency and as dev-dependency (does it?)

It will add to both but I think only the last added one is the one that will be the active one. Adding a package twice is always an error, site-packages does not support multiple versions of one package so it's always a error.

the expected solution is similar to (2)

I'd say the general user expectance is option 1 and that's how prior arts like pip, yarn, npm work too.

[finswimmer]

We cannot ignore the --dev parameter for poetry remove. As @abn said, poetry supports that one have a dependency in both section.

A use case for this would be for example, you are writing a package, which is a wrapper around black. So you define black as a dependency. At the same time, you want to pin black for a specific version during development to format your code. So you define black as a dev dependency with the pinned version.

While this wouldn't make any difference during development (poetry will always install the pinned dependency), it makes a different when you build a wheel/sdist which get's installed somewhere else.

So I agree with @abn suggestions. If --dev is provided, only remove from dev. If --dev is not provided, remove from any section if it's only in one section, otherwise raise an exception or ask the user if they want to completely remove the package.

fin swimmer

[bhaveshpraveen]

@finswimmer @abn

To summarize

1. If --dev is provided, remove only from dev section.
2. If --dev is not provided and dependency is present in only one section, then remove it from that section.
3. If --dev is not provided and dependency is present in both the sections, raise an Exception

Is that correct?

[finswimmer]

If --dev is not provided and dependency is present in both the sections, raise an Exception

Now that I read it like this, we should maybe ask the user, from which section the dependency should be removed (dev, non-dev, both). Otherwise we end up in a situation, where we cannot remove a dependency at all.

[bhaveshpraveen]

Ok, got it.

Just a thought, I was thinking maybe we could add a separate flag, say something like --all or --both, that removes the dependency from both the sections. This way the user can remove the dependency without having to depend on the prompt(interactive input) from poetry.

[bhaveshpraveen]

@abn @finswimmer

Hey, i've coded it up. I couldn't find any tests for remove operation (Maybe I missed it?) To add tests, is there any snippet/example that I can use for reference? And do I have to make changes in the documentation as well?