Click: The leftover arguments are empty in Click 6.0 Context

[ivankravets]

According to documentation args were accessed until Click < 6.

args = None
the leftover arguments.

Is this API removed in 6.0?

[mitsuhiko]

In which situation?

[ivankravets]

Here ctx.args is equal to [] for any commands. For example, platformio serialports monitor --baud 115200.

[mitsuhiko]

So that you got args for children there was a bug. Do you need that? nargs > 1 still captures that.

[ivankravets]

So that you got args for children there was a bug

Exactly, I need args for children.

[ivankravets]

For example,

platformio -f -c eclipse run ...`

I just need run .... Do you have any ideas?

[ivankravets]

Do you need that? nargs > 1 still captures that.

@mitsuhiko don't understand what I should change. nargs option applies to arguments "type". However, the question was how to receive leftover arguments(including options, like valid portion of sys.args) from context using MultiCommand approach.

P.S: I have blocked issue for it https://github.com/platformio/platformio/issues/349 and reduced temporary Click to <6.

Thanks in advance!

[mitsuhiko]

Are you looking for this maybe? http://click.pocoo.org/5/advanced/#forwarding-unknown-options

[ivankravets]

Test code

@click.command(cls=PlatformioCLI,
               context_settings=dict(
                   help_option_names=["-h", "--help"],
                   ignore_unknown_options=True
               ))
@click.version_option(__version__, prog_name="PlatformIO")
@click.option("--force", "-f", is_flag=True,
              help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c", help="Caller ID (service).")
@click.pass_context
def cli(ctx, force, caller):
    print ctx.args   

Click < 6

$ platformio -f -c test  list
['list']
....

$ platformio -f -c test run -h
['run', '-h']
...

Click 6

$ platformio -f -c test run -h
[]

$ platformio run --target upload
[]
...

What is wrong in this code? Thanks.

[mitsuhiko]

I will have a look at that.

[orome]

I'm getting downgrade to click 5.1 (by 2.7.1).

[ivankravets]

@orome It's ok. @PlatformIO requires some functionality which doesn't work in Click 6. See related issue https://github.com/platformio/platformio/issues/349

[mitsuhiko]

Just to leave some comments on this: I would like to somehow undo parts of Click 6 because it just does not work that way. Not entirely sure what the course of action is :(

[ivankravets]

@mitsuhiko do you have any news?

[mitsuhiko]

Not yet, for now please use Click 5 if you need this.

[paxnovem]

I have been trying to reproduce this issue, I have attempted to use Click 5.0, 6.0, 7.0 on both Python 2.7, Python 3.6. and I am getting consistent behavior between versions.

The following code is what I was using to reproduce the issue. If someone is experiencing this, please let me know how to modify this example to hit the broken behavior.

import click

@click.command(context_settings=dict(
                ignore_unknown_options=True,
                allow_extra_args=True
            ))
@click.option("--force", "-f", is_flag=True,
            help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c")
@click.pass_context
def cmd(ctx, force, caller):
        print ctx.args

if __name__ == '__main__':
        cmd()

[jcrotts]

Thanks @paxnovem the above example seems to be working as expected in Click 7 with python 3.7 as well. Please reopen if you are still experiencing the issue.

[ivankravets]

The bug still persists. You don't have access to leftover arguments from any place/subcommands. See updated example to reproduce this issue:

test.py

import click


@click.group(context_settings=dict(ignore_unknown_options=True,
                                   allow_extra_args=True))
@click.option("--force",
              "-f",
              is_flag=True,
              help="Force to accept any confirmation prompts.")
@click.option("--caller", "-c")
@click.pass_context
def cli(ctx, force, caller):
    print(ctx.args)
    # do some preprocessing with leftover args... Do we have `--json-output`? etc.


@cli.command()
def run():
    pass


if __name__ == '__main__':
    cli()

Click 5.0

test.py -f -c foo run

['run']

Click 7.0

test.py -f -c foo run

[]

[jcrotts]

Quite right the two use cases above are slightly different.

I think this issue targets your use case.
https://github.com/pallets/click/issues/347

[ivankravets]

Can you reopen it?

[jcrotts]

Sure, have you been able to find a work around for the behavior you want?

[ivankravets]

@jcrotts Thanks for reopening! The only solution is to parse manually sys.argv which loses sense in using click at all.

Is it difficult to fix it? The issue arose with Click 6.0 release 4 years ago.

[jcrotts]

This seems to be the commit that changed this behavior.
https://github.com/pallets/click/commit/0a2919f34fcbc635d8530b4c5b60bf119b2bcedb

[ivankravets]

You are right, thanks! I removed these lines https://github.com/pallets/click/blob/master/click/core.py#L1122:L1123 and everything works now with Click 7.0.

@mitsuhiko can we remove these L1122:L1123 lines which reset context args?

[jcrotts]

So actually just removing the 2nd line seems to revert the behavior, and then changing the print to print(ctx.protected_args).

Tests pass after the change, but I think this could be overriding a design decision that I'm not fully aware of.

[ivankravets]

I find some workaround. We can actually overload command.invoke() and backups args:

class PlatformioCLI(click.MultiCommand):

    def invoke(self, ctx):
        ctx._args = ctx.args
        if hasattr(ctx, "protected_args"):
            ctx._args = ctx.protected_args + ctx.args
        return super(PlatformioCLI, self).invoke(ctx)

In this case, args will be in ctx._args. I may work in our case. The problem can occur when you need to get leftover arguments in subcommand. In this case, they will be empty.

So, what to do with this issue?

[ivankravets]

http://click.palletsprojects.com/en/7.x/api/#click.Context.args

args = None
the leftover arguments.

It looks like a critical bug.

Another workaround could be ctx.leftover_args and set it to ctx.protected_args + ctx.args before reseting.

[jcrotts]

Seems related to https://github.com/pallets/click/issues/1323