Click: Allow cli to double as python API

[chadrik]

It would be very useful if the python module that defines the click command-line interface could also double as a python api.

Consider your repo.py example; I'd like to do something like this:

import repo
repo.setuser(repo.Repo('/var/tmp'), 'chad', '[email protected]', 'so_amaze')

The problem is that the decorated functions become Command instances, whose __call__ method expects different args, processes sys.argv and calls sys.exit, etc, so the above attempt fails.

I dug through the code a bit and it seems like I could solve this problem by subclassing core.Group to implement a new command() decorator that returns the original, undecorated function:

    def command(self, *args, **kwargs):
        """A shortcut decorator for declaring and attaching a command to
        the group.  This takes the same arguments as :func:`command` but
        immediately registers the created command with this instance by
        calling into :meth:`add_command`.
        """
        def decorator(f):
            cmd = command(*args, **kwargs)(f)
            self.add_command(cmd)
            return f # <--- return original function
        return decorator

It seems to me that it could be quite useful if the default behavior for Group.command was to return the original function: since the Group object is the entry-point of the CLI it seems unnecessary that the individual functions can also serve as entry points.

In case it gives some perspective, celery has a similar decorator-based API, and the default behavior for the __call__ method of its tasks is to execute the original function:

@app.task
def add(x, y):
    return x + y

add(1, 3)  # runs immediately in this process
add.delay(1, 3) # runs on a slave

Is this a change that you think should be an official part of click? Do you see any problems with this approach, or a way to tweak it to make it more acceptable?

[mitsuhiko]

Commands intentionally should not act as Python API. The whole point of click is that it defines command line scripts that are composable. By making them return functions instead of commands it misses half that feature.

Group.command is _not_ the main way to use these things. click.command is. The former is just a special case for one very specific usecase which is hopefully not the default one.

[chadrik]

Ok, forget about returning the original function from Group.command. Bad idea.

Could there not be some configuration variable or decorator arg which makes Command.__call__ call Command.callback instead of Command.main? Then the user can simply call e.g. cli.main() to call their function as a command-line entry point, or cli() to call the original function.

Based on my perusal of the source code, click does not internally rely on the fact that the Command.__call__ calls Command.main. This is just a convenient entry-point for users to initiate their CLI, typically from the if __name__ == '__main__' section of their module.

[chadrik]

For example:

import click

@click.command()
@click.api
@click.option('--count', default=1, help='number of greetings')
@click.option('--name', prompt='Your name',
              help='the person to greet', required=True)
def hello(count, name):
    for x in range(count):
        print('Hello %s!' % name)

if __name__ == '__main__':
    hello(2, 'chad')
    hello.main()

[mitsuhiko]

I'm not a fan of this for various different reasons. The biggest one is that the calling conventions for the function is not that simple. It might accept a context for instance. As such the invocation of callbacks is done through Context.invoke which would require you to have a context object to be universally applicable. However at that point you need to construct it which is non trivial.

Maybe it makes sense to add this at a later point if there are some good use cases for this. Currently I really don't think it's something that would help much. Out of one command you can already invoke another with ctx.invoke.

[chris-hailstorm]

Click is a marvelous tool. It would be lovely to be able to reuse bits of it for non-CLI applications.

The pattern of creating a CLI wrapper to a Python callable that doesn't assume an execution style (could be in CLI, WSGI, JSONRPC, whatever) has to be pretty common. Is there a way to use the Click option / argument validators for more than just the CLI? I think that's the most necessary bit. Dumb example:

def core_function(a):
    ... do stuff ....
    return object  ## friendly for Python caller

@click.command(...)
@click.option('-a', type=click.Path(file_okay=True, dir_okay=False), default='-', prompt=True)
def core_cli(a):
    for o in core_function(a):
        print o  ## friendly for Unix shell

If core_function is called via core_cli, the Click validations will be enforced. But if core_function is imported then called via a different Python module, none of the validations will be enforced. If the same rules should apply, we have to create a pretty un-DRY approach to defining and implementing rules.

I'm imagining something like this:

a_rules = click.validator(type=click.Path(file_okay=True, dir_okay=False), default='-')

@click.validate('a', **a_rules)
def core_function(a):
    ...

@click.command
@click.option('-a', prompt=True, **a_rules)
def core_cli(a):
    ...

Valuable because (a) function is callable via CLI but not just CLI; and (b) entirely DRY.

[Wilfred]

I have a use case for this: writing tests for my click scripts. I end up defining shim functions and decorating those:

def main_inner(x, y, z):
    """An extra function that I can explicitly test.

    """
    pass


@click.command()
@click.option(...)
@click.option(...)
@click.option(...)
def main(x, y, z):
    main_inner(x, y, z)

Have I missed a better way of doing this? I've read the click docs, but I didn't see anything relevant.

[untitaker]

Yes, that is the way to do it.

[eyalev]

I have a Click CLI app and I want to use the same code and options in a Slack bot. Is there a way to call the main Click app method from Python code with the same string I would use in the command line?

[jwilson8767]

Here's the hackiest way to shim this I could come up with:

def unclick(f):
    """aliases function as _function without click's decorators"""
    setattr(sys.modules[f.__module__], '_' + f.__name__, f)
    return f

@click.command()
@click.option('--message')
@unclick
def myfunction(message):
    print(message)

_myfunction('shim worked!')

The best solution would be to have Command expose the original, undecorated function as Command.inner_callback, which could then be used like this:

@click.command()
@click.option('--message')
def myfunction(message):
    print(message)

myfunction.inner_callback('no need to shim!')

My use case is that I want my cli and python apis to live together (without unnecessary shim functions).

Edit: Turns out Command already exposes the callback as 'callback'. So calling myfunction.callback('no need to shim!') already works just fine! Leaving this here to help others looking for a way to call inner functions directly.

[Wilfred]

I learnt today that there is a runner for testing in click:

from click.testing import CliRunner

runner = CliRunner()
result = runner.invoke(your_func, ['your', 'args', 'here'], catch_exceptions=False)

assert result.exit_code == 0

[ghost]

@jwilson8767 Thanks for pointing out that myfunction.callback is exposed. As @mitsuhiko says above this does make it hard to use a command that has been decorated with @click.pass_context.

I use calls to ctx.echo(ctx.get_help()) when catching certain exceptions, which displays the formatted text same as --help when some unexpected options or combinations are used. There's some other useful context methods too.

I tried just assigning ctx = click.get_current_context() inside the function, which doesn't work. If I import the module in some other code and run e.g. mymodule.main.callback(args...) it complains that There is no active click context.

I do like @Wilfred 's solution though, where the Python API is the core and click is just one possible interface, nice. I just can't use any click context methods in the undecorated function. Perhaps I can do some extra handling in the click function interface and then defer to the undecorated core function. In reality the ability to print the --help is meaningless from a Python API perspective anyway, it's only meaningful when invoking from the cli. That's probably true for all the click context methods http://click.pocoo.org/5/api/#context

[jwilson8767]

@davoscollective You could also do checks for if ctx is not None: ctx.get_help()

[ghost]

@jwilson8767 Thanks that would have been a simple solution, but I've gone with what @Wilfred suggested and it works well. Specifically I am using it for some encryption of secrets where I want to be able to encrypt/decrypt some credentials in my development environment from the command line and also import the same function in an application. This is what I am now using : https://gist.github.com/davoscollective/4f28d0e1516bf158720f8ad6339bc510
I have both python 2 (unfortunately) and 3 projects so it works in both.

This example hows that there is a decent way to achieve this requirement, and justifies the closing of this issue. Hope it helps others looking to do the same.