Click: Windows "Usage" incorrect display and difficulty to get a command name correct displayed

Created on 23 Jun 2015 · 25Comments · Source: pallets/click

When you create a command with a setuptools console_scripts entry_points (for instance named mycli) the Usage displayed on POSIX (Linux, Mac,...) when running mycli --help will be:
Usage: mycli

In contrast on Windows, I get the following:
Usage: mycli-script.py

This is annoying because the correct invocation on windows is to use mycli, not mycli-script.py.
The script is the same on all OSes, even though behind the scenes setuptools creates:

on posix, a bin/mycli python script with a correct shebang
on windows, a Scripts/mycli-script.py and Scripts/mycli.exe. This is called by invoking mycli like on POSIX

The obvious way to force this would be to use the name arg in click.command. ("This defaults to the function name."). But it does not default to the function name and always show the mycli-script.py instead of mycli.

It looks like this comes from Context.info_name which is computed somehow, but I could not find a way to get this info_name forced to something fixed.

bug

Source

pombreda

All 25 comments

You're looking for prog_name. It's taken from sys.argv[0] or __file__ by default:

    if prog_name is None:
        prog_name = make_str(os.path.basename(
            sys.argv and sys.argv[0] or __file__))

I wonder what sys.argv is in your case.

untitaker on 25 Jul 2015

From the stdlib docs:

argv[0] is the script name (it is operating system dependent whether this is a full pathname or not)

https://docs.python.org/3/library/sys.html#sys.argv

I start to doubt that this bug is fixable.

untitaker on 25 Jul 2015

I guess I could overwrite main here: https://github.com/mitsuhiko/click/blob/fe2f525e720a6f71e8ac4079807b64df13063e8a/click/core.py#L611
But this is a tad hackish. There should be either a way to work around the quirks of setuptools generated scripts or have a parameter to allow passing a prog_name
Note that in this commit, I subclass Command.main to force a prog_name to self.name and this seems to work fine for this case:
https://github.com/nexB/scancode-toolkit/commit/586a7c767507c952960632cd07279a2022d72e4d

pombredanne on 25 Jul 2015

This was a problem for me too when trying to activate bash completion because the magic environment variable is __COMPLETE by default. So I tried to work around it, hopefully without doing too much monkey-patching. In my case, the application name is "nx_tools".

So prog_name is actually "nx_tools-script.py", and complete_var is "_NX_TOOLS_SCRIPT.PY_COMPLETE". So I tried executing the following:

_NX_TOOLS_SCRIPT.PY_COMPLETE=source nx_tools > complete.sh

only to get an error:

bash: _NX_TOOLS_SCRIPT.PY_COMPLETE=source: command not found

Bash does not allow periods in variable names.

All in all, not that big a deal, but it could be documented. Would you accept a PR on that?

selimb on 28 Jul 2015

Adding a "me-too" on this issue. I was pointed to this bug report from my StackOverflow question. I will document this issue in the help text until the bug is resolved...

gauden on 13 Sep 2015

Because it was mentioned in the SO thread: You can just call your command with an explicit prog_name argument:

@click.command()
def cli():
    pass

cli(prog_name='foo')

No need for overriding main...

untitaker on 13 Sep 2015

That works, as expected, also when cli() is a click.group and I have updated the SO thread with a reference to your answer. Thanks!

gauden on 14 Sep 2015

I'm not sure if we can fix that easily unless someone has an idea how to autodetect it and magically fix it up.

mitsuhiko on 16 Sep 2015

@untitaker wrote:

You can just call your command with an explicit prog_name argument

When doing so you might end up having to write some wrapper function to inject progname and use that instead as an entry point... only if you care about windows. but that becomes a solution for all OS which is not so nice.

@mitsuhiko wrote:

I'm not sure if we can fix that easily unless someone has an idea how to autodetect it and magically fix it up.

The problem is only when using setup tools entry points on Windows. entry points are IMHO a great thing. One approach, since the -script.py addition is only a Windows+entry point thing, could be to detect if we are on windows and if the script name ends with -script.py then strip -script.py from the progname. There may be more introspection that can be done with pkg_resources too.

This would be hackish, but simple enough.

pombredanne on 16 Sep 2015

This definetly seems hackish because of the possibility that your actual program name ends with that string.

A proper solution would be part of setuptools.

On 16 September 2015 16:12:06 CEST, Philippe Ombredanne [email protected] wrote:

@untitaker wrote:

You can just call your command with an explicit prog_name argument

When doing so you might end up having to write some wrapper function to
inject progname and use that instead as an entry point... only if you
care about windows. but that becomes a solution for all OS which is not
so nice.

@mitsuhiko wrote:

I'm not sure if we can fix that easily unless someone has an idea how
to autodetect it and magically fix it up.

The problem is only when using setup tools entry points on Windows.
entry points are IMHO a great thing. One approach, since the
-script.py addition is only a Windows+entry point thing, could be to
detect if we are on windows and if the script name ends with
-script.py then strip -script.py from the progname. There may be
more introspection that can be done with pkg_resources too.

This would be hackish, but simple enough.

Reply to this email directly or view it on GitHub:
https://github.com/mitsuhiko/click/issues/365#issuecomment-140753206

Sent from my phone. Please excuse my brevity.

untitaker on 16 Sep 2015

Not happy to just script -script.py. Is there no better way to detect this?

mitsuhiko on 6 Nov 2015

@mitsuhiko The simplest thing that comes to mind since this is a windows only whart would be to detect if we are on windows, and we are using entry points and there is an XXX.exe and the command comes out as XXX-script.py, then use XXX instead. Not pretty but working in practice.

pombredanne on 6 Nov 2015

@pombreda how do you detect if you were invoked from an entrypoint?

mitsuhiko on 6 Nov 2015

@mitsuhiko
This way:

def is_console_script():
    """
    Return True if the current code was loaded from a setuptools console_script
    entry point. This walks the stack frames up to the root frame and checks if the
    pkg_resources.load_entry_point function is at the root. If yes, we were loaded
    most likely from a console script entry point.
    """
    module = 'pkg_resources'
    ep_func = 'load_entry_point'
    frm = sys._getframe()
    func = None
    while frm is not None:
        func = frm.f_globals.get(ep_func, '')
        frm = frm.f_back
    return func and func.__module__ == module and func.__name__ == ep_func

pombredanne on 6 Nov 2015

and @dstufft suggested this regex (flawed for now because it does not catch the .py case ) too : https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417

pombredanne on 6 Nov 2015

I think if the regex got updated in pip, then you wouldn't need to deal with it at all in click.

dstufft on 6 Nov 2015

@dstufft so your suggestion is to fix this in pip/setuptools instead?

mitsuhiko on 6 Nov 2015

Yes, the purpose of that regex is to make it so that console wrappers have the same sys.argv[0] on both Windows and *nix. It's a bug that it currently isn't doing that.

dstufft on 6 Nov 2015

In that case I will close this here.

mitsuhiko on 6 Nov 2015

excellent! much better! @dstufft Thank!

pombredanne on 6 Nov 2015

@dstufft @mitsuhiko actually things are a tad more complex:

if the entry point launcher script was created by pip from a wheel, there are no issue. If using pip after version 6, pip always creates a wheel first even for sdist. wheel never generates a -script.py, only a .exe launcher. There is no bug in the pip wheel.py and changing the regex at https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417 has no impact, because there is no .py created. Actually the regex could be simplified to only strip .exe . In wheels rather a main.py script is bundled inside the exe launcher and looks like this:

__main__.py
# -*- coding: utf-8 -*-
import re
import sys
from scancode.extract_cli import extractcode
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(extractcode())

This is done here in pip:
https://github.com/pypa/pip/blob/1da0ea13f3f0ab789edd62012d09bf69be045f68/pip/wheel.py#L413

if the entry point launcher script was created by pip from a -e requirement or from a setup.py install then this is setuptools easy_install that takes over and it does generate the -script.py. When this happens I do not think there is a way to distinguish things unless we use the function I proposed above unless things are fixed in setuptools too. This how the -script.py looks like:

``` #!c:\dev\scancode-toolkit\Scripts\python.exe

EASY-INSTALL-ENTRY-SCRIPT: 'scancode-toolkit==1.3.8','console_scripts','extractcode'

__requires__ = 'scancode-toolkit==1.3.8'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
sys.exit(
load_entry_point('scancode-toolkit==1.3.8', 'console_scripts', 'extractcode')()
)
```

This is done here in setuptools:
https://bitbucket.org/pypa/setuptools/src/4ce518784af886e6977fa2dbe58359d0fe161d0d/setuptools/command/easy_install.py?at=default&fileviewer=file-view-default#easy_install.py-2008

There yet another launcher in @vsajip distlib, but that does not seem to apply here:
https://bitbucket.org/pypa/distlib/src/6ada86eea2700f10edcd81f831139c6ae20edb74/distlib/scripts.py?at=default&fileviewer=file-view-default#scripts.py-41

pombredanne on 7 Nov 2015

Perhaps @jaraco would be open to adding the same regex to setuptools script wrappers.

Sent from my iPhone

On Nov 7, 2015, at 10:52 AM, Philippe Ombredanne [email protected] wrote:

@dstufft @mitsuhiko actually things are a tad more complex:

if the entry point launcher script was created by pip from a wheel, there are no issue. If using pip after version 6, pip always creates a wheel first even for sdist. wheel never generates a -script.py, only a .exe launcher. There is no bug in the pip wheel.py and changing the regex at https://github.com/pypa/pip/blob/develop/pip/wheel.py#L410-L417 has no impact, because there is no .py created. Actually the regex could be simplified to only strip .exe . In wheels rather a main.py script is bundled inside the exe launcher and looks like this:

main.py

-_- coding: utf-8 -_-

import re
import sys
from scancode.extract_cli import extractcode
if name == 'main':
sys.argv[0] = re.sub(r'(-script.pyw?|.exe)?$', '', sys.argv[0])
sys.exit(extractcode())
This is done here in pip:
https://github.com/pypa/pip/blob/1da0ea13f3f0ab789edd62012d09bf69be045f68/pip/wheel.py#L413

if the entry point launcher script was created by pip from a -e requirement or from a setup.py install then this is setuptools easy_install that takes over and it does generate the -script.py. When this happens I do not think there is a way to distinguish things unless we use the function I proposed above unless things are fixed in setuptools too. This how the -script.py looks like:

EASY-INSTALL-ENTRY-SCRIPT: 'scancode-toolkit==1.3.8','console_scripts','extractcode'

requires = 'scancode-toolkit==1.3.8'
import sys
from pkg_resources import load_entry_point

if name == 'main':
sys.exit(
load_entry_point('scancode-toolkit==1.3.8', 'console_scripts', 'extractcode')()
)
This is done here in setuptools:
https://bitbucket.org/pypa/setuptools/src/4ce518784af886e6977fa2dbe58359d0fe161d0d/setuptools/command/easy_install.py?at=default&fileviewer=file-view-default#easy_install.py-2008

There yet another launcher in @vsajip distlib, but that does not seem to apply here:
https://bitbucket.org/pypa/distlib/src/6ada86eea2700f10edcd81f831139c6ae20edb74/distlib/scripts.py?at=default&fileviewer=file-view-default#scripts.py-41

—
Reply to this email directly or view it on GitHub.