Picocli: usage help for nested sub listing full command tree

[lgawron]

Currently invoking usageHelp for cli with nested subcommands shows only one level of commands.

Would it be possible to generate a full command listing like:

Commands:

• help Displays help information about the specified command
• mobile Manage device
  
  
  
  • dump-config
  
  
  • disable
  
  
• user Manage user
  
  
  
  • nuke
  
  
• jobs Manage jobs
  
  
  
  • list
  
  
  • trigger
  
  
  • disable
  
  
• version Get server version

and so on?

[remkop]

Currently this may not be easy to achieve. There is an item (#530) on the todo list that will facilitate customizations like this.

An implementation of the ideas in #530 should make it possible for application authors to specify a custom ISectionRenderer for the subcommand list that shows the whole hierarchy for each subcommand instead of just one level.

Application authors would control these section renderers via a IHelpFactory.

This work is planned for picocli 4.0. If you’re interested in helping out with this that’d be great, please let me know.

[remkop]

Update: there's a very promising PR in progress (#567) that will facilitate the above customization. If things go well I can do a 3.9 release with this.

[remkop]

530 (custom usage help message section renderers) has been merged to master.

Please try and see if this API allows you to achieve the layout you have in mind.

[bobtiernay-okta]

This would be a very appreciated addition.

[remkop]

I've added an example that shows how to accomplish this customization with the new API introduced in picocli 3.9:

package picocli.examples.customhelp;

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Help;
import picocli.CommandLine.Help.Column;
import picocli.CommandLine.Help.Column.Overflow;
import picocli.CommandLine.Help.TextTable;
import picocli.CommandLine.IHelpSectionRenderer;
import picocli.CommandLine.Model.CommandSpec;
import picocli.CommandLine.Model.UsageMessageSpec;

import static picocli.CommandLine.Model.UsageMessageSpec.SECTION_KEY_COMMAND_LIST;

/**
 * This example demonstrates how to customize a section of the usage help message.
 * It replaces the standard command list with a custom list that displays
 * not just the immediate subcommands but the full hierarchy of subcommands:
 * <pre>
 * Usage: showall [-hV] [COMMAND]
 * Demonstrates a usage help message that shows not just the subcommands of this
 * command, but also the nested sub-subcommands.
 *   -h, --help      Show this help message and exit.
 *   -V, --version   Print version information and exit.
 * Commands:
 *   sub1           subcommand1 of showall
 *     sub1sub1     subcommand1 of subcommand1 of showall
 *     sub1sub2     subcommand2 of subcommand1 of showall
 *   sub2           subcommand2 of showall
 *     sub2sub1     subcommand1 of subcommand2 of showall
 * </pre>
 *
 * As requested in https://github.com/remkop/picocli/issues/566
 */
@Command(name = "showall", mixinStandardHelpOptions = true,
        version = "from picocli 3.9",
        description = "Demonstrates a usage help message that shows " +
                "not just the subcommands of this command, " +
                "but also the nested sub-subcommands.",
        subcommands = {Subcommand1.class, Subcommand2.class} )
public class ShowAll {
    public static void main(String[] args) {
        CommandLine cmd = new CommandLine(new ShowAll());
        cmd.getHelpSectionMap().put(SECTION_KEY_COMMAND_LIST, new MyCommandListRenderer());
        cmd.usage(System.out);
    }
}

class MyCommandListRenderer implements IHelpSectionRenderer {
    @Override
    public String render(Help help) {
        CommandSpec spec = help.commandSpec();
        if (spec.subcommands().isEmpty()) { return ""; }

        // prepare layout: two columns
        // the left column overflows, the right column wraps if text is too long
        TextTable textTable = TextTable.forColumns(help.ansi(),
                new Column(15, 2, Overflow.SPAN),
                new Column(spec.usageMessage().width() - 15, 2, Overflow.WRAP));

        for (CommandLine subcommand : spec.subcommands().values()) {
            addHierarchy(subcommand, textTable, "");
        }
        return textTable.toString();
    }

    private void addHierarchy(CommandLine cmd, TextTable textTable, String indent) {
        // create comma-separated list of command name and aliases 
        String names = cmd.getCommandSpec().names().toString();
        names = names.substring(1, names.length() - 1); // remove leading '[' and trailing ']'

        // command description is taken from header or description 
        String description = description(cmd.getCommandSpec().usageMessage());

        // add a line for this command to the layout
        textTable.addRowValues(indent + names, description);

        // add its subcommands (if any)
        for (CommandLine sub : cmd.getSubcommands().values()) {
            addHierarchy(sub, textTable, indent + "  ");
        }
    }

    private String description(UsageMessageSpec usageMessage) {
        if (usageMessage.header().length > 0) {
            return usageMessage.header()[0];
        }
        if (usageMessage.description().length > 0) {
            return usageMessage.description()[0];
        }
        return "";
    }
}

@Command(name = "sub1", description = "subcommand1 of showall",
        subcommands = {Subcommand1Sub1.class, Subcommand1Sub2.class})
class Subcommand1 {}

@Command(name = "sub2", description = "subcommand2 of showall",
        subcommands = {Subcommand2Sub1.class})
class Subcommand2 {}

@Command(name = "sub1sub1", description = "subcommand1 of subcommand1 of showall")
class Subcommand1Sub1 {}

@Command(name = "sub1sub2", description = "subcommand2 of subcommand1 of showall")
class Subcommand1Sub2 {}

@Command(name = "sub2sub1", description = "subcommand1 of subcommand2 of showall")
class Subcommand2Sub1 {}

[remkop]

@lgawron, can you check if the above meets your needs?

[lgawron]

I will do so and get back to you soon

On Fri, Jan 4, 2019, 09:31 Remko Popma [e-mail] wrote:

@lgawron https://github.com/lgawron, can you check if the above meets
your needs?

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
https://github.com/remkop/picocli/issues/566#issuecomment-451383200, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AApFh29NC4iEJBB7gPxpP3AqSiLuMu6tks5u_xFrgaJpZM4ZLGvO
.

[remkop]

I went ahead and released picocli-3.9.0. :-)

I believe the new API should enable you to make all kinds of customizations in the usage help message, including the requested one. Please let me know how it is working for you. Also, suggestions for further improvements are always welcome!

[bobtiernay-okta]

Yay! 🎉

[bobtiernay-okta]

So I'm trying to give this a spin and I'm hitting up against not having enough context in IHelpSectionRenderer to reproduce what Help#commandList does. Basically what I want to do is add section groupings between commands (probably a common use case). What's the recommended way of doing this in the API, passing in the CommandLine into my IHelpSectionRenderer?

[bobtiernay-okta]

I was able to reproduce with this:

    static class HelpCommandRender implements IHelpSectionRenderer {

        private final CommandLine commandLine;

        public HelpCommandRender(CommandLine commandLine) {
            this.commandLine = commandLine;
        }

        @Override
        public String render(Help parentHelp) {
            Map<String, CommandLine> subcommands = commandLine.getSubcommands();
            if (subcommands.isEmpty()) {
                return "";
            }

            TextTable textTable = createCommandTable(parentHelp, subcommands);
            subcommands.values().stream().forEach(command -> renderHelp(command, parentHelp, textTable));

            return textTable.toString();
        }

        protected TextTable createCommandTable(Help parentHelp, Map<String, CommandLine> subcommands) {
            int commandLength = maxLength(subcommands.keySet());

            return TextTable.forColumns(parentHelp.ansi(),
                    new Column(commandLength + 2, 2, Column.Overflow.SPAN),
                    new Column(commandLine.getUsageHelpWidth() - (commandLength + 2), 2, Column.Overflow.WRAP));
        }

        private void renderHelp(CommandLine command, Help parentHelp, TextTable textTable) {
            Help help = command.getHelpFactory().create(command.getCommandSpec(), parentHelp.colorScheme());

            UsageMessageSpec usage = help.commandSpec().usageMessage();
            String header = !empty(usage.header()) ? usage.header()[0] : (!empty(usage.description()) ? usage.description()[0] : "");
            Text[] lines = help.ansi().text(format(header)).splitLines();

            for (int i = 0; i < lines.length; i++) {
                textTable.addRowValues(i == 0 ? help.commandNamesText(", ") : parentHelp.ansi().text(""), lines[i]);
            }
        }

        private static String format(String formatString, Object... params) {
            return formatString == null ? "" : String.format(formatString, params);
        }

        private static boolean empty(Object[] array) {
            return array == null || array.length == 0;
        }

        private static int maxLength(Collection<String> any) {
            List<String> strings = new ArrayList<String>(any);
            Collections.sort(strings, Collections.reverseOrder(Help.shortestFirst()));
            return strings.get(0).length();
        }

    }

Is this what one would need to do today? Or am I missing something?

[remkop]

It is possible to pass in additional context like you did, but you should be able to get all you need from the passed-in Help object.

For example, you can get the CommandLine instance with help.commandSpec().commandLine().

It’s unfortunate that you have to construct Help objects for the subcommands. These are available from the passed-in Help object with Help.subcommands(), but this method is protected. I’ll make it public in a next release. (Not sure if semantic versioning allows me to do that in a patch release like 3.9.1...)
A workaround may be to let your renderer subclass Help.

[remkop]

Looks like semver says that bumping visibility of the Help.subcommands() method from protected to public requires a minor version increment, and must not be done in a patch release. There are some bug fixes in the pipeline for a 3.9.1 release soon, but I hesitate to rename this to 3.10.0 just for this API change... Is the workaround acceptable?

[bobtiernay-okta]

I couldn't subclass help because it required a command object which I didn't have in this context.

For now I'm just doing a more customized version of the above approach:

/**
     * Prints command listing help to terminal with categorized sections.
     */
    static class HelpRenderer implements IHelpSectionRenderer {

        private final Map<String, String> sections;

        HelpRenderer(Map<String, String> sections) {
            this.sections = sections;
        }

        @Override
        public String render(Help help) {
            CommandLine commandLine = help.commandSpec().commandLine();
            Map<String, CommandLine> subcommands = commandLine.getSubcommands();
            if (subcommands.isEmpty()) {
                return "";
            }

            TextTable commandsTable = createCommandsTable(help, subcommands);
            for (CommandLine subCommandLine : subcommands.values()) {
                String sectionName = sections.get(subCommandLine.getCommandName());
                if (sectionName != null) {
                    commandsTable.addEmptyRow();
                    commandsTable.addRowValues(formatSection(sectionName));
                }

                addCommandHelp(commandsTable, getHelp(subCommandLine, help.colorScheme()));
            }

            return commandsTable.toString();
        }

        private static TextTable createCommandsTable(Help parentHelp, Map<String, CommandLine> subcommands) {
            int commandLength = maxLength(subcommands.keySet());

            return TextTable.forColumns(parentHelp.ansi(),
                    new Column(commandLength + 2, 2, Overflow.SPAN),
                    new Column(width(parentHelp) - (commandLength + 2), 2, Overflow.WRAP));
        }

        private static void addCommandHelp(TextTable commandsTable, Help help) {
            UsageMessageSpec usage = help.commandSpec().usageMessage();
            String header = !isEmpty(usage.header()) ? usage.header()[0] : !isEmpty(usage.description()) ? usage.description()[0] : "";

            Text[] lines = help.ansi().text(format(header)).splitLines();
            for (int i = 0; i < lines.length; i++) {
                commandsTable.addRowValues(i == 0 ? help.commandNamesText(", ") : help.ansi().text(""), lines[i]);
            }
        }

        private static Help getHelp(CommandLine command, ColorScheme colorScheme) {
            return command.getHelpFactory().create(command.getCommandSpec(), colorScheme);
        }

        private static int width(Help help) {
            return help.commandSpec().commandLine().getUsageHelpWidth();
        }

        private static String formatSection(String sectionName) {
            return "@|" + Theme.SECTION + ",bold " + sectionName + "|@";
        }

        private static String format(String formatString, Object... params) {
            return formatString == null ? "" : String.format(formatString, params);
        }

        private static int maxLength(Collection<String> any) {
            return any.stream()
                    .sorted(reverseOrder(Help.shortestFirst()))
                    .findFirst()
                    .orElse("")
                    .length();
        }

    }

Btw, is there any plans to add annotation based versions of sections? I seem to recall that request in another issue.

[remkop]

There is a plan to support @OptionGroup annotations in picocli 4.0. Option groups can be used for logical grouping (mutually exclusive options or the opposite where specifying one option makes another option required) as well as for creating subsections in the options list section of the usage help message.

Is that what you have in mind?

[remkop]

You can access protected methods in Help by subclassing Help like this:

public class Example {
    public static void main(String[] args) {
        CommandLine cmd = new CommandLine(new Example());
        cmd.setHelpFactory(new MyHelp.Factory());
        cmd.getHelpSectionMap().put(SECTION_KEY_COMMAND_LIST, new MyCommandListRenderer());
        cmd.usage(System.out);
    }
}

class MyHelp extends Help {
    public MyHelp(CommandSpec commandSpec, ColorScheme colorScheme) {
        super(commandSpec, colorScheme);
    }

    static class Factory implements CommandLine.IHelpFactory {
        public Help create(CommandSpec commandSpec, ColorScheme colorScheme) {
            return new MyHelp(commandSpec, colorScheme);
        }
    }
    static class MyRenderer implements IHelpSectionRenderer {
        public String render(Help help) {
            Map<String, Help> subcommands = ((MyHelp) help).subcommands(); // access protected method
            return "do stuff...";
        }
    }
}

[bobtiernay-okta]

Thanks! I think this or my approach is an acceptable workaround for now. Although both suffer from some boilerplate so would be nice to trim the fat eventually.

[lgawron]

Hi,
sorry for delay. The sample code you provided works almost perfecty apart from one thing:

Commands:
  help           Displays help information about the specified command
  mobile         Zarządzanie zdalną aplikacją użytkownika
    databases    Pobranie baz danych użytkownika
    config, cfg  Mobilna konfiguracja użytkownika w danej aplikacji
    config, cfg  Mobilna konfiguracja użytkownika w danej aplikacji
    license      Pobranie informacji o licencji użytkownika
      delete     Usunięcie licencji użytkownika
      token      Pobranie auth tokenu dla celów serwisowych
    dump         Wymuszenie wysłania bazy danych przez urządzenie na serwis
    sync         Wymuszenie synchronizacji
  app            Praca z aplikacją mobilną
    download     Pobranie aplikacji
    run          Uruchomienie aplikacji w emulatorze
  user           Zarządzanie użytkownikiem
    nuke         Zablokowanie użytkownika
  jobs           Harmonogram zadań
    trigger      Uruchomienie zadania poza harmonogramem
  closures       Zarządzanie domknięciami
    generate     Wymuszenie przenegerowania domknięcia
    cleanup      Usunięcie starych tabeli domknięć
  contractor     Zarządzanie kontrahentem
    database, db
                 Pobranie bazy contractor history
    database, db
                 Pobranie bazy contractor history
  version        Wersja serwera
  ui             Uruchom web UI

anything with alias is listed twice .... actually if you specify 2 aliases it is listed 3 times.

[bobtiernay-okta]

@lgawron Ya, for my case I was able to make a shortcut. Certainly my approach is not generalizable as is. I was going to create my own annotation to extract the metadata from within the renderer. That would make it more explicit.

You should be able to tweak the map lookup logic to get the semantics you need though.

[lgawron]

ok, I was not aware that command with aliases is registered under each name separately

        for ( CommandLine subcommand : new LinkedHashSet<>( spec.subcommands().values() ) ) {
            addHierarchy( subcommand, textTable, "" );
        }

that did it. I should probably use some identity hash map but I do not want to introduce another dependency.

Thanks!

[remkop]

Great, thank you both for the verification and the feedback!
It sounds like the current API is workable, so I will close this ticket.

Feel free to re-open or raise a new ticket if there's any problem.

[lgawron]

One more question:

this works:

➜  smart-cli git:(master) ✗ ./smart-cli help
Usage: smart-cli [-hVX] <server> [COMMAND]
      <server>    serwer Gemini (adres lub alias)
  -h, --help      Show this help message and exit.
  -V, --version   Print version information and exit.
  -X              Pokaż dokładniejszy debug
Commands:
  help             Displays help information about the specified command
  mobile           # Zarządzanie zdalną aplikacją użytkownika
    databases, db  Pobranie baz danych użytkownika
    config, cfg    Mobilna konfiguracja użytkownika w danej aplikacji
    license        Pobranie informacji o licencji użytkownika
      delete       Usunięcie licencji użytkownika
      token        Pobranie auth tokenu dla celów serwisowych
    dump           Wymuszenie wysłania bazy danych przez urządzenie na serwis
    sync           Wymuszenie synchronizacji
  app              # Praca z aplikacją mobilną
    download       Pobranie aplikacji
    run            Uruchomienie aplikacji w emulatorze
  user             # Zarządzanie użytkownikiem
    nuke           Zablokowanie użytkownika
  jobs             Harmonogram zadań
    trigger        Uruchomienie zadania poza harmonogramem
  closures         # Zarządzanie domknięciami
    generate       Wymuszenie przenegerowania domknięcia
    cleanup        Usunięcie starych tabeli domknięć
  contractor       # Zarządzanie kontrahentem
    database, db   Pobranie bazy contractor history
  version          Wersja serwera
  ui               Uruchom web UI

so the top level help command, but triggering help some node down the command tree does not:

➜  smart-cli git:(master) ✗ ./smart-cli someServer mobile -h
Missing required option '--user=<user>'
Usage: smart-cli mobile [-hV] [-r=<role>] -u=<user> [COMMAND]
# Zarządzanie zdalną aplikacją użytkownika
  -h, --help             Show this help message and exit.
  -r, --appRole=<role>   Rola aplikacji mobilnej: MOBILE, DRIVER, CATALOG
  -u, --user=<user>      Nazwa użytkownika
  -V, --version          Print version information and exit.
Commands:
  databases, db  Pobranie baz danych użytkownika
  config, cfg    Mobilna konfiguracja użytkownika w danej aplikacji
  license        Pobranie informacji o licencji użytkownika
  dump           Wymuszenie wysłania bazy danych przez urządzenie na serwis
  sync           Wymuszenie synchronizacji

as you see the license comand has subcommands - yet my renderer doesn't seem to get involved.

[remkop]

I suspect you are doing this:

CommandLine cmd = // ...
cmd.getHelpSectionMap().put(SECTION_KEY_COMMAND_LIST, myRenderer);

This only impacts the usage help of that command.

To impact the whole hierarchy, you need to replace the map:

CommandLine cmd = // ...
Map<String, IHelpSectionRenderer> map = cmd.getHelpSectionMap();
map.put(SECTION_KEY_COMMAND_LIST, myRenderer);
cmd.setHelpSectionMap(map);

The convention in picocli is that calling a setter method on CommandLine impacts the whole command hierarchy. See the javadoc for details.

[bobtiernay-okta]

Not sure if this is related to this request (I thought it was originally), but would it be possible to add support to HelpCommand for nested sub-commands? e.g.:

<command> help x y z

Currently it looks like it simply resolves x and ignores it's children and grandchildren.

[remkop]

That should already work correctly when users invoke this:

<command> x y help z

I recommend that all commands in the hierarchy have HelpCommand as (one or their) subcommands.

[bobtiernay-okta]

Hmm, isn't it more typical the way I mentioned? An example that comes to mind is Kubernete's kubectl command:

$ kubectl help --help

Help provides help for any command in the application.
Simply type kubectl help [path to command] for full details.

Usage:
  kubectl help [command] [flags] [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).



md5-04dc109fdc4d920a36800f4c15877d15



$ kubectl help config delete-cluster



md5-04dc109fdc4d920a36800f4c15877d15



Delete the specified cluster from the kubeconfig

Examples:
  # Delete the minikube cluster
  kubectl config delete-cluster minikube

Usage:
  kubectl config delete-cluster NAME [options]

Use "kubectl options" for a list of global command-line options (applies to all commands).

either way, it seems like it could be supported pretty easily

[remkop]

I'm happy with the way HelpCommand works, its semantics are documented in the user manual, the javadoc, and in the usage help message of the HelpCommand itself. Given there are many other things I want to improve in picocli, I don't see myself spending time on this.

That said, I don't object to doing this. If anyone submits a quality PR that implements this (with tests and docs) I don't see any reason not to merge it.