Picocli: Provide convenience API for global options (was: Feature request: inheriting mixins in subcommands)

[garretwilson]

I made a base class BaseCliApplication that all my CLI applications will extend. The base class does all the special picocli setup needed and other things for CLI applications. The class adds a --debug option that turns on debug-level logging:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
  …
}

Unfortunately in my Foo CLI application subclass, I introduce a subcommand bar using a method:

@Command(description = "Do the bar thing.")
public void bar(…) {
  …
}

Picocli doesn't seem to pick up the --debug switch in the subcommand. If I attempt to use the command foo bar, it tells me:

Unknown option: --debug
Usage: foo bar
Do the bar thing.

Is there a @Command parameter that I can use to say "this switch applies to all subcommands"? Or is there a way to specify for a subcommand that the --debug switch should work for it as well?

I looked for documentation for mixins, but I only found information on how to include them if they are already defined. I'm not sure how to define a mixin. How can I say that the setDebug() method is a mixin? (Frankly I'd rather use the first approach: some way to say "this switch applies to all subcommands".)

Thanks in advance.

[remkop]

Picocli provides a @ParentCommand for situations like this, where subcommands need to access "global" options defined on their parent command.

The user manual has an example.

[garretwilson]

Thanks, but I'm not sure how I would use it. In my Foo application, if I call the foo --debug parent command, Picocli automatically calls Foo.setDebug() on the application instance.

Where would I put the @ParentCommand annotation so that Picocli continues to call Foo.setDebug even for foo bar --debug? There is only one example I can find, and it doesn't seem to apply to this situation.

Are you saying I could put this in the annotation for Foo.bar() (the subcommand)?

@Command(description = "Do the bar thing.")
@ParentCommand(Foo.class)
public void bar(…) {
  …
}

Will that work? (I doubt it.) Even if it does, I don't see the point, because it would seem that foo() is already inherently a subcommand of Foo, because it is, after all, Foo.bar(), and both Foo and bar() have @Command annotations. That's how we get foo bar to work, after all.

Perhaps I'm just not understanding what you're suggesting.

[remkop]

The picocli-examples module has this example: https://github.com/remkop/picocli/blob/master/picocli-examples/src/main/java/picocli/examples/subcommands/ParentCommandDemo.java

[remkop]

You could define a --debug option on every subcommand, but I the problem is how to detect that the debug option was specified on any of the parent or grandparent etc commands in the hierarchy on the command line.

For example, if the user invoked:

top-level-command --debug subcommand sub-subcommand --other-option

The SubSubcommand may have its own definition of the --debug option, but this is not set. What was set was the --debug option on the TopLevelCommand. So the business logic of SubSubCommand needs to ask its parent (and potentially its parent's parent) whether the --debug option was set.

This can be implemented something like this:

@Command(name = "top-level-command", subcommands = SubCommand.class)
class TopLevelCommand {
  @Option(name = "--debug") boolean debug;
}

@Command(name = "subcommand", subcommands = SubSubCommand.class)
class SubCommand {
  @Option(name = "--debug") boolean debug;

  @ParentCommand
  TopLevelCommand topLevelCommand;
}

@Command(name = "sub-subcommand")
class SubSubCommand {
  @Option(name = "--debug") boolean debug;

  @ParentCommand
  SubCommand myParent;

  public void run() {
     if (this.debug || myParent.debug || myParent.topLevelCommand.debug) {
        System.out.println("I'm debugging...");
     }
  }
}

[garretwilson]

Yeah, I'm still not getting it. I have the feeling you might not have read my example very closely. The example that was given uses a static subclass for a command. But in my example the subcommand is a method. Where do I put an annotation to an instance variable inside a method? Methods can't have instance variables. They just have variables on the stack.

Let me show all the pieces together:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar(…) {
    …
  }

So where would I use @ParentCommand?

Are you saying that I must change my subcommand to use a static internal class? This is starting to look messy:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(name = "bar", description = "Do the bar thing.")
  static class Bar implements Runnable {

    public void run() {
      //I have to transfer the logic from the bar() method to here
    }
  }

And that was just to refactor. Now I have to go back and add the @ParentCommand:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(name = "bar", description = "Do the bar thing.")
  static class Bar implements Runnable {

    @ParentCommand
    private Foo parent;

    public void run() {
      //I have to transfer the logic from the bar() method to here

      //plus now I have to do this??
      if(parent.isDebug()) {
        //wait, what do I do here? if parent.setDebug() had been called, I wouldn't need any of this
      }
    }
  }

I guess I'm still lost.

[garretwilson]

The SubSubcommand may have its own definition of the --debug option, but this is not set. What was set was the --debug option on the TopLevelCommand.

No, but that's the point—the --debug option was _not_ set for the top-level command. If Picocli had called Foo.setDebug(), then I would be happy, because this is a global setting. The subcommand bar doesn't need to access any "parent command" to find out if Foo.setDebug() was called. The problem is that Foo.setDebug() _isn't even being called_. Even worse, Picocli specifically _prevents_ the --debug switch in the subcommands.

If I add the --debug option to the Foo.bar() subcommand, will Picocli automatically call the _parent command_ Foo.setDebug() method? That would be one workaround, although it would be a lot of work to remember to add @Option(names = {"--debug", "-d"} to every single subcommand for every single application I ever create.

The whole point of creating BaseCliApplication is so that all applications that extend BaseCliApplication would automatically get a --debug switch, and BaseCliApplication.setDebug() would automatically get called if that switch was present. That's what I need.

[remkop]

Ah ok, yes, I missed that the @Command annotation was on a method. This does not support the @ParentCommand annotation. Apologies for not reading more closely.

Methods do not inherit from BaseCliApplication though, as you point out.

How about using a mixin?

class DebugMixin {
    @Option(names = "--debug")
    protected void setDebug(final boolean debug) {
       // one idea is to update static state here, so that the DebugMixin instance does not matter
        …
    }
}

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Mixin
  DebugMixin debugMixin;

  @Command(description = "Do the bar thing.")
  public void bar(@Mixin DebugMixin debugMixin, @Option other…) {
    …
  }

Would that work?

[garretwilson]

OK, so I have good news and bad news.

If I manually add @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug to every single subcommand, then Picocli will automatically call the parent command Foo.setDebug(). Like this:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar(@Option(names = {"--debug", "-d"} boolean debug) {
    …
  }

The bad news is that this defeats half the purpose of having a base application class. Now I'll have to remember to include @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug with every single subcommand I ever create in the future for any application I create. I had wanted this to "just work" automatically.

So this will get me by in the short term, but we really need an option that says inherited=true or something similar for any option, so I could just do this:

@Command(name = "foo", mixinStandardHelpOptions = true)
public class Foo extends BaseCliApplication {

  @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.", inherited=true)
  protected void setDebug(final boolean debug) {
    …
  }

  @Command(description = "Do the bar thing.")
  public void bar() {
    …
  }

That way, any subcommand of any subclass of BaseCliApplication would automatically have its Subclass.setDebug() method called if --debug were present.

Would it be agreeable to add such an inherited flag? (Or something similar? The name is just off the top of my head.)

(BTW I see our replies are sort of going past each other as we both respond. 😄 ) To respond to your @DebugMixin (thanks for the example), that helps but not much. Really it's just a variation of what I say in this reply. In fact I can do the same thing using @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug instead of @Mixin DebugMixin debugMixin, and now that I look at it, that's almost uglier than just spelling out the option. Plus I have to create a completely new class just to pull it off.

And the weird thing is, if I add this @DebugMixin, wouldn't it result in Picocli still calling Foo.setDebug() as well? So really @DebugMixin would be a fake mixin, a lot of work just to get Picocli to call Foo.setDebug() , which is all I really wanted to begin with.

So for the meantime I'll just add @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") boolean debug to every single subcommand, and hope that you'll let me add some sort of inherited flag in the future.

[remkop]

I'm not sure what you mean by

If I manually add @Option(names = {"--debug", "-d"} boolean debug to every single subcommand, then Picocli will automatically call the parent command Foo.setDebug().

Picocli should _only_ call parent command Foo.setDebug() if the user specified foo --debug bar. If the user specified foo bar --debug then Foo.setDebug() should not be invoked (but the bar subcommand will get its --debug option set to true). For foo to be invoked for an option on its bar subcommand would be a bug. I just tried, but could not reproduce this.

Are you sure this is what is happening? Can you share code that reproduces this behaviour?

---

Generally, Picocli offers two mechanisms for reuse: inheritance and mixins. Command methods cannot inherit, so that leaves mixins. For a single option, it may be too much overhead to define a mixin class, I guess that depends on the implementation of the setDebug method.

Mixins allow you to invoke logic when an @Option is set (the setDebug method in your example), while an @Option on a command method is just a parameter, and you would need to have logic inside the command method to do something with the parameter value.

If you do decide to go with a mixin, I like to make mixins reusable components, so it would probably make more sense to move the logic from Foo.setDebug to the DebugMixin implementation, to make it more independent.

[garretwilson]

Picocli should only call parent command Foo.setDebug() if the user specified foo --debug bar. If the user specified foo bar --debug then Foo.setDebug() should not be invoked (but the bar subcommand will get its --debug option set to true). For foo to be invoked for an option on its bar subcommand would be a bug. I just tried, but could not reproduce this.

Oops, you're right. The reason it worked for me is that I had added a hack workaround to get debug turned on by calling setDebug(true) in the subclass constructor. so the only thing I accomplished with the extra @Option was to get Picocli to _allow_ me to send a --debug switch with the bar command. The subclass was what was calling Foo.setDebug().

😞

Generally, Picocli offers two mechanisms for reuse: inheritance and mixins. Command methods cannot inherit, so that leaves mixins.

But even if I refactor the code to switch to internal classes (see above), that doesn't help anything. So this "inheritance" doesn't help my situation, regardless of whether it's for a subcommand or not.

Mixins are bulky and, and require that I remember in every single application and every single command to add this mixin manually.

We need a simply way to say, "this option applies to all subcommands". It's a simple flag. Why do we need to make things complicated? I think this mixin idea is wonderful for more complicated needs that require more flexibility and more access. But this is not a complicated need. It is a simple need.

[remkop]

So, given an option defined in a command (which is bound to a field or method in that command) you’re asking to introduce a mechanism for adding this option to subcommands, while _keeping these subcommand options bound to the parent command field or method_.

I need to think about this. At first glance it seems very application-specific and I have doubts that it is a general enough use case to warrant support in the library. But I could be wrong. I frequently am.

For now I would recommend using a Mixin to allow invoking the setDebug method in any subcommand just by defining an annotated field or method parameter.

[garretwilson]

I need to think about this.

Definitely, and so do I! My proposal was meant to be the impetus for more thinking and coming up with the best design. The initial solution was off the top of my head; we may still think up something better, or realize why this is a bad idea.

On the surface of it, though, it would seem something generally useful.

I'll be thinking more about it too, thanks.

[remkop]

I don't see how this would work for options that are bound to method parameters (when the option is defined on an annotated @Command method). For example:

public class Xxx {

  @Command(subcommands = Baz.class)
  public void bar(@Option(names = "--debug", inherited=true) boolean debug) {
    …
  }

  @Command(name = "baz")
  static class Baz implements Runnable {
    public void run() {
      // how can baz find out whether `bar baz --debug` was specified?
    }
  }
}

[garretwilson]

First let me ask you this (because I haven't used this configuration): in the example you gave, if I issue the command bar baz, is the Xxx.bar() method called, or only Xxx.Baz.run()?

[garretwilson]

how can baz find out whether bar baz --debug was specified?

Your library allows a lot of combinations of things that can work together. We can't use all the combinations together at one time, because they don't always make sense. You already pointed out, for example, that @ParentCommand doesn't work with subcommands that are methods.

So the simple answer to your question seems to be that you can say that the "inherited" flag only works for annotations on accessor methods (e.g. setDebug()) that are guaranteed to present at the time the subcommand is invoked. Otherwise, "inherited" doesn't make sense, as there is "nothing up there" (at the parent command level) to even know about the flag.

That seems like a reasonable answer and not too much of a kludge. Nobody expects all the combinations of features to work with all the available ways there is to put the application together.

[remkop]

First let me ask you this (because I haven't used this configuration): in the example you gave, if I issue the command bar baz, is the Xxx.bar() method called, or only Xxx.Baz.run()?

By default, only Xxx.Baz.run()

[remkop]

I try to avoid providing features that only sometimes work.

One could argue that @ParentCommand is an example of a feature that does not always work, but that one is more like a missing feature that could be added if necessary, and anyway, if that was a mistake, would it be a good idea to repeat it...? :-)

I would only consider introducing a potentially confusing feature like this if it brings considerable benefits when it is applicable. Given that the same (and more) can be achieved with mixins, the only argument for this new mechanism is that it gives more terse syntax than mixins.

To be honest, I am liking this less the more I think about it.

[garretwilson]

But we've only thought about it for an hour or so! haha I don't even know how much I like it either. I think about things for weeks or longer.

So far I'm liking the idea of a mixin that can be declared "inherited". It is self-contained. If a subcommand doesn't need the information passed, then it doesn't need to redeclare the mixin. The mixin would still "do its job" down the hierarchy as long as it was declared "inherited".

I could do @Mixin(inherited=true) DebugMixin debugMixin and all the subcommands would get the mixin. I suppose I could do the same with the version mixin.

It seems that what I'm talking about are options for cross-cutting concerns—things that apply across functionalities. These include:

• logging
• internationalization
• configuration

So I would want to turn on debugging for any command. I would want to set the user interface language for any command. I might want to specify the configuration for any command.

But there is still more thinking to do.

[garretwilson]

To put this into perspective, for the past several years I've created a series of small, loosely-coupled libraries for cross-cutting concerns. (They are refactorings of functionality I've built into applications for over a decade.) These include:

• Rincl (internationalization)
• Clogr (logging)
• Confound (configuration)

Take logging for instance. I have a base application class as I mentioned, BaseCliApplication. All my CLI applications, regardless of commands or subcommands or sub-sub-commands, need a way to initialize the Clogr SLF4J logging so that all the application and all its dependencies auto-magically get their log files put into some file. For example I might want to say --log-file mylogs.log --log-rollover 24h or some sort. Imagine if I had to remember, not only for every single application, but every available subcommand, to include a LoggingMixin!! 😮 But if I could declare LoggingMixin to be inherited at the BaseCliApplication level, that may be all that's needed.

I'm still not sure that's the perfect path forward, but hopefully you understand better that this isn't some esoteric, one-off functionality that's needed for some niche application.

[remkop]

The thing is, all @Option and @Parameters annotations are bound to _something_. This something can be a field, a method, or a method parameter. This binding captures the value that was specified for that option or positional parameter.

If you look at picocli's mechanisms for reuse, it is clear how the binding works for both, and how the command can retrieve the specified values:

• class inheritance: the subclass inherits the fields and methods from its superclass. Command obtains values by inspecting the inherited fields.
• mixins: each mixin class defines its own annotated fields and methods, and the command obtains the values by inspecting the mixin's fields.

Any new mechanism reuse would need to clearly define how the bindings for the reused options and positional parameters would work. With java class inheritance and mixins, it is very easy to understand for users how this works. People can just use the annotations without thinking about how things work under the hood. I doubt we will be able to achieve a similar simplicity with an additional reuse mechanism.

I really hesitate to introduce a 3rd mechanism for reuse. Bear in mind, if you don't like using mixins, perhaps instead of using @Command-annotated methods, you may be better off using @Command-annoted classes, so you can use inheritance as a reuse mechanism.

[garretwilson]

Bear in mind, if you don't like using mixins, perhaps instead of using @Command-annotated methods, you may be better off using @Command-annoted classes, so you can use inheritance as a reuse mechanism.

Could you tell me more about how to do this? My applications already inherit from BaseCliApplication. So you're saying I could somehow add a @Command to BaseCliApplication just to add an option, without knowing what the name of the "root" command will be?

I really hesitate to introduce a 3rd mechanism for reuse.

Well, if Picocli already has a mechanism for reuse, then I'll use that! So here is what I want to know: what can I add to BaseCliApplication so that all subclasses and all their subcommands automatically get a working --debug switch? It could be a mixin, a method, an annotation, a huge subclass—I don't care, as long as I hide it from the base classes and base classes don't have to do any more work in their part to get this functionality. Just let me know which mechanism lets me do that.

[remkop]

Basically what you were doing originally, but using classes for commands instead of @Command-annotated methods.

If BaseCliApplication defines the debug option like this:

@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
  // I don't know what this does but I assume it modifies static state
  // so that it works the same regardless of whether it is set
  // from a subcommand or from a top-level command
}

And all your commands inherit (directly or indirectly) from BaseCliApplication, would that not work? (Note, that means no @Command-annotated methods because these do not inherit from BaseCliApplication.)

For example:

@Command(name = "foo", subcommands = Bar.class)
public class Foo extends BaseCliApplication implements Runnable {
  @Option(name = "--other")
  int other;

  // ...
}

@Command(name = "bar", subcommands = Baz.class)
public class Bar extends BaseCliApplication implements Callable<Void> {
  @Option(name = "--other-option")
  String otherOption;

  public Void call() throws Exception {
    // ...
  }

  @Command(name = "baz")
  static class Baz extends BaseCliApplication implements Callable<Void> {
    @Option(name = "--yet-other-option")
    String yetAnother;

    public Void call() throws Exception {
      // ...
    }
  }
}

[garretwilson]

But bar is not an application! There is only one application, the foo application. It inherits from BaseCliApplication. The base application class comes with lots of extra things related to making a CLI application. That's why its'a base application class.

bar is only a subcommand of foo. It has no reason to inherit the entire application infrastructure.

Maybe I could refactor the debug() option out into a separate ApplicationCommand class, separate from the BaseCliApplication hierarchy, and then have both BaseCliApplication and any subcommand internal class inherit from this ApplicationCommand?

Even if that works (I'll have to try it), still that pollutes the structure of the application a lot. It was so simple, and I was so happy, to be able to add a subcommand just by adding a new method bar() and annotating it. Now I would have to add a new class, implement a call method, etc. And now there's no easy way to pass parameters to the bar() method. Now I have to make instance variables and/or setter methods and then somehow retrieve them from within Bar.call(). So much boilerplate now, for what used to be simple, just so I can get --debug to work across all commands!!

[garretwilson]

@Command(name = "bar", subcommands = Baz.class)
public class Bar extends BaseCliApplication implements Callable<Void> {
  @Option(name = "--other-option")
  String otherOption;

  public Void call() throws Exception {
    // ...
  }

But look how complicate this has become! Before I was able to do this:

  @Command(name = "bar")
  void bar(@Option(name = "--other-option") String option) {
      // ...
    }
  }

[remkop]

I was just trying to answer your question, sorry to upset you.

We've gone over the options that are available to you now for defining a --debug option in multiple commands. None of them are perfect, they all have trade-offs. You are best positioned to select the most palatable trade-off, given your understanding of the requirements, existing components like BaseCliApplication, and what code style you are aiming for.

[garretwilson]

I was just trying to answer your question, sorry to upset you.

I'm not upset. I was just listing the downsides of the current inheritance approaches, because you seemed to imply the existing functionality was sufficient.

None of them are perfect, they all have trade-offs.

And that's why I was proposing a solution that _would_ meet these needs. But you don't want to allow any additional functional here. And it is your library, of course.

But since it's such a big need, and it applies to so many cross-cutting functionality configurations, I'll likely just keep limping along with Picocli until I get some time and then just write my own system (or maybe just expand on the one I wrote 10 or 15 years ago).

Thanks for the discussion.

[remkop]

@garretwilson Reopening as we may be able to accomplish this without introducing a completely new mechanism for reuse.

Let's look further at your idea to add an attribute to the mixin annotation that tells picocli to apply the mixin to subcommands. So it could be used something like this:

@Command 
class BaseCommand {

    @Mixin(inheritedBySubcommands= true)
    DebugMixin debugMixin;
}

This raises some new questions:

• How would the business logic in subcommands be able to inspect the inherited options and positional parameters?
• Do we need a mechanism for more fine-grained control over which subcommands inherit or don’t inherit such Mixins? I’m sure there’ll be use cases for exceptions, like inherit only in the subcommands that are direct subcommands, or exclude specific subcommands.

This last point may be addressed with an explicit exclusion list:

@Mixin(inheritedBySubcommands = true, 
    exclude = { AaaCommand.class, BbbCommand.class, CccCommand.class}

The first point is trickier.

[remkop]

Thinking further about your idea of inherited mixins. It could work, with some restrictions. This is what I've come up with so far...

• Inherited mixins would have a single binding, which is the field or method where the mixin is declared. Individual subcommands that inherit options from such an inherited mixin do not have bindings for these options. (I don't know what else to do here. Let me know if you can think of any way to have individual subcommands have their own binding for an inherited mixin.)
• It will not be possible to use mixins with inheritedBySubcommands = true as @Command method parameters. So, @Command methods can _receive_ inherited mixins from their parent command, but they cannot _produce_ inherited mixins for their subcommands. This is because method parameters are not visible to the subcommands of @Command methods so these subcommands would not be able to inspect the value of options in the mixin.
• If the user specifies options from an inherited mixin multiple times, _the same binding_ will be updated multiple times, once for each occurrence of the option on the command line. For single-valued options this results in an OverwrittenOptionException unless the parser is configured to allow overwriting options.
• Subcommands need to use existing mechanisms like the @ParentCommand annotation to obtain the value of options in the mixin they inherited.

A concrete example:

/** Mixin example. */
public class DebugMixin {
    @Option(names = "--debug")
    public boolean debug; // --debug option is bound to the `debug` field
}

@Command(name = "base", subcommands = Sub2Command.class)
class BaseCommand {

    // if the --debug option is specified on this command or any subcommand,
    // _this_ DebugMixin instance's `debug` field is set,
    // regardless of which subcommand the --debug option was specified on
    @Mixin(inheritedBySubcommands = true)
    public DebugMixin debugMixin;

    @Command
    void sub1(@Option(names = "-x") int x) {
        // @Command methods can access the value of the --debug option
        // by inspecting the mixin in the command class
        if (this.debugMixin.debug) { /* do debuggy things... */ }

        // ... other business logic
    }
}

@Command(name = "sub2")
class Sub2Command implements Runnable {
    @Option(names = "-y") int y;

    @ParentCommand BaseCommand baseCommand;

    public void run() {
        // class subcommands can access the value of the --debug option
        // by pulling in the parent command with the @ParentCommand annotation
        // and inspecting the option in the parent command's mixin
        if (baseCommand.debugMixin.debug) { /* do debuggy things... */ }

        // ... other business logic
    }
}

Possible user input:

base --debug
base sub1 --debug -x=1
base sub2 --debug -y=2

In all three above cases, the DebugMixin.debug field in the BaseCommand.debugMixin instance is set to true.

Thoughts?

[garretwilson]

Do we need a mechanism for more fine-grained control over which subcommands inherit or don’t inherit such Mixins? I’m sure there’ll be use cases for exceptions, like inherit only in the subcommands that are direct subcommands, or exclude specific subcommands.

Whew! I appreciate your thinking of the possibilities, but in my particular case, I'm happy just to have one command be inherited down the hierarchy. We've got by in Java without needing to say, public class Foo extends Bar but only down one level 😉 .

@Mixin(inheritedBySubcommands= true)
DebugMixin debugMixin;

I'm not sure I understand this. I don't need a reference to the mixin. Are you saying I would be forced to create a subclass _and_ include the mixin as an annotation on an instance variable I would never, ever use? (Because the logic of the mixin is self-contained, like the version mixin, right?)

Individual subcommands that inherit options from such an inherited mixin do not have bindings for these options.

If you're saying that the subcommands don't have access to the mixins, but they still take effect, that's fine with me. Presumably the mixins set some globally-accessible thing. And can't he subcommand always inject a @ParentCommand if it really needs to access its parent, and thereby access the inherited information (assuming the parent command exposes it)?

It will not be possible to use mixins with inheritedBySubcommands = true as @Command method parameters.

OK, maybe I can live with this, but see above—can't I still include the @Mixin in the class @Command() annotation, rather than on an instance variable, even with a class subcommand?

Subcommands need to use existing mechanisms like the @ParentCommand annotation to obtain the value of options in the mixin they inherited.

Ah, OK, like I said above. (I'm going through these from the top down, and didn't see this yet.) I have no problem with that.

So I have some questions:

@Command(name = "base", subcommands = Sub2Command.class)
class BaseCommand {

    @Mixin(inheritedBySubcommands = true)
    public DebugMixin debugMixin;

OK, that will work, although it seems cluttered to declare an instance variable I don't want. I would prefer the following (which I could use with the versions mixin, too, which I don't need to access):

@Command(name = "base", mixins={@Mixin(inheritedBySubcommands = true)}

Let me just show you what I would like to be able to do, and you can tell me whether we're getting close to pulling it off. The following are separate source files.

public class DebugMixin {
    @Option(names = "--debug")
    public void setDebug(final boolean debug) {
}

```java
@Command(name = "base", mixins={@Mixin(inheritedBySubcommands = true)})
class BaseCliApplication

```java
@Command(name = "cool")
class MyCoolApplication extends BaseCliApplication {

  @Command
  void sub1(@Option(names = "-x") int x) {
    …
  }
}

This would work great for me, I think. A couple of things to note:

• Note that I'm using a setter method (I have my reasons) rather than an instance variable for DebugMixin, but I assume that would work as well.
• Can I define the inherited mixins at the BaseCliApplication level, but _override the base application name_ at the MyCoolApplication level, yet keeping the inherited mixins? (This would be really helpful.)
• Note that I added this mixins= part of the annotation so that I wouldn't have to "capture" the mixin if I don't need to access it.
• In this scenario, can I still have a subcommand sub1 as a method as shown?
• I assume I can also use an internal class as a subcommand as per #647 ?
• I don't know why I need to use subcommands, but I'll ask about that in #647.

[remkop]

I don't see how this would work:

@Command(name = "base", mixins={@Mixin(inheritedBySubcommands = true)}

There is no reference to the class to mix in, so how would picocli know you intend to mixin from DebugMixin?

I think it would work almost exactly like you specified, with the only difference that the mixin would need to be specified like below. Everything else can stay like in your example.

@Command(name = "base")
class BaseCliApplication {

    @Mixin(inheritedBySubcommands = true) DebugMixin debugMixin;

I understand that in your use case, the setDebug method changes some global state, so in that case the application does not need a reference to the mixin. However, I feel that it would be too limiting to require this from all applications. I am sure there will be use cases where the application _does_ need to access the inherited mixin.

To answer your points:

• yes, setter methods annotated with @Option or @Parameters work the same as annotated instance fields, with the difference that the method is invoked instead of the field value being set
• yes, mixins and subclasses can already be combined like you describe. Any options or @Command attributes that the superclass has (either from mixins or directly declared on that superclass) are inherited by the subclasses.
• When you ask _can I still have a subcommand sub1 as a method_, are you asking whether sub1 would also inherit the --debug option? The answer is yes. I don't know yet whether it makes sense to stop inheriting after one level of subcommands (meaning you would need to specify @Mixin(inheritedBySubcommands = true) again if you need sub-subcommands to also have the --debug option) or whether it should be applied to the full hierarchy, with explicit excludes = ... for subcommands that don't want it.
• yes you can use internal classes as subcommands

[garretwilson]

Sorry for the delay in responding. So many things I'm working on.

There is no reference to the class to mix in, so how would picocli know you intend to mixin from DebugMixin?

Ah… yeah, that was all jotted down off the top of my head. I think I meant to say something like:

@Command(name = "base", mixins={@Mixin(type = DebugMixin.class, inheritedBySubcommands = true)}

There are likely several variations that could work. I think you got the idea.

I understand that in your use case, the setDebug method changes some global state, so in that case the application does not need a reference to the mixin. However, I feel that it would be too limiting to require this from all applications. I am sure there will be use cases where the application does need to access the inherited mixin.

Sure, that's understandable. I was merely pointing out that this is no need to _require_ a reference. But if you require a reference, that's something I can live with. I can hide it in the base class.

When you ask _can I still have a subcommand sub1 as a method_, are you asking whether sub1 would also inherit the --debug option?

I think I was just making sure that the inheritance would work with subcommands that are methods, and not just subclasses. I can't be sure what I was meaning because it was a few days ago (and my mind is a little tired right now to parse through it all). I think something about my example differed a little from your example, so I was just making sure the variation would work.

It sounds very positive from all your responses. You've seen my example, and I requiring an instance variable or a setter method for the inherited option is not a show-stopper.

So what do you think? Are you OK with proceeding with this, and do you have some sort of timeline on when it might be available? (I'm just checking. No pressure—I know we're all super busy.)

Thanks for your consideration of all this.

[remkop]

Great, it sounds like we have arrived at a design for a solution that we are both happy with.

About potential timelines for an implementation, to be honest, this seems quite far away.
Just looking at the todo list for picocli 4.0-alpha2 and the picocli 4.0 milestone, there are 20+ open tickets, some of which are quite chunky pieces of work. I suspect the 4.0-GA release is still several months away.

I won't have time to consider which tickets to start working on next until after that. If it is important to you to get this functionality earlier, your best bet is to contribute a pull request and aim to get this included in the 4.0 release. The higher quality the pull request (unit tests, docs, etc), the easier it is for me to merge, and the sooner it will be available.

[garretwilson]

Drat. I'm afraid I'm super-overloaded to take on a feature of that size at the moment. But I hope the discussion helped.

[remkop]

Definitely! It took me a while to get convinced, but now I believe this will be a very nice feature.

[garretwilson]

So there are some complications with this, and in fact it's not even going to be very simple to create a workaround. Here's the issue. With the debug thing, I don't need to just set a flag; I need to actually call the debugging framework (whether it's logback or whatever) to tell it to set the log level.

Right now I do the following:

1. In the application's constructor, I call setLogLevel() to set a default log level, just in case the user doesn't provide a --debug flag (or provide some other indication of log level).
2. Then if the user uses the --debug flag, Picocli calls the application setDebug(boolean) method which _updates_ the log level.

So while I'm waiting for #649 to be implemented, I thought that I would simply encapsulate this functionality into a DebugMixin class and simply force application subclasses to include the DebugMixin. The problem is: how does the default log level gets set if the user doesn't include a --debug option at all, because DebugMixin never gets instantiated.

I suppose I can have a static block in DebugMixin so that whenever Picocli _loads the class_ that it sets the default log level. Then if --debug is called that value gets updated. But that's a bit of a hack.

On top of it, the log level will be determined not just by DebugMixin, but also by LogMixin, which allows the user to manually set a log level, e.g. using --logLevel. So the log level in the application depends on settings both of DebugMixin and LogMixin If could simply define @Options on setters in the main application class, and indicate that those options are inherited, then all would be fine. The setters would keep track of the state, and updateLogLevel() would take care of all the different combinations.

So in summary it looks like I still need to simply annotate setFoo() with an @Option that applies to all subcommands. And in addition, the workaround is going to look ugly even with a mixin.

[remkop]

Can’t you use @Option(names ="--debug", defaultValue = "INFO") on the annotated method?

Right before parsing, picocli will first call the method with the default value, and if the user specified the --debug option, the method will get called again with the option parameter.

——

Edit: sorry, didn’t read the bit about logLevel, so the above may not work for you. I’ll check again tomorrow. Bedtime for me.

[garretwilson]

Three things you may not have understood from my explanation: 1) There are two distinct options of --debug and --logLevel. 2) The options are not independent; they interact. 3) Setting a default value for an option is not the same as recognizing that the user did not specify the option at all.

Look, let me take a step back and just give you the functional requirements, plain and simple. (This is a subset of the requirements, but it should give you the picture.)

1. If the user indicates --debug, this sets a debug flag _and_ sets the log level to DEBUG.
2. If the user indicates some --logLevel option, the specified log level overrides the --debug option (if present). (Of course, the debug flag is still undisturbed.)
3. If neither option is present, the log level is set to INFO. (Actually I would set it to WARN except if a --verbose option is present, but I don't want to complicate the example.)
4. These options work for all subcommands.

First of all, I think you'll agree that these requirements are logical and reasonable, and that they are in no way exotic, strange, or even complicated. If you find them odd or complicated, we should probably not be having this discussion because we'll get nowhere. To me they seem some of the most obvious and natural settings a CLI application could have.

Yet they are impossible to set up with the current Picocli without tons of repetition and boilerplate. And even with your proposal, I'm not sure they would be easy to implement.

Right now the first three points are super-simple. They look something like this:

@Option(names = {"--debug"})
protected void setDebug(final boolean debug) { … }

@Option(names = {"--logLevel"})
protected void setLogLevel(final String logLevel) { … }

Wow, how easy is that? I can figure out how the options interact by the class internal logic. But I don't want to duplicate all those options for subcommands. All I need is (returning to the top of this ticket):

@Option(names = {"--debug"}, inherited = true)
protected void setDebug(final boolean debug) { … }

@Option(names = {"--logLevel"}, inherited = true)
protected void setLogLevel(final String logLevel) { … }

That's it.

The fact that such logical, useful, natural, common, and simple requirements are so difficult to implement in a base class so that they won't have to be repeated by every concrete application is why I'm pretty sure I'll have to just go implement it myself in a new library, as I indicated earlier. Even your proposal above doesn't really take into account that these things interact, and even if they did, they are much more complicated than a inherited = true flag.

I truly and sincerely appreciate the time and consideration you've given this, but I get the feeling that in some ways I may be taking up more of our time than it would take me just to implement it myself, especially if the solution you propose is several times more complicated and wouldn't even be available any time soon. Sorry, I'm just trying to be realistic here.

[remkop]

_Note to self: don't answer email at 2am before falling asleep... :-)_

Okay, well, if the problem was that picocli doesn't instantiate DebugMixin, then let's just create an instance in the application. Have you tried setting the defaultValue attribute on the options in the mixin?

Here is a demo program that shows how to use default values. I believe it works exactly according to your specifications:

import picocli.CommandLine;
import picocli.CommandLine.Command;
import picocli.CommandLine.Mixin;
import picocli.CommandLine.Option;
import picocli.CommandLine.ParentCommand;

public class Example {
    static class GlobalState {
        static boolean debug;
        static String logLevel;
    }

    static class DebugMixin {
        @Option(names = "--debug", defaultValue = "false")
        public void setDebug(boolean debug) {
            System.out.printf("Setting debug to %s in %s%n", debug, this);
            GlobalState.debug = debug;
            if (debug) {
                setLogLevel("DEBUG");
            }
        }

        @Option(names = "--logLevel", defaultValue = "INFO")
        public void setLogLevel(String logLevel) {
            System.out.printf("Setting logLevel to %s in %s%n", logLevel, this);
            GlobalState.logLevel = logLevel;
        }
    }

    @Command(name = "base", subcommands = Sub2Command.class)
    static class BaseCommand implements Runnable {

        @Mixin // in future: inheritedBySubcommands = true
        DebugMixin debugMixin = new DebugMixin();

        @Command
        void sub1(@Mixin DebugMixin subDebugMixin, // in future: omit because inherited from parent command
                  @Option(names = "-x") int x) {

            System.out.printf("sub1: debug=%s, logLevel=%s (myMixin=%s, parent Mixin=%s)%n",
                    GlobalState.debug, GlobalState.logLevel, subDebugMixin, this.debugMixin);
        }

        public void run() {
            System.out.printf("base: debug=%s, logLevel=%s (myMixin=%s)%n",
                    GlobalState.debug, GlobalState.logLevel, debugMixin);
        }
    }

    @Command(name = "sub2")
    static class Sub2Command implements Runnable {
        @Option(names = "-y") int y;

        @Mixin // in future: omit because inherited from parent command
        DebugMixin debugMixin = new DebugMixin();

        @ParentCommand BaseCommand parent; // for debugging

        public void run() {
            System.out.printf("sub2: debug=%s, logLevel=%s (myMixin=%s, parent Mixin=%s)%n",
                    GlobalState.debug, GlobalState.logLevel, debugMixin, parent.debugMixin);
        }
    }

    public static void main(String[] args) {
        CommandLine.run(new BaseCommand(), args);
    }
}

Verifying the results:

Without any args; you can see the defaults being set (via the parent command's mixin):

Setting debug to false in inherit.Example$DebugMixin@2f7c7260
Setting logLevel to INFO in inherit.Example$DebugMixin@2f7c7260
base: debug=false, logLevel=INFO (myMixin=inherit.Example$DebugMixin@2f7c7260)

With sub1 --logLevel COOL, we can see the default being set (twice, but I assume this is idempotent so hopefully this is not an issue):

Setting logLevel to INFO in inherit.Example$DebugMixin@2f7c7260
Setting debug to false in inherit.Example$DebugMixin@2f7c7260
Setting logLevel to INFO in inherit.Example$DebugMixin@2d209079
Setting debug to false in inherit.Example$DebugMixin@2d209079
Setting logLevel to COOL in inherit.Example$DebugMixin@2d209079
sub1: debug=false, logLevel=COOL (myMixin=inherit.Example$DebugMixin@2d209079, parent Mixin=inherit.Example$DebugMixin@2f7c7260)

With both options: sub2 --debug --logLevel HOT: the logLevel is specified last so overrides the --debug loglevel (but preserves the flag)

Setting debug to false in inherit.Example$DebugMixin@2f7c7260
Setting logLevel to INFO in inherit.Example$DebugMixin@2f7c7260
Setting debug to false in inherit.Example$DebugMixin@2d209079
Setting logLevel to INFO in inherit.Example$DebugMixin@2d209079
Setting debug to true in inherit.Example$DebugMixin@2d209079
Setting logLevel to DEBUG in inherit.Example$DebugMixin@2d209079
Setting logLevel to HOT in inherit.Example$DebugMixin@2d209079
sub2: debug=true, logLevel=HOT (myMixin=inherit.Example$DebugMixin@2d209079, parent Mixin=inherit.Example$DebugMixin@2f7c7260)

With just the sub2 --debug option: the debug flag is set and the log level is DEBUG

Setting logLevel to INFO in inherit.Example$DebugMixin@2f7c7260
Setting debug to false in inherit.Example$DebugMixin@2f7c7260
Setting logLevel to INFO in inherit.Example$DebugMixin@2d209079
Setting debug to false in inherit.Example$DebugMixin@2d209079
Setting debug to true in inherit.Example$DebugMixin@2d209079
Setting logLevel to DEBUG in inherit.Example$DebugMixin@2d209079
sub2: debug=true, logLevel=DEBUG (myMixin=inherit.Example$DebugMixin@2d209079, parent Mixin=inherit.Example$DebugMixin@2f7c7260)

The only boilerplate here is that the mixins need to be specified explicitly until the inherit attribute is implemented, as we discussed.

[garretwilson]

static class GlobalState {
  static boolean debug;
  static String logLevel;
}

But I don't want to use these static global JVM variables!! Besides being ugly (and reminding me of C programs in the 1980s), it's not good practice. I want to use instance variables of my application, just as I was doing.

Besides, what I put in two lines suddenly has been changed to pages of code!!

You're assuming I'm storing the log level with the debug flag. I'm not. And as I mentioned these are not necessarily JVM-wide global variables. If you want some background, you can read the following:

https://csar.io/intro

(I already gave links to Clogr, Rincl, etc.)

But listen, I think I'm getting to the point where I'm bothering you. Once again I really appreciate your listening and thinking about this, but I think at this point it's eating up both our time, and I'm not sure we're getting anywhere. Thanks again for all your consideration, and if some of this discussion helps you make Picocli better, that's great.

But I think I have to move on. For the moment in my application just today I basically duplicated the --debug option and called setDebug() in the main application from within my subcommand method. When I have time, I'll circle back and just implement this the way I expect in my own project.

All the best.

[remkop]

I think you took the GlobalState a bit too literally. As you’ll remember from our previous discussion, when @Mixin(inherit = true) is implemented there will be only one binding, and in this demo I modeled that shared binding in GlobalState. I’m sure I’ll be very different in the actual app.

The example is long because it tries to pull together all topics we discussed previously as well as demonstrate that all tools are available to meet your new requirements.

But as you said, let’s stop here.

[remkop]

@garretwilson

picocli 4.2.0 has just been released.
It adds support for @ParentCommand-annotated fields in mixins. This allows creating mixins with @Option-annotated methods which delegate to the parent command.

Now, to share options between a command and its subcommands:

1. implement the main logic (like setDebug, setLogLevel) once in the parent command
2. define a mixin with the same options using @Option-annotated methods that delegate to the parent
3. subcommands "import" the options with a @Mixin-annotated field or method parameter

I added an example to the user manual.

This is not as compact as what you requested (it requires an extra mixin class), but it has the advantage that maintainers can look at the source for a subcommand and understand at a glance where all options are coming from.

[remkop]

I’m coming around to your idea.

Something like this:

@Option(name = "--debug", global = true)
boolean debug;

Options that have global = true set will be included in all subcommands (and sub-subcommands, up to any depth).
TBD: Should it be possible for positional parameters to be global?

I’m not 100% decided on the name. “inherited” is also a good candidate and strictly speaking more accurate because such an option may be defined on a particular subcommand instead of on the top-level command, in which case it wouldn’t be truly global but only apply to the subtree that has that particular subcommand (where the option is defined) as its root.

However, people talk and think of this concept as “global options”, so users will have a fairly good intuition about what it does. Also there’s no potential confusion with Java inheritance.

Speaking of inheritance, there’s an edge case where global/inherited options conflict with Java inheritance: suppose we have an AbstractCommand class with a global option. We then have two commands, A and B, where B is a subcommand if A. Both A and B are implemented as a Runnable class that extends AbstractCommand. Now we have a problem: the global option is included in B, but B already has that option (because it extends AbstractCommand. I need to think about this more.

[garretwilson]

I’m coming around to your idea.

Thanks for considering it.

I’m not 100% decided on the name. “inherited” is also a good candidate and strictly speaking more accurate because such an option may be defined on a particular subcommand instead of on the top-level command, in which case it wouldn’t be truly global but only apply to the subtree that has that particular subcommand (where the option is defined) as its root.

Use the correct name. I try to teach my students that "global" things are not good things. (Remember C++ programming where we would have things sitting around in the sky that everything could access?) And it isn't global, as you mentioned; I say call it what it is.

However, people talk and think of this concept as “global options”, so users will have a fairly good intuition about what it does.

But looking back in the discussion, it was you who referred to this as "global" (and who renamed the issue, I think). I was against the concept being applied here. This is inheritance plain and simple.

Also there’s no potential confusion with Java inheritance.

Java already has the concept of annotations inheriting separately from the Java class hierarchy. JSP has "inheritance" of scopes, independent of the Java class hierarchy. Maven has POM inheritance, independent of the Java class hierarchy.

Inheritance is a basic feature of software development. You are writing this for programmers, not for some great-aunt-or-uncle who is needing help on understanding how to use their smartphone. Why make the world an even more confusing and inaccurate place? Call it what it is.

Does someone really exist that thinks that by marking an annotation as "inherit" it changes their Java class inheritance hierarchy? If that person exists, they likely haven't finished their first Java course and they are probably in danger of failing it.

[garretwilson]

By the way, @remkop , I hope you understand by now that when I share my opinion I'm not trying to be hostile or derogatory. I'm just thinking out loud. Heck on my own projects I even argue back and forth with myself. 😄 See the discussion in GUISE-88 for example. Thanks again for considering this.

[remkop]

Interesting feedback from a colleague: an enum instead of a boolean may be more flexible (extensible in the future). Some ideas:

@Option(names = "-x", scope = Scope.LOCAL) int x; // not shared
@Option(names = "-v", scope = Scope.GLOBAL) boolean verbose; // shared within subtree

Note to self:
Slightly annoying is that this would cause a small API "clash", since picocli already has an (unrelated) IScope interface, with associated scope() getters on ArgSpec and ArgGroupSpec. IScope is used internally when the parser matches an option (or positional param, or an option group) on the command line: this often results in the annotated field being set or an annotated method being invoked. In this context, the "scope" is the instance of the class whose field is set or whose method is invoked. Perhaps a better name for this would be IBinding (with corresponding binding() getter/setter methods), or IUserObject.

[garretwilson]

I'm cleaning up my inbox and wanted to circle back on this ticket. I hope you are are doing well, and that you are keeping safe and healthy in this world health crisis.

Interesting feedback from a colleague: an enum instead of a boolean may be more flexible (extensible in the future).

Perhaps, but in your example I question the terms "local" and "global". If I set scope = Scope.GLOBAL, will this command be visible in an unrelated command subtree? I think the answer is no. If so, then it's not "global"; it's merely "inherited".

Hopefully within the next month or two I'll be back to the point on my project where I can upgrade to the latest picocli version, so I news on this from time to time is great.

[remkop]

In the latest version (now in master) it’s actually called INHERITED.

[remkop]

Quick update: I am hoping to do a release that includes this feature soon. Relevant snippet from the release notes follows below:

---

Inherited Options

This release adds support for "inherited" options. Options defined with scope = ScopeType.INHERIT are shared with all subcommands (and sub-subcommands, to any level of depth). Applications can define an inherited option on the top-level command, in one place, to allow end users to specify this option anywhere: not only on the top-level command, but also on any of the subcommands and nested sub-subcommands.

Below is an example where an inherited option is used to configure logging.

@Command(name = "app", subcommands = Sub.class)
class App implements Runnable {
    private static Logger logger = LogManager.getLogger(App.class);

    @Option(names = "-x", scope = ScopeType.LOCAL) // option is not shared: this is the default
    int x;

    @Option(names = "-v", scope = ScopeType.INHERIT) // option is shared with subcommands, sub-subcommands, etc
    public void setVerbose(boolean verbose) {
        // Configure log4j.
        // This is a simplistic example: you probably only want to modify the ConsoleAppender level.
        Configurator.setRootLevel(verbose ? Level.DEBUG : Level.INFO);
    }

    public void run() {
        logger.debug("-x={}", x);
    }
}

@Command(name = "sub")
class Sub implements Runnable {
    private static Logger logger = LogManager.getLogger(Sub.class);

    @Option(names = "-y")
    int y;

    public void run() {
        logger.debug("-y={}", y);
    }
}

Users can specify the -v option on either the top-level command or on the subcommand, and it will have the same effect.

# the -v option can be specified on the top-level command
app -x=3 -v sub -y=4

Specifying the -v option on the subcommand will have the same effect. For example:

# specifying the -v option on the subcommand also changes the log level
app -x=3 sub -y=4 -v

[garretwilson]

Good morning! I guess this isn't released yet?

I just finished off a some big features in my main project, so for a break I figured I would come bring in some of the exciting new picocli feature you've been talking about. So this morning I brought up my project and did a search, but Maven Central is still showing v4.2.0 so I guess the version you mentioned is still in the oven. Oh, well. I guess I'll work on something else this morning.

I'm looking forward to the new version, though. Have a good week.

[remkop]

Getting close but not yet.
See the 4.3 roadmap for the remaining items.
Some follow up items came out of the “inherited options” feature, like when to apply default and initial values, and an extra feature where inherited options are hidden (don’t show up in the usage help message) in subcommands.

I also took on some extra work helping out in the https://github.com/petermr/openVirus project which reduced the time I could spend on picocli.

[garretwilson]

Thanks for the update. There's no emergency need for it at the moment. I'll circle back in a week or two. Good luck.

[remkop]

Picocli 4.3 has been released.

Enjoy!

[garretwilson]

As luck would have it I hit a huge milestone a couple of days ago in one of my projects, so today I actually took a day off from my employer, intending to spend the morning in tranquility integrating the new changes. (Right away I ran into a new Eclipse regression bug so I should have known it wouldn't be that easy.)

Here is my original base Picocli application debug setting method, as you've no doubt seen many times:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.")
protected void setDebug(final boolean debug) {
    this.debug = debug;
    updateLogLevel();
}

So with Picocli 4.3 I thought all I needed to do was to add an "inherit" scope:

/**
 * Enables or disables debug mode, which is disabled by default.
 * @param debug The new state of debug mode.
 */
@Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.", scope = ScopeType.INHERIT)
protected void setDebug(final boolean debug) {
    this.debug = debug;
    updateLogLevel();
}

In CLI subclass I had this:

@Command(description = "Does foo and bar.", subcommands = {HelpCommand.class})
public void foobar(
        @Parameters(paramLabel = "<project>", description = "The base directory of the project being served.%nDefaults to the working directory, currently @|bold ${DEFAULT-VALUE}|@.", defaultValue = "${sys:user.dir}", arity = "0..1") @Nullable Path argProjectDirectory,
        …
        @Option(names = {"--debug", "-d"}, description = "Turns on debug level logging.") final boolean debug) throws IOException, LifecycleException {

    setDebug(debug); //TODO inherit from base class; see https://github.com/remkop/picocli/issues/649

So I removed the debug parameter and the setDebug(), which I don't need anymore:

@Command(description = "Does foo and bar.", subcommands = {HelpCommand.class})
public void foobar(
        @Parameters(paramLabel = "<project>", description = "The base directory of the project being served.%nDefaults to the working directory, currently @|bold ${DEFAULT-VALUE}|@.", defaultValue = "${sys:user.dir}", arity = "0..1") @Nullable Path argProjectDirectory,
        …
        ) throws IOException, LifecycleException {

Two little changes, and I should be good to go. But no, I get:

[ERROR] wrong number of arguments

I don't know where it's coming from. I don't know which arguments.

But interestingly if I actually add the --debug flag to the command line, it apparently does turn on debug mode so that I get a full stack trace (because my debug mode changes the log level). I get this:

java.lang.IllegalArgumentException: wrong number of arguments
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at picocli.CommandLine.executeUserObject(CommandLine.java:1872)
    at picocli.CommandLine.access$1100(CommandLine.java:145)
    at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2243)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2237)
    at picocli.CommandLine$RunLast.handle(CommandLine.java:2201)
    at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2068)
    at picocli.CommandLine.execute(CommandLine.java:1978)
    at com.globalmentor.application.BaseCliApplication.execute(BaseCliApplication.java:186)
    at com.globalmentor.application.AbstractApplication.start(AbstractApplication.java:118)
    at com.globalmentor.application.Application.start(Application.java:132)
    at io.guise.cli.GuiseCli.main(GuiseCli.java:88)

The stack trace makes me suspect that your code is calling the subcommand method and, since one of the options was inherited, the code thinks that it should be an argument to the subcommand method (here foobar()) and tries to pass it along, but Java complains because that extra parameter doesn't match the subcommand method signature.

But surely you tested this, so I don't know how this could happen.

In any case, I'm going to suspend work on this for the moment and try to get some tranquility this morning. Maybe I'll try to integrate the new injected Picocli CommandSpec you added in #629.

[remkop]

Away from PC now, will look into this soon.
Would you mind raising a separate ticket?

[remkop]

I raised https://github.com/remkop/picocli/issues/1042

This is a bug. Thank you for raising this!

[garretwilson]

@remkop I've integrated this into my application, and so far it is everything I dreamed it would be!

Admittedly from the outside it doesn't look like a lot technically: it was just a flag to say "inherit this option to subcommands". But I hope you realize how much nicer it makes my application by simply adding that simple scope = ScopeType.INHERIT and have that functionality be turned on for all my applications (by putting it in the base class).

This is really useful; thank you so much.

[remkop]

Thanks for that!
Positive feedback from happy users is what keeps me going. I’m sure it’s the same for other open source maintainers (so drop them a kind word when you get a chance 😉).

What would be really great is if you could do a blog post or article somewhere on how picocli has been useful for you. You’d be surprised how few people know that picocli even exists. Would be good if we could change that.

[garretwilson]

What would be really great is if you could do a blog post or article somewhere on how picocli has been useful for you.

I would absolutely love to!

I haven't been producing blog entries for a while because I'm been working feverishly to migrate my entire personal site to a new Cloud-distributed platform.

• But to do that I needed to write the static site generator (which I've been working on for a year and a half). The CLI of this uses Picocli.
• And to do that I needed to upgrade my metadata framework (which I've been working on for over a decade).

But the new site was just launched, and soon the static site generator and metadata framework will be ready for the next version release. Then I would be enthusiastic to spread the word!

If you don't hear anything further from me on this in about a month, by all means don't hesitate to ping me on this.

[palade]

@remkop I've integrated this into my application, and so far it is everything I dreamed it would be!

Admittedly from the outside it doesn't look like a lot technically: it was just a flag to say "inherit this option to subcommands". But I hope you realize how much nicer it makes my application by simply adding that simple scope = ScopeType.INHERIT and have that functionality be turned on for all my applications (by putting it in the base class).

This is really useful; thank you so much.

@remkop @garretwilson Do you have an example of where you used this? As I get CommandLine$DuplicateOptionAnnotationsException when I set scope = ScopeType.INHERIT in the base class?

If I have something like:
Option1 Option2 SubCommand Option3 -> Option1 and Option2 are ignored if I provide the list of arguments like this. However, if they are declared like:
SubCommand Option1 Option2 Option3 there are not.

The class of SubCommand extends the class of the Command (base class). How do I make Option1 and Option2 available when the order of arguments is specified as above

[remkop]

Why are we getting a DuplicateOption exception?

@palade I am guessing that the DuplicateOptionAnnotationsException occurs when an option is defined with scope = INHERIT in class A, and the application then defines another class B that is both a _subclass of A_ as well as a _subcommand of A_.

When the application uses 2 reuse mechanisms (see sidebar below), the option is added to the subcommand twice... I will look at improving the error message to clarify this.

---

Sidebar:

Picocli now offers 3 methods of reuse:

• java class inheritance: options declared on a superclass are added to all subclasses
• mixins: declare a @Mixin annotated field to import all options declared in the mixin class
  

  * inherited options: options with scope = INHERIT are added to all subcommands (and sub-subcommands)

Are options ignored sometimes?

If I have something like:
Option1 Option2 SubCommand Option3 -> Option1 and Option2 are ignored if I provide the list of arguments like this. However, if they are declared like:
SubCommand Option1 Option2 Option3 there are not.

The class of SubCommand extends the class of the Command (base class). How do I make Option1 and Option2 available when the order of arguments is specified as above

To summarize my understanding: this part of the question is not about using scope = INHERIT, but is about reusing options with Java class inheritance (please correct me if I'm wrong).
Option1 and Option2 are defined in a class, let's call it BaseClass. Then, both the top-level command and the SubCommand extend BaseClass (or perhaps the top-level command _is_ BaseClass).

As a result, now the top-level command has Option1 and Option2 and SubCommand has Option1 and Option2. These are separate instances. So, when the user invokes Option1 Option2 SubCommand Option3, the top-level command's Option1 and Option2 are set, and the Subcommand's Option3 is set.

When you say "Option1 and Option2 are ignored", I am guessing you mean that the SubCommand's copy of Option1 and Option2 are not set. (If you check the top-level command, you will see that the top-level command's Option1 and Option2 _are_ set.)

How to solve this?

So, now we understand what is happening, what shall we do to fix it?

Idea 1: make Option1 and Option2 "inherited options"

Two ways to do this:

• change SubCommand so it no longer extends BaseClass where Option1 and Option2 are defined
• change BaseClass so it no longer defines Option1 and Option2. Separate BaseClass from the top-level command. Something like this;

separate BaseClass from top-level cmd; move Option1/2 out of BaseClass into top-level cmd

                    +-----------+
                    | BaseClass |
                    +-----------+
                          |
                          |
                   +---------------+
                   |               |
                   V               V
+------------------------+    +-------------+
|        TopLevel        |    | SubCommand  |
|         Command        |    +-------------+
+------------------------+    
+ Option1 scope=INHERIT  +    
+ Option2 scope=INHERIT  +
+------------------------+    

This is especially useful if Option1 and Option2 are "standalone" options (like for configuring logging) that subcommands don't need to reference. If a subcommand do need to reference inherited options, it can use the @ParentCommand annotation.

@Command(mixinStandardHelpOptions = true, versionProvider = XX.class)
class BaseCommand {
}

@Command(name = "top", subcommands = SubCommand.class)
class TopCommand extends BaseCommand implements Runnable {
    @Option(name = "Option1", scope = INHERIT) boolean option1;
    @Option(name = "Option2", scope = INHERIT) boolean option2;

    public void run() {
        // business logic of top-level cmd...
    }
}

@Command(name = "sub")
class SubCommand extends BaseCommand implements Runnable {
    @ParentCommand TopCommand parent;

    public void run() {
        if (parent.option1) { doOption1Stuff(); }
        if (parent.option2) { doOption2Stuff(); }
        if (this.option3) { doOption3Stuff(); }
    }
}

Idea 2: Use the @ParentCommand annotation

Instead of using scope=INHERIT, an alternative is to keep the current mechanism of reusing options via java subclassing. In the business logic of Subcommand the application can check both its own copy of Option1 and Option2, as well as the Option1 and Option2 on the parent command. Something like this:

class BaseCommand {
    @Option(name = "Option1") boolean option1;
    @Option(name = "Option2") boolean option2;
}

@Command(name = "top", subcommands = SubCommand.class)
class TopCommand extends BaseCommand implements Runnable {
    public void run() {
        // business logic of top-level cmd...
    }
}

@Command(name = "sub")
class SubCommand extends BaseCommand implements Runnable {
    @ParentCommand TopCommand parent;

    public void run() {
        if (this.option1 || parent.option1) { doOption1Stuff(); }
        if (this.option2 || parent.option2) { doOption2Stuff(); }
        if (this.option3) { doOption3Stuff(); }
    }
}

Idea 3: Refactor Option1 and 2 into a mixin

Mixins allow composition; I prefer mixins over subclassing.

Mixins can also refer to the parent command; or they can be stand-alone.
Please see the manual sections on mixins for more details and several examples.

---

If you want to discuss further, please don't hesitate to create a separate ticket! :-)

[garretwilson]

@palade , I think I remember getting this error at first. My main command is the class Foo itself, and the option ("opt") is a method of that class, like setOpt(). The subcommand is a different method like bar(), and it had a parameter matching the command already marked as inherited; that is, bar(boolean opt) or something. So Picocli gave me this error because the option was duplicated.

That wasn't what I was expected. I had hoped that by using bar(boolean opt) on the subcommand it would override the inherited version of the option setOpt() (even though I might want to call setOpt() from within bar(opt)). But I actually didn't need to override the inherited option, so I didn't mention it and went about my work because I had lots more to do.

Here is the ticket I used to perform the conversion. It should (in the "Development" section) link to a couple of commits and a pull request that should show you clearly the changes I made:

https://globalmentor.atlassian.net/browse/JAVA-195

Oh, actually that is just changing the debug command to an inherited command in the base class. In the subclass CLI application, I had to remove the debug option as a parameter. That was done in this ticket:

https://globalmentor.atlassian.net/browse/GUISE-114

I think these are the important commits:

• https://bitbucket.org/globalmentor/globalmentor-application/commits/66f43dfe5ba47aa3e8a0bfcde3914790038c5656
• https://bitbucket.org/globalmentor/guise/commits/ec6028ff322a1fdadd80b724c746d2bab78b6234

Hope this helps.

[palade]

Thank you both for your replies.

The Option1 and Option2 in my case are still ignored. These are logging info options which could be placed anywhere in the command-line arguments.

Please note that the parse handler is run with the RunLast option. The sub-commands are added programmatically to the top command, and to each is passed the entire array of args. What I don't understand is why the position of these options are important and how to include them in the sub-commands.

[palade]

I see this post on SO that RunLast will invoke only the last command. Still wondering if there is a way to pass the Option1 and Option2 to the sub-command. Otherwise if it is difficult or not possible, would be a way to adjust the USAGE, because at the moment shows like:

topcommand [Option1] [Option2] [COMMANDS]

would be possible to adjust the USAGE to show the exclusive option:

topcommand ( [OPTION1] [OPTION2] | [COMMANDS])

[remkop]

@palade Can you create a new ticket, and in the description show a way for me to reproduce the problem? Either example code or a link to a public repository would work.

[palade]

@remkop Will attempt to modify the usage message as making the above changes would require a larger codebase change. Was wondering if there some examples where I could programatically create the following customSynopsys:

Usage topCommand Option1 Option2
       or topCommand Subcommand 

There are some examples in the manuals, but would like to create this programmtically. I had a look at the help test but it seems that the examples that I found were harding the usage message. Would be possible to create this programmatically (i.e., instead of using the topCommand use the actual name of the top - Command?

[remkop]

@palade I’d be happy to help you accomplish your use case.

I believe it would be useful to create a separate ticket: this ticket ticket is closed, already has 70+ comments, and your use case may result in changes to picocli, which would be easier to track in a separate ticket.

So please indulge me and raise a separate ticket.

Now, I’ve been making guesses about what your code looks like, and I would like to stop guessing. :-) When you mention “my Option1 and Option2 are still being ignored” I have no idea what you tried or what could be the cause.

Please help me help you, and take the time to explain

• Steps to reproduce the issue (there’s no substitute for example code here)
• What you’re actually seeing
• What you expected to see/would like to accomplish

So far I partially understand points 2 and 3 only.

[remkop]

One more thing: there’s no need to create a custom synopsis <cmd> ([Option1] [Option2] | Subcommand). Surely that’s not the solution you really want?

We can make it work so that users can specify these options either before or after the subcommand.

But please create a separate ticket first with your code, so we can discuss more concretely.