Cobra: Option to pass unknown flags to command

[underrun]

related to #623 - but I want the running command to receive rather than ignore unknown flags or return an error.

my use case is loading user defined (in another language) plugin functionality which can accept flags. I can't know these flags at compile time or even at command parsing time because i need the command tell me which plugins to load before i can load it to find out which flags it enables me to accept.

right now my option is to DisableFlagParsing - but that essentially means i need to build my own CLI parser to handle this particular code path and pull out the flags that my command accepts (which cobra could already have done for me).

[felicianotech]

This is already built-in via a Unix "standard". Typically, you append a double dash "--" and everything after that can get passed to a subcommand, another binary, etc rather than being parsed by the Cobra command as a flag.

For example:

my-cli command --normal-flag -- --this-is-ignored-by-cobra

[github-actions[bot]]

This issue is being marked as stale due to a long period of inactivity

[underrun]

I hadn't noticed the reply here - sorry.

The bare "--" doesn't work if the goal is to have a single interface where
the end user doesn't know what's under the hood. It wouldn't be obvious to
the end user what is supposed follow the "--" at all in that case.

The desire is to provide a seamless experience by hanging natively or
proxying help info/error into and flags.

On Thu, Apr 9, 2020, 00:05 github-actions[bot] [e-mail]
wrote:

This issue is being marked as stale due to a long period of inactivity

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/spf13/cobra/issues/739#issuecomment-611255329, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEOR3YESYXDOCZTDEJLKN3RLUGLFANCNFSM4FSSATVQ
.

[jpmcb]

From what I understand of your use case, I believe the double dash -- should work nicely:

POSIX.1-2017

12.2 Utility Syntax Guidelines

Guideline 10:

The first -- argument that is not an option-argument should be accepted as a 
delimiter indicating the end of options. Any following arguments should be 
treated as operands, even if they begin with the '-' character.

From the posix standard

So, each "flag" after the -- can be used as the _unknown operand_ to your application.

The desire is to provide a seamless experience by hanging natively or
proxying help info/error into and flags.

This may be achievable from within your application code. For example, within grep:

$ grep -- -v
-v                  # typed as standard input
-v                  # result of grep searching standard input

This will hang waiting for standard input searching for the string -v literal (which is ignored and _not_ passed as an argument to grep itself)

[macintacos]

I'm a bit confused as to how to get the proposed posix-compatible method to work; when I try the above (passing -- and then a flag to be parsed afterwards) I get unknown command: "--this-is-ignored-by-cobra". Can I get some pointers on this? Any examples I can look at?

[jpmcb]

Hi @macintacos - what have you tried? Can you provide a code snippet of what you're seeing? I believe you will still need to parse the options after the double dash -- in your application code if you are attempting to pass in optional arguments

I was able to get this to work with the base cobra init generated application:

$ go run main.go --not-ignored
Error: unknown flag: --not-ignored
...

$ go run main.go --help -- --ignored-by-cobra
A longer description that spans multiple lines and likely contains
examples and usage of using your application. For example:

Cobra is a CLI library for Go that empowers applications.
This application is a tool to generate the needed files
to quickly create a Cobra application.

Let me know if that is helpful!

[macintacos]

@jpmcb thank you for your quick response!

I think I might've messed up a call to exec.Command, because I shuffled some arguments around and it worked... thanks for at least giving me some hope that it was just something that I was doing wrong and not me running into some weird bug 🙂

Although I think I would agree with the original premise of this issue; it'd be nice if it were possible to tell cobra to blindly accept additional flags/commands if you know what you're doing. In my case, I would take all of the flags and arguments and pass them to exec.Command, since the command I'm creating is essentially running a bash script; it doesn't feel necessary to abstract it in this manner (although I'm fine with the workaround)

[davidovich]

I also think cobra could help here. Something in the lines of python's parse_known_args would allow us to parse known flags and pass other flags to wrapped commands.

[underrun]

@davidovich that's essentially what i was hoping for. again, the -- workaround doesn't work if what you are building is an interface to multiple other underlying things that you want to allow to handle the help/error info themselves.

i completely get that this isn't the common cli use case, but it's so close to doing what is needed to support better delegation that it's frustrating to run into this when it happens.

[jpmcb]

@jharshman @wfernandes thoughts on adding some of this functionality? I'm hesitant since it's baked into the posix standard with the --. And while yes, this may require application developers to include more documentation for their end users and include parsing the unknown args in their application code, I am not sure about subverting standards built into the operating system.

what you are building is an interface to multiple other underlying things ...

I'm also hesitant since the general ethos for unix like commands is to do one thing really well and be enabled to chain in with other commands. For example, instead of a single "line by line" monolithic processor in unix that handles searching, replacing, and cut/pasting lines in files, we have grep, sed, and cut. These chain really well together but also work by themselves. This feels more in line with the spirit of cobra. I don't believe cobra was ever intended to be a tool to generate large, multi-interface applications.

Would it be possible to break apart this monolithic CLI to better handle chaining and different interfaces?

However, I do see the benefit of implementing something like python's parse_known_args. A single interface within cobra to abstract away some of this from the end users would help bring some more complete functionality.

@underrun can you give some examples of what an end user would experience with this "multiple other ulderlying" commands? Maybe I'm having a hard time picturing it.

[davidovich]

I have a use case where I am writing a generic command runner. This runner is configurable by the user, and can call into many utility scripts, but the runner itself cannot know at build time what it will delegate to, only the user knows the flags at invocation time.

A binary plugin system where a driver is responsible for delegating to a subordinate cli is yet another use-case.

I see this kind of usage:

c := &cobra.Command{
    Use:            "run",
    FParseErrWhitelist: cobra.FParseErrWhitelist{UnknownFlags: true},
    RunE: func(cmd *cobra.Command, args []string) error {
        cmd.SilenceUsage = true
        // use the hypothetical UnknownArgs() pflag API
        delegatedArgs := cmd.Flags().UnknownArgs()
        child := exec.Command(args[0], delegatedArgs...)
        return child.Run()
    },
}

[underrun]

This initially came up when working on ksonnet (replaced by tanka) which needed to handle CLI options for managing, using and deploying kubernetes manifests generated from jsonnet (a configuration language) where jsonnet had language specific options we didn't want to replicate because they could change based on the version of jsonnet. We also wanted to maintain a cohesive UX that didn't differentiate between what is was jsonnet and what was ksonnet because the concepts were so very closely related in some cases.

Unix philosophy doesn't preclude doing one conceptual "thing" and doing it well. Git started off as a bunch of individual commands but it makes more conceptual sense as one CLI tool. Sometimes having a CLI for one higher level thing can aide in discoverability of concepts and features, enforcement of best practices, automation of common tasks around that higher level concept and it may be easiest to achieve all that through some composition of lower level tools combined with custom logic.

And I'd love to easily write that custom logic in go :-)

[davidovich]

If anyone wants a workaround, I came up with this atrocity:

func extractUnknownArgs(flags *pflag.FlagSet, args []string) []string {
    unknownArgs := []string{}

    for i := 0; i < len(args); i++ {
        a := args[i]
        var f *pflag.Flag
        if a[0] == '-' {
            if a[1] == '-' {
                f = flags.Lookup(strings.SplitN(a[2:], "=", 2)[0])
            } else {
                for _, s := range a[1:] {
                    f = flags.ShorthandLookup(string(s))
                    if f == nil {
                        break
                    }
                }
            }
        }
        if f != nil {
            if f.NoOptDefVal == "" && i+1 < len(args) && f.Value.String() == args[i+1] {
                i++
            }
            continue
        }
        unknownArgs = append(unknownArgs, a)
    }
    return unknownArgs
}

[freshsun]

I have another user case here. I am developing an assisting tool for hadoop/spark, people use a cobra-developed-cli as a prefix when calling hadoop/spark-submit command, so it can communicate with an custom http server instead of a native one. The logic is implemented within the prefix cobra-cli. Now is the problem, I cannot implement custom flags for cobra-cli, because either I reimplement all hadoop/spark-submit flags again, or let cobra-cli ignore all flags.
What i am doing now is using ENV variables as flag, but it is not very friendly, and functional restricted, and help message is also hard to implement.

[jpmcb]

Thanks for the feedback all! Yes, I think it makes sense to implement this. However, I think to have a streamlined, unified interface, the pflags library will need to include a flag set of unknown flags.

Looks like there's already an open PR for this. Maybe we can generate some traction there:
https://github.com/spf13/pflag/pull/199

Once we have a pflags unknown flag set, we should be able to easily get it back to the cobra application code.

[davidovich]

Once we have a pflags unknown flag set, we should be able to easily get it back to the cobra application code.

In fact, just a module version bump would suffice as Flags() returns a FlagSet already, on which we can call the new SetUnknownFlagsSlice() API.

[github-actions[bot]]

This issue is being marked as stale due to a long period of inactivity

[davidovich]

Let's keep it active.