Influxdb: Reorganize the hierarchy of the CLI to be more action oriented

[russorat]

$ influx get buckets
$ influx export
$ influx apply pkg
$ influx config ...

instead of having the rest hierarchy we have now, we orient it around the action users want to take. I think it supports what you are after more so than cobbling it together in its current shape.

@jsteenb2 can add additional details here.

additional details from @jsteenb2:

Influx CLI proposal

This proposal is advocating for action/verb oriented commands.
At the time of writing this, the CLI is more or less a mirror of the resource heirarchy of the REST API.
I am submitting that we can change up this workflow in the CLI, to minimize command bloat, standardize inputs/outputs, and make it simpler to extend the CLI to support additional features.
In addition, we can support the existing CLI while we transition, but while we are in beta, we can deprecate these and sunset them early in the v2 GA release cycle, if not before.

New Commands

influx list cmd

influx list|ls [resource command] [--name --id]

Both --json and --hide-headers flags are still supported in addition to the --token, --host and org flags.

The list command will have the following subcommands:

influx list buckets

influx list labels

influx list organizations|orgs

influx list secrets

influx list tasks

influx list telegrafs

influx list authorizations|auth

API for list command does not limit how many filters you can use.
The API should allow the user to filter single record or all of them.
Follows same output mechanics with table/json out.

influx list buckets --name=Name1 --name=Name2 --id=ID1 --id=ID2

The CLI should also extend list commands to support args where we inspect the arg to see if it is a valid ID.
If arg is a valid ID then uses it to filter results.
If it is not then the arg is assumed a name and uses it to filter results.
If both args and flags are used, then the CLI should work all the same.

influx list buckets Name1 Name2 ID1 ID2

If no args or flags are provided, then the list command returns all entities.
Paginating as needed, while writing to stdout after each page.

influx list buckets

influx apply command

influx apply [resource command] [pkg flags]

The influx apply command and all its sub commands take piped inputs in the form of a package.

The base command is equivalent to the influx pkg command.

influx apply --file PATH_TO_PKG --url URL_TO_PKG ... other flags

The available sub commands are:

influx apply bucket

influx apply label

influx apply organization|org

influx apply pkg

influx apply secret

influx apply task

influx apply telegraf

influx apply authorization|auth

For all resource sub commands they can provide input via flags for creating or updating a resource.
When an id is provided, the command will update the existing resource identified by its ID and will update it to the specification provided.

# apply and create a bucket from pkg via piped input
cat pkg.yml | influx apply bucket

# apply and create a bucket via file input
influx apply bucket --file PATH_TO_PKG

# apply and create a new bucket via flags
influx apply bucket --name BUCKET_NAME --retention-period 24h

# apply and update existing bucket via flags
influx apply bucket --id BUCKET_ID --name BUCKET_NAME --retention-period 24h  

When providing a pkg, that has more than just buckets in it, the pkg will apply all buckets from that package and any associations outlined in it (aka labels).

influx remove command

influx remove|rm [resource command] [--force]

Available resource sub commands:

influx remove authorization|auth

influx remove bucket

influx remove label

influx remove organization|org

influx remove secret

influx remove stack

influx remove task

influx remove telegraf

Each sub command should support taking args via flags and or arguments.

# remove by --id flag
influx remove bucket --id $BUCKET_ID

# remove by --name flag
influx remove bucket --name $BUCKET_NAME

# remove by input arg being ID
influx remove bucket $BUCKET_ID 

# remove by input arg of bucket name
influx remove bucket $BUCKET_NAME

# mix of names, ids, flags
# this sample removes up to 4 buckets if names and ids are all unique
influx remove bucket \
    --id $BUCKET_ID_1 \
    --name $BUCKET_NAME_1 \
    $BUCKET_ID_2 $BUCKET_NAME_2

All remove commands should ask for confirmation if the --force flag is not provided

influx export command

The export command is used to export existing resources in the form of a package.

influx export [command] [existing flags]

Export has the following sub command(s):

influx export all [existing flags]

# potential sub commands by resource (if desired)
influx export buckets

influx export checks

influx export dashboards

influx export endpoints

influx export labels

influx export tasks

influx export telegraf-configs|telegrafs

influx export variables

influx convert command

The convert command takes a raw input and turns it into a package.
The first example would be to convert a telegraf configuration into a package.
Other examples like converting non-influx v2 platform resources (v1 -> v2 resources, grafana dashboards -> v2 dashboards, etc).

Existing commands which are all action/verb based which will remain unchanged

influx CLI specific commands

influx completion

influx config

influxd ping

influx ping

query commands

influx query

influx repl

influx transpile

setup command

influx setup

write commands

influx backup

influx delete

influx write

[jsteenb2]

@jademcgough this is similar to the discussion we had previously

[mark-rushakoff]

I agree that get buckets reads better, and when you know what you want to do, it writes better; but it is extremely less discoverable.

If I run influx help I want to discover what objects I can operate on. Seeing that I can run get or delete or find is minimally helpful. Seeing that I can operate on buckets, orgs, users etc is actually helpful. And then running influx buckets help to see the operations available, will show me the exact set of operations that apply only to buckets.

If we change from influx <object> <verb> to influx <verb> <object>, how do you see the --help output changing?

[jsteenb2]

@mark-rushakoff I hear you. Right now we support about half the resources we have in the platform. The -h usage is pretty big already and only grows as we extend things. The verb -h would explain the actions themselves and provide what they can do.

$ influx -h

get             retrieve resources ....; Example get buckets
apply           apply resources from a package...
export          export resource(s) to a package...
config          ... yada yada
completions     ... yada yada

don't quote me on that write up, but something to that affect. I'd fully expect the influx get command would show all resources it can get though.

Interestingly, I find the current -h so busy I miss things in it. I like the separation here, filters down the root commands and nukes a bunch of filter commands that do nothing.

For instance:

influx buckets

this command does nothing except display the child commands. Every resource has this structure (minus pkg command).

It also leads to behavioral anomalies which has affected the influx CLI. I want to see it more uniform. And eventually, when the API designs become more standard, be able to generate the entire CLI. Those are my long tail dreams for the CLI :grimacing: . This new design is a step in that direction for standardizing the entire CLI.

In short, it being extremely less discoverable is debatable. I'd wager its more discoverable, extensible, and uniform. With our shell autocompletions in place, we can make these things very easy to discover.

[mark-rushakoff]

I see. Maybe my mental model is stuck back in the >1y ago ways, where we were less consistent on the verbs used. If the list of verbs is small and fixed like get/find/update/delete, then that isn't too bad.

I would still like a clear way to discover what resources you can operate on, to remove ambiguity around "is it get orgs or get organizations"? That seems fine if you can discover that by influx get --help, but there ought to be a more direct way to discover that list. In any case, that can be separate from this issue.

We may need a period of supporting the old ways while warning that it is deprecated, to give people a chance to migrate existing scripts using influx bucket get to influx get bucket.

[jsteenb2]

@mark-rushakoff absolutely agree with you. I think we'll end up with a small number of root commands. I think we'll be able to migrate existing commands over, we can have them supported without them showing up in the usage. That would enable us to move forward and remain backwards capatability.

heads up, we have support for shell completions in existing influx CLI. Running influx completion -h will provide help for setting it up. For me using zsh, all I have to do is source <(influx completion zsh) and everything is gravy. I dumped that line into my dotfile so that my influx ClI completions are always up to date. Works like a charm

[jsteenb2]

@mark-rushakoff I updated the issue here with a more thorough description of what I envisioned. Would greatly appreciate your thoughts.

[rbetts]

_Discovering the resource types_: What about rendering some of the swagger resource definitions via a CLI action? I have a hard time finding the swagger rendering. Having it in the CLI might be useful to others and help around resource discovery?

_Using the term export_: at the risk of bike-shedding, we've use "export" to mean "backup" in 1.x [^1]. Using backup and restore seems fine to me. Just FYI.

_About beta_: We likely have cloud2 customers using the CLI? Also just FYI for transition planning. We can probably identify those users from logs if we make the CLI send an identifiable header / user-agent string?

_Consistency with influxd options_: We have been trying to put all the functionality that runs against a running instance in the influx command and all the functionality that runs against offline data in the influxd command. I don't know that there needs to be any real alignment between the two tools as they are quite different; however, probably worth considering ahead of beta if reviewing CLI structure a final time.

[jsteenb2]

@rbetts great points. Rending swagger via a CLI action, definitely doable. If we have the open api spec, we can even generate curl commands and whatever else we want. Having a spec is incredibly powerful 😃. My long term plan/dream is to have the open api spec generate good chunks of the CLI so it is always up to date. Of course the open api spec itself will be generated from the handlers (always up to date) in this scenario.

This is part and parcel of the service definition work in my mind.... when we have a clear and repeatable pattern, things are much easier to maintain and extend :-)

[mhall119]

There's no reason influx help buckets couldn't return information showing you influx list buckets, influx export buckets, etc.