viper/flag cooperation

[xh3b4sd]

Hey there,

I was looking for a configuration management tool and viper looked promising. I was interested in how viper works with flag/pflag. This is what the README says (BTW it looks broken):

serverCmd.Flags().Int("port", 1138, "Port to run Application server on") viper.BindPFlag("port", serverCmd.Flags().Lookup("port"))

This does not work for me. Maybe a working example would be cool. Anyway, it looks very complicated to me to need to write two lines of code to let config files and command line flags work together. This is what I came up with:

$ cat main.go
package main

import (
    "fmt"
    "github.com/spf13/cobra"
    "github.com/spf13/viper"
    "os"
)

func main() {
    wd, _ := os.Getwd()

    viper.SetConfigName("config") // name of config file (without extension)
    viper.AddConfigPath(wd)       // path to look for the config file in
    viper.ReadInConfig()          // Find and read the config file

    var cmd *cobra.Command
    var s string

    cmd = &cobra.Command{
        Use:   "hugo",
        Short: "Hugo is a very fast static site generator",
        Long: `A Fast and Flexible Static Site Generator built with
            love by spf13 and friends in Go.
            Complete documentation is available at http://hugo.spf13.com`,
        Run: func(cmd *cobra.Command, args []string) {
            fmt.Printf("source: %#v\n", s)
        },
    }

    // Just use config of configuration file provided by `viper` as default value for the command line flag.
    cmd.Flags().StringVarP(&s, "source", "s", viper.GetString("source"), "Source directory to read from")

    cmd.Execute()
}
$ cat config.json
{
  "source": "not modified"
}
$ ./test
source: "not modified"
$ ./test -s foo
source: "foo"

It would be cool to know how this is intended to work and what people think about my proposal.

[biztos]

Hi there. I ran into this too and would like to ask the authors to please update the documentation and remove the mysterious "serverCmd" and also to note that you _must_ import their pflag fork instead of the original. These two things bought me some "lessons learned." ;-)

The following "works for me" and I hope it is useful to anyone following this issue.

// cli-test.go - test viper configs with pflag command-line overrides.
// -----------
// Goal: app should have configurable "foo" read from a config file
// cli-test-config.toml (or .json, et al); or in the environment as
// CLI_FOO; or as a command-line option --foo.

package main

import (
    "fmt"
    "github.com/spf13/pflag" // MUST use this one if using viper!
    "github.com/spf13/viper"
)

func main() {

    // FLAG (PRIMARY):
    pflag.String("foo", "Foo-Override-Sample", "the foo it be")
    viper.BindPFlag( "foo", pflag.Lookup("foo") )

    // ENV (SECONDARY):
    viper.SetEnvPrefix("cli") // e.g. CLI_FOO=bar
    viper.BindEnv("foo")

    // FILE (TERTIARY):
    viper.SetConfigName("cli-test-config") // .toml, .json, et al.
    err := viper.ReadInConfig()
    if err != nil {
        fmt.Println("No configuration file loaded.")
    }

    // DEFAULT:
    viper.SetDefault("foo", "DEFAULT-FOO")

    // DO IT:
    pflag.Parse()

    // What's the result?
    fmt.Println( "foo is", viper.GetString("foo") )

}

Then:

$ go build cli-test.go
$ cat cli-test-config.toml
# no foo
$ ./cli-test
foo is DEFAULT-FOO
$ echo 'foo = "FILEYFOO"' > cli-test-config.toml
$ ./cli-test
foo is FILEYFOO
$ CLI_FOO=enviroFoo ./cli-test
foo is enviroFoo
$ CLI_FOO=enviroFoo ./cli-test --foo=OverRideFoo
foo is OverRideFoo
$

[xh3b4sd]

The problem with the approach of viper.BindPFlag() is that you need to define this for each flag separately. This causes a lot of messy code in CLI tools having more than 10 flags per command or so. Currently we do the following to work around this problem.

package main

import (
    "fmt"
    "github.com/spf13/cobra"
    "github.com/spf13/pflag"
    "github.com/giantswarm/viper"
    "os"
    "strconv"
)

func main() {
    wd, _ := os.Getwd()

    defaultConfig := viper.New()
    defaultConfig.SetConfigName("default_config")
    defaultConfig.AddConfigPath(wd)
    defaultConfig.ReadInConfig()

    envConfig := viper.New()
    envConfig.SetConfigName("env_config")
    envConfig.AddConfigPath(wd)
    envConfig.ReadInConfig() 

    var cmd *cobra.Command
    var s string
    var i uint8
    var fl float64

    cmd = &cobra.Command{
        Use:   "hugo",
        Short: "Hugo is a very fast static site generator",
        Long: `A Fast and Flexible Static Site Generator built with
            love by spf13 and friends in Go.
            Complete documentation is available at http://hugo.spf13.com`,
        Run: func(cmd *cobra.Command, args []string) {
            mergeConfig(cmd.Flags(), envConfig)
            fmt.Printf("source: %#v\n", s)
            fmt.Printf("uint8:    %#v\n", i)
            fmt.Printf("float64: %#v\n", fl)
        },
    }

    // Just use config of configuration file provided by `viper` as default value for the command line flag.
    cmd.Flags().StringVarP(&s, "source", "s", defaultConfig.GetString("source"), "source")
    cmd.Flags().Uint8VarP(&i, "uint8", "i", uint8(defaultConfig.GetInt("uint8")), "uint8")
    cmd.Flags().Float64VarP(&fl, "float64", "f", defaultConfig.GetFloat64("float64"), "int")

    cmd.Execute()
}

func mergeConfig(fs *pflag.FlagSet, v *viper.Viper) {
    fs.VisitAll(func(f *pflag.Flag) {
        if f.Changed {
            return
        }

        flagValue := f.Value.String()

        switch f.Value.Type() {
        case "bool":
            viperValue := strconv.FormatBool(v.GetBool(f.Name))

            if flagValue != viperValue && viperValue != "" {
                f.Value.Set(viperValue)
            }
        case "string":
            viperValue := v.GetString(f.Name)

            if flagValue != viperValue && viperValue != "" {
                f.Value.Set(viperValue)
            }
        case "int64", "int32", "int16", "int8", "int":
            viperValue := strconv.FormatInt(int64(v.GetInt(f.Name)), 10)

            if flagValue != viperValue && viperValue != "" {
                f.Value.Set(viperValue)
            }
        case "uint64", "uint32", "uint16", "uint8", "uint":
            viperValue := strconv.FormatUint(uint64(v.GetInt(f.Name)), 10)

            if flagValue != viperValue && viperValue != "" {
                f.Value.Set(viperValue)
            }
        case "float64":
            viperValue := strconv.FormatFloat(v.GetFloat64(f.Name), 'f', 6, 64)

            if flagValue != viperValue && viperValue != "" {
                f.Value.Set(viperValue)
            }
        default:
            panic(fmt.Sprintf("unsupported flag type %s for flag %s", f.Value.Type(), f.Name))
        }
    })
}

[biztos]

Nice.

I hope I never have to build a tool with >=10 command-line options but I agree in principle, actually I dislike the pflag declarations even for a few flags.

I'm new to Go and this is probably a terrible suggestion, but why not just have some structs and have Viper (or someone) magically _get it done?_

// probably a stupid newbie trick...
package main

import (
    "some-another/viper"
    "fmt"
)

func main() {

    // My expectations, as it were, right out in the open:
    type Config struct {
        Foo string
        Bar int
        Baz bool
        Bat float64
    }
    // As few or as many as you like can be overriden.
    overrides := []viper.Override{
        viper.Override{Key: "Foo", Auto: true}, // FOO, --foo by default
        viper.Override{
            Key: "Bar",
            Env: "CLI_BAR",
            Opt: viper.Opt{
                Long:    "barrister",
                Short:   "b",
                Example: "'Melvin Belli'",
                Help:    "The barrister you wish to retain.",
            },
        },
        viper.Override{Key: "Baz", Env: "CLI_BAZ"}, // no flag.
        // Bat is not overridable
    }
    config := &Config{}
    file := "my-config.toml" // ...etc, more detail obviously

    // Why should I ever have to call anything more than this?
    viper.Config(config, file, overrides)

    fmt.Println(config.Foo) // etc
}

OK, I guess maybe I'm designing a different config manager now. But it seems to me there's a lot of code involved in the setup.

[kzvezdarov]

57 fixes this to an extent. Still only works with pflags, but you can pass it the whole FlagSet at once. I might make it work with the regular flag package, but as far as I can see the only way would be by having a separate binding function for flag and a separate registry for those, or maybe some shared interface if possible.

[spf13]

I think #57 is as far as we should take this. It's most of the solution and in as clean as a way as we can do here.

[AntonioSun]

Can somebody give a working demo code showing what's the suggested way to use the viper package please?

I was trying to make viper and cobra to work together, but was having a difficult time make it work or finding a working example. I don't know if #57 fixes works for cobra or not, but I DO need cobra, its subcommand capability, and fully posix compliant flags and the ability to define your own help and usage feature.

Can we have a demo code for above please?
Thanks