Viper: passing nested struct from environment

[gunjanpatidar]

Is there a way to pass nested slice of arbitrary struct via environment variables? The below code works fine with a yaml config file but I'm not able to get it working via env (for production)

````go
package main

import (
"log"
"strings"

"github.com/spf13/viper"

)

type HostConfig struct {
Host string mapstructure:"host"
Port int mapstructure:"port"
}

type Config struct {
ClusterMode bool mapstructure:"cluster_mode"

Hosts []HostConfig `mapstructure:"hosts"`

}

func main() {
viper.SetEnvPrefix("service")

replacer := strings.NewReplacer(".", "_")
viper.SetEnvKeyReplacer(replacer)

viper.AutomaticEnv()

var cfg Config
err := viper.UnmarshalKey("redis", &cfg)

log.Printf("cfg: %+v\n\nerr: %+v", cfg, err)

}
````

````sh
$ SERVICE_REDIS={"cluster_mode":"true","hosts":[{"host":"abc","port":123}]} go run main.go
2019/02/01 12:50:53 cfg: {ClusterMode:false PoolSize:0 Hosts:[]}

err: '' expected a map, got 'string'
````

[sagikazarmark]

You could try creating a type HostConfigs []HostConfig and implement a text unmarshaler on that type. I would also take a look at what mapstructure provides for custom types.

[gunjanpatidar]

@sagikazarmark That could work but I'm doing this in a larger project where I'm trying to use a yaml config file for local development environment but it's overridden in production by environment variables.

[gunjanpatidar]

Got it working with this, would be better to support this natively though

````golang
package main

import (
"encoding/json"
"log"
"reflect"
"strings"

"github.com/spf13/viper"

)

type HostConfig struct {
Host string mapstructure:"host"
Port int mapstructure:"port"
}

type Config struct {
ClusterMode bool mapstructure:"cluster_mode"

Hosts []HostConfig `mapstructure:"hosts"`

}

func main() {

viper.SetEnvPrefix("service")

replacer := strings.NewReplacer(".", "_")
viper.SetEnvKeyReplacer(replacer)

viper.AutomaticEnv()

var cfg Config
opt := viper.DecodeHook(
    JsonStringToStruct(cfg),
)
err := viper.UnmarshalKey("redis", &cfg, opt)

log.Printf("cfg: %+v\n\nerr: %+v", cfg, err)

}

func JsonStringToStruct(m interface{}) func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
return func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
if rf != reflect.String || rt != reflect.Struct {
return data, nil
}

    raw := data.(string)
    if raw == "" {
        return m, nil
    }

    err := json.Unmarshal([]byte(raw), &m)

    return m, err
}

}
````

````sh
$ SERVICE_REDIS": "{\"cluster_mode\":true,\"hosts\":[{\"host\":\"abc\",\"port\":1234}]}"
2019/02/01 18:46:58 cfg: {ClusterMode:true PoolSize:0 Hosts:[{Host:abc Port:1234}]}

err:
````

[sagikazarmark]

I would say JSON in environment variables is not really common, so I would rather keep this as a recipe if someone needs it.

[ghost]

to support also slice of struct and any kind of var I use this one:

func JsonStringAutoDecode(m interface{}) func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
    return func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
        if rf != reflect.String || rt == reflect.String {
            return data, nil
        }
        raw := data.(string)
        if raw != "" && (raw[0:1] == "{" || raw[0:1] == "[") {
            err := json5.Unmarshal([]byte(raw), &m)
            return m, err
        }
        return data, nil
    }
}

type Config struct {
  Dependencies []Dependency
}
var config Config
opt := viper.DecodeHook(
    JsonStringAutoDecode(config),
)
if err := viper.Unmarshal(&config, opt); err != nil {
    logrus.Fatal("Unable to unmarshal config")
}

I use github.com/yosuke-furukawa/json5/encoding/json5 but you can use encoding/json the same way

[mykter]

The yaml version is the straightforward equivalent:

func yamlStringToStruct(m interface{}) func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
    return func(rf reflect.Kind, rt reflect.Kind, data interface{}) (interface{}, error) {
        if rf != reflect.String || rt != reflect.Struct {
            return data, nil
        }

        raw := data.(string)
        if raw == "" {
            return m, nil
        }

        return m, yaml.UnmarshalStrict([]byte(raw), &m)
    }
}

You can test with something like MYPROG_KEY=$(yq '.key' conf.yaml -y) ./myprog
And for the benefit of anyone searching for the same error I hit: "expected a map, got string".

It would be helpful to reference how to do this in the docs somewhere - it's not obvious to me that the behaviour for unmarshalling environment variables would be so different to the behaviour for config files.