Viper: Addresses of Items in Slice with `Get()`

[jakdept]

Let's say I have a json config that looks like the following, and I load it in:

{
  "slice": [
    {
      "name": "apple",
      "type": "fruit",
      "serving": 1
    },
    {
      "name": "banana",
      "type": "fruit",
      "serving": 1
    },
    {
      "name": "grape",
      "type": "fruit",
      "serving": 6
    },
    {
      "name": "carrot",
      "type": "veggie",
      "serving": 1
    },
    {
      "name": "broccoli",
      "type": "veggie",
      "serving": 1
    }
  ]
}

How, using Get(), would I access slice.2.serving - the serving size for grapes? I would expect that, but I honestly don't know. Is the same string used for GetInt() to access that same location? I didn't see any tests that looked to be referencing an index either.

I've looked through the docs, and I don't really see where it's specified how to get at items within a slice - I saw a lot of named stuff, lots of map[string]interface{}, but no slices.

[torbensky]

I just ran into this as well. It really surprised me, but it looks like Viper doesn't handle lists of objects. You have to call Get and then use reflection to cast it to a slice of interfaces. At least I think so, I hope I'm wrong because this is a really annoying thing to code around.

[torbensky]

The title of this issue is confusing though. I still don't really understand it. IMO the issue title would be more understandable to others if it was called something like:

Support getting list of objects from config

[torbensky]

Given the YAML:

foos:
  - name: foo1
    val: 1
  - name: foo2
    val: 2
  - name: foo3
    val: 3
  - name: foo4
    val: 4

Looking at the type returned by Get

fmt.Printf("%T\n", viper.Get("foos"))

Outputs:

[]interface {}

[torbensky]

Ah-hah, I figured this was too big of a hole in the library to not have a solution. There is a solution!

foos:
  - name: foo1
    val: 1
  - name: foo2
    val: 2
  - name: foo3
    val: 3
  - name: foo4
    val: 4

can be accessed like so:

var foos []struct{
  Name string
  Val int
}
viper.UnmarshalKey("foos", &foos)
fmt.Printf("%v\n", ps) // voila!

[torbensky]

Also, I now understand why the title is worded the way it is. I assumed the problem was being unable to access the slice of objects, but what the OP wanted was a way to address the value directly without deserializing the whole list, which makes sense.

[jakdept]

Yeah - you're right. I was going to change the title so it would fit better (and did because this was a while ago), but you can always do that - so the title makes a bit more sense.

TBH, I've since switched to using kingpin, and just unmarshaling for configs. Viper's not bad, but there's things I don't like.

[moorereason]

I'll admit I'm not a big fan of the GetGobbledyGook func names, but are you suggesting something like this:

func GetSliceStringMap(key string) []map[string]interface{}

Would that solve your use case? And is it worth growing the API when UnmarshalKey is available?

I'm not volunteering to fix--just trying to flesh out the desired feature.

[torbensky]

I think UnmarshalKey is fine and it's serving my needs. However, it would be nice to support array (slice) index access using the key path for all methods. This is I think what jakdept was asking for.

I'd imagine this to work like an XPath for JSON. Sorta like http://goessner.net/articles/JsonPath/ maybe.

So for the YAML:

foos:
  - name: foo1
    val: 1
  - name: foo2
    val: 2
  - name: foo3
    val: 3
  - name: foo4
    val: 4

You could call:

viper.UnmarshalKey("foos[0]", &foo)
viper.Get("foos[2]")
viper.GetString("foos[1].name")

etc...

[jakdept]

Yeah, sorry, I've been on other things. And you're correct - we can currently address by key in multiple different inputs. We cannot address by index though. A lot of that problem is that there's no index.

So, if we take the sample JSON input (i'm more familiar with JSON):

{"menu": {
  "id": "file",
  "value": "File",
  "popup": {
    "menuitem": [
      {"value": "New", "onclick": "CreateNewDoc()"},
      {"value": "Open", "onclick": "OpenDoc()"},
      {"value": "Close", "onclick": "CloseDoc()"}
    ]
  }
}}

When you call GetString("menu,value") you should get the string "File". And you can Unmarshal("menu,popup,menuItem") onto a slice of the right type just fine. But it would also be nice to Get("menu,popup,1") or perhaps Get("menu,popup,[1])") or whatever makes sense.

Of course, whatever syntax is picked should probably fall back to trying a key - because no matter what syntax you have as a string, you can have the same string as a key name.

I guess it only really seems confusing when you've got a slice in the JSON, and then a slice and a map at the same level within that slice - but that seems like a poor reason to avoid doing it in the first place.

[cyberbeast]

Any update on this? Stuck at a similar problem.

[mrngm]

@cyberbeast see this comment from @torbensky. They describe a solution that involves a custom (struct) type and using UnmarshalKey.

Currently having the same problem, but I will try the UnmarshalKey method.

[mrngm]

Using UnmarshalKey() into a slice of map[string]interface{}:

jsonCfg := []byte(`{"slice": [{}, {}]}`) // see first post

config := viper.New()
config.SetConfigType("json")
err := config.ReadConfig(bytes.NewBuffer(jsonCfg))
if err != nil {
    fmt.Printf("err: %v\n", err)
}

var dest []map[string]interface{}
err = config.UnmarshalKey("slice", &dest)
if err != nil {
    fmt.Printf("err: %v\n", err)
}

for i, contents := range dest {
    fmt.Printf("[%d]\n", i)
    for k, v := range contents {
        fmt.Printf("  k: %q, v: %q\n", k, v)
    }
}

output:

[0]
  k: "name", v: "apple"
  k: "type", v: "fruit"
  k: "serving", v: %!q(float64=1)
[1]
  k: "name", v: "banana"
  k: "type", v: "fruit"
  k: "serving", v: %!q(float64=1)
[2]
  k: "name", v: "grape"
  k: "type", v: "fruit"
  k: "serving", v: %!q(float64=6)
[3]
  k: "serving", v: %!q(float64=1)
  k: "name", v: "carrot"
  k: "type", v: "veggie"
[4]
  k: "serving", v: %!q(float64=1)
  k: "name", v: "broccoli"
  k: "type", v: "veggie"

Since the value part of the map is still interface{}, you might need to type assert it first, as you can see in %q!(float64=1).

[dylandreimerink]

I find myself searching for exactly this feature at the moment. There has already been a PR to add support for accessing specific indices a while back which didn't make it at the time. Maybe we can revive it?

[andig]

Fyi- I feel #863 is sort of overlapping with #861 but is less invasive though more indirect when applied to this issue.

[dylandreimerink]

I don't see #863 as overlapping with #861. I think that creating a slice of sub-vipers has use cases like when you want to include multiple configs in one file.

But I don't think it is practical when using arrays as part of a single config like this issue is describing. In terms of user experience, I think it is better to expand the key syntax and thus be able to select all values in a config without extra checks and steps.

[andig]

I think that creating a slice of sub-vipers has use cases like when you want to include multiple configs in one file.

I've mentioned it as you can also use it to workaround the restrictions of this PR as you can address the elements of the sub-vipers slice individually.

But I don't think it is practical when using arrays as part of a single config like this issue is describing.

Agreed, it's nothing more than an (ugly) workaround for this use case.