Vault: Add support for yaml when writing

[philsttr]

vault read currently supports -format of table, json, yaml.

However, vault write only supports key=value pairs or json. Would be cool if yaml was supported as a write format as well. This way we could make apps only aware of yaml (which is much better for configuration, IMHO), and not have to deal with json.

As a secondary concern (maybe worth a separate issue), the vault write -help mentions -format and -field. I haven't quite figured out what these are used for. Wondering if they should be removed from the vault write -help, or if they actually do something.

[jefferai]

Input yml will not be supported. The format and field flags to vault write are there for control of returned output.

[philsttr]

Ok. Can you provide more info on why it won't be supported?

It just seems odd to me that yaml is only supported in one place (cli output). It seems like a second-class citizen, since it's not supported for writes (#1738), and it's not supported in the HTTP API (#1739).

It's a shame, because yaml is so much nicer to deal with than json.

[jefferai]

Supporting more than one input format provides no clear benefit (since no part of Vault's API requires features not present in JSON) but immensely increases the compatibility testing that must be performed and potential corner cases, especially when the various formats provide different features sets.

[philsttr]

Ok.

It may not provide a clear benefit internally for vault, but it definitely provides a clear benefit for clients of the api/cli (i.e. users). I mean, there is benefit for clients to read yaml (otherwise yaml output wouldn't have been supported). And since writing is the exact opposite of reading, it seems like yaml support for writing is an obvious feature, if nothing else but to provide consistency between reads/writes.

Side note: Are you planning on removing yaml support from cli output (to keep consistency, so that others don't expect/assume to be able to write yaml)? (I'm just trying to plan ahead so that I don't end up relying on a deprecated feature.)

[jefferai]

No, we aren't planning on removing it.

[ghost]

I agree with @philsttr that reading yaml would be very useful. yaml is often much easier to hand-craft and visually digest, especially for multiline content like certificates.

[ColinHebert]

@jefferai, is this something that Hashicorp is willing to reconsider?
I agree with you on the fact that it requires extra testing for corner cases and it can be a PITA, but yaml tends to be more flexible for different aspects.

Two examples that come to mind (on top of multiline content as @sushiandbeer said) are the support of comments within the format, the second one (which is more important I think) is the support of binary content in the format (base64 encoded in yaml, read as binary see http://yaml.org/type/binary.html).

JSon is convenient for a lot of things, its simplicity being one of the most important assets, but it tends to be much less flexible than yaml.

[jefferai]

@ColinHebert The two items you listed are characteristics of YAML; in and of themselves they are not use-cases relevant to why YAML should be supported for writing.

Most values in Vault that are human-crafted are in HCL which supports comments, and we have many places throughout Vault's API where base64-encoded binary data is consumed or returned.

[TerraTech]

I concur with @sushiandbeer handling x509 multiline certs are a pain with JSON and much easier to read/use with YAML.

What I do is have my admin tools or manually write them out as YAML, then pipe them through https://github.com/kislyuk/yq for the vault kv put ...

---
crt: |
  multiline cert stuff
key: |
  multiline key stuff
txt: |
  e.g. openssl x509 -noout -text -in x509.crt

$ yq . bundle.yaml | vault kv put kv/cert/foo -
or bash
$ vault kv put kv/cert/foo - < <(yq . bundle.yaml)

So far, this workaround has been working for me.