Yq: In place write strips out comments

[jmreicha]

The CLI strips out comments given the following yaml file:

# comment
a:
  b:

Using the following command for my test:

yaml w -i test.yml a.b.c[0] "test"

[jmreicha]

Apparently it also strips out both single ' and double " quotes from arrays.

a:
  b:
    c:
    - 'test'
    - "test2"

Command I used:

yaml w test.yml a.b.c[1] "test2"

Let me know if you want a separate issue for this.

[mikefarah]

yep this is because it parses the yaml file using https://godoc.org/gopkg.in/yaml.v2, and then spits it out again.

This means it will not parse comments nor will it differentiate between those quotes if it is required to (for instance if the value is double quotes surrounded by singles ('"') ).

[jmreicha]

Got it, thanks.

[nemchik]

@mikefarah is there a way to get yq to NOT strip quotes? I'm using yq for Docker Compose and I need to keep quotes around ports like this

ports:
 - "3000"
 - "3000-3005"
 - "8000:8000"
 - "9090-9091:8080-8081"
 - "49100:22"
 - "127.0.0.1:8001:8001"
 - "127.0.0.1:5000-5010:5000-5010"
 - "6060:6060/udp"

[mikefarah]

Not at the moment - and given how it works with the yaml.v2 I'm not sure it can sorry :(

There may be a workaround I'm not aware of, I'll reopen this issue.

[nemchik]

Ok. Thanks for reopening the issue. Here is the source I'm referencing https://docs.docker.com/compose/compose-file/

Note: When mapping ports in the HOST:CONTAINER format, you may experience erroneous results when using a container port lower than 60, because YAML parses numbers in the format xx:yy as a base-60 value. For this reason, we recommend always explicitly specifying your port mappings as strings.

[IdanAdar]

Is there any way to re-add the quotes via some command, though?

[jeking3]

Too bad it does this, I have to abandon using it because it removes comments.

[devdream02]

@IdanAdar Have you found the solution of re-adding the quotes?
@mikefarah Any update when this functionality be added ?
Thanks!

[IdanAdar]

I have not.

[devdream02]

I have not.

Can you suggest any work around or some other library?

[IdanAdar]

No. In my case it just happened to continue working even without the quotes.

[evanstucker-hates-2fa]

This might be useful...

https://kev.inburke.com/kevin/more-comment-preserving-configuration-parsers/

I don't know Go, but someone wrote a library for it here:

https://gist.github.com/kevinburke/07a21855df700b0acdea9e91c6fde7cf

[danielfbm]

+1 for preserving comments. yq is a great tool, and we currently use it to automatically update our helm charts when a docker image changes

As in most helm chart's values.yaml file, the optional values that have a complex format/complex objects are generally commented, therefore it is really required to keep the comments in order to inform the user on how to customize these values.

I can think of a few workarounds to this using different files in our pipeline etc., but would be really appreciated if yq just preserved the comments

[shaleh]

https://ubuntu.com/blog/api-v3-of-the-yaml-package-for-go-is-available

It looks like moving to the v3 parser is the solution.

[andye2004]

@shaleh reading the release details you linked to I can see that it will solve the comments issue, but I can't see anything about quoted values. Am I missing something?

[mikefarah]

Yup - I've looked into the new library, looks really neat - but it requires significant rework of yq which I just don't have time for atm

[mikefarah]

@andye2004 when I played with it you could preserve quotes

[danielfbm]

any update on this issue? I am willing to collaborate if you will have me

[alexlemaire]

Following up on that: would really love to have comments preserved when updating files in place :+1:

I am using yq to update config files where I'd like to leave comments to give some extra hints to the users of this config file via comments.

[mikefarah]

Just released a new version of yaml that fixes this - it's a pretty significant update so it's still in beta: https://github.com/mikefarah/yq/releases/tag/3.0.0-beta

[jhagege]

Just released a new version of yaml that fixes this - it's a pretty significant update so it's still in beta: https://github.com/mikefarah/yq/releases/tag/3.0.0-beta

Hi, looks great.
Is it possible to update a value with double quotes ?
yq w a.yml services.a.command ""tox -vv -e\${ENVLIST}""

I get the following: command: '"tox -vv -e${ENVLIST}"' (enclosed within single quotes)

[mikefarah]

Yep - if you're wrapping with double quotes (so you can interpolate ENVLIST) then you'll need to escape the inner quotes.

x=thing
yq n services "\"cat${x}\""

will yield

services: '"catthing"'

That this therefore has the value "catthing" when reading the yaml.

If you want a single pair of quotes:

services: "catthing"

Then at the moment v3 does not support this, instead it only quotes values when required. It could support an override by passing in a 'style' flag (https://godoc.org/gopkg.in/yaml.v3#Style).

[mikefarah]

Sidenote:

services: catthing

and

services: "catthing"

Both effectively have the same values...

[jhagege]

Great, thanks for the quick and clear response.

On Wed, Jan 22, 2020, 12:07 AM Mike Farah [e-mail] wrote:

Sidenote:

services: catthing

and

services: "catthing"

Both effectively have the same values...

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/mikefarah/yq/issues/19?email_source=notifications&email_token=ABZIIDEOSR5MJAGHDQ5XQNTQ65W2JA5CNFSM4DX5UYK2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJROMJY#issuecomment-576906791,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABZIIDB5UOYS6QMKOX2DFXTQ65W2JANCNFSM4DX5UYKQ
.

[mikefarah]

Fixed in https://github.com/mikefarah/yq/releases/tag/3.0.1