Spring-boot: Duplicate keys in YAML cause some unexpected results

[dsyer]

If user puts this in application.yml:

spring:
  datasource:
    url: jdbc:mysql://localhost/test
spring:
  main:
    headless: true

They might expect to be able to bind to MySQL and be surprised that the PropertySource only contains the spring.main.* entries. The reason is that there are 2 map keys "spring" and the 2nd one to be parsed wins.

We could fix this by using a custom Constructor in Snake YAML (appending to existing maps instead of overwriting them).

[dsyer]

The offline consensus seems to be that we should throw an exception if there is a duplicate key. The YAML spec says that MappingNodes must have unique keys, but Snake YAML doesn't seem to notice them, so we have to add a custom Constructor anyway.

[philwebb]

See https://jira.spring.io/browse/SPR-12318

[snicoll]

fyi, the related spring framework issue has just been fixed.

[asomov]

I am the developer of SnakeYAML.
Please be aware that the YAML specification explicitly puts the responsibility for keys to be unique to the application (not the parser).
If you think this is incorrect - feel free to raise your voice in the YAML mailing list.

[dsyer]

Andrey, thanks for reaching out, and thanks for SnakeYAML. I don't think we need to discuss it much, unless you are considering making changes in SnakeYAML. It seems like Snake is a parser and an "emitter" (according to your own strap line), so where does the responsibility lie? As far as the consumer of Yaml is concerned he gets a Map that doesn't correspond to what they were expecting, so we feel like we need to deal with it for them (by customizing SnakeYAML). Thanks for making that possible, and let us know if you need anything.

[asomov]

well, reading the description (see above "but Snake YAML doesn't seem to notice") gives a wrong impression that something is wrong in SnakeYAML. There is even an issue in the SnakeYAML's project.
The 1.1 specification explicitly puts the responsibility to the application (user). SnakeYAML should allow users to follow any policy they want. If you think you can survive with the current support in SnakeYAML we can close the topic.
You may give your opinion in the issue for SnakeYAML:
https://code.google.com/p/snakeyaml/issues/detail?id=199

[dsyer]

We can definitely survive with the current support in SnakeYAML, so thanks again for making that possible. I commented on the google issue.

[snicoll]

@dsyer I have moved to RC1 since we're going to upgrade to 4.1.2 that has the fix in it. See #1805

[iolo]

it happens again in spring boot 2.0.0.M3 :'(

[snicoll]

@iolo this issue is closed for almost _three_ years. If you have a question, please ask on Stackoverflow. If you believe you've found an issue, create a separate issue with a sample we can run to reproduce the issue.

[gavenkoa]

YAML v1.2 require uniqueness of keys:

http://yaml.org/spec/1.2/spec.html

JSON's RFC4627 requires that mappings keys merely “SHOULD” be unique, while YAML insists they “MUST” be. Technically, YAML therefore complies with the JSON spec, choosing to treat duplicates as an error. In practice, since JSON is silent on the semantics of such duplicates, the only portable JSON files are those with unique keys, which are therefore valid YAML files.

Mapping nodes are somewhat tricky because their keys are unordered and must be unique.

[asomov]

@gavenkoa : the project which implements YAML 1.2 does require the keys to be unique - snakeyaml-engine