Toml: Proposal: syntax to reset nesting back to top-level

[zimbatm]

Since TOML has no syntax to "close" a section, it's not possible to assign top-level values later in the document. In this proposal, I am asking to extend the syntax to allow us to do that.

Problem statement

I have encountered this issue in various contexts over time. In all these instances, the TOML file is composed by concatenating a sequence of strings together by some tool (Ansible, Bash, Nix...).

For example, let's say that the config is generated by a bash script. You might have a more complicated version of this:

#!/usr/bin/env bash
cpus=$1
max_memory=$2
extra_config=$3

cat <<MAIN_CONFIG > config.toml
cpus = $cpus
threads = 0

[memory]
max = $max_memory
MAIN_CONFIG

cat "$extra_config" >> config.toml

Now let's say that I don't control that script and want to override the top-level threads value by providing it in $extra_config. This is not possible right now.

Proposed solution

Add a notation to "reset" the section. This is probably where most of the bike-shedding will happen.

Various options:

1. []
2. [.]
3. [...]

[eksortso]

There are actually two things going on here, so let me address the more fundamental issue first. It's unlikely that you'll find a solution because you're asking for a way to redefine a key's value. Values can never be changed in TOML. Once threads is defined, that's it. There no changing it. And that would be the same even if threads were to be defined in a subtable.

Sections end when new sections are declared. The top-level section always occurs at the top of the document. That's something else that's unlikely to be changed. But maybe you can use this to your advantage.

So there isn't a way that you could append a new value for threads. But if you can change config.toml before it is used, you could find the first instance of threads = 0 before any line beginning with a bracket (not counting whitespace), and replace it with threads = whatever number you want to replace the zero with.

[zimbatm]

In the context that I am describing, the configuration framework only allows to append a chunk of text to the document. So replacing values is not really doable. Let's change the premise a little bit and say that threads is a valid key in the schema and that it has been left undefined in the original configuration.

[marzer]

You could allow for an explicit [overrides] table at the bottom of the document, that way anything that gets appended to the end of the file is explicitly done so as an override? Seems more in line with the intent.

[ivanjaros]

i wanted to give toml a chance but this is literally the first thing i encountered. unusable :D

edit: i would expect the block to end by new line or possibly with [] header

[zimbatm]

You could allow for an explicit [overrides] table at the bottom of the document, that way anything that gets appended to the end of the file is explicitly done so as an override? Seems more in line with the intent.

This argument comes from the perspective that the software needs to be changed to satisfy the requirements. In the context that I am describing, the user doesn't necessarily have the time or inclination to do it. They are in the context of only configuring said software. Even if they patch the software, that extension mechanism has to be repeated for every program that uses TOML as configuration languages. I understand that it's a trade-off and that it complicates the spec.

[ChristianSi]

@zimbatm I'd say that if you feel the need to spread the same table over several parts of the document, your data is likely organized the wrong way. That will also be confusing for your users.

[zimbatm]

TOML also doesn't support re-defining the same table twice so my use-case would require even more changes to the spec. Abandoning this, thanks for reading!