Toml: Schema defination

[hanabi1224]

Is it good or bad idea to make it possible to define toml schema with sth like (may a subset of) protobuf so that ppl can levarage on pb's infra to generate static types for different languages for toml to be serialized into. The extension point on toml side would a meta tag on a section to map to a strong proto type

[hanabi1224]

Another thing that would be helpful is context based parsing. e.g.

proto:

message Type1
{
  string prop = 1;
}

toml:

[default]
_meta.type = Type1

prop."prod eq true"= "blabla-prod"
prop="blabla-dev"

Given the context { "prod": true } , toml will be deserialize into Type1 with 'prop' value set to blabla-prod, or {"prod": false} then 'prop' value set to blabla-dev

This would be super helpful for some request based scenarios, like travis ci config, web config. Ppl can do

[default]
_meta.type = WebConfig

IsPage1Visible."Role eq 'admin'" = true
IsPage1Visible=false

# use theme 0 on Sunday
Theme."dow eq 0"="theme-0"
# use theme 1 on Monday
Theme."dow eq 1"="theme-1"
...
Theme="theme-default"

Admins=admin1,admin2

[admin1]
_meta.type = WebConfigAdmin

email="[email protected]"
name="aaa"

[admin2]
_meta.type = WebConfigAdmin

email="[email protected]"
name="bbb"

====================

[default]
_meta.type = BuildConfig

Enabled."branch eq 'master or is_pr eq true'"=true
Enabled=false

PublishArtifacts."branch eq 'master' and tag ne ''"=true
PublishArtifacts=false
...

All ppl need is a library that deserialize toml to proto generated strong type with an optional context dictionary, thus in the request handling logic, ppl get a strong typed config object with all values set properly under current context. There need to be a shared context dictionary, on code logic side, ppl just set the context dict property then resolve, and on toml config side, ppl just add conditions using predefined variables defined in context dict

[LongTengDao]

consider proposal #603 :

[default] <BuildConfig>

Because _meta is also a normal string key name, and should not be syntax reserved.

[mmakaay]

If you want protobuf, shouldn't you be using protobuf? Protobuf is for sending messages between different systems, TOML is a configuration file format. Both have their own strengths in their own fields.

The 'O' in TOML stands for 'Obvious' and I would very much like it to adhere to that. Looking at the example code, it's not immediately clear to me what I am looking at. It looks like information that I should be ignoring as a config file user, because it has no meaning configuration-wise. It's metadata that will be used for decoding the TOML data. If it's data that I should ignore, then it is probably data that should not be cluttering the config file.

If you would like a configurable schema definition, I would highly prefer that to be an actual meta document and not part of the actual configuration file. That is such a weird mixture of concepts and
programming responsibilities. Compare for example to XML, where XSD is a separate document and not a built-in part of an XML data file.

Also note that these annotations seem to refer to implementation details of code that will be using the configuration file. Implementation details make the config less portable if you ask me. Also, if I want to change some internal structure's name, all configuration files out in the wild need to be updated, since each configuration file out there has schema metadata in it. When the mapping between configuration file fields and internal data structures is separate from the configuration file, then this coupling is absent and change is easy.

I think it's clear that I am not a supporter of this concept.

[pradyunsg]

context based parsing

Uhm... That's non trivial and TOML won't be gaining such complexity.

If you want schema validation, that's a separate thing that should be done using a separate document. Since #76 and #116 already exist, I'm gonna go ahead and close this.

[hanabi1224]

@mmakaay I think you did not fully get my point on protobuf. The only reason why I choose protobuf here is that protobuf provides code gen tools for all major languages so that I strongly-typed toml config can work with any language with auto-generated strong types that IDE can provide intellesense from. It has nothing to do with data serialization and transport at all (althought that can be a side effect)

[hanabi1224]

@pradyunsg I agree that the context based parsing part is ambitious. If you look at all CI framework, they define their config syntax and implement their own context-based parsing logic. The same applies to all request based system including web api / services. The context-based parsing part is somewhere standard is missing