Lektor: Suggesting new contents.lr file format

[relikd]

I think this can be implemented without a breaking change.
Current format is always terminated with three dashes:

body: 

Multi-line
text content
---
title: single-line content
---
emptyAttr:
---
boolean1: yes
---

I do think this is a lot of boiler plate code espcially if most attributes are single-line.
My suggestion would result it the following:

body:

Multi-line
text content
---
title: single-line content
emptyAttr:
boolean1: yes

There are three scenarios. If the line (after the colon) contains:

1. any non-whitespace → it is considered single-line and terminated by \n and/or \r
2. only whitespace and the following line is also whitespace → it is considered a multi-line attribute and terminated by ---
3. only whitespace and the following line is not whitespace → the attribute is considered empty

This parsing is almost completely compatible with the current file format. In the 1. and 3. case, after a single-line attribute we simply ignore the three dashes in the following line. Case 2 hasn't changed and is processed till the three dashes. So we could use the same parser for both, v1 and v2 content format.

However, there is one case where it might indeed break.
I havent tested these, but I _think_ they are perfectly valid as of now:

tag1: multi-line
starting after tag
---
tag2: 
multi-line
starting in the following line
---

So, in case someone has edited the content file manually and removed the double newline it would break parser v2.
Luckily the UI produces the double newline already, so for almost all projects this should be fine.

We could even recover from a parsing error. In this scenario the parser would assume a new attribute on the 2nd line. If we don't detect an attribute name (v2) or --- (v1) in the next line, then there must be something wrong with the format. So we show the faulty file to the user.

Of course there are edge cases that won't be detected, like:

blogTitle:
Wednesday: Workshop
---

(the parser would assume Wednesday is an attribute and won't complain)

[runfalk]

Before considering this I'd like to know why you want to change the current format. I do agree that it is a bit verbose though. I'm also a bit concerned about the ambiguous cases that are valid today but will be handled differently with the proposed syntax. The syntax is part of our public API and the docs aren't very strict on multi-line fields:

For values with multiple lines it's recommended to insert two newlines after the key.

(https://www.getlektor.com/docs/content/)

[relikd]

Thats the only reason, aesthetics. There is no "greater" benefit or such. But I think its easier to read / parse when reviewing the file manually (with human eyes). Also, you would see more content at the same time, without the need to scroll etc. And you would save 4 bytes per attribute. Which isn't a good reason either, but hey…

I was curious how likely it is that someone would "mess up" the recommendation of using two newlines after the key. So I cloned all open source repositories that are listed here: https://github.com/lektor/lektor/network/dependents and ran a script on all .lr files.

__EDIT:__ ok, my bad. I've removed the wrong statement. There are definitely more cases. Seems to be quite common that people manually edit the files and don't respect the double newline recommendation. I found 1267 attributes in 7013 files that would be affected. Though a single file often contains multiple attribute issues.

[davidism]

Counterpoint to yours: I find the current format easy to read/parse with human eyes. Every field is guaranteed to be separated by --- and be a simple key: value. It requires zero extra parsing, as a human or with Lektor.

That said, there's an issue opened about allowing a project to use a different parser: #751. If that gets through you could write a lektor-non-separated-config package to provide a different parser.

[runfalk]

I found 1267 attributes in 7013 files that would be affected. Though a single file often contains multiple attribute issues.

I really like that you took the time to investigate this. Sounds like we should be very cautious then. In other issues we have discussed exposing the content file format to plugins (https://github.com/lektor/lektor/issues/751#issuecomment-614840711). That may be one way to go about this.

Another idea how to implement this is to include a minimum required Lektor version attribute as part of the project file (like Cargo's https://github.com/rust-lang/rfcs/pull/2495). Some distros like Fedora actually packages old Lektor versions (3.0.1). If that attribute doesn't exist we will parse using the old parser and update the .lr files in place. This migration process would automatically add the minimum required version.

[relikd]

Reading through #541 and the suggestion of including files, we could even enforce single-line attributes. Each attribute that is multi-line (html, markdown, text) will automatically be outsourced into its counterpart file. Without the need of escaping --- on user input. The parser would be even simpler than it is now. One line, one attribute, split by the first colon.

As for the outsourced file, it does not need to be named anywhere. The name could be inferred by the attribute key and model description, e.g., body.md. Enabling syntax highlighting and what not. For flows they could be enumerated, flow-type.1.body.md, etc.

Thats of course only possible with a breaking change, or with the minimum required version as suggested.

[xlotlu]

Counterpoint to yours: I find the current format easy to read/parse with human eyes. Every field is guaranteed to be separated by --- and be a simple key: value. It requires zero extra parsing, as a human or with Lektor.

I'm with @davidism on this one.