Zettlr: [ENHANCEMENT] Support bibliography and csl fields in YAML metadata

[frederik-elwert]

Description

Support per-file bibliography and csl style via the corresponding YAML metadata fields.

Proposed Changes

Currently, Zettlr requires you to define bibliography and CSL style globally through the settings. This becomes cumbersome when switching between projects: One will probably have separate bibliography files for different articles, and CSL styles will also change from journal to journal.

Since Zettlr now somewhat supports YAML metadata blocks, it should respect the entries bibliography and csl in the metadata. Specifically, if either of these variables is defined in the metadata, Zettlr should use the specified files instead of the ones defined globally in the settings.

Caveats

This would be backwards compatible as documents not specifying bibliography or CSL style in the metadata would still use the global settings.

[nathanlesage]

Well, it's not as easy as that. Citeproc currently expects one single database, and switching between them takes some time. I have not actually run benchmarks, but this is not intended by the engine as of now. But I might be able to capture the citeproc processor and return different results based on a specific library.

However, as I already feel very bad for not resolving some of the older issues due to some urgent deadlines at work, this one (for now) has a rather low priority :(

[conig]

Well, it's not as easy as that. Citeproc currently expects one single database, and switching between them takes some time.

Perhaps this is an ugly solution, but multiple databases could be merged and stored as a temp file before being passed to citeproc?

[frederik-elwert]

If I see it correctly, currently the processor reloads the database on every file change. I assume that this can occur quite frequently, when people are adding references as they write. Switching between projects will, in contrast, probably occur much more seldom, as I assume that most people will not work in parallel on multiple articles – at least not those that have a project-specific library.

So I would expect a minor delay during project switching to be tolerable. But of course that’s my outsider’s view, and based on a lot of assumptions, so I understand if things look differently from another perspective.

[nathanlesage]

I assume that this can occur quite frequently, when people are adding references as they write. Switching between projects will, in contrast, probably occur much more seldom, as I assume that most people will not work in parallel on multiple articles – at least not those that have a project-specific library.

Even better: I can make it so that it iteratively reads in all libraries as it encounters them and simply exchanges the citation processor contents every time it should switch the library. However, time and stuff — you know the drill —, so it'll have to wait :D But I already have a very good idea of what I will be doing :)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[the-solipsist]

Adding a comment, as promised in https://github.com/Zettlr/Zettlr/issues/459#ref-issue-705410049.

Zettlr doesn't need special support for any YAML fields that simply modify conversion. The YAML fields that need support in Zettlr are the ones that ~will~ could modify Zettlr's behaviour. Currently, they are:

1. "title"
2. "citation-style" / "csl"
3. "bibliography"
4. "id"
5. "keywords"
6. "lang"

Given that each of these (except ID) could be provided in a Pandoc defaults file / metadata file, I would suggest that Zettlr could also support:

1. "metadata-files"/"metadata-file"
2. "defaults-files"/"defaults-file"

In those files, Zettlr could check whether any of the first ~five~ six fields exists. In case of there are multiple occurrences of a field, then Zettlr should ensure that its behaviour in-editor is the same as what people would get once they export the document. Thus, Zettlr would co-ordinate its behaviour with Pandoc.

The Pandoc rules, as far as I've been able to ascertain them are:

1. "A document may contain multiple metadata blocks. If two metadata blocks attempt to set the same field, the value from the second block will be taken." (Manual)
2. "Metadata values specified inside the document, or by using -M, overwrite values specified with [metadata-files:/--metadata-file] option." (Manual)
3. "This option [metadata-files:/--metadata-file] can be used repeatedly to include multiple metadata files; values in files specified later on the command line will be preferred over those specified in earlier files" (Manual)
4. Values specified in defaults files are treated the same as values provided directly in the commandline, and overwrite values in the in-file YAML and those specified in metadata files. (experimentation)

Thus within the same "level" (i.e., multiple YAML blocks within the same file or multiple metadata files specified in a defaults file), they will accumulate, and in case of a conflict, the last value/string/array will overwrite the first. (There seems to be one exception involving the commandline, but Zettlr doesn't need to concern itself with that, since it doesn't affect Zettlr in any way.)

The order of precedence that Pandoc uses in case of conflicting fields is:

1. Commandline values (-V / -M) = defaults files
2. in-file YAML metadata blocks
3. in-file % pandoc metadata
4. metadata files

Restated: all the metadata gets accumulated, and if there's a conflict in any field/key, then the last value/array from 1 > 2 > 3 > 4.

EDIT: added lang as a possible modifier, as spellcheck exists.