Komga: Open discussion around various metadata containers

[gotson]

Following discussions in #82, this issue is to gather information about the different metadata containers, along with their limitations.

This issue is _not_ about which metadata container to use in Komga, which has its own internal model stored in database. It's a reflexion and thought process about the overall universe of metadata for comics.

Having a single standard would obviously be beneficial, with some caveats:

• The famous xkcd comic about standards
• Even with the best standard, adoption remains key. Not being able to convince people to use it would mean a useless standard.

Nevertheless, it doesn't hurt to summarize the market offerings along with their limitations and adoption.

---

Existing formats:

• ComicRack (known as ComicInfo.xml)
  
  
  
  • Status: part of ComicRack, which is not developed anymore. The single developer, cYo, disappeared from the internet. ComicRack is/was also a shareware model, which could limit the steering capability of the format.
  
  
  • Format: XML, usually stored inside the archive at the root
  
  
  • There is a XML schema available, but there is no versioning (and at least 2 versions of the schema) of the schema, nor usage of XML namespaces
  
  
  • Limitations:
  
  
  • Some fields are missing or limited. Example:
    
    
    
    
    
    • There is a StoryArc field, but no field to indicate the ordering of the book in this StoryArc. It is also impossible to specify more than one StoryArc per book.
    
    
    
  
  
• CoMet
  
  
  
  • Status: unknown. Seems to be a one-man project without actual usage in applications. Not updated since 2015.
  
  
  • Format: XML, apparently stored inside the archive at the root
  
  
  • There should be a XML schema available, but I can't find it. There is also no versioning of the schema, but there are XML namespaces defined.
  
  
  • Limitations:
  
  
  • Some fields are missing or limited. Example:
    
    
    
    
    
    • No difference between Series and Story Arc
    
    
    
  
  
• ACBF
  
  
  
  • Status: has no history on their wiki since 2018, but there is a commit from Dec 2019 in their repo. Not sure who uses this format either.
  
  
  • Format: XML
  
  
  • There is a XML schema available for download, it is versioned and uses XML namespaces.
  
  
  • Limitations: no yet evaluated
  
  
• OPF
  
  
  
  • Status: industry standard used in ebooks. It relies on the Dublin Core Metadata Initiative for the metadata fields.
  
  
  • Format: XML, with schema, versioned, using namespaces.
  
  
  • Limitations: not yet evaluated
  
  

[jevcleem]

I've been reading about metadata being stored in the zip comment (Source: https://wiki.mobileread.com/wiki/CBR_and_CBZ#Metadata). Apparently it's not a common practice but this would have some benefits imo. The zip comment is an uncompressed section of the file such that unzipping is not necessary to change it. I think writing a json object to the comment section of the zip file would be easy to implement and computationally efficient.
Positives:

• Easy to implement
• Efficient and fast (due to unzipping not necessary)

Negatives:

• Additional dependecy necessary for writing/reading to zip comment
• New standard
• Metadata lost when unzipping and zipping again
• Not possible if not a zip file, e.g. epubs/pdf

Other suggestions:

• I think if the xml route is chosen, it would be best to store it outside the zip file.
• Maybe write the software in such a way that the metadata implementation can be a server setting.

[gotson]

Thanks for the input. You are mentioning 2 things here:

• the location of data, the zip comment, which could actually hold any of the XML mentionned above.
• the serialization format, json. Actually JSON is poorly equipped compared to XML for schema definition and validation. They do achieve the same purpose though : serialization.

Those 2 aspects are important, but to me the most important one is about the content, or semantics, ie what can you represent with those formats. 2 mentioned above already are not exhaustive enough to represent the information accurately.

I think I should start to gather data about what a complete model would look like for comics, and compare it to the existing formats to see the gaps.

[theotocopulitos]

I've read other threads on this topic, and know about your thoughts regarding comicinfo.xml . But having been around this world even before Comicrack became a reality, I was the author of the first comicvine scrapper implented in Comicrack, I would like to add some thoughts for you to ponder:

• comicinfo.xml is, no doubt, incomplete, badly implemented and not documented...

• however, it is the 'de-facto' standard... almost every comic related project reads it and many write it (namely, comictagger, comixed, ubooquity, mylar...)

• thousands of users used this protocol: convert cbr to cbz, scrape from CV, and embed comicinfo in the cbz...

• this allows for data portability when you switch computer, reader, organizer...

Many efforts were made to find a better replacement for comicinfo.xml, so far, without any success. I guess not all of them were backed by a code as functional and polished as yours,,, So this might be the place to tackle such endeavor...

So, I get your points: comicinfo is far from optimum, you are exposing an API that would allow to get the info to anyone and deal it however we like... but in the interim, while a better option is created, I feel full support to read/write comicinfo would be most welcome by the community,

[gotson]

This issue is not much to talk about reading or writing from those formats, but more to discuss the formats themselves.

As for comicinfo.xml, Komga can already read it (#54). Writing is logged under #82.

[gotson]

Closing in favor of https://github.com/anansi-project/rfcs