Epub-specs: Do 3.1 attributes need to be dropped by 3.0.2

[RachelComerford]

Debate over use of @refine - Need to bring back @refine there are too many EPUB3 publications
that heavily use ＠refines and this is extremely unlikely to change.

@murata2makoto: On 2016-01-15, Digital Publishing Initiatives Japan Co., Ltd. had 294,275 commercial EPUB3 publications. All of them heavily use @refine. More about this, see the EBPAJ guideline and examples, which are in a ZIP file.

New versions incompatible with these EPUB3 publications will be rejected by the Japanese market.

(Markus wrote "in education US, use of refines heavily in their production". See https://docs.google.com/document/d/1OoepgLPldn0V1dehA4ldzoJLeppQtfYkUQA3vUdOD8Y/edit?ts=56f15ecc)

@mattgarrish : If you're going to try and add the 3.1 attributes into 3.0, we need to be serious about support for new features before a specification can become a recommendation (something we should do regardless). If we can't guarantee support, we're just taking a bad situation and making it worse.
We also shouldn't release standards until we can validate them.

We may not have to follow strict W3C procedures, but if we want to improve people's faith in our standards there are processes we should start following.

@murata2makoto : One solution is to drop 3.1 attributes from 3.0.2.

@mattgarrish : Speaking of no support, we should also remove specific mention of the satellite specs from epub, with the possible exception of CFI. And take alt style tags out of the core, where it never belonged.
They're zombies that linger on without prospects for support. They also aren't likely to be workable with EPUB 4.

@murata2makoto : I am not saying that refines is a good thing. I know that it is a last minute hack. But longevity of hundreds of thousands EPUB 3 publications containing refines is more important than anything else.

@mattgarrish : The really major change anticipated for 3.1 never happened: allowing html content documents. Aside from that, a driving force of 3.1 was to undo legacy messes we brought forward from 2.0 and features of 3.0 that were going nowhere in anticipation of better compatibility with 4.0 and to try and reduce the confusion around 3.0 and what it actually supports. That's where the break necessitating a new version number came from, as we're bound to no errors/warnings because of distribution requirements.

This proposal is going to undo all of that, but there are still improvements worth keeping (getting the HTML/SVG/Unicode off specific references, improving CSS, etc.). They just aren't very major anymore, which is why it can be formulated as a 3.0.2. Keeping 3.1 as the basis for 3.0.2 still makes a lot of sense, as we put significant effort into cleaning up the documents outside of these few contentious points (not just the restructure, but clarifying a lot of ambiguities). Going back to square one would be a painful and unnecessary thing to do, whether we want to expand on this revision or not.

@therealglazou: The @refines history is worth introspection: ID/IDREF mechanisms are well known, since the SGML era, by all markup experts as a potential trap for editability and maintainability. I was already discussing this problem with Jean Paoli back in 1992, when we were working on "paired elements" and "paired processing instructions" for the internal controls of GRIF's rendering engine. ID/IDREF is worth it only if there is never any human intervention in the ID selection/modification. This is not the case with the refines attribute because most EPUB production chains in the world still contain human manipulations. I have to add the fact that EPUB 3 metadata allow nested refinements adds a unbelievable level of complexity, and turns any potential UI for metadata editing into UX hell. All in all, I am extremely disappointed and sad to see we could not kill this ugly design, bad choice and financial pit. If there is one single reason why EPUB is sick, it's that one.

[murata2makoto]

The EBPAJ guideline is in this ZIP file.

[GarthConboy]

Note language from original proposal on this topic: "Use of the “refines” attribute is “discouraged” rather than “deprecated” in EPUB 3.0.2. However, addition of the new duration, opf:alt-rep, opf:authority, opf:file-as, opf:role, opf:scheme and opf:term attributes is retained in EPUB 3.0.2."

[therealglazou]

Note language from original proposal on this topic: "Use of the “refines” attribute is “discouraged” rather than “deprecated” in EPUB 3.0.2. However, addition of the new duration, opf:alt-rep, opf:authority, opf:file-as, opf:role, opf:scheme and opf:term attributes is retained in EPUB 3.0.2."

Which implies the new spec needs to handle ALL collisions (not only for refines but also for all deprecated/removed stuff we authorize again): for example, what happens when there are BOTH a file-as refinement and a opf:file-as attribute for some given metadata?

[therealglazou]

I think the spec also needs prose (informative level) saying if it is ok to preserve "discouraged" stuff when a package is modified or if editing environments and filters have to update to the new solutions, or even if it's expected a choice must be presented to the user (I can imagine the face of a non-tech-savvy author or contributor in front of such a question...)

[dauwhe]

Sounds like we need an actual object model for package/metadata, given that we have all these different serializations of what is presumably the same information?

[therealglazou]

Sounds like we need an actual object model for package/metadata, given that we have all these different serializations of what is presumably the same information?

Well, I doubt it would help; the production of that spec would be a nightmare. The main questions are:

1. is something discouraged ever serialized?
2. what if some discouraged feature and its 3.1 equivalent are both present, possibly with different and incompatible values?

Sigh.

[GarthConboy]

Clearly we will need some rules for Daniel's #2 above. I'd generally think newer/non-discouraged wins in the event of both.

[laudrain]

@murata2makoto : One solution is to drop 3.1 attributes from 3.0.2.

I support this idea : less work also for epubcheck.

[therealglazou]

Clearly we will need some rules for Daniel's #2 above. I'd generally think newer/non-discouraged wins in the event of both.

Not that obvious: in that case, editing for typos a document with the discouraged features will serialize/save the package without; probably unacceptable for the JP industry?

[GarthConboy]

I don't quite get Daniel's comment of "editing for typos a document with the discouraged features will serialize/save the package without" -- without what?

Though, regardless, in this case, I could also live with Makoto's/Luc's suggestion of dropping said 3.1 attributes in 3.0.2.

[mattgarrish]

Sounds like we need an actual object model for package/metadata

3.0 metadata are roughly triples with the subject being the publication, but using ID/IDREF because of problems the un-nestable nature of the meta elements introduced. That's why the original 3.0 publications specification had an RDFa vocabulary baked into it for the properties.

We discussed trying to reformulate the meta elements as a proper RDFa Lite implementation during 3.1, but the attribute approach was ultimately adopted for simplicity and some measure of familiarity with 2.0. I don't know if we can get a completely coherent model out of the two, or it would take some work.

As little a fan of refines as I am, I don't see the value in introducing a replacement set of attributes when they are no longer replacing refines. If we're owning the mistakes of 3.0 again, then own them. Even if we were to "discourage" refines, those moves are often pointless when validators don't report anything. You have to be "in the know". (insert plug for a strict profile here)

[murata2makoto]

@mattgarrish So, should we simply reintroduce refines (without any reservations) and drop 3.1 attributes for replacing them? I think that this is a practical solution.

[mattgarrish]

I assume you meant me here, and, yes, that's my argument. The attributes were added to handle deprecating refines. If we aren't deprecating refines, it invalidates the case for the attributes and adds confusion.

We're too far into the life of 3.0 to be tinkering in such a significant way. As ugly as refines is, it's ingrained in 3.0 and there are too many cases where the attributes fail to replicate its functionality (e.g., adding structured schema.org metadata).

[kevinhendricks]

FWIW, having epubcheck warn/deprecate refines of refines would be a big help to discourage their use. The recursive nature of refines on refines makes the whole concept flakey at best. Does the Japanese publishing industry make heavy use of refines on refines or just refines of major dc style elements?

[therealglazou]

The recursive nature of refines on refines makes the whole concept flakey at best

Even simple refines were a original sin I reported at that time: the whole thing is based on a ID/IDREF mechanism and ID/IDREF is well known, since the early days of SGML, to be a potential big problem if humans edit the document... In that case, who do you chose the ID? Randomly, and it's hardy editable. Manually, and how do you prompt a non-tech-savvy user with such a silly question? All in all, refines are plain wrong when nested elements can do the trick.

[dauwhe]

On today's CG call, we discussed the possibility of removing the new 3.1 attributes. Is anyone aware of people using these in publicly-available EPUB files?

[GarthConboy]

Not for us. I'm willing to drop.

[laudrain]

Not for us neither in production. I'm ok to drop.

[mattgarrish]

Closing this with the merge of PR #1040. If there any items were missed, please open new issues.