specification 🚀 - Specify/advise semantics of default Container resources (index.html/.ttl)

There is rough consensus in https://github.com/solid/solid-spec/issues/134 that handling of LDPC resources is an implementation detail.

I've created https://github.com/solid/specification/issues/109 to provide some answers to the general issue around representation handling.

Regular Accept rule should apply (with optional q parameters) for reading. For writing, it'll fallback on the interaction rules around LDPC specified by applicable methods.

What kind of non-normative text do you think may help? If we can, I prefer to not include such text.

csarven on 5 Nov 2019

3. "I accidentally created an index.html but forgot to add an .acl, plus that kicks me out of the Data browser, so anyways what do I do now."

I had made a comment about creating a /public/index.html file in a pod on gitter and that it can cause problems for some users so, I wanted to give an update on how to restore your public folder if someone had done this.

Log into your pod, select "your stuff" on the drop down menu, select the "your storage" tab, locate and open the index.html file, click the gear icon and select delete, and your public page will be restored to the default view.
Hope this is helpful.

mikeadams1 on 16 Nov 2019

Documenting a clarification on this issue (based on meeting with @dmitrizagidulin ):

If /index.* exists, what happens when / is requested? Common practice: where /index.* placed in the system and / eventually resolves with a message body including the contents of of index.*. There is some overlap with https://github.com/solid/specification/issues/109 . For example:

must updating /index.* happen through /?

I think we should first have agreement on the interaction, ie. should request to /:

redirect to /index.*
include contents of index.* in the message body
include contents of index.* in the message body and Content-Location: index.* in the header
or something else?

/index.* will inherit the authorization policies from /'s ACL (or ancestor's).

csarven on 17 Nov 2019

👎1

Seen in isolation, I feel that this issue should have been resolved without exposing the existence of index.* to the client at all, it feels much like an filesystemy anachronism, that the use of /index.* is just a way to store a representation of / internally.

The risk that people will use both / and /index.html to refer to the same resource, and therefore complicate URI Normalization is great. That is, it creates a mess for us when we try to determine if two resources are the same, and complicate query and caching.

If we do come up with a consistent way to deal with representations more generally, (#109), then it does make sense to apply that to this problem too. I think it is crucial that the minimal container triples and containment triples are in all representations, and that we minimize the risk that people will use different URIs for what is actually the same resource.

kjetilk on 18 Nov 2019

👍1

Forgive me if I'm off topic, but I think this is the same issue:

It seems to me like there are 2 types of resources in a Solid pod.

Server/Pod-Provider managed things (e.g. the pod management interface)
User managed things (i.e. LDP resources)

To me, it seems the problems described in this issue are caused by the lack of this distinction in NSS and in Solid in general right now. The Server managed resources should not be able to be managed by the user.

I think this paradigm of separating the User managed from the Server managed is helpful. It could even be advertised by the server. For example, servers could list their server-managed resource(s) (i.e. their "entry point" for users) when receiving a OPTION * request.

JordanShurmer on 27 Nov 2019

I discussed this briefly with @timbl , and he said that in the databrowser's use of index.ttl, there was no expectation that it is a default container resource, to the contrary, it is a normal resource, contained as usual.

kjetilk on 5 Dec 2019

👍1

I also discussed index.html briefly with @timbl . First, there is no expectation that all index.* will behave the same. index.ttl is merely a databrowser convention, it is just another resource.

index.html OTOH is a HTML representation of the container, as per old Apache convention. @timbl intended this to be governed by the Accept header.

There are some problems that I see with this approach (that we didn't have time to discuss further), since there are some things that are important in a container's representation, notably the containment triples, the minimal server-managed LDP triples, and the "POSIX metadata", i.e. metadata to indicate ctime, mtime, atime, etc, which would come in handy if Solid was used as an actual file system, e.g. through FUSE.

So, I am uncomfortable about not having them in the representation, even though people could get at them by adjusting the Accept header.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source, so it breaks LDP's model in which LDPC isa LDP-RS.

One possible resolution to this problem is to require the inclusion of the RDF as RDFa. Even though people might PUT index.html without the RDFa, the RDFa could be added to the DOM by the server.

I see three ways to resolve this:

Live with that the HTML representation is not a representation of the container with its triples, but return HTML only if it is requested through the Accept header.
Try to work out something that behaves in the browser like things used to do in the past, but does so in a way that makes it clear that ./index.html is not a representation of ./
Require (or at least say SHOULD) returning the RDF representation of a container as RDFa in the HTML.

My preference would be the latter.

kjetilk on 9 Dec 2019

index.html OTOH is a HTML representation of the container

Possibly. To actually resolve that, I was hoping to get an answer to https://github.com/solid/specification/issues/69#issuecomment-554760518 , https://github.com/solid/specification/issues/109#issuecomment-549805258 eg. Content-Location?

That aside..

Any representation of a container is required to be RDF bearing; RDF Source; LDP-RS... having triples that can be parsed by an RDF parser... Full stop. Whether it has a certain number of triples or even the "right" triples is orthogonal. That is a different requirement about representations and equivalences we can work out.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source

Not necessarily. If an HTML has no RDFa in it, an RDFa parser will obviously find 0 triples. That's exactly the same as a Turtle, if there are no triples in the document. Attempting to creat or update / with text/turtle or text/html with no RDF can be expected to have the same treatment - basically 0 triples.

https://github.com/solid/specification/issues/45#issuecomment-541614170 proposes how to handle equivalent representations:

Servers supporting Turtle, JSON-LD and optionally other RDF serializations for LDP-RS SHOULD provide different RDF serializations based on client's proactive negotiation. For example, if a server allows the creation of an LDP-RS in text/html (including RDFa), it SHOULD respond to GET with Accept: text/turtle requests with 200.

csarven on 9 Dec 2019

index.html OTOH is a HTML representation of the container

Possibly. To actually resolve that, I was hoping to get an answer to #69 (comment) , #109 (comment) eg. Content-Location?

Right, so, @timbl saw this as behaving like Apache does, i.e., there's nothing on the surface, the server just silently takes the index.html file on the server and returns it's content as a representation of ./. No Content-Location or anything.

That aside..

_Any_ representation of a container is required to be RDF bearing; RDF Source; LDP-RS... having triples that can be parsed by an RDF parser... Full stop.

That'd be the LDP purist's view, ;-) this stems from the LDP's hierarchy of interaction models, it is not a concern of Solid as currently designed.

Whether it has a certain number of triples or even the "right" triples is orthogonal. That is a different requirement about representations and equivalences we can work out.

Right, but I take the pragmatic view: What are the useful pieces of LDP that we actually need? The mentioned RDF is something we need, and from that, it follows that they must be included with the container representation.

Now, I'm sufficiently purist myself to be wary of a design that doesn't include the container representation with all possible representations, regardless of Accept header. From that, it follows that indeed, any representation is required to be RDF bearing, so we arrive at the same conclusion, but through purism on different parts of the architecture. :-)

However, neither of these views are consistent with the view that @timbl gave me of his motivation behind the index.html, we need to work out something that is.

From an LDP purist's point of view (which I am not), the HTML representation is also a non-RDF source

Not necessarily. If an HTML has no RDFa in it, an RDFa parser will obviously find 0 triples. That's exactly the same as a Turtle, if there are no triples in the document. Attempting to creat or update / with text/turtle or text/html with no RDF can be expected to have the same treatment - basically 0 triples.

Right, but my (unstated) assumption is that the HTML would also contain content that is not represented as RDF, that's the whole point of putting HTML there, which makes it non-RDF in LDPs (flawed) model where LDP-RS and LDP-NR are mutually exclusive, regardless of whether there is RDF content hosted in it.

#45 (comment) proposes how to handle equivalent representations:

Servers supporting Turtle, JSON-LD and optionally other RDF serializations for LDP-RS SHOULD provide different RDF serializations based on client's proactive negotiation. For example, if a server allows the creation of an LDP-RS in text/html (including RDFa), it SHOULD respond to GET with Accept: text/turtle requests with 200.

No, that does not appear to address the problem, the point is that the index.html would contain content that is not represented in RDF, that's the whole point, if it was fully represented with RDF, it would not be put as HTML in the first place.

Moreover, the UNIX filesystem analogy extends to containers too, in that the container just contains resources, it is not intended to have an extensive representation on its own.

kjetilk on 9 Dec 2019

Edit/Note: I wrote the prior to seeing Kjetil's comment above / in https://github.com/solid/specification/issues/69#issuecomment-563198708

This issue was intended to work out whether index.* can be observed or interacted with.

I still think the handling of index.* is ultimately an implementation detail. If it is a container's representation with its own URL, so be it.

https://github.com/solid/specification/issues/109#issuecomment-554760602 actually tries to resolve this:

where index.* exists by other means, we need to resolve #119

The point of that was to have a clear split between resources that come to existence via Solid's prescribed interaction vs. by other means, and so re "exists by other means" ie. some implementation happens to have index.* with its own accessible URL where it didn't go through an interaction with the container, then it is of course a separate resource (and certainly not a representation of /). Out of Solid spec by default. But the point with 119 is that if there are some unique cases that should have special handling, and so that would help to address:

if /index.* exists, what happens when / is requested, we need to resolve #69

What databrowser, Apache are doing with /index.* are implementation details. So, requesting / should have nothing to do with /index.* because they are different resources. Just because Apache can be configured to handle / and processing/serving /index.* is not something that Solid needs to adopt. Again, if for example a Solid server implementation wants to accept text/html or text/turtle.. at / and store the representations at /index.*, then that's its own decision. It naturally needs to address "if /index.* exists, what happens when / is requested".

csarven on 9 Dec 2019

RDF 1.1 says that RDFa in markup languages qualifies it as RDF. If there is any information in there that's not part of an RDF graph, then it doesn't suddenly become non-RDF. If two RDF graphs are isomorphic, that's all that counts towards representation equivalence.

If / is intended to be in RDF, then by minimum server can take Turtle and JSON-LD (as previously roughly agreed for minimum serialisations). If a server wants to accept any other media type, it needs to be an RDF bearing document. Similarly, if it wants to serve a container in text/html, it needs to encapsulate the information in RDFa.

csarven on 9 Dec 2019

Firstly, this resource is now only about index.html as indeed all other index.* are just normal resources, and thus implementation details.

kjetilk on 9 Dec 2019

RDF 1.1 says that RDFa in markup languages qualifies it as RDF. If there is any information in there that's not part of an RDF graph, then it doesn't suddenly become non-RDF.

Eh, well, it makes it a not RDF source, at least, from LDP:

Linked Data Platform RDF Source (LDP-RS)
An LDPR whose state is fully represented in RDF, corresponding to an RDF graph. See also the term RDF Source from [rdf11-concepts].

Linked Data Platform Non-RDF Source (LDP-NR)
An LDPR whose state is not represented in RDF. For example, these can be binary or text documents that do not have useful RDF representations.

So, HTML+RDFa is an RDF Source iff it is fully represented by the hosted RDF. If it contains information that is not represented by the hosted RDF, it is neither an LDP-RS nor an LDP-NR. Then, my proposition is that the whole point with having an index.html file there is to have human-readable information that is not fully contained in RDF. It is still an LDPR though, but it is not consistent with the LDPC interaction model, which doesn't worry me too much, as long as we can represent the stuff that should be in there as RDFa.

If two RDF graphs are isomorphic, that's all that counts towards representation equivalence.

I don't think so, because what happens then if you round-trip between the HTML+RDFa and Turtle? index.html is can't be an RDF source. It is a special case that doesn't fit with LDP's flawed model. Which, BTW, is flawed for all HTML+RDFa that contains information not fully represented with RDF.

If / is intended to be in RDF, then by minimum server can take Turtle and JSON-LD (as previously roughly agreed for minimum serialisations). If a server wants to accept any other media type, it needs to be an RDF bearing document. Similarly, if it wants to serve a container in text/html, it needs to encapsulate the information in RDFa.

Right. So we agree that it is a good thing if the index.html returns RDFa, but we still have to the define this as a special case, and also whether it is a MUST in Solid to return RDFa, but we cannot do that with reference to LDP as LDP doesn't deal with this case.

kjetilk on 9 Dec 2019

The intended state of an HTML+RDFa representation is what corresponds to an RDF graph. Round-tripping is a non-issue because information that's not marked in RDFa is neither intended or expected to preserve. HTML+RDFa is an LDP-RS.

There is no need to specify index.html as a special case anymore than specifying index.ttl. I suggest to close this issue in favour of resolving https://github.com/solid/specification/issues/109

I think the original issue title reflects what's discussed and linked. I find the one you've changed to omits key information. Can we revert?

csarven on 9 Dec 2019

[@csarven] information that's not marked in RDFa is neither intended or expected to preserve

Really? I don't think many if any HTML+RDFa creators would agree with you. Certainly, I would find it a very large problem if my HTML+RDFa documents were to suddenly lose all HTML content and be reduced to RDF in any serialization.

TallTed on 9 Dec 2019

👍1

The intended state of an HTML+RDFa representation is what corresponds to an RDF graph. Round-tripping is a non-issue because information that's not marked in RDFa is neither intended or expected to preserve. HTML+RDFa is an LDP-RS.

No, it is not! Please comment on each of my points if you disagree!

There is no need to specify index.html as a special case anymore than specifying index.ttl. I suggest to close this issue in favour of resolving #109

I strongly disagree! This is very much a special issue, as it describes a feature that is in Solid and has been in Solid since the dawn of ages. You will then have to argue for the removal of a feature that people rely on and that the Director has voiced a clear opinion on.

I think the original issue title reflects what's discussed and linked. I find the one you've changed to omits key information. Can we revert?

Sure, but then, please be a little sensitive to the fact that some of this happened in a F2F discussion that you didn't attend to. I have tried to explain it in clear terms, but you seem to simply dismiss the discussion without considering the merits of the arguments.

kjetilk on 9 Dec 2019

👍1

@TallTed ,

Really? I don't think many if any HTML+RDFa creators would agree with you. Certainly, I would find it a very large problem if my HTML+RDFa documents were to suddenly lose all HTML content and be reduced to RDF in any serialization.

If the intention is to preserve content in the RDF graph universe of things, it needs to emit itself to get picked up. What wrote HTML+RDFa and what decisions did it make?

@kjetilk ,

Dmitri's initial comment is the original issue covering a bunch of related stuff, hence the title!

Right, so, @timbl saw this as behaving like Apache does, i.e., there's nothing on the surface, the server just silently takes the index.html file on the server and returns it's content as a representation of ./. No Content-Location or anything.

When you say "No Content-Location or anything", that can be taken as a response to https://github.com/solid/specification/issues/69#issuecomment-554760518 :

I think we should first have agreement on the interaction, ie. should request to /:
include contents of index.* in the message body

being the preferred interaction from that list. [Perhaps that's another way of looking at "Please comment on each of my points if you disagree!" ;)]

Note how not requiring Content-Location in this case is contrary to our preference of requiring it for the general case https://github.com/solid/specification/issues/109 . I'd like to resolve that.

The risk that people will use both / and /index.html to refer to the same resource

The point of resolving issues like #119 is so that we understand the scope and methods in which resources make their way into a system. How did index.html materialise? Was it created as a representation of / or as a resource different than /? Contained or not? Anything referring to it?

Require (or at least say SHOULD) returning the RDF representation of a container as RDFa in the HTML.

If an implementation accepts text/html on a resource eg. /, with Accept and Content-Type, I prefer this (even as a MUST) - needs to have containment information and following the common criteria on affecting server-managed triples.

csarven on 10 Dec 2019

So, HTML+RDFa is an RDF Source iff it is fully represented by the hosted RDF. If it contains information that is not represented by the hosted RDF, it is neither an LDP-RS nor an LDP-NR.

That's a misinterpretation of LDP-RS, LDP-NR and the RDF Source that it links to (RDF 1.1). RDF 1.1 is clear about RDFa:

An RDF document is a document that encodes an RDF graph or RDF dataset in a concrete RDF syntax, such as Turtle [TURTLE], RDFa [RDFA-PRIMER], JSON-LD [JSON-LD], or TriG [TRIG]. RDF documents enable the exchange of RDF graphs and RDF datasets between systems.

A concrete RDF syntax may offer many different ways to encode the same RDF graph or RDF dataset, for example through the use of namespace prefixes, relative IRIs, blank node identifiers, and different ordering of statements. While these aspects can have great effect on the convenience of working with the RDF document, they are not significant for its meaning.

We informally use the term RDF source to refer to a persistent yet mutable source or container of RDF graphs. An RDF source is a resource that may be said to have a state that can change over time. A snapshot of the state can be expressed as an RDF graph. For example, any web document that has an RDF-bearing representation may be considered an RDF source.

I think you're mistakenly overloading the term "fully". LDP-RS uses that term to differentiate from LDP-NR. Just as LDP-NR uses "do not have useful RDF". The use of the term "fully" alone is inadequate to cover all the intricacies or even to create a new constraint with a whole set of ramifications without definitions. It is not LDP's place to do that because the simplest explanation is that it respects spec orthogonality. LDP is merely classifying the kind of documents for its interaction model - the intended semantics being "RDF-bearing" or not.

Proposing that (HTML+)RDFa is somehow incompatible with, falls between, or depends on conditions (?) for RS and NR is nonsensical and renders things useless for no practical benefit.

If it helps to be sure, use the LDP-RS interaction model when communicating representations with RDFa - something I've already suggested elsewhere. If we don't need LDP's interaction models in the end, nothing fundamentally changes, so there is nothing else to do here. Great. If / is intended to have RDF-bearing representations, then servers permitting media types that could potentially contain RDFa needs to decide what is acceptable given the underlying semantics eg. involving server-managed triples.

csarven on 10 Dec 2019

This is going to go down in history as an example of why some stuff should be agreed on F2F, as we now ended up generating more heat than illumination ;-) So, we're in "violent agreement" for the most part, I guess. There's just one thing I still feel like responding to here:

Proposing that (HTML+)RDFa is somehow incompatible with, falls between, or depends on conditions (?) for RS and NR is nonsensical and renders things useless for no practical benefit.

since it does have a practical consequence, that "fully" means it is round-tripable, i.e. you can choose any RDF serialization, and the semantics will be the same.

That aside, I think we have an agreement on the following:

That all operations on the container will use the container request-URI.
That the actual use of index.html is an implementation detail local to the server, which will not be exposed to the client.
That the use of HTML+RDFa will need to follow the same rules as any other RDF serialization, i.e. it cannot modify server-managed triples such as containment triples.
That said, HTML+RDFa may contain more information than is captured in RDF, and therefore isn't round-tripable, the HTML therefore needs separate storage.
The server will need to be able to modify the triples included as RDFa, since at the very least, it needs ot manage server-managed triples.
It is thus not an actual "Representation URL" in the sense of #109, since it doesn't have a different URL than the container itself, it is merely an augmented representation of the same resource.
Therefore, it also does not have its own ACL, etc.

Any misrepresentation in the above? If not, I think the main question is whether the HTML representation MUST have the container's triples as RDFa or if it should be a SHOULD.

kjetilk on 10 Dec 2019

Agree.
Agree, server is not prohibited (whether as a representation of a container or as an independent resource with that name).
Agree.
Disagree, only the RDF graph is intended to be round-tripable. That is the agreement that's being committed to with RDFa use.
Agree, server will have the same behaviour for all supported RDF serializations.
Agree.
Agree.

Any misrepresentation in the above? If not, I think the main question is whether the HTML representation MUST have the container's triples as RDFa or if it should be a SHOULD.

If an implementation accepts text/html on a resource eg. /, with Accept and Content-Type, I prefer this (even as a MUST) - needs to have containment information and following the common criteria on affecting server-managed triples.

csarven on 11 Dec 2019

[@csarven] only the RDF graph is intended to be round-tripable. That is the agreement that's being committed to with RDFa use.

I could not disagree more strongly with the above, which I do not believe can be found in any RDFa specification nor guidance.

You are approaching RDFa from the wrong side. HTML+RDFa is embellished HTML (hence, _HTML plus RDFa_), it is not embellished RDF (which would be _RDFa plus HTML_).

TallTed on 11 Dec 2019

Regarding LDP-NR vs LDP-RS classification —

As a member of the LDP WG, I understood us to be saying that LDP-NR _might_ include RDF content, but always include non-RDF content which is meant to be preserved, so the document must be preserved as PUT or POSTed.

LDP-RS are 100% RDF, and might be stored by the back-end in their original form, or transformed into another RDF serialization, or loaded into a graph store and not preserved as a document per se — though always retrievable in either Turtle or JSON-LD serialization.

(I was a minority in thinking that Turtle — which may include out-of-band, non-RDF comments and statement order — should be considered LDP-NR, and thus should be preserved entirely.)

TallTed on 11 Dec 2019

👍1

HTML+RDFa is embellished HTML (hence, HTML plus RDFa), it is not embellished RDF (which would be RDFa plus HTML).

Pardon me but as accurate as that may be it is being pedantic. RDFa extends the host language, and as a result it is considered to be "RDF-bearing". I'll defer to RDF 1.1 once again.

As a member of the LDP WG, I understood us to be saying that LDP-NR might include RDF content, but always include non-RDF content which is meant to be preserved, so the document must be preserved as PUT or POSTed.

No objection.

LDP-RS are 100% RDF, and might be stored by the back-end in their original form, or transformed into another RDF serialization, or loaded into a graph store and not preserved as a document per se — though always retrievable in either Turtle or JSON-LD serialization.

No objection.

Neither of those statements conflict or exclude RDFa in markup languages. There is no test that I'm aware of that can differentiate between the strict or perceived definitions of "fully" and "do not have useful RDF" above and beyond what's generally intended through HTTP+RDF 1.1.

The arguments portraying RDFa as though it yields non RDF Source is a direct conflict with RDF 1.1. FWIW, LDP intentionally defines the concepts imprecisely, yet, what's argued here should somehow be accepted as a precise interpretation.

csarven on 11 Dec 2019

"RDF-bearing" is not the same as "RDF only" which is what you're asserting with "only the RDF graph is intended to be round-tripable. That is the agreement that's being committed to with RDFa use".

You're effectively asserting that the RDF content is the only thing of value in HTML+RDFa documents, and that the creators of those documents have acceded to that assertion by using that document format.

I do not believe that many, if any, and certainly not all, creators of such documents would agree.

TallTed on 11 Dec 2019

Also -- RDF 1.1 does not define _LDP-RS_, though it may well define _RDF Source_. These are distinct, and pretending that they are not will cause problems in many interactions.

TallTed on 11 Dec 2019

You're effectively asserting that the RDF content is the only thing of value in HTML+RDFa documents, and that the creators of those documents have acceded to that assertion by using that document format.

That goes without saying because it is precisely what happens with any RDF reserialisation - similar to handling of whitespaces, comments, unused prefixes, order etc. that gets dropped or is rendered insignificant for the resulting RDF graph.

If a resource like / is intended to be an an RDF Source, allowed media type on that resource MUST uniformly adhere to the same criteria. Put another way, if an application wants to update an RDF Source like /, it needs to meet that requirement in its request in addition to other optional content negotiation - whether that's Turtle, JSON-LD at minimum or optionally with HTML+RDFa etc.

There is no need to assume the contrary because it moves away from the simplest explanation based on the specs.

RDF 1.1 does not define LDP-RS, though it may well define RDF Source. These are distinct, and pretending that they are not will cause problems in many interactions.

Yes, evidently, but I don't understand the point you're trying to make. What's important is that LDP-RS inherits RDF 1.1 notion of RDF Source.

Aside: concerning that LDP's interaction model may already be on thin ice with Solid any way, we could drop the LDP speak. It doesn't change the bottom line any way.

csarven on 11 Dec 2019

Please reread the LDP spec (as I have just done). From 2. Terminology —

Linked Data Platform Resource (LDPR)
A HTTP resource whose state is represented in any way that conforms to the simple lifecycle patterns and conventions in section 4. Linked Data Platform Resources.

Linked Data Platform RDF Source (LDP-RS)
An LDPR whose state is fully represented in RDF, corresponding to an RDF graph. See also the term RDF Source from [rdf11-concepts].

Linked Data Platform Non-RDF Source (LDP-NR)
An LDPR whose state is not represented in RDF. For example, these can be binary or text documents that do not have useful RDF representations.

In other words -- LDP Resources have ~~three~~ two subsets --

LDP-RS
LDP-NR
~~neither LDP-RS nor LDP-NR~~

HTML+RDFa docs are NOT fully represented in RDF, thus they are NOT LDP-RSs. HTML+RDFa docs do have _partial_ representation in RDF, thus when they are LDPRs, they are LDP-NRs. ~~not LDP-NRs. But they may still be LDPRs.~~

Yes, LDP references the RDF 1.1 notion of RDF Source -- but as a see also, not as a same as. Further, the linked notion is from a _non-normative_ section of RDF 1.1 (1.5 RDF and Change over Time, a subsection of 1. Introduction), and reads --

We informally use the term _RDF source_ to refer to a persistent yet mutable source or container of RDF graphs. An RDF source is a resource that may be said to have a state that can change over time. A snapshot of the state can be expressed as an RDF graph. For example, any web document that has an RDF-bearing representation may be considered an RDF source. Like all resources, RDF sources may be named with IRIs and therefore described in other RDF graphs.

Note the VERY IMPORTANT word, "informally".

You are taking this description from RDF 1.1 as both normative and formal, when it is neither.

You are also saying that these specs say something they don't — i.e., that any RDF source is only and purely RDF, and thus can be losslessly — or at least, without loss of anything of any importance — transformed from any serialization into any other. _Neither the RDF nor LDP spec says this._

ETA -- I've reread down a little further. The terminology section should have had a slightly different definition for Non-RDF Source, which expansion is found in the normative section, 4.4 Non-RDF Source —

LDP Non-RDF Sources may not be able to fully express their state using RDF

In other words — the state of LDP-NRs _may_ be partially expressed in RDF, as with RDFa (and, I contend, as with Turtle.)

TallTed on 11 Dec 2019

@csarven ,

Disagree, only the RDF graph is intended to be round-tripable. That is the agreement that's being committed to with RDFa use.

OK, so we're drilled down now, good. Aside from the remarks of @TallTed , which I think is clarifying (I suppose I tend to read the LDP spec as the devil reads the bible, but I cannot guarantee I will stop doing it :-) ), how can we practically resolve the present issue with your interpretation?

The point is that when you add a HTML representation of the container, you do so to add information that carries HTML semantics, and it carries human-readable content, like the root of the Solid server does nowadays. If you allow roundtrip, then AFAICS, you allow information which is the very reason for this feature to exist to be lost, and that defeats the purpose of the feature. With conneg, you can always get a pure RDF representation of the same RDF that is hosted as RDFa in HTML in other serializations, but you'd still have to preserve the HTML, so it isn't roundtripable.

I don't think we should progress further down the LDP rabbit hole, we need to focus on how we can define the feature so that it does what is expected from it.

The root, e.g. https://kjetiltest2.dev.inrupt.net/ can be taken as an example. This feature is really about tightening up the spec around that, and then we need to decide if RDFa is a MUST or a SHOULD.

kjetilk on 12 Dec 2019

Ted, masterful hairsplitting =)

You are taking this description from RDF 1.1 as both normative and formal, when it is neither.

I'm completely taking it in good faith because of the fact that LDP refers to it. But if you insist and I have no objection to go with LDP's RDF Source is not "same as" RDF 1.1's RDF Source. So, what is LDP's "RDF Source"... just RDF 1.1's RDF but not "RDF Source"? A "pure" definition of RDF? constraint on RDF 1.1? Something else?

Neither the RDF nor LDP spec says this.

Again, taken in good faith because the definitions are confined to what makes it into an RDF graph. Naturally what doesn't make it in is "lost" but I'm not framing it as such. You're. I see it in terms of intentions. An HTML+RDFa document can live just fine on its own as a plain ol' HTML if you will. Given what RDFa affords in context of RDF 1.1, what is emitted is the RDF graph. If a consumer is handling it with the RDF goggles on, it is not at all about lossy. Why would a RDFa publisher ever assume that anything but the RDFy bits should remain? That is completely out of scope.

What I gather from your explanation of LDP and its interaction models is that its classification of the sources can be fuzzy. There is an element of heuristics involved by a server. Again, are there algorithms or tests that can determine whether a payload is "fully", "partially", "do not have useful", and so forth? If not, then how is interop guaranteed if servers/clients could potentially do and expect different things?

Moreover, what you describe about how some (RDF 1.1) RDF Sources could be fall into different LDP interaction models only supports the idea/possibility to not bother with LDP's interaction models for Solid. IMHO.

csarven on 12 Dec 2019

how can we practically resolve the present issue with your interpretation?

If an implementation accepts text/html on a resource eg. /, with Accept and Content-Type, I prefer this (even as a MUST) - needs to have containment information and following the common criteria on affecting server-managed triples.

?

The point is that when you add a HTML representation of the container, you do so to add information that carries HTML semantics, and it carries human-readable content, like the root of the Solid server does nowadays.

There is a very specific expectation that's overlooked. The whole point of interacting with / and that it is an RDF Source precisely sets the constraints. The basic expectation is that it needs to a have representation in which the semantics reflect/resemble a container, containment triples etc. That's is a contract. If a server does not want to accept text/html at / or doesn't meet the same criteria about the RDF bits (as with any RDF serialization) it should reject it.

I hate to say it but we can't have our cake and eat it too on this one. We can't expect and design / (eg. a homepage) act like an regular resource (plain ol' HTML when we want it to) and at the same expect that it should be a "pure" RDF container. If HTML is allowed and the "interaction model" (but not necessarily LDP's "interaction model") on the resource is RDF, then the intention is clear. It will be handled as an RDF. If the document contains RDFa and able to cover all the container material (just like with other RDF serializations), super great!

Moreover, we can't on one hand expect that HTML can make its way into / and at the same time complain that some "content" is lost when re-serialized. As already noted, whitespaces, comments, unused prefixes, order etc. are part of the state.. but no one seems to be complaining about the "pure" RDF losing track of that.

Aside: FWIW, an experimental server like https://github.com/csarven/mayktso/ will take HTML+RDFa and store it as is. It allows conneg to other RDF. If text/html is requested, it will serve the document (which is saved as is). If text/turtle is requested, it will transform and serve that. If a Turtle document is sent to the server, it will save it as is. If text/html is requested, it will reject (because it doesn't want to serialize HTML+RDFa - although nothing in particular is stopping it from listing a bunch of triples). If text/turtle is requested, it will serve it right back out.

I don't think we should progress further down the LDP rabbit hole, we need to focus on how we can define the feature so that it does what is expected from it.

Agree. Especially involving interaction models. Especially it seems to introduce more complexity to RDF 1.1 than actually helping.

The root, e.g. https://kjetiltest2.dev.inrupt.net/ can be taken as an example. This feature is really about tightening up the spec around that

That's simply broken. Accept text/html and text/turtle return completely different "semantics". Having said that, I will entertain the idea that those are equivalent - which will open up another can of worms :)

csarven on 12 Dec 2019

You know, this is starting to sound like the Council of Chalcedon or something! ;-) And it is not even F2F.

This is not a difficult problem to solve, it is only LDP orthodoxy that prevents us from solving this issue, and that orthodoxy is entirely misplaced, since LDP is neither valid dogma nor considered axiomatic for Solid. The situation is simply a spec that didn't take into account the possible situation we're in, which is pardonable, since writing specs for every strange thing that people might do is hard.

So, if you purge the LDP-RS definition from your mind, how would you then do it?

kjetilk on 12 Dec 2019

❤1

So, if you purge the LDP-RS definition from your mind, how would you then do it?

My understanding of LDP didn't venture from RDF so what I've initially proposed is workable just the same without LDP(-RS). Pasting again for now (but I do think that we can improve the language):

If an implementation accepts text/html on a resource eg. /, with Accept and Content-Type, I prefer this (even as a MUST) - needs to have containment information and following the common criteria on affecting server-managed triples.

If anyone would like to make a simpler or a more appropriate proposal, just make it so.

csarven on 12 Dec 2019

Right, but this doesn't resolve the round-trip problem, I may have misunderstood you. The point is that non-RDF content in the HTML MUST be preserved, and the understanding that I have gotten from you is that you insist it cannot.

kjetilk on 12 Dec 2019

Is it not the case that this basically depends on whether a client identifies the resource as an LDP-RS or as an LDP-NR?

As an (RDFa-based) LDP-RS, the RDF-ness of the resource would seem to be of paramount importance. The same document (i.e. the bytes) could, alternatively, be stored as an LDP-NR, in which case the non-RDF characteristics of the resource would be paramount.

If you buy that argument, then the semantics of these resources would depend on the interaction model provided by the client when it first creates the resource. In the absence of a client-provided interaction model, the issue can be reframed as https://github.com/solid/specification/issues/128.

acoburn on 12 Dec 2019

@csarven -

The definition of an LDP-RS in the LDP spec is crystal clear: "An LDPR _whose state is fully represented in RDF_, corresponding to an RDF graph." (emphasis added)

Any other LDPR is an LDP-NR, and should be preserved inits original form -- whether or not RDF can be extracted, distilled, generated, or otherwise produced from that LDPR. Such generated RDF may be freely written to an RDF store, and transformed from one serialization to another.

LDP is not, was not, will never be, focused solely on RDF resources.

TallTed on 12 Dec 2019

Kjetil,

Right, but this doesn't resolve the round-trip problem, I may have misunderstood you. The point is that non-RDF content in the HTML MUST be preserved, and the understanding that I have gotten from you is that you insist it cannot.

Right. The "agreement" on / is the RDF-bearing bits. If a client wants to write to /, the server only needs to check against that criteria, and only needs to transmit the RDF-bearing bits when it is requested again. If there is any expectation beyond that, then we are talking about the possibility of / being any kind of resource (not strictly RDF-bearing).

I can't stress enough the point that if a server doesn't want to handle HTML(+RDFa) because it is yucky, or concerned about not being "fully" or "round-tripable" or "pure RDF" or not pigeonholing well into LDP-RS..., there is a solution to that: 415. No hard feelings.

Aaron,

Is it not the case that this basically depends on whether a client identifies the resource as an LDP-RS or as an LDP-NR?

Right. Moreover, going with the LDP interaction model, the target resource eg. / is an LDP-RS, so the client is only allowed to provide an LDP-RS representation. If a server accepts text/html, it is definitely about the RDF graph encoded in HTML+RDFa. If a server doesn't want to accept text/html (or application/xhtml+xml, image/svg+xml..), they should return 415.

If you buy that argument, then the semantics of these resources would depend on the interaction model provided by the client when it first creates the resource. In the absence of a client-provided interaction model, the issue can be reframed as #128.

Yes, I came to the same conclusion which prompted me to open https://github.com/solid/specification/issues/105 (what 128 is based on).

Ted,

LDP-NR, and should be preserved inits original form

I don't think that's explicitly stated but I would agree with you that is an reasonably clear intention. Just as there are other intentions - some of which we seem to disagree.

LDP is not, was not, will never be, focused solely on RDF resources.

I don't think anyone argued for or against that.

csarven on 12 Dec 2019

@acoburn :

Is it not the case that this basically depends on whether a client identifies the resource as an LDP-RS or as an LDP-NR?

Yes, you could make a case for that.

As an (RDFa-based) LDP-RS, the RDF-ness of the resource would seem to be of paramount importance. The same document (i.e. the bytes) could, alternatively, be stored as an LDP-NR, in which case the non-RDF characteristics of the resource would be paramount.

Yes, if the client explicitly declares as an LDP-RS, then it isn't a unreasonable expectation is that it is fully round-tripable. However, per LDP, LDP-C isa LDP-RS, so the requirements of this issue has broken that already. :-) Moreover, @timbl said that LDP's interaction models weren't something he'd consider influencing Solid.

Back in the olden days, we used to serialize RDF graphs to HTML+RDFa on the server without adding anything, but in our day, where there is client-side code ready to do stuff with the RDF regardless of serialization, I don't see much value anymore to use RDFa as YA round-triplable RDF serialization, it is first when you add something that is not represented by RDF that HTML gets interesting.

I suppose we could find ways to support round-tripable HTML+RDFa alongside "enriched HTML+RDFa", but I consider that to be a different issue, and one that I would give very low urgency. The actual problem here is that LDP-C has a representation that is not only RDF, which is a departure from the LDP interaction model hierarchy.

kjetilk on 12 Dec 2019

Right. The "agreement" on / is the RDF-bearing bits. If a client wants to write to /, the server only needs to check against that criteria, and only needs to transmit the RDF-bearing bits when it is requested again. If there is any expectation beyond that, then we are talking about the possibility of / being any kind of resource (not strictly RDF-bearing).

Yes, but there is a very explicit expectation beyond that, but not as broad as "any kind of resource", it is HTML, and it must be able to carry content beyond RDF, but it may or may not contain a representation of the container RDF data.

kjetilk on 12 Dec 2019

Alternatively, requiring only RDFa would enable transmitting richer information to different consumers than any other RDF. But, we are of course not discussing things at that level. Instead, we are approaching the design problems from "RDF purity".

Curious to see what comes out.

csarven on 12 Dec 2019

Curious to see what comes out.

I'm curious to hear what you propose, @csarven , because my understanding now is that you reject the premise of issue, that the HTML has to be able to carry content beyond the RDF representation. I may misunderstand, and it may be something fundamental that I have escaped me, but we really need to get something on this...

kjetilk on 13 Dec 2019

Let me see if I can write something down myself. Beyond the what we do agree on (which may still be contentious, since the current implementation seems to rely on an actual index.html), the current behaviour is that it simply returns HTML if told so by conneg. This boils down to one requirement, where we can discuss the requirement level. So, something like:

_A Solid Server MUST/SHOULD/MAY/MUST NOT/SHOULD NOT accept and persist a HTML representation for any container that extends beyond the RDF representation of the container._

My understanding is that this is currently a MUST for the sake of interoperability, but for something that just has to conform with the current behaviour, SHOULD or MAY would also do (in which case, the server could say 415, like you proposed, @csarven ). That is, AFAICS, the only thing required to work with the current implementation.

However, we might also want to have a representation of the RDF of the container, which could result in another requirement, something like

_If a HTML representation of a container exists for a given container resource, the full representation of a container MUST/SHOULD/MAY/MUST NOT/SHOULD NOT be the container's RDF embedded in the HTML representation._

A MUST here places an addition burden on the server implementations, albeit a small one, as you can easily inject the RDFa into the HTML representation's DOM in the head element before it is sent to the client. This would take care of my concern that a representation of the container isn't really a representation without the RDF...

Perhaps we also need to say something about what happens when you do an update with PUT, PATCH or POST, or is that clear enough?

kjetilk on 13 Dec 2019

I gave my response in https://github.com/solid/specification/issues/69#issuecomment-564640467 .

A container MUST be RDF-bearing.
A container MAY carry non-RDF emitting content.

Prohibiting that would entail something like, you can't have an HTML document, eg. a typical homepage, beyond listing (but not even necessarily human-visible) of the containment triples in RDFa.

csarven on 13 Dec 2019

Based on 2019-12-13 meting, the rough proposal is:

A container representation MUST be RDF-bearing.
Container representations MUST persist existing non-RDF emitting content.

csarven on 13 Dec 2019

🎉1

Now there is convergence! :-) So, this takes care of that the container representation always has the RDF it needs. It also ensures that if somebody makes a large HTML page, it won't be overwritten when simply adding a triple using RDF. The latter has the somewhat weird effect though, that an PUT with RDF does not replace the entire representation of the resource, but I think it is a reasonable compromise to make.

Now, we can always discuss the requirement levels (e.g. if it must be MUST, or it should be SHOULD) :-) And then, objections are also welcome!

kjetilk on 13 Dec 2019

Noting here that we need to clarify intended persistence or use specific terms to cover different scenarios. One general simplification may be by relaxing the requirement (use SHOULD or MAY) but that may or may not be sufficient, so we should take a closer look at eliminating ambiguities. If we can prescribe, great. Otherwise, the spec should at least supplement with non-normative information.

Some thoughts:

Ideally all information of significance should be encoded in RDF but the intention with that "persist" was to simply take non-RDF content into account on the basis that it is useful to retain in some cases. It is however equally legitimate and preferable to replace an entire representation when for example an author wants to redo their homepage or change the description for the kind of content they have under a particular container. Provided that it doesn't conflict with other criteria eg. URI re-use recommendations.

General considerations: Clients need to better indicate their intentions to servers, and servers should have a default behaviour. Servers that do not want to or capable of persisting the non-RDF may want to error and indicate why.

Restrictions on altering server-managed triples remain. Creates are straight-forward as there are no existing alternative representations to modify. For the purpose of updating:

PATCH application/sparql-update isolates updates to the RDF graph, so the server may want to update representations without changing existing non-RDF content.
PUT text/turtle could indeed have a weird effect, so it may be permissible to not force the server to persist the non-RDF content in the other representations. After all, it is PUT and the behaviour should be uniform, but side-effects are not entirely ruled out.
POST is a bit fuzzy - may need to resolve other issues first eg. RDF Merge? Possible to change non-RDF content?

csarven on 14 Dec 2019

Yeah, so short version is really now that we need to sort out all the other resource access and lifecycle stuff, and then this issue can be formulated as a special case of those.

kjetilk on 16 Dec 2019

PATCH application/sparql-update isolates updates to the RDF graph, so the server may want to update representations without changing existing non-RDF content.

I should think that PATCH application/sparql-update should only be accepted when the target is pure RDF -- whether in a graph store, or in an RDF serialization with no (or, at least, minimal) out-of-band data.

Applying such a PATCH to an RDFa document, or to another HTML document with embedded "data islands" (typically in <script> elements) is fraught with peril, as the patching tool needs to understand every serialization in the existing document -- including where the RDF content may potentially repeat in embedded Turtle and JSON-LD and RDFa (and maybe more).

PUT text/turtle could indeed have a weird effect, so it may be permissible to not force the server to persist the non-RDF content in the other representations. After all, it is PUT and the behaviour should be uniform, but side-effects are not entirely ruled out.

HTTP spec says PUT replaces a resource in its entirety. In other words, PUT text/turtle may be understood as _DELETE the old {RDFa, HTML, XLS, TXT, etc.} resource, and INSERT this Turtle resource in its place_.

It _may_ be reasonable for Solid to require that PUT text/turtle (or any PUT) be preceded by a DELETE or similar where an existing target is not pure RDF (as I expanded above), or even not already text/turtle. In other words, forbid PUT to replace targets which don't closely match what's being PUT.

Indeed, it may be reasonable to disallow PUT except where the target does not exist!

Possibly, warnings or other explicit user interaction may be sufficient here.

POST is a bit fuzzy - may need to resolve other issues first eg. RDF Merge? Possible to change non-RDF content?

POST is inherently fuzzy -- because an HTTP server is normally free to do all sorts of things with the content POSTed by the client, including creating a new resource (with a new name) alongside the existing (which remains unchanged); or moving the existing resource (which remains unchanged) to a new name alongside a new resource which is created with the new content; or replacing the old entirely with the new (no matter if or how they are mismatched); or rejecting the new content entirely; etc.

TallTed on 16 Dec 2019

I should think that PATCH application/sparql-update should only be accepted when the target is pure RDF

It is intended to affect a resource URI as opposed to a representation URL.

PUT text/turtle may be understood as DELETE and [..] INSERT this Turtle resource in its place

The provided document is intended to atomically replace the representation which effectively updates the resource's RDF graph.

csarven on 16 Dec 2019

It is intended to affect a resource URI as opposed to a representation URL.
...
The provided document is intended to atomically replace the representation

Wait, are you replacing a representation or a resource? I think that last should be "atomically replace the resource".

And in the first above, I think you meant "affect a resource as opposed to a representation" (and not only because the effects are on the things identified/located by the URI/URL, not on the URI/URL strings themselves).

Also, I did not say anything about applying PATCH application/sparql-update to a representation, just to a target.

If an HTML+RDFa resource (which non-RDF content must be preserved as it is an LDP-NR) is targeted by a PATCH application/sparql-update with a small DELETE/INSERT of triples, what do you expect to happen? Personally, I would generally expect a 409 Conflict response, unless the server was fully capable of mapping the DELETE/UPDATE/INSERT into the RDFa (which is far more advanced than most if not all I'm aware of).

Finally, please remember that all URLs are URIs (while many URIs are _not_ URLs). A URL might identify an LDP-RS just as well as any other URI would. Drawing false distinctions (as between "representation URL" and "resource URI") needlessly complicates already challenging discussions.

TallTed on 16 Dec 2019

PATCH application/sparql-update isolates updates to the RDF graph, so the server may want to update representations without changing existing non-RDF content.

I should think that PATCH application/sparql-update should only be accepted when the target is pure RDF -- whether in a graph store, or in an RDF serialization with no (or, at least, minimal) out-of-band data.

OK, I can see that point. If I were to implement this, they way that I would have done it is that I would have extracted all the RDF from the document and put that in a triple store, and then, when the document was again dereferenced, I would have injected it back into the document, possibly in the head, making no attempt to preserve the structure, only the semantics. If done this way, patching with SPARQL wouldn't be a problem, but it would also not be exactly the same document.

Trying to keep the HTML intact while running an update query does sound somewhat difficult in the general case, indeed.

I think it should be a MAY requirement level thing, implementors may choose to reject SPARQL on HTML+RDFa.

HTTP spec says PUT replaces a resource in its entirety. In other words, PUT text/turtle may be understood as _DELETE the old {RDFa, HTML, XLS, TXT, etc.} resource, and INSERT this Turtle resource in its place_.

Yeah, there is certainly tension on this point, I think we came to a lesser evil on this point.

It _may_ be reasonable for Solid to require that PUT text/turtle (or any PUT) be preceded by a DELETE or similar where an existing target is not pure RDF (as I expanded above), or even not already text/turtle. In other words, forbid PUT to replace targets which don't closely match what's being PUT.

Right, that is a good idea, I think. Even though it adds some complexity, it also makes the idea here even lesser of an evil.

Indeed, it may be reasonable to disallow PUT except where the target does not exist!

Hmmm, interesting... Yes, since it is a container, perhaps that makes sense.

kjetilk on 17 Dec 2019

In that context I meant "replace the representation" as to the intention over the wire. I've already said "affect a resource" prior to that.

re affects/effects, I was trying to make the point that "pure RDF" is irrelevant because the "target" is intended to be the resource, not the representation. Manipulating the graph.

I'm not aware of a definition where HTML+RDFa is an LDP-NR. A representation can be "fully" stated in "pure RDF" in HTML+RDFa, therefore it can be an LDP-RS.

csarven on 17 Dec 2019

A representation can be "fully" stated in "pure RDF" in HTML+RDFa, therefore it can be an LDP-RS.

But @TallTed is referring to the situations where it is not fully represented in RDF. If it is fully represented in RDF, then it is easy, but that's not the interesting situation.

kjetilk on 17 Dec 2019

But if you look at how "pure RDF" is marketed at the moment, a generalisation is being made in that any representation with RDFa is an LDP-NR (or LDPR).

If you want to hold off, wait until the representation is inspected. Just as we should inspect claimed Turtle to make sure that it is actually Turtle and not gobbledygook PDF soup.

csarven on 17 Dec 2019

😄1

re affects/effects, I was trying to make the point that "pure RDF" is irrelevant because the "target" is intended to be the resource, not the representation. Manipulating the graph.

I have explicitly raised scenarios where the TARGET RESOURCE is NOT PURE RDF, is NOT just "a graph". You are declaring that my scenarios -- which are based on real-world experience -- do not exist. Please stop that.

I'm not aware of a definition where HTML+RDFa is an LDP-NR. A representation can be "fully" stated in "pure RDF" in HTML+RDFa, therefore it can be an LDP-RS.

Definitions were unfortunately written unclearly in the LDP spec. I am sorry about that. The error comes down to a missing "fully" in the LDP-NR definition. I believe the intent is as I have stated multiple times, and my belief is based on having been a participant in the rooms (and concalls) which resulted in the LDP spec.

I don't think I have ever said that it is _impossible_ to have an HTML+RDFa document which fully represents its state within the RDF. I have said (or strongly implied) that this is _unlikely_, in practice is uncommon in the extreme, and is not so for the hypothetical RDFa documents to which I've been referring in the conversation above.

It would be fine to treat an RDFa document which is fully represented internally as RDF as an LDP-RS -- but because this is not the case for all (nor, I say again, for most) RDFa documents, this should not be taken as the general nor the determinant case.

The rule here must be either "treat RDFa as LDP-NR because they cannot be guaranteed to be LDP-RS" or "test all RDFa files for RDF representation; where 100%, treat as LDP-RS; where <100%, treat as LDP-NR". As the latter rule provides no assurance of anything, I submit that it is problematic, and the former rule should be ratified.

Please note that nothing says that there cannot be RDF content within an LDP-NR! In other words, LDP-NR (including RDFa, PDFs, TXT, XLST, etc.) can be RDF bearing -- and RDFa (a/k/a HTML+RDFa) documents definitely do bear RDF, as that's the whole difference between then and HTML docs.

TallTed on 17 Dec 2019

Ted, we're trying to acknowledge some of the common real-world publishing scenarios as well as implementations in the wild in order to see how far the existing specs gets us and where to go next.

I'll repeat the running scenario so you can tell us the right way of doing it with respect to LDP:

People will want to publish a homepage in HTML. That's typically at site root: /, and that's generally accepted (or at least intended) to be a container from both LDP and Solid's perspective. We can look at many variations on this eg. whether / can be any kind of resource (re interaction model), or must only be one, or differ depending on how it is observed, etc. but for now let's stick to the mainline.

An actual example (but feel free to throw this out and use a different one): my website is at http://csarven.ca/ and WebID is http://csarven.ca/#i . How can I update that (WebID Profile) document so that it is available in HTML+RDFa, Turtle, and JSON-LD representations?

From your responses so far, I haven't understood your solution or how exactly can LDP help to accomplish what we are after without conflicting with LDP; introducing boatloads of conditions; or awkward cases to deal with.

It is not a problem if there is no simple solution here while conforming to LDP, but we'd just like to understand that. I don't want us to propose a convoluted solution just so that the Solid spec somehow is compatible with LDP. As previously agreed, the Solid spec can take on whatever is of use but not more.

csarven on 17 Dec 2019

An actual example (but feel free to throw this out and use a different one): my website is at http://csarven.ca/ and WebID is http://csarven.ca/#i . How can I update that (WebID Profile) document so that it is available in HTML+RDFa, Turtle, and JSON-LD representations?

The way this has been handled by Trellis is that, for a given LDP-RS (whether container, root resource or whatever), that resource can be content-negotiated in the following ways:

Read requests: by default (with no Accept header), Turtle is produced. Other types of RDF-based conneg are supported (JSON-LD, as required by LDP, and also N-Triples), but an HTML representation is also made available when a client includes, for example, Accept: text/html. That HTML version is, in fact, RDFa with all of the same triples one would find in the other RDF-based serializations. The application generates the HTML based on a configurable template, so a server operator can modify the look and feel of the RDFa.

Write requests: these come in two flavors: sparql-update for PATCH interactions, and (relevant for this discussion) RDF serializations for PUT and POST. However, (notably) PUT and PATCH only accept text/turtle, application/n-triples and application/ld+json. Write requests where the incoming entity body is text/html are not accepted as an LDP-RS (they would be for an LDP-NR)

acoburn on 17 Dec 2019

👀1

Thanks for sharing @acoburn .

If I understand correctly, Trellis will only let an application create or update a homepage such that when requested as text/html, it will only provide a HTML+RDFa serialization as "pure RDF" (or "fully" as per LDP-RS). More concretely, it is not possible to create, update or generate a document like my homepage (which contains some content that is not intended to be part of the RDF graph).

csarven on 17 Dec 2019

@csarven given the current implementation of Trellis' RDFa writer, one _could_ create and update a homepage such as yours, but the HTML template in use would be applied to _all_ LDP-RS resources. So, for example, you could update the data on that homepage via SPARQL-Update (or PUT with a Turtle payload), but in order to change the color scheme or the HTML layout, you'd have to modify the template which is not something you can do through the HTTP interface.

acoburn on 17 Dec 2019

I'm very relieved to hear that, @acoburn, since it sounds like Trellis is already quite close to the emergent (I wouldn't say rough yet :-) ) consensus, because at a fundamental level, the changes should be relatively minor to support this, right?

I mean, in the sense that you could have HTML documents that incorporates the template and therefore the RDFa with relative ease, right?

kjetilk on 18 Dec 2019

What Trellis is doing is perfectly fine. Its RDFa writer serves as a baseline for representations equivalent with respect to the RDF graph. In the Solid ecosystem, we are expecting applications to create/update specialised documents, whereas the server will only do the minimal, so the single template to help generate the HTML+RDFa serialization is fine and in what shape or form is an implementation detail (as long as the triples are there).

Having said that, what LDP affords and Trellis implements is I think limited in that it doesn't clearly handle a category of use cases related to creating and updating human- and machine- readable resources eg. a homepage, index, or any article for that matter, driven by client applications.

We need to bridge applications like dokieli and Trellis so that they can work with desired states of resources.

@acoburn :

if POST/PUT text/html is accepted as LDP-NR, what then happens on GET text/turtle?
If / is LDP-BC (LDP-RS), is its representation in text/html limited to the RDFa writer (excluding any client involvement)?

Do we expect multiple representations of a resource to be equivalent (in some reasonably defined or predictable way eg. isomorphic RDF graphs imply information equivalence)? If not, can representations be kept tracked and updated independently - is this too complex?

csarven on 18 Dec 2019

I don't think I have ever said that it is impossible to have an HTML+RDFa document which fully represents its state within the RDF. I have said (or strongly implied) that this is unlikely, in practice is uncommon in the extreme, and is not so for the hypothetical RDFa documents to which I've been referring in the conversation above.

It would be fine to treat an RDFa document which is fully represented internally as RDF as an LDP-RS -- but because this is not the case for all (nor, I say again, for most) RDFa documents, this should not be taken as the general nor the determinant case.

I fully agree with above, I can't really imagine scenario where anyone would go with RDFa intending to only serialize RDF graph. I would strongly prefer to treat HTML as Non-RDF Source and wherever we consider using RDFa just enable use of plain HTML.

People will want to publish a homepage in HTML. That's typically at site root: /, and that's generally accepted (or at least intended) to be a container from both LDP and Solid's perspective. We can look at many variations on this eg. whether / can be any kind of resource (re interaction model), or must only be one, or differ depending on how it is observed, etc. but for now let's stick to the mainline.

While I agree with recognizing it as a common use case. I think we should take this use case and consider various approaches. For example / when requested with text/html could redirect to something user (or application they use) can configure eg. /home. Most frontend framework if used can easily change browser location bar to / using History API.

An actual example (but feel free to throw this out and use a different one): my website is at http://csarven.ca/ and WebID is http://csarven.ca/#i . How can I update that (WebID Profile) document so that it is available in HTML+RDFa, Turtle, and JSON-LD representations?

I think approach above would work in this case, if you don't want to fiddle with History API you would just endup with extra slug of your choice. eg. /home or /hello. Myself I will most likely use History API to set it to bare /.
WebID draft makes text/trutle a MUST and client's have not reason to expect any other representation. I don't see reason for making publishing WebID Profile as HTML+RDFa a requirement for Solid.

Frankly, if we discuss website use cases, we should gather few common use cases. One common patter used by many SPA/PWA would have common shell / layout returned from any /**/* and all the navigation would happen client side. Not sure if here one would even go with Solid Storage for deploying it or use something better tailored for it (including pre rendering etc.).

My main point goes back to the top of this comment:

HTML should be treated as not intended to represent RDF and wherever we want to enable use of HTML we must not assume use of HTML+RDFa and support plain HTML.

elf-pavlik on 19 Dec 2019

I can't really imagine scenario where anyone would go with RDFa intending to only serialize RDF graph.

You can imagine a "scenario" for format x but not y?

Trellis has a RDFa writer outputting something useful for human consumers, like a directory index that's also machine-readable.

The utility of RDFa in the Solid ecosystem doesn't depend on LDP classification.

We expect the applications to understand the purpose of the RDF graphs and provide a useful environment.

For example / when requested with text/html could redirect to something user (or application they use) can configure eg. /home. Most frontend framework if used can easily change browser location bar to / using History API.

Is that a common publishing behaviour?

Why shouldn't people have their homepage at /?

I don't see reason for making publishing WebID Profile as HTML+RDFa a requirement for Solid.

No one proposed that should be a requirement.

We want to specify right enough things for the ecosystem so that people can do different stuff with it. We shouldn't prohibit such possibilities and choices.

HTML should be treated as not intended to represent RDF and wherever we want to enable use of HTML we must not assume use of HTML+RDFa and support plain HTML.

Why?

We want to enable the creation of applications to advance human/machine-readable resources.

csarven on 19 Dec 2019

For example / when requested with text/html could redirect to something user (or application they use) can configure eg. /home. Most frontend framework if used can easily change browser location bar to / using History API.

Is that a common publishing behaviour?
Why shouldn't people have their homepage at /?

HTML should be treated as not intended to represent RDF and wherever we want to enable use of HTML we must not assume use of HTML+RDFa and support plain HTML.

Why?

IMO if we want to enable people to have their homepage at / (without using History API) I think we should also allow them to use plain HTML and don't put HTML+RDFa requirement. Can we do that?

In https://github.com/solid/specification/issues/107#issuecomment-567981204 I notice that one may want to publish plain HTML at any URI ending with trailing /.

elf-pavlik on 20 Dec 2019

You might want to respond to this:

Do we expect multiple representations of a resource to be equivalent (in some reasonably defined or predictable way eg. isomorphic RDF graphs imply information equivalence)? If not, can representations be kept tracked and updated independently - is this too complex?

The History API has no bearing on this issue; HTTP resource access. It is about server and client negotiation as to what happens at / on the HTTP layer. How to keep the interactions uniform while handling multiple representations. This is not about permitting format x and prohibiting y based on opinions or arbitrarily pigeonholing. Assumptions need to be in check. I would argue that publishing and implementation experience ought to trump personal preferences. Can we do that?

csarven on 20 Dec 2019

Do we expect multiple representations of a resource to be equivalent (in some reasonably defined or predictable way eg. isomorphic RDF graphs imply information equivalence)? If not, can representations be kept tracked and updated independently - is this too complex?

I agree with what @TallTed said:

HTTP spec says PUT replaces a resource in its entirety. In other words, PUT text/turtle may be understood as DELETE the old {RDFa, HTML, XLS, TXT, etc.} resource, and INSERT this Turtle resource in its place.

I don't think we should handle different representations denoted by the same IRI independently and only use different representations for RDF Sources, in that case server would not be expected to preserve any serialization specific artifacts.

Trellis has a RDFa writer outputting something useful for human consumers, like a directory index that's also machine-readable.

Can implementation X conform to the spec if when text/html requested it responds with plain HTML (no RDFa)? If yes I think it should use Content-Location header and if we want to give clients control over it we most likely will need to define dedicated client-server interface to manage it.

elf-pavlik on 20 Dec 2019

👍1

https://httpwg.org/http-core/draft-ietf-httpbis-semantics-latest.html#rfc.section.6.2.5.p.9

For example, if a client makes a PUT request on a negotiated resource and the origin server accepts that PUT (without redirection), then the new state of that resource is expected to be consistent with the one representation supplied in that PUT; the Content-Location cannot be used as a form of reverse content selection identifier to update only one of the negotiated representations. If the user agent had wanted the latter semantics, it would have applied the PUT directly to the Content-Location URI.

elf-pavlik on 20 Dec 2019

👍1

Good point, @elf-pavlik . I'm generally in line with that draft, and I think the easy way out of that conondrum is to prohibit PUT on container (which could be extended to any resource that has server-managed partial representation). However, that discussion is better suited in #40 , I think. And as you can see, I'm open to several takes on this.

kjetilk on 20 Dec 2019

👍1

re PUT quote - which is by the way carried from 7231 - the Content-Location discussion (109) wasn't about client's request for reverse content selection. It was about whether to have any expectation in a response.

I don't think we should handle different representations denoted by the same IRI independently and only use different representations for RDF Sources,

I don't understand why the state of a resource is constrained to having only representations as RDF Sources.

in that case server would not be expected to preserve any serialization specific artifacts.

That's just encoded RDF graph in RDF Sources and that it can be serialized in different ways without having any expectation to persist information beyond the graph. Which is why it is okay to focus on the RDF graph in HTML+RDFa.

Can implementation X conform to the spec if when text/html requested it responds with plain HTML (no RDFa)? If yes I think it should use Content-Location header and if we want to give clients control over it we most likely will need to define dedicated client-server interface to manage it.

By "it should use", I assume you mean the server. I don't understand why HTML without RDFa needs to have its own URL. Updates can go through the resource URI just as it did when it was created - same effective request URIs. Server may use Content-Location for any Content-Type.

csarven on 7 Jan 2020

It seems like we haven't completely narrowed in the problem space yet, there are too many options open. I'd like to narrow in on a few.

Seemingly current state: index.html is a resource in its own right, returned as a representation of / when Accept header says so, otherwise manipulated like any other resource. It is not clearly defined which ACL applies to it, but it seems to mostly have its own, creating an unclear situation when it is returned as the representation of /. Possible improvement is to clearly define which ACL applies. This is what we originally encountered in https://github.com/solid/web-access-control-spec/issues/36
Work to resolve #109 in a way that allows index.html to exist and defined to be the representation of /, when the Accept header says so.
index.html is not a resource in its own right, but an internal implementation detail that is never exposed to the client. Within this, there are several possibilities:
1. The HTML representation is manipulated completely separate from the RDF representation based on the Content-Type and Accept headers. No requirements to represent the container RDF.
2. The HTML representation MUST bring along the representation of the container RDF. There are several possibilities within this:
  1. PUT isn't allowed to update the container, POST is used to replace the HTML representation and to append the RDF representation.
  2. PUT is allowed to replace the HTML representation, but not the RDF representation.
  3. PUT is allowed for both representations, but server-managed triples are ignored.
  4. Triples in the RDFa are ignored when updating a container.
  5. Triples in the RDFa are merged into the RDF representation of a container.
  6. Triples in the RDFa stays a part of the HTML representation, will not influence the RDF representation of the container.

The main reason I'm reluctant to address #109 now (it needs to be addressed, but not necessarily now), is the rather messy situation I've seen with the application of ACLs to index.html. I'm afraid it difficult to get right, thus resulting in bad security usability. That one resource is a representation of another is a common pattern on the Web, but not in a UNIX filesystem, which also creates some tension. That's why I am most inclined to not have index.html as a resource in its own right, making #109 an orthogonal issue that does not need to be addressed before later.

I am also most inclined to make sure the container representation contains the RDF, some of which is required to be represented. In #108 and #40 , I have advocated for that PUT isn't allowed to update a container, but I'm open to other interpretations.

kjetilk on 7 Jan 2020

I don't think we should handle different representations denoted by the same IRI independently and only use different representations for RDF Sources,

I don't understand why the state of a resource is constrained to having only representations as RDF Sources.

Thinking in terms of read-write web, using different RDF serializations for RDF Sources seems pretty straight forward for read-write operations using IRI denoting a resource. Here one should not have any expectations for server to preserve any serializations specific artifacts (comments, prefixes, json-ld context etc.)
As soon as one would want to have consistent read-write operations for different representations, where serialization specific artifacts get preserved, i think each representation should have IRI denoting that specific representation. This approach also seems applicable whenever one would want to have different representations for Non-RDF Sources.

https://developer.mozilla.org/en-US/docs/Web/HTML/Link_types

alternate
Otherwise, the link defines an alternative page, of one of these types:
[...]
in another format, such as a PDF (if the type attribute is set)

For the case where someone needs consistent read-write operation using IRI denoting an LDP Conainer, I think giving HTML representation more specific IRI would enable straight forward way of doing it. HTTP GET to that IRI with Accept: text/html could return Content-Location header as well as Link header with `rel="alternate"; type="text/html" could clearly advertise IRI for that specific representation.

I see it similar to example in https://www.w3.org/TR/cooluris/#r303gendocument

r303gendocument

Except that any RDF based representation of the container would not need more specific IRI. Of course as long as use case doesn't require to persist rdf serialization specific artifacts.

elf-pavlik on 8 Jan 2020

The error in the Cool URIs example is in labeling the circle for application/rdf+xml as RDF instead of as RDF/XML.

It's a common blur from the early days of RDF, where RDF/XML was treated as the only serialization -- and RDF and RDF/XML were therefore treated as synonyms.

Today, text/turtle or application/ld+json would be a more likely serialization media type for such examples -- and the circle would be more likely to be labeled Turtle or JSON-LD, respectively, than simply RDF.

TallTed on 8 Jan 2020

👍1

I agree with all the comments regarding application/rdf+xml which also unfortunately registers .rdf file extension. For solid I think of something in lines of diagram below:

solid-conneg

Where https://solid.example/team/ gets handled as RDF Source but it also has more specific IRI https://solid.example/team/page.html for Non-RDF Source representation, in this case text/html. Any read request would simply get response based on content type directly from https://solid.example/team/, while write would need to use more specific IRI of representation. Relying on Content-Location instead of redirecting would also help with using vanity URLs, person navigating we browser to https://solid.example/team/ would not even see that more specific https://solid.example/team/page.html.

To address full resource lifecycle it would require specifying interface to add that alternate relation

Link: <https://solid.example/team/page.html>; rel="alternate"; type="text/html"

I think @csarven referred in some other issue to HTTP LINK method draft https://tools.ietf.org/html/draft-snell-link-method

elf-pavlik on 10 Jan 2020

[Draft: I may edit to clarify.]

We have some common ground derived from several issues that can help to address this one. Here is one way:

Read, Append, Write operations on a container should go through the canonical / (slash semantics).

The handling of container representations is optional. A server may use the /index.* convention as representations of a /. They may expose the representation URL through Content-Location. A container may have RDF bearing and non-RDF bearing representations. The representations of a container may be listed in its containment triples.

ACL is set on the container and applicable to all of container's representations.

Deleting a resource removes its representations.

RDF bearing representations of a container should not update the server-managed triples.

csarven on 20 Jan 2020

👍1

They may expose the representation URL through Content-Location.

If server sends in response representation, which has URL denoting that specific representation, I think at least it SHOULD expose it.

Deleting a resource removes its representations.

At least for portability we very likely need to define how to express 'different representation of', I think rel="alternate"; type="text/html" might just work.

RDF bearing representations of a container should not update the server-managed triples.

LDP spec doesn't define what RDF bearing means. Could we base it on RDF Source instead? People working on implementations need to properly handle it and in solid ecosystem can rely on consistent support for both text/turtle and application/ld+json representations, plus PATCH with application/sparql-update when interacting with RDF Sources.

Read, Append, Write operations on a container should go through the canonical / (slash semantics)

I think we may need to clarify that ldp:Container as subclass of ldp:RDFSource expects supported RDFSource representations when interacting with it.

To support use case of Non-RDF Source representations like any text/html, we still need to specify how to create initial version of such representation.

We may still need to clarify what happens if more specific representations exist as RDF Sources. In that case Write operation on alt.ttl doesn't affect representation returned by GET on / with Accept: text/turtle even if /alt.ttl stays designed as rel="alternate"; type="text/turtle" of /. Only Write/Append on / affect RDF Source representation returned on Read.

A server may use the /index.* convention as representations of a /.

If we provide mechanism to denote and manage more specific representations I don't see when server would also use internal convention as above. For example if client creates /page.html as rel="alternate"; type="text/html" of /, server can't use /index.html any more.

elf-pavlik on 21 Jan 2020

If server sends in response representation, which has URL denoting that specific representation, I think at least it SHOULD expose it.

Please take that up in https://github.com/solid/specification/issues/109 and explain why.

At least for portability we very likely need to define how to express 'different representation of', I think rel="alternate"; type="text/html" might just work.

That has no bearing on the delete behaviour.

LDP spec doesn't define what RDF bearing means. Could we base it on RDF Source instead?

I wasn't referring to LDP. RDF 1.1's "RDF-bearing" and "RDF source" are sufficient for Solid.

I think we may need to clarify that ldp:Container as subclass of ldp:RDFSource expects supported RDFSource representations when interacting with it.

No need. We've already determined that Solid / is not "pure RDF" like in LDP.

If we provide mechanism to denote and manage more specific representations I don't see when server would also use internal convention as above.

We are not specifying the identifier of the representation.

csarven on 21 Jan 2020

We're having a gathering in Inrupt this week, so I had a chat with @timbl on this. We found that we need to take it up F2F the next time more of us meet, on the basis on what we've discussed here.

I briefly outlined my preference, which is option 3.ii.e. with some variations of the PUT behaviour above, and he did appreciated that solution, but it is easier to convey more options F2F. I will note, however, that the absence of the 303 behaviour in Solid is a deliberate design.

kjetilk on 22 Jan 2020

/ is an information resource, 303 is not applicable. 200 and Content-Location is suitable for possible use of index.*.

I'll respond to 3.ii.e. for the record. 3.ii.e. is relatively aligned with https://github.com/solid/specification/issues/69#issuecomment-576060661 but it leaves out key behaviours part of the wider mechanism:

3. "never exposed to the client" entails Content-Location or redirects are forbidden, which is a stronger constraint than RFC 7231's. A bit more relaxed version would be that Content-Location is not required but may be used (basically 7231 but that should be addressed in https://github.com/solid/specification/issues/109 in any case). In any case, we've already highlighted use cases where some implementations may want to expose representation URLs.

ii. Existing server-managed triples are assumed to be in HTML+RDFa (as per general discussion for the most part) but doesn't particularly exclude fuzzy (clumsy handling of?) RDF formats in script. It raises potential complexity in any case.

e. seems fine if intention is to append to the RDF graph, provided that existing server-managed triples are not affected or new ones added. This is rather taken as a global rule, along the lines of LDP 5.2.4.1.

I've tried to reconcile as best as I could based on things we agree on, as well as what's happening in on some implementation behaviour in https://github.com/solid/specification/issues/69#issuecomment-576060661 . I think it is important digest it as a whole. I'm not sure if we'll have consensus that's more fitting but happy to examine it along with the other options further in a F2F and/or a call.

csarven on 23 Jan 2020

ii) The HTML representation MUST bring along the representation of the container RDF. There are several possibilities within this.

:-1: I think this would prevent people to publish just plain HTML representation. Even if we would have take this path I think we should not require RDFa but also allow including text/turtle and application/ld+json in script tags.

I also notice that NSS by default uses data browser (unless run with --suppress-data-browser). In that case it returns plain text/html representation if requested. This representation has no intention of accepting plain text/html writes. I think we may need to properly document different use cases and requirements coming from them. Then we can clearly evaluate how different approaches can address those requirements in clean way.

elf-pavlik on 23 Jan 2020

I'll try to comment just on the behalf of myself, and since I woke up early and couldn't sleep. :-)

First, @elf-pavlik ,

I think this would prevent people to publish just plain HTML representation.

Yes, but that is I think, an important feature, as a container has, per definition, an RDF representation, which will always be significant. Moreover, it is important for Databrowser, because it can easily embed itself in the HTML document at an appropriate place if the RDF is there, which I also think is desireable. However, I think we can allow more embeddable RDF formats, it doesn't need to be constrained to RDFa.

I'm all for being use case driven, but I am concerned that we are spending too much time on this pretty edge-case feature (this will be the 79th comment), so I would much prefer to find an urgent resolution to it. I think the use case is pretty clear, people have maintained HTML representations of containers since the dawn of ages, and we don't want to break that even when there is a client-side generated view.

@csarven ,

I'm afraid you didn't capture the discussion very well, because I fundamentally disagree with the design ;-)

What I do agree upon is:

Read, Append, Write operations on a container should go through the canonical / (slash semantics).

However,

The handling of container representations is optional. A server may use the /index.* convention as representations of a /.

index.* was put outside of the design space already in December, we should not go in more circles on this, this is specifically about the behaviour of an HTML representation.

They may expose the representation URL through Content-Location. A container may have RDF bearing and non-RDF bearing representations. The representations of a container may be listed in its containment triples.

I also think this is inconsistent with the design that manipulations go through /. With this design, index.html is a resource in its own right, and then we must allow it to be manipulated as any other resource, with all the problematic side effects it has with the applicability of ACLs and possibly other metadata resources, as well as problems it might cause for queries and stuff down the road.

index.html is either a resource in its own right, or it isn't. It can't be something in between. If it is not a resource in its own right, your concerns as to additional constraints to the RFC is not applicable.

ACL is set on the container and applicable to all of container's representations.

Yes, but if you insist that index.html is something that can be referenced as a member of the container and Content-Location, it places that in a special situation. It can be solved, but it is a more complicated solution, and therefore more error prone.

RDF bearing representations of a container should not update the server-managed triples.

Yeah, but it needs to be stronger, since it just cannot update the server-managed triples, so it doesn't capture the nuances.

So, this proposal is not consistent with Trellis (which is in my 3.ii space), NSS (which doesn't manipulate through /), the design constraints we had earlier (it is only about HTML), nor internally.

kjetilk on 24 Jan 2020

index.* was put outside of the design space already in December, we should not go in more circles on this, this is specifically about the behaviour of an HTML representation.

No, once again, the original issue including the title and the comment that Dmitri created is about and I quote: "index.html and/or index.ttl". If you're only interested in focusing on the index.html bit, that's fine, but the issue still needs to address index.ttl or at least see it index.* as a specialisation of https://github.com/solid/specification/issues/109 for starters - there is a reason why I've created that first so we can revisit this (also mentioned that before). "index.*" was used as an alias to both of those indexes (and others obviously). There are also numerous references to both (if not more) index formats elsewhere.

I also think this is inconsistent with the design that manipulations go through /. With this design, index.html is a resource in its own right, and then we must allow it to be manipulated as any other resource, with all the problematic side effects it has with the applicability of ACLs and possibly other metadata resources, as well as problems it might cause for queries and stuff down the road.

index.html is either a resource in its own right, or it isn't. It can't be something in between. If it is not a resource in its own right, your concerns as to additional constraints to the RFC is not applicable.

Yes, but if you insist that index.html is something that can be referenced as a member of the container and Content-Location, it places that in a special situation. It can be solved, but it is a more complicated solution, and therefore more error prone.

You've misunderstood. The base requirement is that interactions go through /. If however a server exposes the representation URLs (see issue 109), interactions can still go through /. That doesn't exclude index.* being their own resources... and whether a read or write can happen. Authz policy on the representations is still.. literally what's set for /. Issue 109 and issues involving ACL and representations is clear about being set on the primary resource (as opposed to the representation).

Yeah, but it needs to be stronger, since it just cannot update the server-managed triples, so it doesn't capture the nuances.

Yeah, I've proposed stuff.. please see how these relate https://github.com/solid/specification/issues/40#issuecomment-573358652 , https://github.com/solid/specification/issues/45#issuecomment-541614170 , https://github.com/solid/specification/issues/40#issuecomment-567417005 .. The repo is littered with possible ways forward.

The whole point of "RDF bearing" was if client/server deems a resource to be so, the rules on containment applies. Heck, we can even go all the way back to this: https://github.com/solid/solid-spec/issues/202#issuecomment-512902223 . However server handles / with text/html, the rest follows. It would literally allow /'s text/html representation to be treated as RDF bearing or non-RDF bearing.

the design constraints we had earlier (it is only about HTML)

Clearly you are mistaken:

https://github.com/solid/specification/issues/69#issue-497863441 questions .html , .ttl
https://github.com/solid/solid-spec/issues/134#issue-402668817 ibid
https://github.com/solid/web-access-control-spec/issues/36#issue-402670605 ibid
https://github.com/solid/solid/issues/213 ibid

Obviously not all implementations are doing the same. Some of the informal criteria that I've mentioned is what NSS does and what Trellis either does or can do. Your 3. is a non-starter based on... guess what "we've already highlighted use cases where some implementations may want to expose representation URLs." So, you can't just ignore that and still try to force your preference.

I've suggested that I can clarify and expand. I've also suggested that you should take the comment as a whole and see how it connects with the agreements made elsewhere. That wasn't an arbitrary decision and I didn't mention all that for fun.

I'm reverting the issue title until there is consensus without obvious objections or at least minimal approval from the creator of this comment.

Clearly we are talking past each other. I am as frustrated (if not more) as anyone else. We can pick this issue up in a call or a F2F :)

csarven on 24 Jan 2020

@timbl and I resolved to go for option 1. i.e. something close to the current NSS behaviour. I'll do a writeup.

kjetilk on 24 Jan 2020

Please specify diff with https://github.com/solid/specification/issues/69#issuecomment-576060661 .

csarven on 24 Jan 2020

Let me first apologize for the fast turn of the events here. It was truly not intentional: This issue is interesting as it have brought out pretty much all the tensions between the different components and philosophies of Solid, the LDP, the UNIX file system, the roles of representations on the Web, etc. However, it is also an issue quite far on the edges, and we cannot keep it open just for the undeniable intellectual exercise it provides. With more than 80 comments, I think it has been on an extensive hearing, and suddenly today, I had the rare opportunity to take it up with Tim in real life, and so I hope people aren't too annoyed if we can take that conversation as the guide. Moreover, you will quickly notice that my own favorite, the "3.ii." direction was quickly dismissed. So, here we go:

index.html is a resource in its own right, and will be manipulated as any other resource. That is, it can be read, and all update operations to the HTML representation will be done on index.html itself. index.html will be contained in the container as any other resource. It may have its own ACL that will apply when the index.html resource itself is dereferenced.

Only when a read operation is executed on / with an Accept header that indicates that a HTML representation is wanted will the contents of index.html be returned (there's some wiggle room around q factors here). It MUST then include the RDF representation of the container embedded in the HTML (the details of this was not discussed, but the idea is that the databrowser can then be embedded in the HTML). Content-Location MUST be set.

The ACLs will be applied as follows: First, the container's ACL will be applied. If the client has read access to the container, an internal redirect is made. If the index.html has its own ACL, then that too will need to indicate that read is authorized for the content to be returned.

Time ran out as we started to discuss what should be done if the client is authorized to read the container, but not the index.html. I think the natural thing to do would be to return to the Accept header to check if there are other representations (i.e. RDF) that are acceptable, if not, I would suggest a 406, but a case could also be made for always having an RDF fallback in that case, since an RDF representation of a container always exist.

The diff to @csarven's comment is that index.* is not considered. index.ttl has had a different mission in Solid and has had so for some time (it has certainly nothing to do with my preference, I have not even been aware of this before Tim told me about it, we've just basically had a collective misunderstanding around it). Most operations on / will not be affected by the presence of index.html, only Read will. The Content-Location and containment is required, the ACL algo is different.

So, this wasn't my favorite resolution, but with the clarification on how the ACLs are applied, it resolves the initial issues that prompted many of the reports on this. It also eases some tensions on the method definitions, as index.html is relevant only for read operations. I think it should be helpful to settle it.

kjetilk on 25 Jan 2020

Housekeeping: The reference to "index.ttl" in this issue should be left as a representation. The data augmentation case raised by data browser (which happens to use index.ttl) will be addressed in https://github.com/solid/specification/issues/144 .

it is also an issue quite far on the edges

The use cases that are brought up are are among the most common practices on the Web. Solid must be able to address them.

With the proposal that's brought up:

It MUST then include the RDF representation of the container embedded in the HTML (the details of this was not discussed, but the idea is that the databrowser can then be embedded in the HTML). Content-Location MUST be set.

I'll respond to the questions I've raised in https://github.com/solid/specification/issues/69#issuecomment-566974645 :

Do we expect multiple representations of a resource to be equivalent (in some reasonably defined or predictable way eg. isomorphic RDF graphs imply information equivalence)? If not, can representations be kept tracked and updated independently - is this too complex?

The proposal suggests that the representations are at the very least expected to be equivalent based on RDF graph. The proposal also suggests that they will be tracked and updated independently.

The details indeed need to be worked out:

The discussion in https://github.com/solid/specification/issues/108 shows that server interference ie. injection of containment information into HTML is not particularly practical or mature. There is also no implementation experience. Moreover, when updating non-RDFa RDF bearing resources, server interference is not expected, that is the server will either allow the request or reject. Explained further below based on existing consensus.

The intended state of / in text/html should be controllable by a client without server interference provided that the representation conforms to server-imposed constraints. Rough consensus in https://github.com/solid/specification/issues/40#issuecomment-573358652 :

The criteria from 5.2.4.1 can [in addition to PUT] be applied to POST (#108) , PATCH (#85), index.html (#69).

helps to clarify client and server expectations. That is, client needs to ensure the integrity of the containment information in an HTML with RDF bearing representation when making changes and the server verifies the request. This is the same criteria for all RDF bearing representations.

As no details are provided on equivalence or particular information persistence beyond encoded RDF graph, it can be deemed to be compatible with https://github.com/solid/specification/issues/69#issuecomment-563224708 :

If two RDF graphs are isomorphic, that's all that counts towards representation equivalence.

In https://github.com/solid/specification/issues/69#issuecomment-576060661 , I proposed a relaxed version of representation equivalence which can be determined by the agreement between a server and a client:

A container may have RDF bearing and non-RDF bearing representations. The representations of a container may be listed in its containment triples.

It meant that a representation in HTML may or may not be RDF bearing. If deemed to be RDF bearing, there are specific expectations. This is a far simpler design for both servers and clients. For instance, updating an RDF bearing representation does not entail that the non-RDF bearing representations needs to be updated for some "equivalence", and vice-versa.

To summarise suggestions (preferably selecting one from this order):

A representation being RDF bearing or not is an agreement between a server (verification) and a client (intention).
Client ensures that the representation is RDF bearing (in addition to adhering to server constraints) and server verifies.

Aside: Either of those options can allow any application (eg. data browser, dokieli) to be embedded in HTML. One of the difference between those applications may be that data browser's primary reason to have a container's representation in HTML (and the whole design on having index.html) is so that its JavaScript can be embedded and it doesn't care about the underlying information (at this time). In dokieli, resource's content matters and its JavaScript is only intended to act as a way to introduce interactions on the resource - if JavaScript is unavailable, the resource can still be expected to be human and machine-readable (useful).

Content-Location MUST be set.

No objection, however I think that should be inherited from the general case in https://github.com/solid/specification/issues/109 .

The ACLs will be applied as follows: First, the container's ACL will be applied. If the client has read access to the container, an internal redirect is made. If the index.html has its own ACL, then that too will need to indicate that read is authorized for the content to be returned.

Do I understand you correctly in that the order of this check is different to resources in general ie. if a resource in a container has its own ACL, it will be applied, otherwise, the inheritance algorithm is applied. What I'm not clear about the proposal is if index.html's ACL exists it will be applied instead of container's ACL. Can you clarify that bit?

what should be done if the client is authorized to read the container, but not the index.html. I think the natural thing to do would be to return to the Accept header to check if there are other representations (i.e. RDF) that are acceptable, if not, I would suggest a 406, but a case could also be made for always having an RDF fallback in that case, since an RDF representation of a container always exist.

I don't particularly see why index.html's ACL needs to exist (as opposed to just inheriting container's). It opens up more complications than it actually helps. While a container's representations are resources in their own right, it doesn't mean that they must have their own ACL. Simply use container's (fixed reference). What's the actual use case to be different?

csarven on 31 Jan 2020

I suggest we remove this from the FPWD as we will not be able to resolve this in foreseeable future.

kjetilk on 31 Jan 2020

I just want to comment very briefly on this:

injection of containment information into HTML is not particularly practical or mature. There is also no implementation experience.

That's wrong. Trellis already supports it as indicated above. We also support it trivially with Perl:

my $gen = RDF::RDFa::Generator->new;
$gen->inject_document($dom, $model);

where $model contains the RDF you wish to inject, and $dom contains the DOM object of the XHTML document you inject into. This is code that has been in production for a decade. I quickhacked a little script that takes an XHTML document on the STDIN and outputs the injected document to STDOUT last night: inject_rdfa.pl.txt

There is plenty of implementation experience, and it is very mature. I would be very surprised if it is not equally simple in JS, it is just about generating the RDF and adding it to the DOM tree, that's all there is to it.

kjetilk on 3 Feb 2020

Edit: Didn't notice your comment before sending mine, so here is a quick reply:

That's wrong. Trellis already supports it as indicated above. We also support it trivially with Perl:

It is not an arbitrary injection. Obviously triples can always be thrown in somehow. Even possible with sed but that's probably not a good idea, right? The whole document needs to be properly serialised and be coherent, and anything added or removed from the containment triples should not interfere with everything else, including ideally structure and rendering. Having code to inject is one thing but I'd like to see actual HTML documents in practice that's subject to updates.

In any case, I've noted below that this part of the update is an implementation detail.

Revisiting this.. there is more commonality in the approaches than they seem.

Appending a resource to a container:

PUT /index.html
POST /
Slug: index.html

It should include a triple like:

[about=""]
rel="ldp:contains" href="index.html"

Following are equivalent:

GET /index.html
GET /

Content-Location: index.html

[about=""]
rel="ldp:contains" href="index.html"

[
RDF in HTML best practice:
Don't set base URI in the HTML representation (index.html), but if set, it should end with /. This is so that the RDF graph in index.html is same as /.
]

Is this correct:

[/ or /index.html] MUST then include the RDF representation of the container embedded in the HTML.

Effectively establishes container's HTML to be RDF bearing.

Influences required RDF serializations: https://github.com/solid/specification/issues/45 ie. adding RDFa (or script with RDF) to Turtle and JSON-LD.

When a new resource is appended or removed from a container, all of its representations (including HTML) needs to incorporate the changes to the containment.

Appending another resource:

POST /

Location: foo

GET /index.html
GET /

Content-Location: index.html

[about=""]
rel="ldp:contains" href="index.html"
rel="ldp:contains" href="foo"

How exactly a server includes the containment triples is an implementation detail. Same level of requirement for Turtle and JSON-LD.

This may be a less of an issue when /index.html is updated directly (eg. PUT /index.html) because the container's integrity falls on the server-imposed constraint ie. listing containment triples.

Server should reject if update to /index.html changes containment triples (aligned with global rule on updating containers). That entails that an RDFa parser is required. Having an RDFa parser makes it possible to serialize to other RDF.

If we treat all representations of a container as equivalent (based on underlying RDF graph), then adding new resources to a container ultimately requires an update to container's description. So the representations need to include the containment triples.

The following criteria can work for updating resources but not sufficient for appending or deleting a resource from a container:

Client ensures that the representation is RDF bearing (in addition to adhering to server constraints) and server verifies.

So, we do need the following any way:

[/ or /index.html] MUST then include the RDF representation of the container embedded in the HTML.

Having said that, there is the question of whether the HTML representation of / may be exempt from this and so a SHOULD instead of MUST. Relaxing the requirement definitely simplifies server and client implementations and allow more flexibility on / (eg. as arbitrary homepage or directory listing).. Keeping it strict means more consistency but also there is a chance that the resulting HTML is not necessarily what the client would like to see/interpret.

csarven on 3 Feb 2020

TL;DR: Please explain benefit motivating the requirement of embedding RDF in HTML representation of a container.

I honestly don't understand benefit of requiring HTML representation of a container to include RDF (embedded via <script> tag or RDFa). In use case of 'home page', I think user often would want to use some site generator and let it PUT / POST that HTML representing container. If any client needs RDF it can always request text/turlte or application/ld+json. No one stops users to embed RDF in HTML if they have reason for that, still requiring it would force them to regenerate the home page every time they add or remove something from that container. I think that requirement adds burden without any clear benefit, again if someone wants to embed RDF in HTML they can always choose to do it.

elf-pavlik on 4 Feb 2020

There is much repetition at this point. I'm responding below but it'd be great if we can continue the recurring themes and ideas in public chat, calls, F2F etc.

TL;DR: Please explain benefit motivating the requirement of embedding RDF in HTML representation of a container.

The exact same question can be asked for any RDF format. There is no difference between them if the exchange language is RDF and the sources are RDF bearing. Any one can do the job. We can also discuss why HTML+RDFa alone may suffice, and can in fact handle more use cases than the alternatives. It can be very accessible and neither would it require JavaScript to read (or use) a homepage or a directory index. So, why even bother introducing the others? As appealing or appalling as that may be to some, framing as such may not be fruitful.

Once again, if the representations are expected to encode equivalent RDF graphs, then being consistent is a reasonable design decision. The contrary is easy to raise: why should format x and y be expected equivalent but not z, especially when the resource is expected to be RDF bearing with containment information to begin with. This case is not to be conflated with a resource that's deemed to be non-RDF bearing (like an image) and then also providing an RDF-bearing representation. That's not what we want or should be practiced. The homepage case not only predates but is widely deployed than root storage or container. Hence, if the distinction between a homepage and root container is so important, then it only makes sense to leave the homepage or a directory index alone at / and simply use another URI for the root container. Certainly that's not a nice option to some people and it only complicates the situation. So, for the time being, we have to look into how to accommodate both cases - which are actually quite similar - using the same URI.

In use case of 'home page', I think user often would want to use some site generator and let it PUT / POST that HTML representing container.

I am a user. That's not what I want. I do not want to switch between applications just to update different aspects of a resource, or worse, have it switch perspectives based on a particular representation - the classic "pure RDF" and "not so pure RDF". Nothing like that is in practice or can be considered a good design. I want to be able to use an application like dokieli to update / where it works as storage root (including containment information) as well as a human and machine-readable homepage including my WebID Profile. I consider my WebID Profile in a HTML+RDFa to be canonical because that gives the most utility. I want to be able to authenticate using that WebID. If a server can't provide an RDF bearing representation of my WebID Profile or a client unable to parse an RDF bearing representation (as in RDFa), I can't authenticate. Currently, only Turtle (and JSON-LD) are acknowledged by some servers and authentication clients. We need to clarify this gap so that people are not prevented from publishing as they wish while adhering to minimal global requirements.

csarven on 5 Feb 2020

Proposal for a new solution in https://github.com/solid/specification/issues/198, which considers “index” representations part of a compound state of a container (and considers the index.html and index.ttl behavior implementation-specific details).

RubenVerborgh on 16 Sep 2020

Specification: Specify/advise semantics of default Container resources (index.html/.ttl)

All 90 comments

Related issues