Wot-thing-description: Consider adding DataSchema to response and additionalResponses

[mmccool]

When there are additionalResponses, and maybe also when there is just a "response", the output DataSchema may not be the same as the input DataSchema. For example, even when getting a property, an error response may not contain the property but an error code and message instead.

However, right now ExpectedResponse only has a ContentType. This is OK if the response is unstructured (eg JPEG image), but it the response is JSON or XML a DataSchema is needed.

I propose adding an optional "data" field with type "DataSchema" to both "response" and "additionalResponses"

[mmccool]

One problem: actions already have an "input" and "output" data schema, so adding another data schema inside "response" would be redundant and confusing. We can just avoid using it for properties since if the response is not the same as the input then we can just call it an action. So we could do it only in additionalResponses, but then I would have to create a subclass of ExpectedResponse in the ontology, etc. to model this.

[sebastiankb]

from today's TD call: @mjkoster can support here since they had the same issue in OneDM

[mjkoster]

It seems like the processing exception (error report) case is the most important to allow additional response types, where "the usual programming is interrupted" and a different payload is sent, for example in the case of Modbus errors. The important distinction is that there is special header information like a response code that tells the consumer to look for the special data payload.

In the case of diverse payloads that occur "normally", the anyOf construct might be more suitable.

In other cases, we want to model optionality in the TM but will choose one for the instance and TD.

[farshidtz]

One problem: actions already have an "input" and "output" data schema, so adding another data schema inside "response" would be redundant and confusing.

Also, events already have a data field for output data schema, equivalent to actions' output.

Seems like all interactions need a schema for the default response and this could go into Interaction Affordance, deprecating event data and action output.

[mjkoster]

I'd like to consider the logical structures here, and try to drive to a concrete proposal.

There are input and output schemas for actions, and data schemas for properties. Events have a notification schema. These schemas are all part of the application data model for interactions. Only the formatting is expected to change when the protocol binding changes (array vs. object for multiple items, key names, contentType...).

Subscription schemas and cancellation schemas for events are arguably part of the protocol binding, because they would be different for different protocols. I recommend that we try to work subscription and cancellation into these schema extensions for forms, also bc that's where the ops for them are defined.

As for request and response, these are also protocol artifacts. NB for some protocols, the action output data will be packaged in the response that goes with the request that sends the input data. For other protocols, the action output data will not be in the response. Likewise, sometimes the response will contain a location from which to get the output data or status data for long running actions. We may also need a separate getStatus op and schema for these. The response may contain different information in the case of errors, as we note elsewhere.

I recommend we not try to collapse the action output and event notification schemas into responses, but we do need to provide a way to indicate that a response contains action output data or event notification data for those protocols where they are packed into responses. Putting the contentType and location, plus any other header information, is also a logical choice for response definitions.

It might be a good practice to specify response/ExpectedResponse only for the cases that return application data specified in the dataschema, output, or notification, and always use additionalResponses for errors and exceptions. I think MMC has proposed along these lines. Then responses would never contain schemas when there is a dataschema, or output or notification schema in the interaction definition.

Then, additionalResponses could always be used for error cases or where alternate control schemas are needed. A protocol may have an observe or subscribe response that isn't part of the application data model, so we could use additionalResponses to indicate that case. For example, the new CoAP pubsub protocol passes control documents before any data are available.

Finally, we could extend requests in forms to accommodate subscription and cancellation, and other cases like aborting a long-running action; maybe additionalRequests would be an appropriate vocabulary term here, since they would need to have bespoke payload schemas like additionalResponses do.

To recall another discussion, if there are cases where there are multiple schemas as normal data model options, these will be specified in the interaction definition using an optionality construct like "anyOf" or sdfChoice type pattern.

We could sketch up some examples if any of this is not well enough defined.

PS. This is a good place to mention that my original protocol binding strawman from Santa Clara 2017 had requests and responses with all of the on-the-air schemas in the binding section.

[egekorkan]

Subscription schemas and cancellation schemas for events are arguably part of the protocol binding, because they would be different for different protocols. I recommend that we try to work subscription and cancellation into these schema extensions for forms, also bc that's where the ops for them are defined.

There is a related issue in Scripting: https://github.com/w3c/wot-scripting-api/issues/208

I also agree that we can deprecate these two fields since even the TDs provided that use them are actually using WebSub and putting the payload descriptions actually does not help with the implementation unless there is a mention of WebSub somewhere (in subprotocols) since the consumer needs to keep track of some parts of the payload which are not related to the application logic (where scripting API lives) but should be actually handled by the implementation.

[relu91]

I too tend to agree that what should be handled transparently by the runtime should not be exposed by the TD. However, about the subscription payload, I had a use case in mind. TDDs you can have events about added or removed Thing Descriptions. Therefore one client can subscribe to one or the other to obtain useful information about what is currently available in its environment. Translating that to a TD is pretty straightforward:

{
    /*other TD fields */
    "events": {
        "added":{
            "title": "Thing Description added",
            "description": "Fired every time a new TD is added to the TDD"
        },
        "removed":{
            "title": "Thing Description removed",
            "description": "Fired every time a new TD is removed from the TDD",
        }
    }
}

Now consider that I can use JSONPath to filter the notifications and be notified only for things with @type : "sosa:Sensor". How could I describe this use case without the subscription field? I think this is strong evidence that there might be subscriptions that depend on _applciation data_ rather than just protocol configuration and prelude.

Not sure if we can say the same about cancellation.

concerning this issue:
I think the current approach of the TD is to keep protocol and logical interaction levels separated. So what it is really wired is that with additionalResponses we are mixing protocols and DataSchemas, concepts that are kept in different abstract categories so far (e.g. we have a response and we have output).

Since we are gradually introducing JSONPointers in TDs it might worth exploring the idea of having additionalSchemas and point to one named schema from additionalResponses or simply pointing to a portion of the output schema without introducing a new field.

[egekorkan]

PS. This is a good place to mention that my original protocol binding strawman from Santa Clara 2017 had requests and responses with all of the on-the-air schemas in the binding section.

@mjkoster could you share it in this issue?

[sebastiankb]

@mjkoster

I recommend we not try to collapse the action output and event notification schemas into responses, but we do need to provide a way to indicate that a response contains action output data or event notification data for those protocols where they are packed into responses. Putting the contentType and location, plus any other header information, is also a logical choice for response definitions.

+1

I think, a good way is to simple define multiple schemas with anyOf and a JSON Pointer points to a schema entry. Maybe a more clean way can be that we use a new data schema field like additionalSchemas at interaction level (similar we have for uriVariable):

    "properties": {
        "foo": {
            "type": "string",            // for default response
            "additionalSchemas": {
                "anyOf": [
                    {"type": "string"},  // for additional response I
                    {"type": "number"}   // for additional response II
                ]
            }
        },
        "forms": [
            {

                "href": "https://mylamp.example.com/status",
                // for default response
                "contentType" : "text/plain;charset=UTF-8",

                // for additional response I and II
                 "additionalResponses" : [{
                    "contentType" : "application/json",
                    "schemaPointer" : "$/properties/foo/additionalSchemas[0]"
                },
                {
                    "contentType" : "application/json",
                    "schemaPointer" : "$/properties/foo/additionalSchemas[1]"
                }
                ]
            }
        ]
    }

[egekorkan]

I like the above proposal. Just that I don't think we need to use anyOf in additionalSchemas, the value of additionalSchemas can simply be an array or an object. In the object case, we can predefine some possible values.

Also while we are at it, we can solve this issue (I cannot find the exact one) where a platform or protocol can contain predefined schema constructs that are reproduced in every interaction. Since that needed use of JSON Pointers.

[sebastiankb]

From today's TD call:

• we discussed this example
• the $ have to be removed since this is related to JSON Schema
• the pointer approach should be similar as applied for security
• the additionalSchemas term should be introduced at the interaction level
• improved version can be that the additionalSchema is a map:

    "properties": {
        "foo": {
            "type": "string",                           // for default response
            "additionalSchemas": {
                "additionalRes_I":  {"type": "string"},   // for additional response I
                "additionalRes_II"  {"type": "number"}    // for additional response II
            }
        },
        "forms": [
            {

                "href": "https://mylamp.example.com/status",
                // for default response
                "contentType" : "text/plain;charset=UTF-8",

                // for additional response I and II
                 "additionalResponses" : [{
                    "contentType" : "application/json",
                    "schema" : "additionalRes_I"
                },
                {
                    "contentType" : "application/json",
                    "schema" : "additionalRes_II"
                }
                ]
            }
        ]
    }

• @mmccool will provide a PR and update the current approach as described in Section AdditionalExpectedResponse

[mmccool]

So I am blocked on making a PR for this. I wanted to cut-and-paste the mechanism used for securityDefinitions BUT somehow that got broken! When it has been fixed I can proceed.

Again, I want to introduce a Map, perhaps "schemaDefinitions", that define some named data schemas that can be used in additional Responses.

One question in general though: is this ever useful in other places?

[benfrancis]

schemaDefinitions appears to have landed in https://github.com/w3c/wot-thing-description/commit/100f0de1d8d608e7c3c3420b1c4e5f68a5afa628. Is this feature now considered complete?

I have added an example using schemaDefinitions to describe asynchronous action requests in https://github.com/w3c/wot-thing-description/issues/302#issuecomment-884867648. This example supports the idea of separating the output schema of an action from the response schema of a Form, since the initial response to the invokeaction request doesn't include the output of the action. It therefore also suggests that the schema member might be useful for an ExpectedResponse as well as an AdditionalResponse, which isn't currently included in the specification.

[egekorkan]

@benfrancis schemaDefinitions is not currently complete but it will be very soon. I am preparing a PR for it to be used anywhere