Wot-thing-description: [vocabulary] Consider adding keyword to describe synchronous actions

[egekorkan]

Related to #302 and also to https://github.com/w3c/wot-binding-templates/issues/2

Use Case: A Thing is used by a Consumer. Consumer sends requests sequentially after their response to invoke multiple actions. A more tangible version, Consumer sends an HTTP POST request to invoke an action, once the Thing replies with the response, the Consumer sends another one

Problem: Currently, it is not possible to describe whether an action is synchronous or asynchronous, i.e. whether the Thing responds once the action is finished or once it is accepted and started by the Thing.

Proposal: I would suggest adding a new keyword for Action Affordance called something like synchronous or sync that has a boolean value. Since claiming that the action is synchronous is "more assertive", the default value of this keyword should be false, similar to the keywords safe and idempotent.

Examples: I have two devices that already exhibit this behavior:

• TD of a pan tilt hat: This is a platform that can hold a camera and can be rotated. An action starts the rotation and the response is received once the movement is started.
• TD of a robot arm: The robot arm receives an action request, goes and does the action (which can take a couple of seconds) and then responds. This one already has this keyword in its Action Affordances since we wanted to remember the behavior for ourselves.
  Edit: see the illustration in the comment below.

It is important to note that the reason they are programmed like is due to the underlying library that is provided by the manufacturers.

Alternatives: I have looked at SSN vocabulary but couldn't find a corresponding term.

Note: This will have some dependence on the protocol. For example, an MQTT based Thing cannot have synchronous actions unless it implements the response topic feature introduced in MQTT V5.0. However, we already have such protocol dependence sometimes, like readmultipleproperties which shouldn't be done with an HTTP GET request; observable properties and events with HTTP without subprotocol.

[sebastiankb]

Based on today's TD call @egekorkan will reformulate this issue.

[sebastiankb]

@ashimura mentions to check the SMIL 3.0 standard

[egekorkan]

I had a look at the SMIL standard but my issue is way simpler than what is trying to be solved. It would be incredibly useful in the future when we discuss heavier eventing based communication and want to guarantee timing behavior on both sides.

[egekorkan]

Here is an illustration of the issue I have mentioned above

[benfrancis]

I'm not sure how much value an annotation to say whether an action is synchronous or asynchronous really adds on its own. As described in #302, what the client really needs to know is:

1. How to invoke an action
2. How to query the status of an action once invoked, or find out whether it succeeded or failed
   and optionally:
3. How to cancel an action after being invoked

Telling the client that an action is synchronous might give it a hint that in the case of an action invoked via an HTTP POST request any success/failure state may be expected in the HTTP response, but it doesn't specify what status codes or payload to expect in either case.

Telling the client that an action is asynchronous might give it a hint that in the case of an action invoked via an HTTP POST request the result can not be expected in the HTTP response, but it doesn't tell the client how it should query the status of or cancel the request at a later point (e.g. through a GET on a newly created resource which represents that particular action request).

In another protocol (e.g. a WebSocket sub-protocol) there may be no built-in request/response mechanism which connects a particular outgoing message with a particular incoming message, so a synchronous action doesn't really make any sense. The client does still need to know how to invoke an action, how to query its state and how to cancel it though.

The more we dig into these complexities the more I think that protocol details like these should be defined in an external protocol specification rather than trying to describe any possible protocol declaratively in the Thing Description. These protocol details could be described in an expanded set of defaults in section 8.3.1, default profile or sub-protocol specification.

[egekorkan]

I totally agree with the fact that it is only a hint. If such a hint is not there, how a Consumer can know how to build its application without first interacting and checking out how the Thing works? It is similar to safe and idempotent, which are also hints.

I also agree that some protocols may not provide a way to do it but neither can HTTP do eventing natively but do it via long polling. For the case of MQTT, it is a recent change that you can do request-response but before that (and still) we were using it to do readproperty, which is, implicitly, a synchronous interaction.

I also think that we should not think how the protocols effect the application logic. For me, this has nothing to do with the protocol in the first place. Once one starts thinking how to concretely implement it, the protocols will come into question.

Also for the hypermedia case in #302, this would still be a valid hint, whether you should query the state of the action while waiting for the response or after you get the response. So for the synchronous case, the Thing can reject the invoke request or have an internal error while executing it (like lifting a heavy object inducing overcurrent protection) and the Consumer knows that it should query the action before getting the response in the very first place. In the asynchronous case, the Consumer knows that until it gets the response, it doesn't need to query anything but after that it should immediately start querying because it doesn't know when the action will finish.

[egekorkan]

For the asynchronous case illustrated above, to have the same behavior as the synchronous case, the script would need to change to something like:

await robot.invokeaction("moveDown"); // resolves quickly
// the function below can be change to observerproperty or a polling based
// readaction hypermedia operation
robot.subscribeEvent("position", (x) => { // start getting position updates
  console.info("current position:", x);
  if ( x == 0 ){ // 0 being where the block to grip is
    await robot.invokeaction("closeGrip"); // resolves quickly
  }
});

[egekorkan]

Is there any development on this being included in the TD vocabulary? I have existing implementations, examples and enough explanation. I have also edited the asynchronous case's illustration and showed that it cannot grab the cube

[sebastiankb]

Lets make a decision in today's TD session.

[sebastiankb]

@egekorkan for today's discussion it would be good what kind of term you like to introduce and where + what is its default assumption

[egekorkan]

The answers would be:

• Where: Action affordance level, same as safe and idempotent
• Exact term: I have 4 similar proposals: synchronous, sync, asynchronous, async. Probably a short version is better. I find sync better, reason below.
• Default behavior: If we choose sync we would give an assurance to the Consumer that action has physically happened before returning the response. It would be similar to saying idempotent, i.e. giving the Consumer the assurance that no matter how many times you send the input, I will return the same output. Thus, with sync, the default behavior would be false and would be in line with safe and idempotent. If we choose async, then the default behavior would be true.

[sebastiankb]

From today's TD call:

• it seems useful to have such a hint in the forms container, it will not break any TD 1.0 rules
• sync can be used as term
• there is a default value with the value false
• @egekorkan and @mlagally can have a short exchange and will provide a PR

[ashimura]

I think what the diagrams above shows is not the need for synchronous Action invocation but what kind of granularity level of Actions would be required.

For example, the Robot Arm Thing should simply have a "grab" Action instead of the combination of atomic Actions of "move-down" and "close-grip". Or maybe the Robot Arm Thing should have an additional sensor to detect the distance between the arm and the target object, and should stop its movement when it reaches the object before invoking the "close-grip" Action.

Even if we really have to use the combination of the smaller atomic Actions, we still can expect some additional communication mechanisms between the Consumer (=client) and the Robot Arm Thing (=server), e.g., polling or callback, so that the Robot Arm can detect the right timing to grab the target object based on the response of the Robot Arm.

In addition, technically, synchronous communications can be implemented even with asynchronous protocols, and also asynchronous communications can be implemented even with synchronous protocols.

So I still don't understand the actual need for the additional "sync" keyword for the Action affordance. Maybe it would be needed when we want to use multi-processing to orchestrate multiple independent devices, e.g., the Robot Arm A handing the target object to another Robot Arm B. However, in that case we would rather need an additional mechanism to synchronize those two robot arms.

That's why I really want to know about the possible industry use cases a bit more in detail to clarify the actual need for this additional "sync" keyword.

Please note that here I assume the terms of "synchronous", "asynchronous", "polling" and "callback" based on the definition in the Siemens Termcenter Gateway.

[egekorkan]

Even if we really have to use the combination of the smaller atomic Actions, we still can expect some additional communication mechanisms between the Consumer (=client) and the Robot Arm Thing (=server), e.g., polling or callback, so that the Robot Arm can detect the right timing to grab the target object based on the response of the Robot Arm.

I agree with this. However, this would mean something going into the discussion in #899 which ended being a dead-end for now. We seem to not have the resources as the WG to tackle this.

For example, the Robot Arm Thing should simply have a "grab" Action instead of the combination of atomic Actions of "move-down" and "close-grip". Or maybe the Robot Arm Thing should have an additional sensor to detect the distance between the arm and the target object, and should stop its movement when it reaches the object before invoking the "close-grip" Action.

This is a design choice. If a grab action is programmed in the Thing, then that Thing would be constrained to that application scenario whereas making it more granular would make it possible to use it in different applications. We cannot advise Things to be built one way or another.

So I still don't understand the actual need for the additional "sync" keyword for the Action affordance.

In this case, assuming we have the following action affordances snippet of a Thing that has platform with a camera attached that can also take photos:

"panTo": {
  "description": "Moves the pan platform to the angle specific in the input",
  "input": {
    "unit": "degrees",
    "type": "number",
    "minimum": -90,
    "maximum": 90
  },
  "forms":[...]
},
"takePhoto":{
  "forms":[...]
}

A human developer wants to turn the camera to a certain angle and take a picture. How can he/she/they program one of the following:

1.

// alternative when action is sync
await camera.invokeaction("panTo",45);
await camera.invokeAction("takePhoto");

2.

//alternative when action is async
await camera.invokeaction("panTo",45);
sleep(5000); //sleeps 5 seconds to make sure it is there
await camera.invokeAction("takePhoto");

3.

//alternative when there is an observable property that tells the pan position 
await camera.invokeaction("panTo",45);
camera.observeProperty("panPosition", (x) => { // start getting position updates
  console.info("current position:", x);
  if ( x == 45 ){
    await camera.invokeAction("takePhoto");
  }
});

What @ashimura is saying that unless there is a way to ensure that the 3rd alternative can exist, we shouldn't add a new keyword. I beg to differ.

[sebastiankb]

what is the status here?

[egekorkan]

@sebastiankb I honestly do not remember what was to be discussed with @mlagally ? @mlagally do you? :)

[sebastiankb]

see here #213

[mlagally]

@egekorkan let's discuss and conclude this topic in the profile call.

[ashimura]

From my viewpoint, the core problem here is rather:

1. The gap between (1) the expected granularity of the atomic Actions on the Consumer's side and (2) the actual granularity of the atomic Actions exposed by the Thing

and

1. Which side would fill the gap, (a) the Consumer's calling a set of Actions as an expected granularity of Acton or (b) the Thing's exposing a set of Actions directly

For example, in the use case of a robot arm above, the robot arm as a Thing exposes very low-level of atomic Actions of go down and grab without synchronization while the application expects a higher-level of atomic Actions like pick up the target which consists of going down and grabbing in synchronization.

So I've been repeatedly suggesting that we should collect more information about the gaps between the expectation on the Thing side and the expectation on the Consumer side based on concrete use cases from industries rather than simply adding an additional flag on sync/async to TD Actions.

For exaample, Matsuda-san's slides for March vF2F (P13) mentioned ECHONET Web API's capability of grouping multiple devices as an integrated device, and it would make sense to have a sync/async flag to handle that kind of meta-device including multiple devices (which handles multiple Actions in sequence).

[egekorkan]

@mlagally wrote:

@egekorkan let's discuss and conclude this topic in the profile call.

I can join a profile call but I need to know in advance which one since I generally have a collision on that spot. However, if we want to have a discussion on what word to add, that should be in a TD call. In the profile call (and then in the spec) we can however say that all actions of the X profile MUST use synchronous actions.

[egekorkan]

So I've been repeatedly suggesting that we should collect more information about the gaps between the expectation on the Thing side and the expectation on the Consumer side based on concrete use cases from industries rather than simply adding an additional flag on sync/async to TD Actions.

So the example I have above is from our lab that is teaching students WoT since 3 semesters. I would thus consider it a use case, unless when you say an industry, you mean some company's products needing this information.

[ashimura]

So I've been repeatedly suggesting that we should collect more information about the gaps between the expectation on the Thing side and the expectation on the Consumer side based on concrete use cases from industries rather than simply adding an additional flag on sync/async to TD Actions.

So the example I have above is from our lab that is teaching students WoT since 3 semesters. I would thus consider it a use case, unless when you say an industry, you mean some company's products needing this information.

I'm not saying your use case is not a use case :)

However, for actual deployment of WoT for IoT industries, we should work with the liaison SDOs and get implementations from the IoT vendors as well.

[k-toumura]

I think the question is not whether synchronous or asynchronous style is more common in the industry.
Both style exist and we need to describe these style properly in TD.

For example, for reading a file in JavaScript, there are two APIs: fs.readFile() (asynchronous) and fs.readFileSync() (synchronous). Both of which are equally useful, and information about whether the API is synchronous or asynchronous is also useful to the API user.

[ashimura]

I think the question is not whether synchronous or asynchronous style is more common in the industry.
Both style exist and we need to describe these style properly in TD.

For example, for reading a file in JavaScript, there are two APIs: fs.readFile() (asynchronous) and fs.readFileSync() (synchronous). Both of which are equally useful, and information about whether the API is synchronous or asynchronous is also useful to the API user.

Exactly, I believe we should document how to use TD and Binding Template for actual IoT service implementations. I personally think it would be better to have a separate WoT Implementation Guideline but the description could be part of the TD spec and the Binding Template Note, etc.

[mlagally]

I was reviewing the Echonet slides, but they are not having these details on the actions. Is there a public documentation where we can find details?

[ashimura]

I was reviewing the Echonet slides, but they are not having these details on the actions. Is there a public documentation where we can find details?

There is information about their specs on their Web site but I think we should continue to talk about the detail on the potential gaps collaboratively with them via our liaison :)

[k-toumura]

Exactly, I believe we should document how to use TD and Binding Template for actual IoT service implementations. I personally think it would be better to have a separate WoT Implementation Guideline but the description could be part of the TD spec and the Binding Template Note, etc.

Yes. We can define sync/async in a table of 5.3.1.4 ActionAffordance in TD spec, then write a implementation guideline which explains how to write a TD for a robot (or anything) which has synchronous (or asynchronous) Web API.

[mlagally]

I look at the action semantics from the Profile perspective, i.e. with the objective of defining a mechanism that uses the TD (as of today) and define a model on top of that. This means that we have to describe the response payload and how to interpret it.
In the sync case it is returning the action response in the output data schema, in the async case the output schema contains a processing status and "a way" to query the status of the action.
Note that I'm not talking about notification mechanisms for now.

[mlagally]

@ashimura
Can you please ask over the liaison channel, whether they can help us with a description of the gaps (in English)?
Unfortunately the Echonet specification is in Japanese.

[egekorkan]

Removing the PR Needed label for now. More discussion needed after the profile spec is in a feature freeze regarding this discussion