Sdl_evolution: [Accepted with Revisions] SDL 0173 Revisions - Read-Generic-Network-Signal-data

[theresalech]

Hello SDL community,

The current review of "Revision 0173-Read-Generic-Network-Signal-data.md" begins now and runs through July 23, 2019. The last review of these revisions took place June 19 - July 9, 2019.

This is a review of proposed revisions to a previously accepted but not yet implemented proposal, SDL 0173.

The pull request outlining the revisions under review is available here:

https://github.com/smartdevicelink/sdl_evolution/pull/764

Reviews are an important part of the SDL evolution process. All reviews should be sent to the associated Github issue at:

https://github.com/smartdevicelink/sdl_evolution/issues/768

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of SDL. When writing your review, here are some questions you might want to answer in your review:

• Is the problem being addressed significant enough to warrant a change to SDL?
• Does this proposal fit well with the feel and direction of SDL?
• If you have used competitors with a similar feature, how do you feel that this proposal compares to those?
• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
  Please state explicitly whether you believe that the proposal should be accepted into SDL.

More information about the SDL evolution process is available at

https://github.com/smartdevicelink/sdl_evolution/blob/master/process.md

Thank you,
Theresa Lech

Program Manager - Livio
[email protected]

[joeljfischer]

First, many of these changes seem like good changes and even needed for this proposal to work correctly. However, I have several serious concerns:

1. This proposal is scheduled for the next release. Even though it's a donation, I don't believe this has been fully thought out.

a. The SDL Server tag was not present on the initial proposal and despite the text of this change mentioning Server, the tag was not added to the proposal. In addition, I don't believe the full impact of SDL Server changes have been laid out. UI additions will be needed. VehicleDataItems will need to be stored and reconstructed. Versioning will have to be implemented, and the Server will have to have an endpoint to retrieve the correct generic signal map. @nickschwab Correct or amplify on my comments if needed.

b. Even though this is a donation, the changes here expand upon the original proposal extensively. I think the maintainer's review time for this item will increase dramatically if this is accepted. @joeygrover @JackLivio correct me if I'm wrong.

[atiwari9]

a. The SDL Server tag was not present on the initial proposal and despite the text of this change mentioning Server, the tag was not added to the proposal. In addition, I don't believe the full impact of SDL Server changes have been laid out. UI additions will be needed. VehicleDataItems will need to be stored and reconstructed. Versioning will have to be implemented, and the Server will have to have an endpoint to retrieve the correct generic signal map. @nickschwab Correct or amplify on my comments if needed.

Nick and i did discuss the SDL server side changes in last proposal extensively and Nick had noted down the changes needed for SDL Server. Of course the new changes around versioning should be evaluated in terms of how much work would that be server side. IMO, it should be small delta to original proposal. But i'd let @nickschwab comment on that.

[JackLivio]

1a. I just want to comment that implementing the versioning mentioned for the schema is almost identical to how the consumer friendly messages are implemented. If the policy server has implemented versioning for the consumer friendly messages, I think the work would be comparable.

1b. Any time versioning is introduced it adds complexity to all effected SDL components. More code, more corner cases, and yes more review time.

1. I do not think OEM_SPECIFIC should be the request type used in the system request. OEM_SPECIFiC was created for a different purpose. There should be a new request type created just for this part of the feature so that it is clear to Core/HMI that the incoming information is related to the new vehicle data mapping.

2. I think GetPolicyConfigurationData needs to be thought out more. Using a single string parameter is not enough. There should at least be an additional parameter designating what section of the policy table the value you are looking for is. Sending something like GetPolicyConfigurationData:"SomeVehicleDataParameter" is not enough for this to be a generalized RPC to have future value for other policy aspects. Also there is not mention of the HMI interface that this RPC should be included in.

3. SDL would need to provide a way for HMI to read oem_mapping_version so that HMI can decided whether to request OEM mapping file download using the endpoint mentioned in PTU for the OEM table mapping file, using SystemRequest type OEM_SPECIFIC

Can you elaborate on this?

5.

{
    "HEAD_LAMP_STATUS" : {
        "HEAD_LAMP_STATUS_LEFT" : "true",
        "HEAD_LAMP_STATUS_RIGHT" : "true",
        "HEAD_LAMP_STATUS_RIGHT" : "true"
    }
}

Response 
{
    "HEAD_LAMP_STATUS" : {
        "HEAD_LAMP_STATUS_LEFT" : true,
        "HEAD_LAMP_STATUS_RIGHT" : true,
        "HEAD_LAMP_STATUS_RIGHT" : "TWILIGHT_1"
    }
}

HEAD_LAMP_STATUS_RIGHT is in the request and response twice, and is two different types in the response. I don't think you can do that using JSON. Im assuming this is a copy paste error?

1. Regarding backwards compatibility, does your gps example support that? In the mobile api there are a lot of GPS parameters that were mandatory before 5.0 and then made non-mandatory in 5.0. I am not sure if these old vehicle data structs apply to this proposal or not but I wanted to point this out. Could you shed some light how we handle future changes in parameters becoming mandatory or non-mandatory? I know there is versioning implemented for Core & the policy server, but does this extend to Core and mobile communication?

[joeygrover]

The PR is extremely difficult to read and follow. There are a lot of different terminologies thrown around along with fragmented examples which makes it hard to read through. I would recommend returning this for revisions which would include some added flow charts of how data is sent to each of the sections so that the flow is extremely clear and there is no room for misunderstandings. I would also include all HMI and RCP messages necessary in with their full definitions including existing params. For such a large scale change with many corner cases and expectations I feel this is mandatory. In the current state, I do not feel confident I could review the code and ensure it follows the proposal.

[atiwari9]

@JackLivio

1a. I just want to comment that implementing the versioning mentioned for the schema is almost identical to how the consumer friendly messages are implemented. If the policy server has implemented versioning for the consumer friendly messages, I think the work would be comparable.

That is great!!

1b. Any time versioning is introduced it adds complexity to all effected SDL components. More code, more corner cases, and yes more review time.

Are we expecting to keep the history of files changed as well? If so, I believe we don't need to do that for OEM mapping table as OEM is responsible to maintain that. For schema file, we should already have versioning since this would be part of preload pt as well.

I do not think OEM_SPECIFIC should be the request type used in the system request. OEM_SPECIFiC was created for a different purpose. There should be a new request type created just for this part of the feature so that it is clear to Core/HMI that the incoming information is related to the new vehicle data mapping.

I believe OEM_SPECIFIC was purpose built to be used for requests/data items specific to OEM. Since OEM mapping table contains vehicle data signal definitions, we planned to use SystemRequest with _requestType_: _OEM_SPECIFIC_ and _requestSubType_: “VEHICLE_DATA_MAPPING”. HMI would already know about this since this request would be originated from HMI. SDL just needs to act as pass thru and does not care about it since SDL has nothing to parse/use from this file. So creating a new request type for this is redundant in my opinion.

I think GetPolicyConfigurationData needs to be thought out more. Using a single string parameter is not enough. There should at least be an additional parameter designating what section of the policy table the value you are looking for is. Sending something like GetPolicyConfigurationData:"SomeVehicleDataParameter" is not enough for this to be a generalized RPC to have future value for other policy aspects. Also there is not mention of the HMI interface that this RPC should be included in.

Agreed that we can possible think of more params to make it scalable to future requests, that can fetch data from a table. Well, since we do not have specific use cases at this point, we'd need to make it generic enough. I am ok to add a supplemental param which could be called, let's say _subIdentifier_ . If you have some other suggestions on name or number of params, please let me know.

Can you elaborate on this?

Well that is just summarizing the point2, quoted below:

To specify versioning and endpoint for OEM mapping table and to provide HMI a way to read OEM mapping table version

HEAD_LAMP_STATUS_RIGHT is in the request and response twice, and is two different types in the response. I don't think you can do that using JSON. Im assuming this is a copy paste error?

Yes this is typo, i had fixed that in actual proposal here but this comment in PR was left out. Updated the comment now.

Regarding backwards compatibility, does your gps example support that? In the mobile api there are a lot of GPS parameters that were mandatory before 5.0 and then made non-mandatory in 5.0. I am not sure if these old vehicle data structs apply to this proposal or not but I wanted to point this out. Could you shed some light how we handle future changes in parameters becoming mandatory or non-mandatory? I know there is versioning implemented for Core & the policy server, but does this extend to Core and mobile communication?

I believe that is covered by original proposal by following rule:

Root-Level vehicle data type items such as rpm are always mandatory:false as defined by the RPC Spec for the vehicle data RPCs. If a data item is of type Struct, that struct can have mandatory parameters but the struct itself cannot be mandatory for the root-level item.

Also the example mentioned here shows combinations of mandatory and non-mandatory params. Since all the changes in schema (from approved items in core) would be applied to all SDL versions with Generic Network Signal Data, SDL would be able to dynamically adapt itself. But in case of conflict with API, API takes priority.

Let me know if we need to talk on this further or if i missed some point.

[atiwari9]

@joeygrover

The PR is extremely difficult to read and follow. There are a lot of different terminologies thrown around along with fragmented examples which makes it hard to read through. I would recommend returning this for revisions which would include some added flow charts of how data is sent to each of the sections so that the flow is extremely clear and there is no room for misunderstandings. I would also include all HMI and RCP messages necessary in with their full definitions including existing params. For such a large scale change with many corner cases and expectations I feel this is mandatory. In the current state, I do not feel confident I could review the code and ensure it follows the proposal.

Only real change in this revision is to add versioning for schema file, OEM mapping table and new API between SDL and HMI to read OEM mapping table(or generic policy configuration data). Everything else is implementation details(core of which is already defined in approved proposal) which we discussed with LuxOft. I can provide the flow diagrams for versioning part if that is what you are referring to. Still that would be a simple flow diagram from PTU to SDL policy DB to HMI.

[joeygrover]

The issue is that other OEMs will need to understand how this feature works, so while "implementation details" can take care of some items, larger pieces need to be included in a clear, readable format. I'm talking about each flow including that type of documentation.

[atiwari9]

The issue is that other OEMs will need to understand how this feature works, so while "implementation details" can take care of some items, larger pieces need to be included in a clear, readable format. I'm talking about each flow including that type of documentation.

Ok, will provide some flow diagrams around versioning.

[joeygrover]

Each of these sections should have clear flows and definitions:

• To include versioning for Vehicle Data Schema
• To specify versioning and endpoint for OEM mapping table and to provide HMI a way to read OEM mapping table version
• To define SDL request structure for Get/Subscribe vehicle data
• To define use cases for vehicle data schema update

[nickschwab]

a. The SDL Server tag was not present on the initial proposal and despite the text of this change mentioning Server, the tag was not added to the proposal. In addition, I don't believe the full impact of SDL Server changes have been laid out. UI additions will be needed. VehicleDataItems will need to be stored and reconstructed. Versioning will have to be implemented, and the Server will have to have an endpoint to retrieve the correct generic signal map. @nickschwab Correct or amplify on my comments if needed.

Nick and i did discuss the SDL server side changes in last proposal extensively and Nick had noted down the changes needed for SDL Server. Of course the new changes around versioning should be evaluated in terms of how much work would that be server side. IMO, it should be small delta to original proposal. But i'd let @nickschwab comment on that.

@atiwari9 The addition of version checking to determine whether or not SDL Server should respond back with a vehicle_data array isn't a substantial change to the required additions to SDL Server, however, I do have some concerns/questions about the overall proposal. Forgive me if some of these are addressed in the revised proposal or in this comment thread, as this proposal is a bit difficult to follow.

1a. As @joeljfischer mentioned, this proposal needs to be updated to tag SHAID and SDL Server as affected projects. The amount of work required for both of these projects to accommodate this proposal is substantial.

1. While the proposed behavior in Use cases for vehicle data schema update does a decent job at outlining when and what VehicleData items should be over/written in Core, I think the rare frequency of which these updates would occur does not justify the more complex handling logic. To simplify this process, I believe Core should simply overwrite all of it's known VehicleData items (maybe only custom items?) if an array is present in the received PTU.
2. There are many references to an OEM mapping table and an endpoint, but it is not clear what this "endpoint" actually is. Is it an RPC? Is it an API endpoint on SDL Server? Is it an OEM-defined URL stored and returned in the module_config portion of a Policy Table? Please define it in further detail.
3. Is the intent for SDL Server to do a simple equality check on the vehicle_data_schema_version value, or is it possible for the incoming version to be more recent than what SDL Server has available which would require some form of semantic version checking?
4. Since we are considering revisions, I would like the author and participating members to reconsider whether or not it makes sense to include all VehicleData parameters in the Policy Table rather than just the custom ones.
5. General note: please reference particular pieces of SDL (such as "Core" or "Server") when discussing a change or behavior rather than making generic references to "SDL"

[JackLivio]

@atiwari9 Could you please keep numbering in responses to keep track of discussion items?

1bi.

Are we expecting to keep the history of files changed as well?

No, I dont think so.

1bii.

For schema file, we should already have versioning since this would be part of preload pt as well.

This is throwing me off. Is there a schema file separate from the policy table entries?

2.

Since OEM mapping table contains vehicle data signal definitions, we planned to use SystemRequest with requestType: OEM_SPECIFIC and requestSubType: “VEHICLE_DATA_MAPPING”.

Ok the requestSubType should probably be mentioned in the proposal. I don't remember seeing that.

1. OK i agree with using a subidentifier. We should call the parameter policyType and they should correspond to the classes/structs seen in this file

Also I suggest renaming identifier to "property"

Example would be:

Request

GetPolicyConfigurationData: {
  policyType: "ModuleConfig",
  property: "vehicle_year"
}

Response

GetPolicyConfigurationData: {
  value: "2017"
}

This might need some work because there are varying policy table types, not just strings. Also some of these fields are arrays.

1. OK I think I understand but I believe talking about a mapping table that is this secret piece of proprietary only information is causing a lot of the confusion for this proposal. Ford might have an idea on how to implement it, but I doubt other OEMs would know how to set this up? I could be wrong there.

2. Ok I see the correct changes thank you.

3. OK thank you I might need to review this piece again.

[theresalech]

The Steering Committee voted to defer this proposal, keeping it in review until our next meeting on 2019-07-09, to allow for additional time for discussion on the review issue.

[atiwari9]

@nickschwab -

1a. As @joeljfischer mentioned, this proposal needs to be updated to tag SHAID and SDL Server as affected projects. The amount of work required for both of these projects to accommodate this proposal is substantial.

We discussed a bit on this in meeting yesterday to clarify that you were referring to original proposal for "substantial changes" comment. I agreed and we have had discussions on the changes needed at SDL server side during original proposal approval process. Still if there are concerns around changes proposed in new revisions, feel free to comment here and we can discuss.

1. While the proposed behavior in Use cases for vehicle data schema update does a decent job at outlining when and what VehicleData items should be over/written in Core, I think the rare frequency of which these updates would occur does not justify the more complex handling logic. To simplify this process, I believe Core should simply overwrite all of it's known VehicleData items (maybe only custom items?) if an array is present in the received PTU.

We touched upon this as well in meeting, well i just wanted to clarify that the intention is to keep the SDL approved vehicle data items immutable as agreed upon in original proposal. We'd potentially add new SDL approved vehicle data items to the schema as well as that is the direction for Get/Subscribe/unsubscribe vehicle data going forward. And those data items should not be changed by OEM. The rules i defined in revision proposal just reflect this and in a way same as what you have mentioned. Again, to reiterate, this logic is only for core to implement. SDL server would still server the entirety of the schema

1. There are many references to an OEM mapping table and an endpoint, but it is not clear what this "endpoint" actually is. Is it an RPC? Is it an API endpoint on SDL Server? Is it an OEM-defined URL stored and returned in the module_config portion of a Policy Table? Please define it in further detail.

You are correct, _endpoint_ is just the URL to OEM mapping table. That does not need to be on SDL Server and it is up to the OEM to support that. This value would be stored in _endpoints_ table.

@nickschwab - _I do have a separate question on this though, i have defined the key(_service_) for this to be String(_oem_mapping_file_), but as per HMI_API.xml, the data type is Integer as seen here. But preload table already has some string entries defined. So do i need to change it to integer or is String fine as well?_

1. Is the intent for SDL Server to do a simple equality check on the vehicle_data_schema_version value, or is it possible for the incoming version to be more recent than what SDL Server has available which would require some form of semantic version checking?

Since only SDL server has the latest copy of schema, it is certain that vehicle_data_schema_version received from sdl_snapshot would be less than or equal to that. So SDL server just needs to perform equality check and server the schema in case versions mismatch.

1. Since we are considering revisions, I would like the author and participating members to reconsider whether or not it makes sense to include all VehicleData parameters in the Policy Table rather than just the custom ones.

We had discussed this in past, and we came up with the hybrid approach i.e. to keep the existing vehicle data in API as it is and to use schema approach for newer vehicle data items(both OEM and standardized ones). Just that for new standardized items, we'd still need to add the required enums to define the proper return types.

[atiwari9]

1a. As @joeljfischer mentioned, this proposal needs to be updated to tag SHAID and SDL Server as affected projects. The amount of work required for both of these projects to accommodate this proposal is substantial.

We discussed a bit on this in meeting yesterday to clarify that you were referring to original proposal for "substantial changes" comment. I agreed and we have had discussions on the changes needed at SDL server side during original proposal approval process. Still if there are concerns around changes proposed in new revisions, feel free to comment here and we can discuss.

@theresalech - We have discussed the SDL server side changes in original proposal itself so can you please add the correct tags as suggested by @joeljfischer ?

[theresalech]

@atiwari9 I've updated the original proposal file to include SDL Server as an impacted platform. It looks like the original review issue already had a label for sdl policy server Please let me know if anything else is needed for this request!

[atiwari9]

@JackLivio

1bii.

For schema file, we should already have versioning since this would be part of preload pt as well.

This is throwing me off. Is there a schema file separate from the policy table entries?

Well, so i guess by policy table entries you mean the entries which govern permissions for RPCs. That will be there, depending on combinations of functional groups and app permissions, the same way it is managed for rest of vehicle data items currently.

There is a new _vehicle_data_ (_vehicle_data_items_) field added to preload table and we call that schema. That schema would be part of policy table update if it is needed to be served. For local copy of preload table, this field would always be present and updated when we add new standardized vehicle data items per SDLC approval process.

2.

Ok the requestSubType should probably be mentioned in the proposal. I don't remember seeing that.

Will do.

(#ToUpdate) - i am just adding a tag in comments so that i can come back and update everything in one go.

• _requestSubType_

3.

OK i agree with using a subidentifier. We should call the parameter policyType and they should correspond to the classes/structs seen in this file
Also I suggest renaming identifier to "property"
Example would be:
Request
GetPolicyConfigurationData: {
policyType: "ModuleConfig",
property: "vehicle_year"
}
Response
GetPolicyConfigurationData: {
value: "2017"
}
This might need some work because there are varying policy table types, not just strings. Also some of these fields are arrays.

Ok, this sounds good. I guess we can keep the return type as String array then and HMI can cast the value back. (#ToUpdate)

Something like:

<function name="GetPolicyConfigurationData" messagetype="request" scope="internal">
    <description>Request from HMI to SDL to get policy configuration data (i.e. OEM mapping table file version etc.) from Policy Table.</description>
    <param name="policyType" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the Struct from where configuration data to be received according to Policy Table types i.e. module_config etc.</description>
    </param>  
    <param name="property" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the variable for which the configuration data to be received according to Policy Table types i.e. vehicle_year etc.</description>
    </param>  
</function>
<function name="GetPolicyConfigurationData" messagetype="response">
    <param name="value" type="String" array="true" maxlength="1000" minsize="1" maxsize="100" mandatory="false">
        <description>Value of requested property from polocyType in PT. If no value is found in PT for specified policyType and property, this parameter will be omitted.</description>
    </param>
</function>

4.

OK I think I understand but I believe talking about a mapping table that is this secret piece of proprietary only information is causing a lot of the confusion for this proposal. Ford might have an idea on how to implement it, but I doubt other OEMs would know how to set this up? I could be wrong there.

Yeah i see what you mean. This piece is dependent on OEMs and has always been since only OEM are aware of vehicle signal data source. Today this information is hardcoded in HMI SW. So OEMs can either choose to do that(or just use separate update process if they like). Or just use the mechanism we provide to download the OEM mapping table reference file from OEM's servers. I am going to provide a flow diagram as well for this to provide more clarity in my next comment to Joey.

EDIT: Referring the flow diagram from next comment here:

[atiwari9]

@joeygrover -

To include versioning for Vehicle Data Schema

Here is flow for vehicle_data_schema_version update. This flow also touches on oem_mapping_version
field update as well. Details can be found here

To specify versioning and endpoint for OEM mapping table and to provide HMI a way to read OEM mapping table version

Here is the flow diagram for OEM mapping table update from this section.

To define SDL request structure for Get/Subscribe vehicle data

Here is the flow diagram in relation to this section.

To define use cases for vehicle data schema update

I guess your question on this might be related to terms used:

• _local schema vehicle data items_ - Are the vehicle data items defined in local copy of schema. It could be from preload table(supplied with SDL core release) or from previous schema update along with PTU.
• _Standard vehicle data items_ - These are vehicle data items which are either defined in HMI_API.xml, or are provided in preload schema with SDL core release. Since these vehicle data items are approved vehicle data items as per SDLC, these are immutable to any OEM side changes i.e. schema update over PTU.

Hope this clarified your questions and addressed the concerns, if you still have some question, feel free to ask in this thread.

[JackLivio]

1bii. Thank you for the clarification.

1. 👍

3.

Do you think we should include optional value parameters for each primitive type?

<function name="GetPolicyConfigurationData" messagetype="response">
    <param name="stringValue" type="String" array="true" maxlength="1000" minsize="1" maxsize="100" mandatory="false"/>
    <param name="boolValue" type="Boolean" array="true" maxlength="1000" minsize="1" maxsize="100" mandatory="false"/>
    <param name="intValue" type="Integer" array="true" maxlength="1000" minsize="1" maxsize="100" mandatory="false"/>
    <param name="jsonValue" type="String" maxlength="1000"  mandatory="false"/>
</function>

I included an additional param "jsonValue" that would be a string type but could return the full JSON equivalent of a requested parameter that is a struct and not a primitive data type. If we use this, we should probably restrict the possible values that can be returned as a JSON value to prevent Core from sending the HMI the entire policy table.

1. Regarding your flow chart, you have written that mobile sends the mapping file as a putfile? Did you mean for this to say SystemRequest? Mobile to Core SystemRequest has the ability to transfer the data received from the server endpoint, putfile is not needed.

[atiwari9]

@JackLivio

3.

I included an additional param "jsonValue" that would be a string type but could return the full JSON equivalent of a requested parameter that is a struct and not a primitive data type. If we use this, we should probably restrict the possible values that can be returned as a JSON value to prevent Core from sending the HMI the entire policy table.

_jsonValue_ is a good idea i believe but as you stated it might need additional processing on core side to determine what structure to send and/or how to restrict what can be sent. Since core organizes data in DB itself it might need to process the data back to JSON for this as well. I need to confirm this though.

Don't you think it should be HMI's responsibility to determine the data type and then cast it as per what is expected. e.g. HMI should know that return type of URL should be String, steal_focus should be boolean and exchange_after_x_days is integer. Even if all this data is return as String, HMI should be able to cast it back. This saves logic on SDL side as well to populate different params in response based on data type.

1. 
   >Regarding your flow chart, you have written that mobile sends the mapping file as a putfile? Did you mean for this to say SystemRequest? Mobile to Core SystemRequest has the ability to transfer the data received from the server endpoint, putfile is not needed.

Yes, i stated two options/ways app can send the data. It could be sent as _bulkData_ as part of _SystemRquest_ or just a _PutFile_ itself. This provides some flexibility to handle large files.

[JackLivio]

1. I can't find the mention of putfile anywhere in the proposal until this diagram. How would core know the app sending the putfile means to update the oem mapping table as opposed to a seperate app specific file? The diagram also does not specify SDL could use the system request or a putfile to fulfill the request, it makes it seem like the putfile needs to happen happen after the system request.

[atiwari9]

@JackLivio

4.

I can't find the mention of putfile anywhere in the proposal until this diagram. How would core know the app sending the putfile means to update the oem mapping table as opposed to a seperate app specific file? The diagram also does not specify SDL could use the system request or a putfile to fulfill the request, it makes it seem like the putfile needs to happen happen after the system request.

App would send the file(if need be due to size) as _systemFile_ . Since OEM mapping file is supposed to be processed by HMI only, core is not affected by this file at all.

If HMI did send _PutFile_ (as _systemFile_) instead of bulkdata , then _SystemRequest_ would have fileName as well which HMI would use to load the file from _systemFile_directory. I have updated the diagram to make it more legible.

In nutshell, both _SystemRequest_ and _PutFile_ type _SystemFile_ save to same system directory so for HMI point of view, both methods achieve same results and core is just acting as a pass thru for both.

[JackLivio]

1. OK using putfile for large files does have advantages, but SystemRequest technically does not have a limitation to the fileSize that it can send to core. I just want to point out that using putfile is a completely new idea not mentioned in the proposal and this should be included as a revision to the proposal if it is included.

If we proceed with using Putfile for sending the mapping table there should be 1 defined flow, not two optional ways of sending the OEM mapping table to Core. Can you please update the flow chart to show Putfile being sent by mobile to core after the onSystemRequest, but before the SystemRequest? It must also be noted that the putfile MUST complete successfully before mobile notifies Core about the contents of the file upload via SystemRequest. The SystemRequest will only include the fileName, requestType, and requestSubType relevant to the mapping table putfile, no binary data related to the mapping table will be included in the systemRequest.

[JackLivio]

1. Sorry Didn't address this point right away.

Don't you think it should be HMI's responsibility to determine the data type and then cast it as per what is expected.

Yes i think this is reasonable actually. Core and HMI should always be compatible versions so my previous concerns on breaking changes to policies are not valid. I think your original proposition is ok.

<function name="GetPolicyConfigurationData" messagetype="request" scope="internal">
    <description>Request from HMI to SDL to get policy configuration data (i.e. OEM mapping table file version etc.) from Policy Table.</description>
    <param name="policyType" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the Struct from where configuration data to be received according to Policy Table types i.e. module_config etc.</description>
    </param>  
    <param name="property" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the variable for which the configuration data to be received according to Policy Table types i.e. vehicle_year etc.</description>
    </param>  
</function>
<function name="GetPolicyConfigurationData" messagetype="response">
    <param name="value" type="String" array="true" maxlength="1000" minsize="1" maxsize="100" mandatory="false">
        <description>Value of requested property from polocyType in PT. If no value is found in PT for specified policyType and property, this parameter will be omitted.</description>
    </param>
</function>

[atiwari9]

@JackLivio

1. 
   >Yes i think this is reasonable actually. Core and HMI should always be compatible versions so my previous concerns on breaking changes to policies are not valid. I think your original proposition is ok.

OK

4.

OK using putfile for large files does have advantages, but SystemRequest technically does not have a limitation to the fileSize that it can send to core. I just want to point out that using putfile is a completely new idea not mentioned in the proposal and this should be included as a revision to the proposal if it is included.

Since SystemRequest(BulkData) and PutFule(SystemFile) save to same system directory, the mechanism for HMI would be same in either case. Also core would not need any additional changes for both mechanisms as functionality exists today for both modes. But still, if there is consensus that _PutFile_ would add additional complexity for this proposal(which IMO is existing functionality and won't need any core changes), i am willing to just specify _SytemRequest_ flow and rescind _PutFile_ mechanism. Please confirm.

[joeygrover]

4. It doesn't just matter that "core wouldn't need any updates" the mobile libraries have to adhere to a specification that is known. By adding items that "could work this way" or "could work that way" we introduce additional logic into the libraries that must be maintained and tested for their lifespan. So choices like that can't be arbitrary or vague. They require clear definitions of when certain logic must take place.

However, based on this comment OEM mapping table version would be included in PTU,, the mapping updates would come through a policy table JSON file. Since these can be encrypted, the mobile library can't split these items apart. Therefore I do not think PutFile can be used in this scenario and should have any reference removed. The standard PTU flow should be used.

[atiwari9]

@joeygrover

1. Ok fair point, i have updated the flow diagram:
   

   ToUpdate

CC: @JackLivio

[nickschwab]

@atiwari9 -

1a. Seems that comments regarding SHAID being an affected platform are being overlooked. SDL Server receives information about supported vehicleData properties from SHAID. Please make sure that this proposal lists both SHAID and SDL Server as affected platforms.

8. My note here is about how Core interprets the vehicle data schema changes, not how they're sent in a PTU response. If @JackLivio has no concerns about the complexity of the proposed method of checking each item rather than doing a simple truncate-and-insert, then this item number can be ignored.

9. Can you please provide a clear JSON example of the proposed changes to the endpoints struct in the Policy Table?

13. Should the OEM mapping file be uploadable via the SDL Server UI, or should OEMs be responsible for hosting it elsewhere then entering its fully-qualified URL on their SDL Server? Allowing file uploads would require additional work to support writing to one or more centralized file storage solutions, such as Amazon S3, which may be problematic for OEMs which do not already use AWS in production.

14. How will OEMs update their oem_mapping_version number on SDL Server? A change to the UI? By changing an environment variable? Please describe (and illustrate, if appropriate) in detail.

15. The proposed keys inside the vehicle_data struct have redundant names. Please avoid prefixing key names with the key name of their parent.

15a. vehicle_data_schema_version should be changed to schema_version

15b. vehicle_data_items should be changed to schema_items

[JackLivio]

1. I would prefer to use a "truncate-and-insert" / full replacement of the schema data. It makes the implementation of core a lot easier instead of having 4 extra cases to look out for when parsing the update.

If the concern is around the future of making nonstandard vehicle items to be standard items, Cores GetVehicleData request should be checking if the param exists in the mobile api xmls before referencing the vehicle data schema mapping anyways since the mobile/hmi apis are the single point of truth of what is a "standard" vehicle data item? This would ensure existing standard vehicle data parameters overrule any entries in the vehicle data schema updates.

[joeygrover]

16. I think the proposal could benifit from some clarifying edits as some of the terminology used doesn't follow the standard for aspects of SDL. For instance:

Instead of validating vehicle data items against APIs XML, SDL would rely on a dedicated vehicle data items schema as below:

I _believe_ the author means:

Instead of validating vehicle data items against __the RPC Spec__, SDL would rely on a __new dedicated and updatable spec that would contain vehicle data items in a__ schema as below:

This can lead to the proposal being difficult to read while trying to translate the author's thoughts into the respective SDL concepts. I realized that some of these were present in the original proposal, however, the proposed edits have exasperated the problem. Any reference should be made clear to the RPC spec, the HMI interface between Core and HMI, policy server JSON additions, which mapping applies where. The proposal could start with a few definitions of the pieces that will be talked about, for example:

• OEM Network Mapping: This file referse to key value pairs that the OEM module will use to retrieve vehicle data off its network
• Vehicle Data Schema: This is a JSON object that contains vehicle data defintions for both requests and responses that can be sent using

Then use those given names throughout the proposal so it can easily be identified as to which element is being discussed rather than the high level terms (scehma, map, etc).

17. So rereading the proposal I'm having difficulting remembering what this means:

A base JSON schema file for SDL core and proxy will be kept locally and an additional one in the OEM cloud

Does this just mean for proxy we will start including a JSON file for vehicle data that is already defined in the RPC spec? Not sure that is necessary, but again maybe this is just a piece that needs to be clarified.

18. Versioning for Vehicle Data Schema: There should be a note here that the current policy server is setup in a way that the same policy table items will be supplied for every request. Is it acceptable that a single vehicle data schema be used for all requests? Or does the author intend for the policy server to dynamically pick the schema based on other variables (model/year/etc)? If the latter, this would require a major reworking of how the policy server operates.

19. This section should have the included min and max definitions included into the mapping.

20. Just looking over the method names again setGenericNetworkData and setCustomDataType, does it make sense to have these names better aligned for mobile? Generic network data doesn't really make much sense to a developer. It would probably be more intuitive to say Custom vehicle data for all the get/set methods, but would need confirmation from other members.

[atiwari9]

@nickschwab
1a.

Seems that comments regarding SHAID being an affected platform are being overlooked. SDL Server receives information about supported vehicleData properties from SHAID. Please make sure that this proposal lists both SHAID and SDL Server as affected platforms.

@theresalech - Please take note

1. 
   >Can you please provide a clear JSON example of the proposed changes to the endpoints struct in the Policy Table?

here is an example, _GetURLS_ request passes Integer as per HMI_API but in below examples we can see Strings as well.

"endpoints": {
                "0x07": {
                    "default": ["http://x.x.x.x:3000/api/1/policies/proprietary"]
                },
                "0x04": {
                    "default": ["http://x.x.x.x:3000/api/1/softwareUpdate"]
                },
                "queryAppsUrl": {
                    "default": ["http://sdl.shaid.server"]
                },
                "lock_screen_icon_url": {
                    "default": ["http://i.imgur.com/TgkvOIZ.png"]
                },
               "oem_mapping_file": {
                    "default": ["<http://oemMappingUrl.com>"]
                }

13

Should the OEM mapping file be uploadable via the SDL Server UI, or should OEMs be responsible for hosting it elsewhere then entering its fully-qualified URL on their SDL Server? Allowing file uploads would require additional work to support writing to one or more centralized file storage solutions, such as Amazon S3, which may be problematic for OEMs which do not already use AWS in production.

It is OEM's responsibility to host the OEM mapping table file. SDL Server should just provide a way to input version number by OEM, which needs to be included in PTU.

14

How will OEMs update their oem_mapping_version number on SDL Server? A change to the UI? By changing an environment variable? Please describe (and illustrate, if appropriate) in detail.

There should be a UI field to update _OEM mapping table_ version number at SDL server side. I do not have any specific UI/blueprint around it. I am hoping that this could be accommodated as an additional field in policy configuration HMI. Let me know your thoughts.

15

1 The proposed keys inside the vehicle_data struct have redundant names. Please avoid prefixing key names with the key name of their parent.

Right, these are just examples. Actual data items would need to be defined by:

• SDLC for new standard vehicle data items which will be added to schema
• OEMs for OEM custom data items

15a,b - Is there specific reason to change the names? I'd say vehicle data prefix makes it distinctive, unless there are other apprehensions like length of fields etc.?

[atiwari9]

@JackLivio

8.

If the concern is around the future of making nonstandard vehicle items to be standard items, Cores GetVehicleData request should be checking if the param exists in the mobile api xmls before referencing the vehicle data schema mapping anyways since the mobile/hmi apis are the single point of truth of what is a "standard" vehicle data item? This would ensure existing standard vehicle data parameters overrule any entries in the vehicle data schema updates.

Yes, that is what i was getting at. Well as per my understanding, we are going to maintain new standard vehicle data items only in schema and not in API as per original proposal, with exception of structs needed by new standard vehicle data which would still be part of API spec.

I agree to your comment that core should just check against the list of standard vehicle data items. But we should really define where new standard data items would be maintained going forward.

[atiwari9]

@joeygrover

1. Agreed that the terms used have evolved over the implementation discussion, i'd update the proposal revision with standard terms for mapping and schema as you mentioned for wider audience (#ToUpdate).
   Hope the proposal is understandable still in current form for you at least for the things we are trying to revise as i had provided more details around that in my last comment addressed to you.

17.

Does this just mean for proxy we will start including a JSON file for vehicle data that is already defined in the RPC spec? Not sure that is necessary, but again maybe this is just a piece that needs to be clarified.

This again goes back to original proposal where we tried to maintain a logic where both core and proxy would get schema updates OTA. But due to complexity in dynamically generating proxy methods based on schema updates, OTA update for schema at proxy side was dropped leaving it only with a local copy of schema.

Now why do we still need a local copy of schema at proxy side:

• We had also discussed at some point to add new standard vehicle data items in schema itself instead of adding the new standard vehicle data items to HMI/Mobile APIs (except enums/structs for data return types).
• Local copy of schema would be updated with every proxy/core release, this would keep the base schema same for both core/proxy. OEMs then can append to schema with custom vehicle data items which SDL core would download OTA. In order to be able to request custom data items from proxy, we added following changes to proxy side.

18.

Versioning for Vehicle Data Schema: There should be a note here that the current policy server is setup in a way that the same policy table items will be supplied for every request. Is it acceptable that a single vehicle data schema be used for all requests? Or does the author intend for the policy server to dynamically pick the schema based on other variables (model/year/etc)? If the latter, this would require a major reworking of how the policy server operates.

Nope, SDL server maintains and serves only single copy of schema. Signal reference changes based on vehicle MY, powertrain etc are managed in _OEM Network Mapping_ file.

1. Agreed, but these are just examples and actual vehicle data items would need to be provided by:
2. SDLC for new standard vehicle data items which will be added to schema
3. OEMs for OEM custom data items

4. Current Vehicle data items defined in API spec need not be included in schema

5. It has been long we discussed proxy side changes/methods. But if members want to change it, i dont see any reason to not change it to setCustomVehicleData & getCustomVehicleData

[joeygrover]

17. When you say "local schema" are you referring to the RPC spec? The proxy library will always be updated with agreed upon vehicle data items that get added to the RPC spec with supporting classes, methods, etc. The proposal makes it sound like there is this other file that the libraries read from and do something with. That is what I am trying to clear up and why using the SDL specific terminology is so important.

18. Ok just wanted to make sure. So for the OEM Network Mapping document, that will be hosted by some other server/endpoint and there is no plan for an SDL supported sample of that project that would serve up all those different OEM Network Mapping documents, correct?

19. It is even more important because it is an example that it be correct.

20. Yes, we might have just been fatigued at that point of actually getting to some of the naming vs functionality. @joeljfischer thoughts here on this section

[joeljfischer]

20.

Once core has downloaded and processed the new vehicle data params, it'd send an onPermissionsChange notification to the connected app with new vehicle data params.

I assume the OnPermissionsChange requires no MOBILE_API XML changes? There's no new parameter we're adding that will contain this data, it'll all be OEM specific?

I suppose I'd advocate a better name than setGenericNetworkData like setOEMCustomVehicleData and setOEMCustomVehicleDataType

[nickschwab]

@atiwari9 -

9. Since this is a fully-qualified URL, I recommend changing the key name of the new endpoint to oem_mapping_url to comply with the existing naming convention set by lock_screen_icon_url.

13. I support requiring an OEM to host the mapping file themselves and provide a fully-qualified URL to its location, but this still requires some way to save the URL in SDL Server so that it may be returned in a PTU. This process has not yet been made clear through the proposal or the comments. Since the author's intent is that the mapping file be made as a new endpoint, it would be logical to add an input field for this URL to be defined on the "Module Config" UI tab of SDL Server. Screenshot below.

14. Since the oem_mapping_version value is directly related to the oem_mapping_url endpoint URL, it would be logical to be able to update it in the same place as the endpoint URL itself (within "Module Config"). With that in mind, I think it makes sense to relocate and rename oem_mapping_version to version within the oem_mapping_url endpoint struct, as follows:

"endpoints": {
                "0x07": {
                    "default": ["http://x.x.x.x:3000/api/1/policies/proprietary"]
                },
                "0x04": {
                    "default": ["http://x.x.x.x:3000/api/1/softwareUpdate"]
                },
                "queryAppsUrl": {
                    "default": ["http://sdl.shaid.server"]
                },
                "lock_screen_icon_url": {
                    "default": ["http://i.imgur.com/TgkvOIZ.png"]
                },
               "oem_mapping_url": {
                    "default": ["http://hmi.oem.com/vehicle_data_mapping"],
                    "version": "00.00.01"
                }
}

Note: the presence of version within any endpoint should be considered optional.

15. I think you may have misinterpreted my comment. I am highlighting the higher-level vehicle_data_schema_version and vehicle_data_items keys (not their values/children). Both of these keys are direct properties of vehicle_data, therefore it is redundant to prefix them with vehicle_data_. JSON keys should not be universally unique, but rather unique to their local scope to create easier-to-read data, establish predictable key names, and consume less bandwidth.

[joeygrover]

14. "oem_mapping_url" is a little vague, should likely be qualified a bit more to something like "oem_specific_vehicle_data_mapping_url".

[nickschwab]

14. @joeygrover that's fair. Prefixing with oem is probably overkill since Module Config is entirely custom to OEMs anyway. How about custom_vehicle_data_mapping_url?

21. If an OEM does not set custom_vehicle_data_mapping_url (name subject to agreement) in their Module Config, what should be sent back in the Policy Table in response to a PTU request? Omit custom_vehicle_data_mapping_url entirely?

[atiwari9]

@joeygrover

1. Local schema is included in preload policy table for core. This is needed in case we add new standard vehicle data items. Again i'd refer to my previous comment which needs confirmation.

I can share a sample as well if you like

1. Correct

2. Ok, i'd update the original proposal as per API examples it maps to. (#ToUpdate)

20..

I assume the OnPermissionsChange requires no MOBILE_API XML changes? There's no new parameter we're adding that will contain this data, it'll all be OEM specific?

Yes, API has params listed as Strings and yes these are OEM specific.

I suppose I'd advocate a better name than setGenericNetworkData like setOEMCustomVehicleData and setOEMCustomVehicleDataType

I suppose this is OK as well. I'd update this too. (#ToUpdate)

cc: @joeljfischer

[joeygrover]

17. I am specifically talking about the proxy side changes. Please confirm there is no additional file or json the __proxy library__ will be using. It only implements the RPC spec defined messages.

[atiwari9]

@nickschwab

1. 
   >Since this is a fully-qualified URL, I recommend changing the key name of the new endpoint to oem_mapping_url to comply with the existing naming convention set by lock_screen_icon_url.

Would HMI be able to request this given that API only supports Integer type for _GetURLs_ ? Are you expecting changes in _HMI_API_ _GetURLs_ as well?

1. Wouldn't adding Version here need changes to current _endPoint_ DB table?

Or did you mean to just consolidate the UI and JSON but save the version to _module_config_ table still as _oem_mapping_version_

Also, when i see version within endpoint, it seems to be url version. But that can be subjective too.

1. Yes i was referring to same fields, it is OK to shorten it but i'd still prefer to keep the full form vehicle_data_schema_version when this is saved to _module_config_ DB table.

[atiwari9]

@nickschwab

1. I'd say omit. Since it is OEM's responsibility to set this URL, we do not need to provide some workaround.

[theresalech]

Here's a summary of the current status of this discussion, for review during our call.

Author to Update
1. Author to ensure proposal mentions SHAID and SDL Server as affected platforms.
2. Author to add a requestSubType mention to proposal.
3. Author to update GetPolicyConfiguration RPC in the proposal.
4. Author to update proposal with SystemRequest flow chart and description of the process.
8. Author to update proposal to simplify Core behavior when a vehicle data schema item update occurs (truncate-and-insert rather than search-delete-and-insert).
9. Author to include Policy Table endpoints sample JSON in proposal to clarify endpoint changes and use recommended naming.
10. Author to update proposal to clarify that vehicle data schema version checking is a simple equality check (not semantic versioning).
12. See 16.
13. Author to update proposal to indicate an additional field be added to the “Module Config” tab “Endpoints” section of SDL Server to allow an OEM to set custom_vehicle_data_mapping_url.
15. Author to update proposal to remove redundant Policy Table key names/prefixes.
16. Author to update proposal to use SDL specific and consistent, defined terminology throughout proposal. This will include the addition of flow charts supplied in the comments.
17. Author should revise proposal to state proxy library changes will include only those represented in the RPC spec; remove reference to a “local schema” for the proxy library.
18. Author to consider including this information as scoped work into the proposal
19. Author to update example to be correct.
20. Author to updated method names to be conforming so that feature is easily traceable to all projects. Based on URL rename, suggest to go with customVehicleData.

No Action Required
5. No action.
6. No action.
7. No action.
11. No action.

Additional Input Needed
14. Awaiting agreement from members regarding naming and key location (determines how this key is visually represented in the SDL Server UI). Author to update proposal when outcome reached with the determined Policy Table key name, key location, and SDL Server UI changes.
21. Additional input needed from Core team for the expected endpoints response when a specific endpoint is not provided. Author to update proposal when outcome reached.

[atiwari9]

@joeygrover

1. 
   >I am specifically talking about the proxy side changes. Please confirm there is no additional file or json the proxy library will be using. It only implements the RPC spec defined messages.

Short answer, at this point NO since we do not have any standard vehicle data items in schema yet.

But we'd need proxy side static schema(json) as well in case there is agreement/confirmation on using schema to add new standard vehicle data items instead of API spec. I'd again refer back to my comment asking for a confirmation on this.

As per what we had discussed in original proposal, we are going to add new standard vehicle data items in schema only. But we should all agree on that since there seems to be confusion around this.

[atiwari9]

@nickschwab
We had resolution on most of the points in SDLC meeting yesterday. @theresalech would update the summary notes as needed. We still need inputs on two points below:

14

• Agreed to change oem_mapping_url to custom_vehicle_data_mapping_url for endpoint
• Agreed to change oem_mapping_version to custom_vehicle_data_mapping_version while saving to policy DB
• Whether to shorten oem_mapping_version to version would depend on if we include this with in url endpoint json. That bring us to below question:

Where should the _oem_mapping_version_ & _vehicle_data_schema_version_ be stored in policy DB. As per earlier comments, we agreed to save it as part of _module_config_ table. @LuxoftAKutsan has some ideas around keeping _oem_mapping_version_ and _vehicle_data_schema_version_ to a new table in policy DB. @LuxoftAKutsan - Please provide details.

Refer comment

15 Agreed to shorten the names to schema_version and schema_items in _JSON_, but we'd use full form version names for custom_vehicle_data_mapping_version and vehicle_data_schema_version

21 It was discussed in meeting to check server behavior when policy url is not provided. IMO, SDL server won't know what to send anyways since only OEM knows the correct URL. So SDL server shall just omit this endpoint field in PTU

[nickschwab]

@atiwari9 -

9. I'm not familiar with HMI APIs. Perhaps @JackLivio can shed some light on this.

14. Yes, it would require a new String(version) column to be added to the endpoint table. The version is directly associated with the endpoint, so it would be an architectural mistake to place it anywhere else, in my opinion. Also, I strongly advise against using different table column names in Core's database than the keys defined in a Policy Table, as doing so would create technical debt and confusion to developers.

15. The vehicle data schema_version is not part of the Policy Table module_config struct, therefore it should not be placed there in Core's SQLite database. I suggest a new table called vehicle_data to match the key name in the Policy Table (assuming that doesn't conflict with an existing table) with a column String(schema_version). See comment in 14 regarding the use of different names between Policy Table JSON and Core's SQLite database.

21. I agree.

[atiwari9]

@nickschwab

1. it was decided in SDLC meeting to use integer (0x10) as service id.

1. So that'd mean an endpoint JSON string would be returned with all endpoints for GetPolicyConfigurationData when sent with:

GetPolicyConfigurationData: {
  policyType: "ModuleConfig",
  property: "endpoints"
}

That'd have all the endpoints and HMI would need to logically parse to read version from here. This could also need possible changes in GetURLS method implementation as today this method sends only urls in response. We'd need to update method in API spec to accommodate new version field.

<function name="GetURLS" messagetype="request" scope="internal">
      <description>Sent by system to SDL to get list of URL for specified service type and optionally policy's application id.</description>
      <param name="service" type="Integer" minvalue="0" maxvalue="100" mandatory="true"/>
</function>
<function name="GetURLS" messagetype="response" scope="internal">
      <param name="urls" type="Common.ServiceInfo" array="true" mandatory="false" minsize="1" maxsize="100"/>
</function>

That is the reason is am hesitant with approach you suggested.

1. Fine with the name as long as it does not conflict or confuse anyone. But give that we'd need additional table, i recommend to add oem version to that table as well to avoid changes to GetURLs method in HMI api

[nickschwab]

@atiwari9 -

14 & 15. I understand that this would likely require a change to GetURLS, but we need to take a thoughtful and consistent approach to data schema design. The tables and columns in Core should reflect the design of the Policy Table JSON. There are many design issues with Core's SQLite schema today, but we shouldn't exacerbate the problem just because something is convenient or less work in the short-term.

[nickschwab]

9 & 14. Action item: endpoint name to be 0x10 rather than custom_vehicle_data_mapping_url.

[theresalech]

The Steering Committee voted to return this proposal for revisions, noted in the Author to Update section of this comment. The following points were also discussed on the Steering Committee call:

• 9. Will use 0x10 as endpoint
• 14. Further discussion on the review issue will be required to confirm next steps. The review issue will be left unlocked so that this conversation can take place while the author is revising the proposal in a separate PR.
• 17. Standard RPCs found in the RPC spec will need to be included in the vehicle data schema used by Core, so that if the RPC Spec is updated with a new vehicle data item, the old schema with OEM-specific vehicle data items can be updated to use the new version of the defintions of those vehicle data items as added in the RPC Spec. This will allow Core to dynamically add new vehicle data items as they become available for all apps using the accepted, standard vehicle data items. Therefore, the schema will be part of preloaded and updated schema Core uses, however the mobile side will not need this schema as the libraries will always implement the latest agreed upon vehicle data items.
• 21. Further discussion on the review issue will be required, once the Project Maintainer has confirmed Core’s response when a specific endpoint is not provided.

[atiwari9]

14 & 15. I understand that this would likely require a change to GetURLS, but we need to take a thoughtful and consistent approach to data schema design. The tables and columns in Core should reflect the design of the Policy Table JSON. There are many design issues with Core's SQLite schema today, but we shouldn't exacerbate the problem just because something is convenient or less work in the short-term.

cc: @JackLivio @joeygrover - As per @nickschwab 's comment above, How do you guys feel about update to GetURLS method as discussed in this comment , Also to be noted that GetPolicyConfigurationData also would just return all URL json structures and it'd be up to HMI to parse the correct output.

[LuxoftAKutsan]

@nickschwab @atiwari9

• 9 0x numbers usually mean Hexadecimal numbers. So 10 as an integer in API should be reflected as 0x0A in the policy table.

• 14

  ( nickschwab ) I suggest a new table called vehicle_data to match the key name in the Policy Table (assuming that doesn't conflict with an existing table) with a column String(schema_version).

That's completely matches the approach that I would like to propose.

[atiwari9]

@JackLivio @joeygrover - Can you please take a look at this comment

1. @nickschwab agrees to omit, any apprehensions on that?

[JackLivio]

14.

@atiwari9
I am ok with using GetPolicyConfigurationData to get the url information.

If we go this route I would like to further suggest we deprecate the GetUrls HMI RPC in favor of GetPolicyConfigurationData. Also this means we can name the url to something descriptive like "custom_vehicle_data_mapping_url" like we had previously instead of "0xA".

To satisfy the placement of the mapping version we could extend the module config section to have a new object called "endpoint properties".

 {
  "module_config": {
    "full_app_id_supported": true,
    "exchange_after_x_ignition_cycles": 100,
    "exchange_after_x_kilometers": 1800,
    "exchange_after_x_days": 30,
    "timeout_after_x_seconds": 60,
    "seconds_between_retries": [
      1,
      5,
      25,
      125,
      625
    ],
    "endpoints": {
      "0x07": {
        "default": [
          "http://192.168.1.143:3001/api/v1/staging/policy"
        ]
      },
      "0x04": {
        "default": [
          "http://localhost:3000/api/1/softwareUpdate"
        ]
      },
      "queryAppsUrl": {
        "default": [
          "http://localhost:3000/api/1/queryApps"
        ]
      },
      "lock_screen_icon_url": {
        "default": [
          "https://i.imgur.com/TgkvOIZ.png"
        ]
      },
      "custom_vehicle_data_mapping_url": {
        "default": [
          "http://localhost:3000/api/1/vehicleDataMap"
        ]
      }
    },
    "endpoint_properties": {
      "custom_vehicle_data_mapping_url": {
        "version": "4.5.0"
      }
    },
    "notifications_per_minute_by_priority": {
      "EMERGENCY": 60,
      "NAVIGATION": 15,
      "VOICECOM": 20,
      "COMMUNICATION": 6,
      "NORMAL": 4,
      "NONE": 0
    }
  },  // module_config
  "vehicle_data":{
        "schema_version":"00.00.01",    
        "schema_items":
        [
            {
                "name":"rpm",
                "type":"Integer",
                "key":"OEM_REF_RPM",
                "array":false,
                "mandatory":false,
                "params":[]
            }
        ]
    }
}

This would provide flexibility to add properties to future endpoints that might need them.

Note the vehicle data schema version is still contained within the vehicle_data object. We did not think it made sense to include the schema version in the module config section separate from the vehicle data object.

[atiwari9]

1. Need to update GetPolicyConfigurationData response:
2. Remove maxlength so that String size limit is not applied and it takes max allowed by system
3. In case data is not available, core should respond with _DATA_NOT_AVAILABLE_

<function name="GetPolicyConfigurationData" messagetype="request" scope="internal">
    <description>Request from HMI to SDL to get policy configuration data (i.e. OEM mapping table file version etc.) from Policy Table.</description>
    <param name="policyType" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the Struct from where configuration data to be received according to Policy Table types i.e. module_config etc.</description>
    </param>  
    <param name="property" type="String" maxlength="1000" minlength="1" mandatory="true">
        <description>Name of the variable for which the configuration data to be received according to Policy Table types i.e. vehicle_year etc.</description>
    </param>  
</function>
<function name="GetPolicyConfigurationData" messagetype="response">
    <param name="value" type="String" array="true" minsize="1" maxsize="100" mandatory="false">
        <description>Value of requested property from polocyType in PT. If no value is found in PT for specified policyType and property, core should respond to HMI with DATA_NOT_AVAILABLE</description>
    </param>
</function>

[JackLivio]

👍 Please note in the api description the maxlength of the value parameter is system dependent. Also why do you have scope="internal" in the function definition?

[JackLivio]

New Comment 22.

SDL core would need to provide a way for HMI to read OEM Network Mapping version so that HMI can decide whether or not to request OEM Network Mapping file download using the endpoint mentioned in PTU for the OEM table mapping file, using SystemRequest requestType OEM_SPECIFIC and requestSubType VEHICLE_DATA_MAPPING.

I believe that GetPolicyConfigurationData provides the "way" of getting the mapping version. This point should be updated to describe the use of that RPC.

[JackLivio]

New Comment 23.

I don't believe the "Impact on existing code" section accurately represents the rest of the proposal. Considering the size of this proposal, I think this section should be a little more thought out.

APIs would need a new Request/Response API to act as the Generic Request Response for GetVehicleData/SubscribeVehicleData/UnsubscribeVehicleData

What do you mean by this? Aren't these the same APIS as the existing vehicle data RPCs? ie GetVehicleData/SubscribeVehicleData/UnsubscribeVehicleData

New API is needed to provide HMI a way to read OEM Network Mapping table version

Specify the RPC is GetPolicyConfigurationData

SDL core would need to persist OEM and schema versions in policy DB

What about the rest of the vehicle data schema that is downloaded in a PTU. Is that information also supposed to be stored in the policy DB?

There are plenty of other important items that are mentioned in the proposal that should at least be summarized in this section. Normally some sample code is supplied in this section but is not always necessary. There should at least be a list of the "to-dos" an engineer assigned this feature could take and get started developing with.

Some items to add:

• Modifying Core's message validation process to check for known parameters in the vehicle data schema if validation fails against the RPC Spec.
• Adding a new endpoint custom_vehicle_data_mapping_url and updating Core's policy classes to support the parsing, setting, and retrieving of a new module config field endpoint_properties.
• Modifying the PTU process to parse the vehicle data schema items.
• Defining the persistent storage of the vehicle data schema items.
• Implementing new HMI RPC GetPolicyConfigurationData and removing GetURLS.

[atiwari9]

1. #ToUpdate with this and this comment

@JackLivio -

1. It is defined already if you look at following statement in proposal:

SDL core still needs to provide an API for HMI to read this value. We can do so by adding a new API to read OEM Network Mapping file version; we can make it generic so that it can also be used in the future to read some other configuration params from policy. e.g. following new API can be used to read OEM Network Mapping table version from module_config -> endpoint_properties for custom_vehicle_data_mapping_url
<GetPolicyConfigurationData function definition xml>

Still i'd mention it in the statement you pointed out.

23.

What do you mean by this? Aren't these the same APIS as the existing vehicle data RPCs? ie GetVehicleData/SubscribeVehicleData/UnsubscribeVehicleData

It actually was meant to refer to proxy side changes and i believe text is old. New API is not needed rather existing Get/Sub/Unsub/OnVehicleData needs update. #ToUpdate

Specify the RPC is GetPolicyConfigurationData

Ok #ToUpdate

What about the rest of the vehicle data schema that is downloaded in a PTU. Is that information also supposed to be stored in the policy DB?

schema_items are stored in policy DB as well in new tables (vehicle_data_item, vehicle_data_item_parameters), will update the comment as well. #ToUpdate

Modifying Core's message validation process to check for known parameters in the vehicle data schema if validation fails against the RPC Spec.
Adding a new endpoint custom_vehicle_data_mapping_url and updating Core's policy classes to support the parsing, setting, and retrieving of a new module config field endpoint_properties.
Modifying the PTU process to parse the vehicle data schema items.
Defining the persistent storage of the vehicle data schema items.
Implementing new HMI RPC GetPolicyConfigurationData and removing GetURLS.

Ok, i'd add. #ToUpdate . Though the "persistent storage for schema items" will be covered in earlier comment itself.

@theresalech - Summarized the updates in this comment. If there are no more comments, i propose to accept this proposal with revisions.

[theresalech]

The Steering Committee voted to accept this proposal with the revisions noted by "#ToUpdate" in this comment from the author.

[theresalech]

@atiwari9 please let me know when your PR #764 has been updated to reflect the agreed upon revisions. I'll then merge into the proposal file so that it is up to date, and note the changes on the corresponding implementation issues for the feature.

[theresalech]

Proposal file has been updated and implementation issues have been updated to reflect accepted revisions:
Core
SDL HMI
RPC Spec
Java Suite
iOS
Server