Lorawan-stack: Use a wizard for creating end devices

[kschiffer]

Summary:
The add device form (added in #573) is currently not composing fields based on constraints (except for ABP / OTAA selection). For example, it could hide the fields that are only relevant for specific LoRaWAN Versions, or rename fields accordingly (eg. NwkSKey vs. FNwkSIntKey).

Why do we need this?
For better UX, avoid faulty inputs

What is already there? What do you see now?
The add device form, with conditional fields based on OTAA/ABP

What is missing? What do you want to see?
More sophisticated checks and constraints applied into the form, preventing faulty inputs

Environment:
Console in browser

How do you propose to implement this?
Likely hook into field change events and compose fields accordingly

Can you do this yourself and submit a Pull Request?
Yes.

[bafonins]

The main problem with the current implementation of the device form is that everything is coupled together. This causes

1. Complex and long validation schema
2. No straightforward way to disable certain fields based on the stack configuration
3. No straightforward way to change the form fields/field titles accoring to selected values
4. Unnecessary complexity in the js sdk for batched creation/update/deletion as well as creation rollback logic

I propose to implement the device form as the multi-step form. This can look something like this:

Such approach addresses all issues mentioned above:

1. Instead of one complex schema we define a small one for each step.
2. Simply disable the whole step if a certain reponsible component is not available in the stack. What is more, we can inform the user about this via description/notification.
3. Adapt each step depending on the previously submitted values. For example, change Join EUI label to App EUI for lorawan version 1.0.x.
4. No need for batched requests. The user becomes responsible for creating device in different components. However we might want to keep batched deletion for convenience.

Editing devices can be implemented as a stack of accordions:

Where each accordion expands with a standalone form.

Besides the device form we can use wizard for the application form as well to:

1. Create the application in the is
2. Link the application to as

cc @kschiffer @johanstokking @htdvisser

[johanstokking]

Sounds good to me, especially if we can group fields for components together in a step. That way, we can simply skip/disable steps when a component is not available.

Referencing https://github.com/TheThingsNetwork/lorawan-stack/issues/1234 also; even if JS is available, the user can skip the JS fields.

[bafonins]

I came up with such diagram:

Every node is a step

• Steps with dotted outline do not submit the form but keep it in the local state for the next steps.
• Others send requests to the server on submission.

It looks complicated, but it the current implementation has even bigger state space regarding validation/submission/field mask generation/etc.

Consider the diagram as an initial proposition to start the discussion.

[htdvisser]

I think this is a good start.

• Maybe the flow would be better if the Join Server came first.
• An important thing we haven't really made any progress on yet is pre-filling fields from device templates in the device repository #263. I think that could really simplify the process for production deployments.

And some small things:

• A DevEUI is not forbidden for ABP devices (it can optionally be set)
• LoRaWAN 1.0.x devices don't have a NwkKey, only an AppKey
• FNwkSIntKey is called NwkSKey (towards the user)
• The Application Server does not get the NwkKey or AppKey

[johanstokking]

Yes, great start.

• Maybe the flow would be better if the Join Server came first.

I agree with this. If the activation mode is OTAA and the cluster JS is enabled, on the JS page users get the choice to enter root keys and store them on the cluster JS. (if they disable that, NS uses JS lookup, just like when there is no JS in the cluster enabled)

[rvolosatovs]

• "Activation mode" is in payload - in your diagram it corresponds to actually 2 fields - multicast bool and supports_join bool. There are 3 possible options, since multicast && supports_join is not valid.
• frequency_plan -> frequency_plan_id

• resets_f_cnt is optional, false by default.

• FNwkSIntKey should be presented as NwkKey only for <1.1

• FNwkSIntKey is missing in ABP NS settings for >= 1.1
• multicast devices only need AppSKey - NS does not need any keys for multicast devices to work, only AS

[bafonins]

@htdvisser

Maybe the flow would be better if the Join Server came first.

Why?

An important thing we haven't really made any progress on yet is pre-filling fields from device templates in the device repository #263. I think that could really simplify the process for production deployments.

For me it seems outside of the scope of this issue

A DevEUI is not forbidden for ABP devices (it can optionally be set)

Do we want to set it for ABP devices as well? I would say the fewer fields we have, the better. However, it is needed we can add it.

FNwkSIntKey is called NwkSKey (towards the user)

So the label should be NwkSKey for both 1.0.x and 1.1.x ? Here is what we have atm in the console:

• LoRaWAN 1.0.x devices don't have a NwkKey, only an AppKey
• The Application Server does not get the NwkKey or AppKey

Fixed 👌

@johanstokking

If the activation mode is OTAA and the cluster JS is enabled, on the JS page users get the choice to enter root keys and store them on the cluster JS. (if they disable that, NS uses JS lookup, just like when there is no JS in the cluster enabled).

So this is just the matter of allowing users to skip the join server submission step? No requests to js at all?

Regarding https://github.com/TheThingsNetwork/lorawan-stack/issues/1134, could you also specify where do these fields belong in the diagram?

@rvolosatovs

Activation mode" is in payload - in your diagram it corresponds to actually 2 fields - multicast bool and supports_join bool. There are 3 possible options, since multicast && supports_join is not valid.

Just to clarify:

1. supports_join=true - OTAA
2. supports_join=false - ABP
3. multicast=true && **no** supports_join - multicast
   Is this correct?

resets_f_cnt is optional, false by default.

I think it is still fine to show it to the users. Should allow the users to set it for multicast devices?

FNwkSIntKey should be presented as NwkKey only for <1.1

You mean NwkSKey ?

frequency_plan -> frequency_plan_id
FNwkSIntKey is missing in ABP NS settings for >= 1.1

Fixed 👌

[bafonins]

Updated diagram:

[johanstokking]

@htdvisser

Maybe the flow would be better if the Join Server came first.

Why?

Yeah I'm coming back on this; I think it makes more sense to enter MAC versions before entering keys. So I'm supporting the current flow. If MAC version is entered (when NS is enabled), we don't have to ask for NwkKey since that's 1.1.x.

An important thing we haven't really made any progress on yet is pre-filling fields from device templates in the device repository #263. I think that could really simplify the process for production deployments.

For me it seems outside of the scope of this issue

It is out of scope indeed.

A DevEUI is not forbidden for ABP devices (it can optionally be set)

Do we want to set it for ABP devices as well? I would say the fewer fields we have, the better. However, it is needed we can add it.

Yes, we want to optionally ask for it. The more identification information we have about end devices, the better. It is also required in stateful passive roaming. The reason we don't force users to enter the DevEUI, is because if there is no DevEUI, we don't want users to enter bogus values.

FNwkSIntKey is called NwkSKey (towards the user)

So the label should be NwkSKey for both 1.0.x and 1.1.x ? Here is what we have atm in the console:

It's NwkSKey for 1.0.x and FNwkSIntKey for 1.1.x.

If the activation mode is OTAA and the cluster JS is enabled, on the JS page users get the choice to enter root keys and store them on the cluster JS. (if they disable that, NS uses JS lookup, just like when there is no JS in the cluster enabled).

So this is just the matter of allowing users to skip the join server submission step? No requests to js at all?

Indeed.

An example is a device featuring Semtech's modem which uses the Semtech Join Server. We only need to know EUIs in that case.

Regarding #1134, could you also specify where do these fields belong in the diagram?

They belong in the JS step; these fields to go JS, if the JS is enabled in the cluster and if the user wants to provision the devices on the JS.

These fields are optional.

Just to clarify:

1. supports_join=true - OTAA
2. supports_join=false - ABP
3. multicast=true && **no** supports_join - multicast
   Is this correct?

Strictly:

1. OTAA: supports_join
2. ABP: !supports_join && !multicast
3. Multicast: multicast

Invalid is supports_join && multicast, but this is (or should be) validated at NS.

resets_f_cnt is optional, false by default.

I think it is still fine to show it to the users. Should allow the users to set it for multicast devices?

No, this is not for multicast. resets_f_cnt refers to uplink, but there ain't uplink in multicast.

FNwkSIntKey should be presented as NwkKey only for <1.1

You mean NwkSKey ?

Should be NwkSKey indeed.

[rvolosatovs]

@bafonins please see @johanstokking 's answer.
In regards to multicast - device address is required as well

[bafonins]

• Added fields from https://github.com/TheThingsNetwork/lorawan-stack/issues/1134
• Added DevEUI to abp/multicast devices
• Added Device Address to multicast devices
  

[rvolosatovs]

Device Address still missing from multicast device Network Server settings

[bafonins]

Device Address still missing from multicast device Network Server settings

Updated 👌

[johanstokking]

Multicast can be 1.0.x and 1.1.x. Entering the session information (DevAddr and keys) is the same for multicast as for ABP.

[johanstokking]

Also resets_join_nonces can be set in JS with 1.1.x

[kschiffer]

Splitting up the device creation is definitely needed and a nice way to get the flow and implementation closer to the requirements of the backend, while reducing complexity for the user.

Some thoughts and challenges that I see:

• We should add functionality to abort the whole flow, resulting in a deletion of whatever registry entries have already been created.
  
  
  
  • Likewise, going to the previous step needs to put the form in "update mode" (should be relatively easy as the endpoints are the same, except for IS)
  
  
• One problem I see is the Application Server step being very shallow (will this likely change later?) and adding the payload format options there is also not really in line with the user's needs while creating a device.
  
  
  
  • A solution might be to merge the AS and JS step given that both are pretty straightforward and not interdependent. I realize this goes against the separation by component but I think this would simplify the whole flow with a reasonable abstraction. Especially given that we don't communicate any connection to the respective stack components within the steps.
  
  
• Depending on the stack config we might end up with a wizard that only has one or two steps, which is an anti-pattern for wizards.
  
  
  
  • For the case of just one step, we can simply hide/remove the wizard aspect
  
  
  • For two steps, I think we have to live with the issue 🤷‍♂
  
  
• It should be considered that the wizard solution will increase the _time-to-complete_ for the user story quite a bit
  
  
  
  • This could become an issue in situations where a lot of devices need to be created by hand, though we will introduce batch creation features for this use case and the CLI/scripting can help with such situations as well
  
  
  • Still, consider the difference with the V2 console device creation and how users might perceive this change
  
  
• I like the corresponding "segmented edit" solution for general settings
• The flow diagram is very helpful and we should keep it updated 👍

Unnecessary complexity in the js sdk for batched creation/update/deletion as well as creation rollback logic

I think the functionality there in the SDK for splitting and merging device requests is still pretty valuable, even if we don't use it in this case

[rvolosatovs]

@johanstokking for multicast devices NS only needs the SNwkSIntKey in 1.1 for MIC calculation. FNwkSIntKey and NwkSEncKey, in fact, should not be allowed to set IMO.

[johanstokking]

• We should add functionality to abort the whole flow, resulting in a deletion of whatever registry entries have already been created.
  
  
  
  • Likewise, going to the previous step needs to put the form in "update mode" (should be relatively easy as the endpoints are the same, except for IS)
  
  

I don't think we should be creating anything before hitting Finish or something. Going back and forth in the wizard and closing the browser window should not result in half-created devices.

• One problem I see is the Application Server step being very shallow (will this likely change later?) and adding the payload format options there is also not really in line with the user's needs while creating a device.

We will add configuration for application packages here (i.e. remote multicast setup, fragmented data block transfer, Semtech modem options, etc). So yes it shallow now but it will be extended.

• A solution might be to merge the AS and JS step given that both are pretty straightforward and not interdependent. I realize this goes against the separation by component but I think this would simplify the whole flow with a reasonable abstraction. Especially given that we don't communicate any connection to the respective stack components within the steps.

For our future selves I would recommend keeping them separate. Also combining them makes rendering stuff on those pages conditional on component availability which adds complexity to an already complex flow.

• Depending on the stack config we might end up with a wizard that only has one or two steps, which is an anti-pattern for wizards.

  • For the case of just one step, we can simply hide/remove the wizard aspect
  • For two steps, I think we have to live with the issue 🤷‍♂

• It should be considered that the wizard solution will increase the _time-to-complete_ for the user story quite a bit

  • This could become an issue in situations where a lot of devices need to be created by hand, though we will introduce batch creation features for this use case and the CLI/scripting can help with such situations as well
  • Still, consider the difference with the V2 console device creation and how users might perceive this change

The separation of components and the flexible deployment scenarios is one of the key changes compared to V2, so it makes total sense that this is effective in V3 Console.

Indeed, for creating large amounts of devices, people should use APIs and CLI anyway.

There is no scenario with one step, it's always at least two steps.

How about rendering tabs instead of pages? That way it's still one page but things are easily accessible without going back and forth in steps.

[bafonins]

@johanstokking

I don't think we should be creating anything before hitting Finish or something.

If we make the actual request only at the last step, then it means that we would make 3-4 request. How do we handle errors returned by different components? Handling this, navigating the user to the errored step/resetting store/etc. is complex. That is why I suggest submitting values at every step, because every successive step can rely on submitted values as valid.

Going back and forth in the wizard and closing the browser window should not result in half-created devices.

I am against allowing users to edit data in the wizard (for the steps that result in submission). Mb, only optional fields that are stored in a single compoent (and no other components need those), say name, description. Otherwise handling this becomes quite complicated.

@kschiffer

Depending on the stack config we might end up with a wizard that only has one or two steps, which is an anti-pattern for wizards.

Well, we are open to alternative solutions. Any propositions?

[johanstokking]

If we make the actual request only at the last step, then it means that we would make 3-4 request. How do we handle errors returned by different components? Handling this, navigating the user to the errored step/resetting store/etc. is complex. That is why I suggest submitting values at every step, because every successive step can rely on submitted values as valid.

OK. That's fine, as long as the Console can handle devices that are in ER but cannot be found in JS/NS/AS because that step failed or the user closed the tab or something.

I am against allowing users to edit data in the wizard (for the steps that result in submission). Mb, only optional fields that are stored in a single compoent (and no other components need those), say name, description. Otherwise handling this becomes quite complicated.

What do you mean?

Note that that AS, for example, doesn't need any fields, it just needs the device to exist. So if the AS is there, even if the user doesn't enter any values for AS, it should still create an (empty) device in AS.

[bafonins]

What do you mean?

Sorry. This was towards @kschiffer suggestion:

Likewise, going to the previous step needs to put the form in "update mode" (should be relatively easy as the endpoints are the same, except for IS)

Going back and forth in the wizard and closing the browser window should not result in half-created devices.

I would consider this as a future improvement. For now we can just show a notification to the user on uncompleted flow. Later, the user can edit/delete the device in the general settings page.

[kschiffer]

Well, we are open to alternative solutions. Any propositions?

Well, I think that given the _edge-casiness_ of this particular situation, it is ok to live with this issue. Your wording suggests that I should not word concerns without proposed solutions, but I like to give others the opportunity to come up with some if I can't immediately find any.

I am against allowing users to edit data in the wizard (for the steps that result in submission). Mb, only optional fields that are stored in a single compoent (and no other components need those), say name, description. Otherwise handling this becomes quite complicated.

This would again break with the best practices for the wizard pattern. Users will likely want to revise or simply inspect earlier fields, especially given that these are interdependent. I'm okay with not including this functionality initially, but it should be added eventually (as in: kept track of in an issue).

Otherwise handling this becomes quite complicated.

Generally speaking, the necessity to implement complex frontend logic should not affect our commitment to UX.

I would consider this as a future improvement. For now, we can just show a notification to the user on uncompleted flow. Later, the user can edit/delete the device in the general settings page.

Not optimal but acceptable for me, as long as we will enhance this eventually.

[johanstokking]

As discussed offline, here's the initial proposal;

Create device

1. General settings
   
   
   
   • Fields
     
     
     
     
     
     • ids
     
     
     
     • Optional name
     
     
     
     • Optional description
     
     
     
     • Optional attributes
     
     
     
     • Required activation mode: OTAA, ABP or Multicast
     
     
     
     • If NS in cluster
       
       
       
       
       
       
       
       • lorawan_version
       
       
       
       
       • lorawan_phy_version
       
       
       
       
     
     
     
   
   
   • This step creates the device only in IS. That way, we know that the identifiers are free
   
   
   • Activation mode is used in the session state
   
   
   • If not NS in cluster, for simplicity in the wizard, you may use the highest known versions (i.e. now 1.1.0 and 1.1.0-A respectively) in the session state
   
   

2. LoRaWAN settings

   • If NS in cluster: Fields
     
     
     
     • lorawan_version (required)
     
     
     • lorawan_phy_version (required)
     
     
     • supports_join set if Activation Mode is OTAA (required)
     
     
     • multicast set if Activation Mode is multicast
     
     
     • supports_class_b
     
     
     • supports_class_c
     
     
     • frequency_plan_id (required)
     
     
     • mac_settings.supports_32_bit_f_cnt
     
     
     • mac_settings.factory_preset_frequencies
     
     
     • mac_settings.ping_slot_data_rate_index.value
     
     
     • mac_settings.ping_slot_frequency
     
     
     • mac_settings.ping_slot_periodicity.value
     
     
     • mac_settings.rx2_data_rate_index.value
     
     
     • min_frequency
     
     
     • max_frequency
     
     
   • If NS in cluster: Fields if activation mode is OTAA
     
     
     
     • Checkbox whether to use external JS (default checked)
     
     
   • Fields if activation mode is ABP or Multicast
     
     
     
     • If NS or AS in cluster: session.dev_addr
     
     
     • If NS or AS in cluster: generate a session.keys.session_key_id
     
     
     • If NS in cluster: session.keys.f_nwk_s_int_key.key - required
     
     
     • If NS in cluster: session.keys.s_nwk_s_int_key.key (if LW >= 1.1.0) - required
     
     
     • If NS in cluster: session.keys.nwk_s_enc_key.key (if LW >= 1.1.0) - required
     
     
     • If AS in cluster: session.keys.app_s_key.key - required
     
     
     • If NS in cluster: session.last_conf_f_cnt_down
     
     
     • If NS in cluster: session.last_n_f_cnt_down
     
     
   • If NS in cluster: Fields if activation mode is ABP
     
     
     
     • mac_settings.resets_f_cnt
     
     
   • If NS or AS in cluster: Fields if activation mode is ABP
     
     
     
     • session.last_f_cnt_up - this is required for security
     
     

   • If NS in cluster: Fields if activation mode is ABP or OTAA

     • mac_settings.adr_margin
     • mac_settings.class_b_timeout
     • mac_settings.class_c_timeout
     • mac_settings.desired_adr_ack_delay_exponent.value
     • mac_settings.desired_adr_ack_limit_exponent.value
     • mac_settings.desired_max_duty_cycle.value
     • mac_settings.desired_rx1_data_rate_offset
     • mac_settings.desired_rx1_delay.value
     • mac_settings.desired_rx2_data_rate_index.value
     • mac_settings.desired_rx2_frequency
     • mac_settings.max_duty_cycle.value
     • mac_settings.rx1_data_rate_offset
     • mac_settings.rx1_delay.value
     • mac_settings.status_count_periodicity
     • mac_settings.status_time_periodicity
     • mac_settings.supports_32_bit_f_cnt
     • mac_settings.use_adr

   • This sets the device in NS and AS (if in cluster). If we have an OTAA device, we don't have anything to set in AS at this point, but I would still make the call for consistency

3. If AS in cluster: Application settings
   
   
   
   • Fields
     
     
     
     
     
     • payload_formatters
     
     
     
   
   
   • This sets the device in AS
   
   
4. If JS in cluster and if OTAA and if not using external JS: Security settings
   
   
   
   • Fields
     
     
     
     
     
     • Generate root_keys.root_key_id
     
     
     
     • root_keys.app_key.key
     
     
     
     • root_keys.nwk_key.key (if LW >= 1.1.0)
     
     
     
     • resets_join_nonces
     
     
     
     • home_net_id
     
     
     
     • network_server_kek_label
     
     
     
     • application_server_kek_label
     
     
     
     • application_server_id
     
     
     
   
   
   • This sets the device in JS
   
   

Update device

Here, I would go with a tabbed approach, basically with wizard steps as tabs.

Key here is that;

• ids, supports_join, multicast are read-only
• Activation mode is evaluated in this order:
  
  
  
  • OTAA: if no NS in cluster, or if NS in cluster and NS says supports_join
  
  
  • ABP: needs NS in cluster (and is only relevant in that case): !supports_join && !multicast
  
  
  • Multicast: needs NS in cluster (and is only relevant in that case): !supports_join && multicast (although multicast should be enough)
  
  

Some use cases to support in updating devices:

• The Console can have NS, AS and JS in the cluster, but the device may not be in the NS, AS or JS (Console gets 404). This is a valid case and Console should handle this. An example case is a claimed device, which has the device set in ER and JS, but not (yet) in NS and AS
• Building upon the previous: set the LoRaWAN and Application settings of a device that is only in ER and JS (i.e. set it in NS and AS)
• Changing lorawan_version and lorawan_phy_version (this changes constraints)
• Changing the "external JS" option; i.e. let's say you have an existing device but you want to configure root keys in the cluster-JS. Then, you uncheck this checkbox and the user should be able to set the root keys in the Security settings tab
• Via bulk import, the device may be created on JS and have root keys, but they are not exposed. Like today, the user should be able to see that root keys are there (root_keys != nil) but they're not exposed (root_keys.app_key == null etc). The user should be able to leave the root keys intact

General stuff

• For keys, an input field or a generate random button
• The difference between external JS and unexposed root keys;
  
  
  
  • External JS is where the JS is not in the same cluster as the NS. This allows for creating an OTAA device but not having to enter the Security settings, since the keys are configured somewhere else
  
  
  • Unexposed root keys is where the device is in the cluster-local JS, but the JS does not expose the root keys. This is for security, i.e. in case of secure elements, where JS has access to root keys but does not expose them
  
  

[rvolosatovs]

I think https://github.com/TheThingsNetwork/lorawan-stack/issues/579#issuecomment-525719408 already contains everything necessary for NS.
I think all mac_settings should be available to be set for all non-multicast devices.
For multicast devices only the following make sense:

• "mac_settings.factory_preset_frequencies"
• "mac_settings.ping_slot_data_rate_index.value"
• "mac_settings.ping_slot_frequency"
• "mac_settings.ping_slot_periodicity.value"
• "mac_settings.rx2_data_rate_index.value"
• "mac_settings.rx2_frequency"

[htdvisser]

Maybe some things to clarify

Create:

1. General Settings
   a. When we create the device in the ER, we don't set the NS/AS/JS addresses. Do we update the ER registration when the device is set in those? What if we're using an external JS?
   b. Do we store the activation mode in the ER?
   c. Do we store the phy/mac versions in the ER?
2. LoRaWAN settings
   a. How about supports_class_b?
3. Application settings
   a. Yes, this could mean that we set the device on the AS twice

Update:

• Activation mode: would be a lot easier if we store this in ER
• Do we only look at 404's or also at the NS/AS/JS addresses in the ER?
• I think it would be nice to be able to delete an individual NS/AS/JS registration for example when changing "external JS" to true, or to reset device state in the NS

General stuff:

• _"External JS is where the JS is not in the same cluster as the NS"_: I think this should be something like "join_server_address is not same as JS address in console config"

[rvolosatovs]

If we're talking about top-level fields:
https://github.com/TheThingsNetwork/lorawan-stack/blob/375c82cc068bbadb72b887e25631f8f2dc03a366/api/end_device.proto#L395-L418 this whole chunk belongs in NS(but not all of these are required)

m{in,ax}_frequency is not applicable to multicast

[johanstokking]

@rvolosatovs

I think #579 (comment) already contains everything necessary for NS.

I think all mac_settings should be available to be set for all non-multicast devices.
For multicast devices only the following make sense:

One of the problems with this issue is the amount of information buried in comments.

Can you please add _exactly_ the fields in the right places? I'm fine if you copy my whole list of bullet points so we incrementally work on that until we have a final version.

---

@htdvisser

1. General Settings
   a. When we create the device in the ER, we don't set the NS/AS/JS addresses. Do we update the ER registration when the device is set in those? What if we're using an external JS?

I think we should update the addresses when we set the devices in AS/NS/JS.

b. Do we store the activation mode in the ER?

No, we don't have a field for that at the moment, we don't really need it and it creates room for inconsistencies.

c. Do we store the phy/mac versions in the ER?

No, see API documentation. Again, I don't see the need and it creates room for inconsistencies.

a. How about supports_class_b?

Yeah I guess we should add that, pending #19. @rvolosatovs please include this in an updated version if relevant.

1. Application settings
   a. Yes, this could mean that we set the device on the AS twice

Yes, that doesn't hurt

Update:

• Activation mode: would be a lot easier if we store this in ER

Yes but I'm not sure if we should go for easiness and if that fully applies. We need to have it available in the hot path.

Also, it would only be easy if we can start from a clean slate. However, we need to be backward compatible, adding heuristics when lorawan_version is not in ER, introducing conditional gets from NS etc.

• Do we only look at 404's or also at the NS/AS/JS addresses in the ER?

We can only get a 404 if there's an address set, otherwise there's nothing to get. We should interpret both as "not in that registry". We cannot error here (like we did before), exactly for the reason that creation in IS and AS/NS/JS does not happen at the same time, and users should be able to recover inconsistencies created by the Console.

• I think it would be nice to be able to delete an individual NS/AS/JS registration for example when changing "external JS" to true, or to reset device state in the NS

Yes, let's do that later

• _"External JS is where the JS is not in the same cluster as the NS"_: I think this should be something like "join_server_address is not same as JS address in console config"

Yes, that's indeed the way to check it. join_server_address can be empty also, using interop.

[rvolosatovs]

@bafonins do you still have the source for the graph?
Since a lot of work went into constructing that already, I think we should just update that to make it complete to have a clear representation?
We can also work on updating the NS part offline to not clutter the discussion here.

[rvolosatovs]

I updated the graph with all possible NS fields
The fields in red are mandatory

[johanstokking]

@rvolosatovs we're aiming to define the flow and fields that we're presenting in the Console when creating and updating devices.

As such,

• We should not be allowing changing mac_state.lorawan_version
• We can do mac_state.ping_slot_periodicity through CLI for now
• Please think about how the user is going to set supports_class_b, supports_class_c and mac_state.device_class; which is part of create, which is part of update, how does one relate to each other?
• We should not be changing individual counters; a "reset frame counters" button would do (which afaik can be a separate action, like we have in V2 Console, that's setting the counters to 0)
• We should carefully select which settings we want in the Console of mac_settings and mac_state.desired_parameters; @htdvisser can you think along here?

I changed the order in https://github.com/TheThingsNetwork/lorawan-stack/issues/579#issuecomment-553347858. Please work incrementally on that list.

[rvolosatovs]

Can you please add exactly the fields in the right places?

I answered this question - i.e. I presented all fields, that can be set and should be set on NS. Or at least that's how I understood your question.

Regarding what should be in the console, everything mac_state should probably only be settable through CLI, however we may require that for ABP/multicast devices and/or OTAA devices, which are being registered with an existing session, and, hence MAC state.
I will update your comment.

[rvolosatovs]

We should not be changing individual counters; a "reset frame counters" button would do (which afaik can be a separate action, like we have in V2 Console, that's setting the counters to 0)

We do need to be able to set downlink frame counters for ABP and multicast - otherwise NS cannot send downlinks
For uplink frame counters are also very useful from security standpoint for ABP

[kschiffer]

Update device

Here, I would go with a tabbed approach, basically with wizard steps as tabs.

Let's use the accordion approach here, as presented in the first comment from @bafonins:

It will give us fewer issues with horizontal space, and allows to distinguish modes with Edit / Create buttons.

[johanstokking]

@rvolosatovs

We do need to be able to set downlink frame counters for ABP and multicast - otherwise NS cannot send downlinks
For uplink frame counters are also very useful from security standpoint for ABP

If this is simple we can do it. In fact, two/three input boxes may be simpler than a reset button which triggers a specific action.

Regarding what should be in the console, everything mac_state should probably only be settable through CLI, however we may require that for ABP/multicast devices and/or OTAA devices, which are being registered with an existing session, and, hence MAC state.

I understand. I think we should separate "new and reset ABP devices" from "existing ABP devices that have already been on a network".

The former should be complete, so it should include some mac_state (i.e. factory reset frequencies, RX1 delay, RX2 settings, etc) but not mac_state.desired_parameters.

The latter should be done via migration and we can add the fine tuning settings later in the Console; for now CLI should be used.

---

Thanks for updating

LoRaWAN settings

• If NS in cluster: Fields
  
  
  
  • mac_settings.factory_preset_frequencies
  
  
  • mac_settings.ping_slot_data_rate_index.value
  
  
  • mac_settings.ping_slot_frequency
  
  
  • mac_settings.ping_slot_periodicity.value
  
  
  • mac_settings.rx2_data_rate_index.value
  
  

Are these necessary for OTAA? Isn't this only for ABP and Multicast?

[rvolosatovs]

@johanstokking

LoRaWAN settings

• If NS in cluster: Fields
  
  
  
  • mac_settings.factory_preset_frequencies
  
  
  • mac_settings.ping_slot_data_rate_index.value
  
  
  • mac_settings.ping_slot_frequency
  
  
  • mac_settings.ping_slot_periodicity.value
  
  
  • mac_settings.rx2_data_rate_index.value
  
  

Are these necessary for OTAA? Isn't this only for ABP and Multicast?

All MAC settings are optional by design.
The only case where these are "required" are class B multicast devices, that is due to the class B operation specifics.
Apart from that, factory_preset_frequencies is likely to be required for ABP for best results, but nothing prevents OTAA device to have that set.

[johanstokking]

All MAC settings are optional by design.
The only case where these are "required" are class B multicast devices, that is due to the class B operation specifics.

OK

Apart from that, factory_preset_frequencies is likely to be required for ABP for best results, but nothing prevents OTAA device to have that set.

It's ineffective for OTAA; the default join frequencies are required by spec, and the channels come in via join-accept and MAC commands. There should not be out-of-band configuration of preset frequencies for OTAA devices.

[kschiffer]

@bafonins can you give a status update and timeline on this issue?

[johanstokking]

For app_s_key and skip_payload_crypto, here's the thing;

• AS respects the field skip_payload_crypto of the end device. If true, AS does not perform payload encryption and decryption
  
  
  
  • AS will get a skip_payload_crypto on the application-level too (via Link, like payload formatters), which takes precedence over the end device's setting
  
  
• There's probably always a app_s_key in AS, but it may be wrapped, i.e. session.keys.app_s_key.key is not set (but encrypted_key and kek_label is set)
  
  
  
  • When AS doesn't have the KEK, i.e. it can't unwrap the encrypted key. Now, that errors. We will probably just return the encrypted app_s_key as is to clients
  
  
• In the Console, if skip_payload_crypto is set true (on end device or application link), don't bother with app_s_key: disable it, don't care