Mbed-os: LoRaWAN HAL should have function to read EUI if device has one

[janjongboom]

Description

• Type: Enhancement

Enhancement

Reason to enhance or problem with existing solution

Some targets - like the Multi-Tech ones - have an EUI printed on the module. Very useful as you don't have to buy one yourself. However, we cannot read this EUI at the moment.

Suggested enhancement

Add a HAL function to read EUI. Maybe in the LoRaWAN HAL, but I assume we have similar things for BLE.

Pros

Users don't have to configure their EUI in mbed_app.json anymore.

Cons

Could also be in userspace. Also I don't know how to read the EUI on MT devices, because the code is in a statically compiled binary.

cc @hasnainvirk

[hasnainvirk]

@janjongboom Users need to add AppEUI and AppKey in their json anyway so why not DevEUI too ?
The problem with providing a mechanism to read from the chip does not scale as we are targeting many chips. In a production environment, user may use neither of our proposed solutions. Keys will live in a secure element and and retrieved from there using an application extension. As much as I like your idea, I am afraid it will cause more nuances rather than solving a problem.
[Mirrored to Jira]

[hasnainvirk]

@AnttiKauppila @kivaisan what do you think ?
[Mirrored to Jira]

[janjongboom]

DevEUI needs to be readable regardless. It's not held in secure element. It's already in the module, why not read it from there.
[Mirrored to Jira]

[hasnainvirk]

@janjongboom In production, you can do it by whatever means in Application like:

lorawan_connect_t my_conn;
my_conn.dev_eui = read_from_chip();

We are giving an example only. We don't need to read from the chip necessarily.
[Mirrored to Jira]

[kivaisan]

I have to agree with Hasnain. I cannot see any additional benefit from providing just an API to get data as all device/application developers will have their own API to get this info anyway. And we cannot know what kind of API they need for it (it can be for example that they fetch all kind of other data with the same API).
[Mirrored to Jira]

[loverdeg]

@janjongboom @hasnainvirk @kivaisan

I detest the idea of compiling devEUI into an application. Even if it were for hobbyist development I'd still use whatever unique key I could find in hardware.

Furthermore when developing an application that communicates via LoRaWAN I do not want to manually grab the devEUI and give it to the stack. I especially wouldn't want to do this for every application I write.

In my opinion, any board intended to be used for LoRaWAN should be pre-loaded with devEUI either in hardware, ROM, or some sort of pseudo-ROM.
An api would then be provided to get devEUI somewhere in the target layer and ultimately mounted up to the stack.

Always thought it was rather unfortunate that it's not defined in the radio and IPv6 EUI-64 official. Even if LoRaWAN isn't IP based.

[Mirrored to Jira]

[loverdeg]

I guess this comes down to whether a stack is provided API's by a board support package or if the application must hand them to the stack manually.

As someone working for a company that designs and produces LoRa endpoints I know that we will always provide an API to get devEUI as part of the BSP. It likely follows that we would be happy to abstract this API for usage by the stack.
[Mirrored to Jira]

[janjongboom]

@hasnainvirk @kivaisan I think this is very short-sighted. Should we remove get_mac_address() function from NetworkStack too then?
[Mirrored to Jira]

[janjongboom]

For reference, this is how we read EUI on xDot:

    #include "xdot_eeprom.h"

    uint8_t eui[8] = { 0 };

    int v = xdot_eeprom_read_buf(0x401, eui, 8);
    for (size_t ix = 0; ix < 8; ix++) {
        printf("%02x ", eui[ix]);
    }
    printf("\n");

[Mirrored to Jira]

[AnttiKauppila]

I totally agree with Jan here. All device information needed must be read from the device whenever possible. Manually setting e.g. EUI works only in projects having a few devices. If a manufacturer needs to create e.g. 1 million devices, this must be automatically handled by the SW.
[Mirrored to Jira]

[kivaisan]

I don't think the issue here is that do we read the value from HW or set it in application. Of course the value should be read from HW if it has it (most probably has). The question just is that who will read it, application or some lora component.

The issue I see here is that there is no standard way to read it. So can application just do the read and give the value as connect() parameters, or should stack read it internally? If the stack should read it internally, I assume the API should be then in LoRaRadio since that's the only LoRA HAL specific API. In latter case, I think connect parameters should not even have DevEUI option.

[Mirrored to Jira]

[janjongboom]

@kivaisan I believe this should be added to device HAL for any device that has LoRa radio on board. If no EUI can be read we can fall back to whatever is in mbed_app.json.
[Mirrored to Jira]

[hasnainvirk]

@janjongboom get_mac_address() queries ethernet driver for mac address. LoRa Radios do not provide such a hook. Module manufacturers add their custom ROMs where DevEUI may be stored, abstracting that would mean we need pre_main handlers for hooking up EUI read and the generic API that must be provided by the module HAL. Next steps could be:

1) Doing an example abstraction, maybe with xdot
2) Writing a porting guide for the module vendors that they should provide this API if they can

It all seems to me that an application extension could handle easily itself rather than the stack.
[Mirrored to Jira]

[loverdeg]

@hasnainvirk @kivaisan Ahoy, Please review my comments further up.

I believe this is best handled at the target/BSP level; which I do not consider part of the application.

Having the user implement this as part of the application may only require an additional line or two of code, but it will still place the burden on them to go look up the particular API to get devEUI from their board and provide it to the stack.

Their are far more users than there are boards and the collective time inefficiency for users would far outweigh that of us board vendors.

I think this may uncover the rabbit hole of whether board vendors should be allowed to integrate board level drivers into the OS as part of their BSPs.
Unless BSP drivers are imported alongside, but outside, the OS things might get bloated real quick.

I suppose this is one of the reasons I preferred the less 'monolithic' nature of mbed OS 3.

[Mirrored to Jira]

[hasnainvirk]

@loverdeg-ep I like what you are saying. The point here is not about Mbed-OS being monolithic or not. The point here is that MCU abstraction code comes from the third party vendor. It's a hard expectation that they will provide a standard abstraction for reading something from the chip like DevEUI. Even the example given above by Jan, https://github.com/ARMmbed/mbed-os/issues/6132#issuecomment-366886905 shows APIs given by one manufacturer which is custom to that port only.

I totally understand your point and we had done such stuff in past like onboard_modem_api for cellular https://github.com/ARMmbed/mbed-os/blob/master/targets/TARGET_NXP/TARGET_LPC176X/TARGET_UBLOX_C027/onboard_modem_api.c

We have a porting guide which highlights what this API means and why we need it. But we didn't get a lot of followup for that. Drivers for cellular boards kept on coming without onboard_modem_api support. That's my only fear. It's not straight forward to assume that everyone will follow the suit.
[Mirrored to Jira]

[loverdeg]

@hasnainvirk I see your point.

However, to officially add board support to the OS do you not need to be a partner and follow certain standards?

Edit:
Additionally certain tests must pass before code can be added to master as well correct?
[Mirrored to Jira]

[hasnainvirk]

@loverdeg-ep I am not 100% sure but I think official board support to OS does not include providing stuff like onboard_modem_api. I would assume that official board support means HAL for basic peripherals. Other perks like 'onboard_modem_api' or 'dev_eui_read' :) are not enforced but encouraged through various porting guides and discussions like what we are having here. I will take pointers from this discussion and let's brainstorm what can be done.
[Mirrored to Jira]

[adbridge]

Internal Jira reference: https://jira.arm.com/browse/IOTCELL-1117

[loverdeg]

@hasnainvirk @janjongboom
We are working on a new LoRaWAN endpoint so I wanted to restart this conversation.

Per https://github.com/ARMmbed/mbed-os-example-lorawan/issues/55 there is a lot of ignorance to the fact that the LoRaWAN specification very specifically defines what EUI is and how it should be assigned.

I think implementing some form of what @janjongboom is suggesting would make the EUI concept a little more intuitive for new developers, as well as make those who have read the spec feel better about their software/hardware design diagrams.

[janjongboom]

@loverdeg-ep @hasnainvirk Yes, I believe this should be a HAL function which targets could implement.

[loverdeg]

@janjongboom Any other LoRaWAN authorities we should involve?

[loverdeg]

@adbridge this is not resolved

@janjongboom @BlackstoneEngineering
We are moving quickly on our BSP, and ASP.
I'd like to get this addressed quickly as well.

In terms of developer experience this stands out like a sore thumb to me. Not quite as bad as the black eye that is compiled in developer certificates but...

[adbridge]

@loverdeg-ep This was marked as resolved internally by the LoRaWAN team. @mhiljane could you please comment as to why?

[hasnainvirk]

@loverdeg-ep @adbridge This falls in HAL team's domain. First they need to provide generic HAL APIs which the stack can harness to provide this information. Adding APIs in the stack to read an EUI is trivial. Hooking APIs from HAL layer to actually achieve this, is not.

[kjbracey-arm]

Conversation seems to have gone down something of a rabbit-hole. The model used already to obtain EUI-64 for 802.15.4 radios is pretty effective. If I recall correctly

1) Radio driver has API that can provide suggested EUI-64, if the radio has an on-board one.
2) Radio driver has API to set the EUI-64 for traffic, allowing app to override for any test purpose.
3) If app does not set EUI-64, default stack behaviour is to use the EUI-64 from the radio. If the radio doesn't have one, get a EUI-64 from the system.

(EMAC API also has the same basic set-up, but for EUI-48)

The major hole is that there is no "get EUI-64 from the system" call, but there is "get EUI-48", ie mbed-mac-address. That can be used fine if just extended.

The minor hole is that there is no way to reverse the plumbing such that a system EUI can be obtained from a particular driver. We've yet to see a need for this - I guess it would allow say a LoRAWAN driver to provide an EUI-64 which is then used by a 802.15.4 radio. And this wrinkle seems to be what a lot of the discussion above is about? I'm not convinced we currently need this.

So my suggestion:
1) LoraWAN follows model of NanostackRfPhy and EMAC, as above
2) Look in to adding a system call to get an EUI-64 from the target - 64-bit version of mbed_mac_address. NanostackRfPhy could also use this
3) Think about how to restructure so an individual driver can provide EUI-64 to the rest of the system - consider if this is really needed.

[kjbracey-arm]

From above:

get_mac_address() queries ethernet driver for mac address.

Conceptually, this isn't quite right.

It queries the target for a mac address that could be used by an Ethernet driver (or any other driver) - it is usually something derived from a unique identifier in the SoC. This is not used if the driver has its own mac address.

However, in cases where the Ethernet is built-in to the SoC, it is sometimes the case that the "Ethernet driver" code does provide this function, because the Ethernet driver can be viewed as target code. But that's a special case, and we don't rely on it.

So when searching for mbed_mac_address, you do see a mixture of implementations inside netsocket/emac-drivers/TARGET_xxx and targets/TARGET_xxx. But it is still the "target" providing it either way, regardless of the code location - it's always inside TARGET_xxx.

[loverdeg]

For reference:

Embedded Planet has a block of MACs we can use for our endpoints. We may burn these in, or ROM them somehow.

However since our board has an NRF52840 I would tend to just use the 802.15.4/Bluetooth MAC, so I don't need to write up an operations procedure. Our BSP will never officially support LoRaWAN and BT radios being active at the same time.

[kjbracey-arm]

On a known target, you should certainly be able to do:

  stack->set_mac_address(interfaceA, interfaceB->get_mac_address());

or similar in the application, if you know which way round you want it.

The extra system support would only be needed if you wanted "interfaceB" to magically supply the MAC address to interfaceA without the app having to write anything - automatically if B had its own MAC address but A didn't.

Part of me thinks that any such system magic might be more confusing than just telling people to do the above manually.

[loverdeg]

@kjbracey-arm if part of you thinks what I mentioned above is too voodoo then I'm happy to write the ops procedure and use our assigned MAC's for LoRaWAN.

Edit: versus reading MAC from the NRF52840 802.15.4 radio

[kjbracey-arm]

If I'm understanding right, you are saying you do have a use-case for the "deluxe" integrated implementation where you take an EUI from the 802.15.4 driver and use it for your LoRAWAN?

That's two steps up from the original issue here, which was just demanding the basic functionality of "use the EUI from the LoRAWAN module - don't make the app specify it".

I'm generally in principle of having an integrated mechanism - it would probably make sense to make it work like the XXX::get_default_instance and XXX::get_target_default_instance mechanism we've used several times already - providing hook points for the target or a library or the application to specify a system choice.

But the design work and discussion on that is bigger than just doing what Jan's asking for, which is possibly why his simple issue has been somewhat delayed.

Jan's needs are totally covered by:
1) EUI-64 to be read from radio driver
2) EUI-64 from radio to be used by stack (if not app overridden)
3) Stack uses target EUI-48 instead if no radio-supplied EUI-64

That's just making LoRAWAN work like Ethernet or 802.15.4.

You would also need:
4) Target to be able to provide EUI-64 as well as EUI-48.
5) Mechanism for drivers to override target EUI-64 (and/or EUI-48).

Those are orthogonal, and the second is not LoRAWAN-specific, so maybe should go to a separate issue, leaving this one to deal with the lack of LoRAWAN EUI handling.

[loverdeg]

1. EUI-64 to be read from radio driver

As far as I know, there are no LoRa radios that have any sort of unique identifier whatsoever.

Otherwise, yes I think your breakdown is correct regarding us possibly needing a 2nd issue.

Further Background Info:

• We are not using one of the standard modules that comes with the extra cost of an STM32 MCU.
• LoRaWAN MAC is running on NRF52840 via Mbed OS.
• We will either use the NRF52840 MAC or burn a MAC from our range into NRF52840.
• This all means our TARGET_EP_XXX folder will probably contain more than just PinNames.h and device.h

@kjbracey-arm If you'd like more design details we are partners, have NDA, and you can contact me at the email listed in my profile.

[loverdeg]

@kjbracey-arm Regardless of my needs is this something that should be unmarked as resolved in jira?

[cmonr]

Looks like that's either a @hasnainvirk or @mhiljane question.

To the two tagged, mind mentioning why this is internally marked as a "won't fix"?

Looping in @ARMmbed/mbed-os-hal since they were mentioned.

[loverdeg]

Still think this needs a resolution.

@snelson-senet Do you have any thoughts on the topic?

@janjongboom, @linlingao

[ciarmcom]

Thank you for raising this issue. Please note we have updated our policies and
now only defects should be raised directly in GitHub. Going forward questions and
enhancements will be considered in our forums, https://forums.mbed.com/ . If this
issue is still relevant please re-raise it there.
This GitHub issue will now be closed.