Zephyr: BLE throughput of DFU by Mcumgr is too slow

[icephyr]

Is your enhancement proposal related to a problem? Please describe.
Hi guys, I met a problem about the DFU of BLE Chip, hope anyone can help on this issue, thanks.

I tried to upgrade the firmware of nRF52832 running zephyr project, with Mcumgr, the tool under linux. The Mcumgr communicates with BLE chip by BLE connection. The firmware can be successfully transferred via BLE, as shown below:

However，the transmission speed is really too slow, only 2.4 KiB/s as shown in the picture above, and it will cost 1mintue and 3 seconds to transfer the whole firmware file, which will be a disaster when I need to upgrade many many BLE devices.
So I wonder if any methods can help to speed up the throughput of BLE transmission in DFU process. By the way, the zephyr project version I am using is V1.13, not the LTS version yet. Has the problem been resolved in the lastest newest branch ? I will have a try if so.

Thanks.

[aescolar]

@carlescufi Can you help @icephyr ?

[bfrog]

@icephyr There are a few Kconfig tweaks you can try to improve BLE transfers in general, not sure if you tried increasing the number and size of the BLE RX bufs and setting the MTU higher but those can change things drastically. It still has to write to flash though, there's no avoiding that.

[carlescufi]

@icephyr there are a lot of factors that can influence that speed. Among them there will also be your Linux host settings, as well as the Kconfig options that were mentioned by @bfrog. There is a throughput sample that you can try as a starting point for configuring the BLE stack for throughput in here.

[icephyr]

Thanks you guys.
As @bfrog and @carlescufi mentioned, I have not tried increasing the number and size of the BLE RX bufs and setting the MTU higher yet, I will have a try first.
So increase the number and size of the BLE RX bufs just on the slave side, while as for setting the MTU higher, the MTU need to be set on both slave side and master side, if I did't misunderstand what you means, right ?

[icephyr]

@icephyr there are a lot of factors that can influence that speed. Among them there will also be your Linux host settings, as well as the Kconfig options that were mentioned by @bfrog. There is a throughput sample that you can try as a starting point for configuring the BLE stack for throughput in here.

Hi @carlescufi , I add the config as you mentioned, but it didn't seem to work, the throughput is about 2.5KB/s.

the configs I added are :

CONFIG_HEAP_MEM_POOL_SIZE=2048
CONFIG_BT_RX_BUF_LEN=258
CONFIG_BT_GATT_CLIENT=y
CONFIG_BT_ATT_TX_MAX=10
CONFIG_BT_ATT_PREPARE_COUNT=2
CONFIG_BT_CONN_TX_MAX=10
CONFIG_BT_L2CAP_TX_BUF_COUNT=10
CONFIG_BT_L2CAP_TX_MTU=247
CONFIG_BT_L2CAP_DYNAMIC_CHANNEL=y
CONFIG_BT_CTLR_PHY=y
CONFIG_BT_CTLR_PHY_2M=y
CONFIG_BT_CTLR_RX_BUFFERS=2
CONFIG_BT_CTLR_TX_BUFFERS=10
CONFIG_BT_CTLR_TX_BUFFER_SIZE=251
CONFIG_BT_CTLR_DATA_LENGTH_MAX=251
CONFIG_BT_CTLR_ADVANCED_FEATURES=y
CONFIG_BT_CTLR_XTAL_THRESHOLD=500000

[icephyr]

Update what I have tried.

I add the configs you mentioned above, on zephyr V1.13.0 version, but it seems not work. The througput is still about 2.5KB/s. I'm not sure if it is normal.
Then I realized that maybe stopping the applications(mainly scanning) before the DFU starts will help on speed up the througput. So I stopped the scanning behaviour of BLE chip before the file transportation by mcumgr, then the throughput reached 5.6 KB/s.
I need more test on the throughput optimization issue and will continue to update the results, thanks you guys.

[carlescufi]

@nwsetec could you please add your observations here, from the email thread?

[nwsetec]

Link to Zephyr mailing lists thread: https://lists.zephyrproject.org/g/users/topic/34540601#1741

After turning off scanning, reducing the BLE connection interval, increasing receive buffer sizes and enabling dynamic length extensions (DLE) I had a similar upper DFU speed of roughly 6kB/s.

I did some packet sniffing logs and shared with @cvinayak and he confirmed that BLE connection was not the bottleneck. @cvinayak also shared a video of Zephyr to Zephyr throughput of ~1.3Mbit/s (162kB/s)

I also spoke with @bgiori (mcumgr Android and iOS libraries) and he confirmed that it's the protocol's round trip time that is likely the current limit on the speed of the DFU upload, i.e. new data is only sent after the DFU sender receives the latest offset response from the DFU receiver.

Also notable is the nRF devices halt the CPU during a flash write which stops BLE use during this time, so DFU's to SoC/internal flash will have somewhat reduced BLE throughput but DFU throughput achievable should still likely be much higher than 6kB/s.

[icephyr]

Thanks you guys, I agree that the upper limit of DFU throughput should still likely be much higher than 6kB/s. I think it's a big issue, since such rate will take very long, especially for massive firmware upgrade, like BLE Mesh OTA. Hope this issue can be solved finally. I will study on the mcumgr round trip mechanism first and will share my results later.

[vikrant8051]

Hi @nwsetec,

After turning off scanning, reducing the BLE connection interval, increasing receive buffer sizes and enabling dynamic length extensions (DLE) I had a similar upper DFU speed of roughly 6kB/s.

Could you please guide me what exact configuration Parameters we have to set in context of samples/boards/nrf52/mesh/onoff_level_lighting_vnd_app to achieve 6KB+ throughput ?
(This app supports out of the box DFU if build with prj_smp_svr.conf)

[nwsetec]

Hi @vikrant8051,

Apologies for the late reply.

I basically checked my project configuration against this Nordic project sample:
https://github.com/NordicPlayground/fw-nrfconnect-nrf/blob/master/samples/bluetooth/throughput/prj.conf#L28

In addition this is with BLE scanning turned off. If your mesh device is configured as a repeater I'm not sure how you will turn off the listening/scanning for mesh packets (if DFU speed optimisation is more important). I also used the smallest connection interval of 7.5ms but note that this (unfortunately) only worked for some connecting centrals. From memory another connecting v4.0 central device achieved a similar result with only a 15ms connection interval.

[Flo3561]

Hi there,

Are there any new findings? e.g. with new releases 2.0.0 or 2.1.0?

[lukasbrchl]

Hi guys, does anyone have new findings on this issue?

[cvinayak]

I am not an expert on the DFU protocol, but the BLE controller is capable of doing upto ~1.3Mbps of GATT Write commands. With a 1Mbps UART, and features provided in this PR https://github.com/zephyrproject-rtos/zephyr/pull/28089, a throughput of ~690 Kbps should be achievable.

[DMotocus]

@carlescufi Can you help @icephyr ?

⚰️

[carlescufi]

@carlescufi Can you help @icephyr ?

⚰️

There's not that much I can add to the existing thread @DMotocus. It ultimately all boils down to connection interval, GATT MTU and whether DLE and 2Mbit/s PHY is enabled or not. Since we are using a 3rd-party stack to send the image there are too many variables out of our control.

[lukasbrchl]

We were able to get only 1kB/s throughput from common Android smartphones and mcumgr-android library (or nRF Device Manager if you prefer). If anyone finds out how to make it faster, we would be glad to implement it in our solution.

[bgiori]

Hi all, we recently started an effort to improve the image upload speeds by updating the mcumgr client side to support sending multiple concurrent requests. Allowing even 2 concurrent requests has significantly improved upload speeds. Unfortunately I don't have anything concrete about the max upload speeds this new client can achieve, but it should be better than the current send-and-wait regardless.

The original improvement to the newtmgr (mcumgr) cli tool is here: https://github.com/apache/mynewt-newtmgr/pull/162

I am currently working on an implementation for mcumgr-android and eventually mcumgr-ios.

[bgiori]

My initial implementation on android is getting ~24kB/s without any connection parameter tuning besides setting the att mtu to 515

[github-actions[bot]]

This issue has been marked as stale because it has been open (more than) 60 days with no activity. Remove the stale label or add a comment saying that you would like to have the label removed otherwise this issue will automatically be closed in 14 days. Note, that you can always re-open a closed issue at any time.