Zigbee2mqtt: Update_available on devices which already up to date

[b-reich]

What happened

My TRADFRI bulb E14 WS 470 reports "update_available": true

What did you expect to happen

They should report "update_available": false

How to reproduce it (minimal and precise)

zigbee2mqtt/bridge/ota_update/update  with payload "Esszimmer Lampe1"

{"message":"Esszimmer Lampe1","meta":{"device":"Esszimmer Lampe1","from":{"date_code":"20190312","software_build_id":"2.1.022"},"status":"update_succeeded","to":{"date_code":"20190312","software_build_id":"2.1.022"}},"type":"ota_update"}

Debug info

Zigbee2MQTT version: 1.15.0 on docker
Adapter hardware: CC2531
Adapter firmware version: 20190608

[b-reich]

After the update its still reports "update_available": true

[fr4ncis3]

Zigbee2MQTT version: 1.15.0 on Rpi3
Adapter hardware: CC2531
Adapter firmware version: 20190608

Same behaviour here for a:

IKEA TRÅDFRI LED Bulb E27 806 lumen
Code LED1836G9
Ikea code 904.087.97

https://www.zigbee2mqtt.io/devices/LED1836G9.html

Pretty new to OTA updates with zigbee2mqtt. Tried Tradfri pruducts only.
Always the same result as described by the OP.

Regards,
fr4ncis3

[TilmanK]

Can you share the debug log when requesting the update check?

[fr4ncis3]

Here is mine.

Client mosqsub|30566-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/bridge/ota_update/check', ... (12 bytes)) tradfri_bulb Client mosqsub|30566-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/bridge/log', ... (147 bytes)) {"message":"Checking if update available for 'tradfri_bulb'","meta":{"device":"tradfri_bulb","status":"checking_if_available"},"type":"ota_update"} Client mosqsub|30566-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/bridge/log', ... (123 bytes)) {"message":"Update available for 'tradfri_bulb'","meta":{"device":"tradfri_bulb","status":"available"},"type":"ota_update"} Client mosqsub|30566-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/tradfri_bulb', ... (58 bytes)) {"brightness":127.5,"state":"OFF","update_available":true}

Hope this helps

Regards,
fr4ncis3

[b-reich]

```
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:52: Received MQTT message on 'zigbee2mqtt/bridge/ota_update/check' with data 'Esszimmer Lampe1'
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:52: Checking if update available for 'Esszimmer Lampe1'
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:52: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Checking if update available for 'Esszimmer Lampe1'","meta":{"device":"Esszimmer Lampe1","status":"checking_if_available"},"type":"ota_update"}'
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:52: Check if update available for '0x842e14fffe6680d8' (TRADFRI bulb E14 WS 470lm)
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:52: Using endpoint '1'
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:52: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:52: Got OTA request '{"fieldControl":1,"manufacturerCode":4476,"imageType":16901,"fileVersion":553788977}'
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:53: Is new image available for '0x842e14fffe6680d8', current '{"fieldControl":1,"manufacturerCode":4476,"imageType":16901,"fileVersion":553788977}', latest meta '{"fileVersion":587531825,"url":"http://fw.ota.homesmart.ikea.net/Tradfri_OTA_release_signed_2020_09_08_141305/bin/10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed"}'
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:53: Update available for '0x842e14fffe6680d8': YES
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:53: Update available for 'Esszimmer Lampe1'
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:53: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Update available for 'Esszimmer Lampe1'","meta":{"device":"Esszimmer Lampe1","status":"available"},"type":"ota_update"}'
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:53: MQTT publish: topic 'zigbee2mqtt/Esszimmer Lampe', payload '{"brightness":1,"color":{"x":0.4599,"y":0.4106},"color_temp":454,"state":"OFF"}'
zigbee2mqtt | Zigbee2MQTT:info 2020-10-05 16:56:53: MQTT publish: topic 'zigbee2mqtt/Esszimmer Lampe1', payload '{"brightness":1,"color":{"x":0.4599,"y":0.4106},"color_temp":454,"linkquality":42,"state":"OFF","update_available":true}'
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:55: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:56:58: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:57:01: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
zigbee2mqtt | Zigbee2MQTT:debug 2020-10-05 16:57:04: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
````

[ZenSmartHome]

The same issue here with TRADFRI bulb E27 WW clear 250lm
Current version: 2.1.022
Available version: 2.3.050
The OTA update went smoothly as per the logs, but nothing changed at the end, the bulb still using the old firmware
Zigbee2MQTT:debug 2020-10-05 08:15:46: Received MQTT message on 'zigbee2mqtt/bridge/ota_update/update' with data 'Table Lamp 2' Zigbee2MQTT:info 2020-10-05 08:15:46: Updating 'Table Lamp 2' to latest firmware Zigbee2MQTT:info 2020-10-05 08:15:46: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"meta":{"device":"Table Lamp 2","status":"update_in_progress"},"msg":"Updating 'Table Lamp 2' to latest firmware","type":"ota_update"}' Zigbee2MQTT:debug 2020-10-05 08:15:46: Received Zigbee message from 'Table Lamp 2', type 'readResponse', cluster 'genBasic', data '{"dateCode":"20181203","swBuildId":"2.1.022"}' from endpoint 1 with groupID 0 Zigbee2MQTT:debug 2020-10-05 08:15:46: Updating to latest '0x14b457fffe4a6d1e' (TRADFRI bulb E27 WW clear 250lm) Zigbee2MQTT:debug 2020-10-05 08:15:46: Using endpoint '1' Zigbee2MQTT:debug 2020-10-05 08:15:46: Received Zigbee message from 'Table Lamp 2', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16643,"manufacturerCode":4476}' from endpoint 1 with groupID 0 Zigbee2MQTT:debug 2020-10-05 08:15:46: Got OTA request '{"fieldControl":1,"manufacturerCode":4476,"imageType":16643,"fileVersion":553788977}' Zigbee2MQTT:debug 2020-10-05 08:15:47: getNewImage for '0x14b457fffe4a6d1e', meta {"fileVersion":587531825,"url":"http://fw.ota.homesmart.ikea.net/Tradfri_OTA_release_signed_2020_09_08_141305/bin/10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed"} Zigbee2MQTT:debug 2020-10-05 08:15:48: getNewImage for '0x14b457fffe4a6d1e', image header {"otaUpgradeFileIdentifier":{"type":"Buffer","data":[30,241,238,11]},"otaHeaderVersion":256,"otaHeaderLength":56,"otaHeaderFieldControl":0,"manufacturerCode":4476,"imageType":16643,"fileVersion":587531825,"zigbeeStackVersion":2,"otaHeaderString":"GBL GBL_tradfri_light_unified_w\u0000","totalImageSize":203378} Zigbee2MQTT:debug 2020-10-05 08:15:48: Got new image for '0x14b457fffe4a6d1e' Zigbee2MQTT:debug 2020-10-05 08:15:48: Starting upgrade Zigbee2MQTT:debug 2020-10-05 08:15:48: Received Zigbee message from 'Table Lamp 2', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16643,"manufacturerCode":4476}' from endpoint 1 with groupID 0 Zigbee2MQTT:debug 2020-10-05 08:15:56: OTA update at 0%, remaining Infinity seconds Zigbee2MQTT:info 2020-10-05 08:15:56: Update of 'Table Lamp 2' at 0.00%

Zigbee2MQTT:debug 2020-10-05 08:36:17: Got upgrade end request for '0x14b457fffe4a6d1e': {"status":0,"manufacturerCode":4476,"imageType":16643,"fileVersion":587531825} Zigbee2MQTT:debug 2020-10-05 08:36:17: Update succeeded, waiting for device to restart

[Koenkk]

Can you try switching to the latest dev branch and try again? (https://www.zigbee2mqtt.io/how_tos/how-to-switch-to-dev-branch.html). @PedroLamas has implemented an integrity check for IKEA OTA files, I'm wondering wether these IKEA OTA files pass this check.

[b-reich]

I try the latest (latest-dev) container

zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:19: Check if update available for '0x842e14fffe6680d8' (TRADFRI bulb E14 WS 470lm)
zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:19: Using endpoint '1'
zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:21: Received Zigbee message from 'Esszimmer Lampe1', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16901,"manufacturerCode":4476}' from endpoint 1 with groupID 0
zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:21: Got OTA request '{"fieldControl":1,"manufacturerCode":4476,"imageType":16901,"fileVersion":553788977}'
zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:22: Is new image available for '0x842e14fffe6680d8', current '{"fieldControl":1,"manufacturerCode":4476,"imageType":16901,"fileVersion":553788977}', latest meta '{"fileVersion":587531825,"url":"http://fw.ota.homesmart.ikea.net/Tradfri_OTA_release_signed_2020_09_08_141305/bin/10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed"}'
zigbee2mqtt    | Zigbee2MQTT:debug 2020-10-05 20:03:22: Update available for '0x842e14fffe6680d8': YES
zigbee2mqtt    | Zigbee2MQTT:info  2020-10-05 20:03:22: Update available for 'Esszimmer Lampe1'
zigbee2mqtt    | Zigbee2MQTT:info  2020-10-05 20:03:22: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Update available for 'Esszimmer Lampe1'","meta":{"device":"Esszimmer Lampe1","status":"available"},"type":"ota_update"}'

[pedrolamas]

FYI, I've downloaded all IKEA firmware files they have on that json file and used the new CRC32 check I added to verify them all, these ones failed:

• 10005777-6.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed
• 10005778-10.1-TRADFRI-onoff-shortcut-control-2.2.010.ota.ota.signed
• 10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.029.ota.ota.signed
• 10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed
• 10043101-3.1-TRADFRI-dimmer-2.1.024.ota.ota.signed
• 10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed
• 10047227-1.2-TRADFRI-cv-cct-unified-2.3.050.ota.ota.signed
• 191100-4.1-TRADFRI-sy5882-driver-ws-2.0.029.ota.ota.signed

There's a total of 23 files, the above shows 8 files that failed the CRC32 check and I marked the one you are trying to flash.

I'm now verifying if my code is correct or if I made a mistake here!

[MattWestb]

@PedroLamas All in your list is the "new" old OTA files that is all broken (corrupted on the server).
Use the test url "http://fw.test.ota.homesmart.ikea.net/feed/version_info.json" for getting the working "old" one (have the same metadata but working).

[pedrolamas]

@MattWestb I downloaded all files from the test url, passed them through the new validation and these are the ones that failed:

• 10005777-4.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed
• 10005778-10.1-TRADFRI-onoff-shortcut-control-2.2.010.ota.ota.signed
• 10032198-2.1-TRADFRI-gateway-1.11.51.elf.sig.ota.signed
• 10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.029.ota.ota.signed
• 10040611-3.2-TRADFRI-sy5882-unified-2.3.050.ota.ota.signed
• 10043101-3.1-TRADFRI-dimmer-2.1.024.ota.ota.signed
• 10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed
• 10047227-1.2-TRADFRI-cv-cct-unified-2.3.050.ota.ota.signed
• 191100-4.1-TRADFRI-sy5882-driver-ws-2.0.029.ota.ota.signed

[pedrolamas]

Not that it will help much, but by using a Hex editor to manually debug those files, I found out that the EBL End Tag (0xfc 0x4 0x0 0x4 followed by 4 bytes for the CRC32) is on the file but only after the Total Length indicated on the OTA header...

The last EBL sub-element is currently being truncated (the size indicated is 2052, yet that would take it over the available data on the OTA sub-element, so all it gets to read is just 1752)

[MattWestb]

Ok then IKEA have "updated" the test branch with the "new" files (was working for one week ago) and asllo one new GW firmware (51).
Its make no sense way its not standard format of the OTA files and its not helping the users if not having deCONZ and have downloaded the old version of the files (I have all old saved in deCONZ so no problem for my) but all of the "new" is not working on the "normal" devices.
If its only the end that is no standard and the "firmware" is intact i wonder if IKEA have start using Simplicity commander 5 and is one bug in the OTA making process in it.

[pedrolamas]

Further checks confirmed my suspicion: the full EBL data is in the upgrade file and can be verified by looking at the CRC32:

As shown in the image, the EBL data total length is 208468, however the OTA sub-element only reports a length of 207376 (so there's 1092 bytes after the declared end of the image).

Bottom line is that there's a bug somewhere, but I don't believe it is on our side or something we can (or should) fix.

[Koenkk]

Bottom line is that there's a bug somewhere, but I don't believe it is on our side or something we can (or should) fix.

I agree with this. I expect that OTA updates of firmwares listed here: https://github.com/Koenkk/zigbee2mqtt/issues/4559#issuecomment-703937390 should even fail when done via the TRADFRI gateway (or they apply some hacky magic there). Lets see if the issue is fixed once IKEA updates their firmwares again.

[MattWestb]

@PedroLamas Is it some tags that pointing the that airia after the normal OTA elements (but before then EBL end tag or the last CRC32) ?
It is possible updating more memory regions / elements with GBL file but not at the same time if its one bootloader image and also the lock bit / HW setting and user data airia thru patching.
Then IKEAs contractor / manufacturer have done false data setting in some series of devices (device name was truncated in the "userdata" memory airia) its can being patch information that is burned to the device user airia (patch command is documented in the gecko bootloader, simplicity commander and manufacturing guidelines manuals).

One possible updating element is the signing token / cert in the devices if they is changing it its must being done thru patching.

As long the devices dont accepting the "new" OTA files its no reason sending then the the devices.

Interesting if the IKEAs GW is doing some magic and upgrading the devices with the new OTA images.
Perhaps I should restoring one ICC-1-A module with original firmware and data setting from one GU10 WW 2.0 and pairing it with the IKEA GW and see if it updating it and then dumping it and looking how its looking after being upgraded.

[pedrolamas]

@MattWestb Here's how it works on that specific file:

• OTA image data MUST start with the bytes 0x1E, 0xF1, 0xEE, 0x0B, that is found on offset 456
• if from that point you offset 52 (so 508 from the beginning) and read the next 4 bytes, you get the OTA Total Image Size field value and that is 207438 for this file.

If you open this document and search for "11.4.2.10" referring to the Total Image Size, you will see this:

The value represents the total image size in bytes. This is the total of data in bytes that SHALL be transferred over the-air from the server to the client

What this means is that this is the amount of data that the coordinator should send to the device, yet I now know for a fact that the EBL data is 208468 and that is larger than the declared OTA Total Image Size field.

[tabnul]

Comparible issue here with tradfri outlet.
Device version: 2.0.022
Available update: 2.0.024

Although it states it will update to latest;

debug 2020-10-05 19:26:21: getNewImage for '0x588e81fffe656a24', meta {"fileVersion":537019939,"url":"http://fw.ota.homesmart.ikea.net/Tradfri_OTA_release_signed_2020_09_08_141305/bin/10005777-6.1-TRADFRI-control-outlet-2.0.024.ota.ota.signed"}

It updates to the same version. strange behaviour.

Finished update of 'router_gang', from '{"dateCode":"20190308","softwareBuildID":"2.0.022"}' to '{"dateCode":"20190308","softwareBuildID":"2.0.022"}'

[b-reich]

I understand why the OTA fail. But why is an update offered although this is the same version?

[pedrolamas]

@Koenkk @MattWestb today I downloaded all the firmware files from the IKEA test url, ran them through my new ebl/gbl parser (WIP here) and they all passed! :)

There are quite a few GBL format ones there now!

[sliwma]

Same issue with two LED1736G9 bulbs.
Zigbee2MQTT:info 2020-10-12 22:39:42: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Bulb-salon-main1","meta":{"device":"Bulb-salon-main1","from":{"date_code":"20190312\r\n","software_build_id":"2.1.022"},"status":"update_succeeded","to":{"date_code":"20190312\r\n","software_build_id":"2.1.022"}},"type":"ota_update"}'
Zigbee2MQTT version: 1.15.0
Adapter hardware: CC2531
Adapter firmware version: 20190608

[fedebruni84]

Same issue with two LED1736G9 bulbs.
Zigbee2MQTT:info 2020-10-12 22:39:42: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Bulb-salon-main1","meta":{"device":"Bulb-salon-main1","from":{"date_code":"20190312\r\n","software_build_id":"2.1.022"},"status":"update_succeeded","to":{"date_code":"20190312\r\n","software_build_id":"2.1.022"}},"type":"ota_update"}'
Zigbee2MQTT version: 1.15.0
Adapter hardware: CC2531
Adapter firmware version: 20190608

Same here with on/off button E1743

{"message":"announce","meta":{"friendly_name":"ikea_tradfri_switch02"},"type":"device_announced"}
{"message":"Update of 'ikea_tradfri_switch02' at 100.00%","meta":{"device":"ikea_tradfri_switch02","progress":100,"status":"update_progress"},"type":"ota_update"}
{"message":"ikea_tradfri_switch02","meta":{"device":"ikea_tradfri_switch02","from":{"date_code":"20190410","software_build_id":"2.2.008"},"status":"update_succeeded","to":{"date_code":"20190410","software_build_id"   :"2.2.008"}},"type":"ota_update"}

[fbr1969]

Same issue with TRADFRI control outlet E1603/E1702

INFO: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"HA_IKEA_Prise01","meta":{"device":"HA_IKEA_Prise01","from":{"date_code":"20190308","software_build_id":"2.0.022"},"status":"update_succeeded","to":{"date_code":"20190308","software_build_id":"2.0.022"}},"type":"ota_update"}'
INFO: Update available for 'HA_IKEA_Prise01' INFO: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Update available for 'HA_IKEA_Prise01'","meta":{"device":"HA_IKEA_Prise01","status":"available"},"type":"ota_update"}'

Zigbee2MQTT version: 1.15.0
Adapter hardware: CC2531
Adapter firmware version: 20190608

[DenisBY]

Here is another error:

debug 2020-10-23 18:58:41: Received MQTT message on 'zigbee2mqtt/bridge/ota_update/update' with data '0x680ae2fffe355e1e'
info  2020-10-23 18:58:41: Updating '0x680ae2fffe355e1e' to latest firmware
info  2020-10-23 18:58:41: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"meta":{"device":"0x680ae2fffe355e1e","status":"update_in_progress"},"msg":"Updating '0x680ae2fffe355e1e' to latest firmware","type":"ota_update"}'
debug 2020-10-23 18:58:41: Received Zigbee message from '0x680ae2fffe355e1e', type 'readResponse', cluster 'genBasic', data '{"dateCode":"20181203","swBuildId":"2.1.022"}' from endpoint 1 with groupID null
debug 2020-10-23 18:58:41: Updating to latest '0x680ae2fffe355e1e' (TRADFRI bulb E27 WW 806lm)
debug 2020-10-23 18:58:41: Using endpoint '1'
debug 2020-10-23 18:58:43: Received Zigbee message from '0x680ae2fffe355e1e', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":553788977,"imageType":16643,"manufacturerCode":4476}' from endpoint 1 with groupID null
debug 2020-10-23 18:58:43: Got OTA request '{"fieldControl":1,"manufacturerCode":4476,"imageType":16643,"fileVersion":553788977}'
debug 2020-10-23 18:58:44: getNewImage for '0x680ae2fffe355e1e', meta {"fileVersion":587531825,"url":"http://fw.ota.homesmart.ikea.net/global/GW1.0/01.11.051/bin/10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed"}
debug 2020-10-23 18:58:44: getNewImage for '0x680ae2fffe355e1e', image header {"otaUpgradeFileIdentifier":{"type":"Buffer","data":[30,241,238,11]},"otaHeaderVersion":256,"otaHeaderLength":56,"otaHeaderFieldControl":0,"manufacturerCode":4476,"imageType":16643,"fileVersion":587531825,"zigbeeStackVersion":2,"otaHeaderString":"GBL GBL_tradfri_light_unified_w\u0000","totalImageSize":203378}
info  2020-10-23 18:58:44: MQTT publish: topic 'zigbee2mqtt/0x680ae2fffe355e1e', payload '{"state":"ON","update_available":true}'
info  2020-10-23 18:58:44: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Update of '0x680ae2fffe355e1e' failed (Image is truncated: not long enough to contain a valid tag)","meta":{"device":"0x680ae2fffe355e1e","status":"update_failed"},"type":"ota_update"}'
error 2020-10-23 18:58:44: Update of '0x680ae2fffe355e1e' failed (Image is truncated: not long enough to contain a valid tag)

I see "totalImageSize":203378 however ls -l reports 205446 bytes of downloaded file.

[pedrolamas]

10046695-1.1-TRADFRI-light-unified-w-2.3.050.ota.ota.signed

@DenisBY This is part of the list I posted above of IKEA images with errors.

These should also be failing with the official IKEA bridge (unless they are using some non-standard process to upgrade these devices)

The IKEA beta images are all ok, so once those IKEA images are released outside of beta, this issue should go away.

[murjam]

The IKEA beta images are all ok, so once those IKEA images are released outside of beta, this issue should go away.

@pedrolamas Where do you find the beta images? Is there a way to feed those to the device (i.e by manually swapping the content/URLs in zigbee-herdsman-converters)?
I happen to have a E1743 that eats the battery within a few days, would love to upgrade the FW.

[MattWestb]

@murjam The test url have the same version of all devices only the NCP image for the gateway is newer but normally we do not flashing that from Z2M ;-))
Also its no new on the way then looking on zigbee alliance certifications portal.

[MattWestb]

@murjam If you have some spare dongle that is working with ZHA you can do one fast install with docker image and upgrading it from it only need putting the test url in the config of ZHA :-))

zha:
  zigpy_config:


    ota:
      otau_directory: /config/zigpy_ota
      ikea_provider: true
      ikea_update_url: http://fw.test.ota.homesmart.ikea.net/feed/version_info.json

Or you can downloading the OTA file and putting it in the /config/zigpy_ota directory.

[fr4ncis3]

Tried again today with the new Zigbee2MQTT release.

Zigbee2MQTT version: 1.16.0 on Rpi3
Adapter hardware: CC2531
Adapter firmware version: 20190608

and

IKEA TRÅDFRI LED Bulb E27 806 lumen
Code LED1836G9
Ikea code 904.087.97

https://www.zigbee2mqtt.io/devices/LED1836G9.html

I get the following json payload back:

{"message":"Update of 'tradfri_bulb' failed (Image is truncated: not long enough to contain a valid
tag)","meta":{"device":"tradfri_bulb","status":"update_failed"},"type":"ota_update"}

Regards,
fr4ncis3

[pedrolamas]

@fr4ncis3 that error message is correct, as that image is part of the list I posted above of IKEA images with errors.

It's up to IKEA to deploy a fix for that.

[pedrolamas]

The test url have the same version of all devices only the NCP image for the gateway is newer but normally we do not flashing that from Z2M ;-))

@MattWestb correct, they have the same version but seem to have different content!

Some of the images on the production feed are failing (this list), but the same ones on the beta feed are ok!

One can even notice they are different just by checking their size:

[MattWestb]

I dont have the problems then all my devices is up to date before the "server problems" in the cloud (and also some is manual "downgrade" for testing purpose).
I can also using the IKEA GW, deCONZ and ZHA for doing OTA updates that is working with all IKEA devices.

But i finding the question from @murjam is very relevant for "normal users".
If it working changing the server URL in the code to using the test url.
If doing it in the release code it wold not being big issue then the IKEA test feed is stabile for the moment (no mutch new betat coming in the test stream) and Z2M have one short release cycle for reversing it then the problems is solver on the server side.
If not liking changing the URL then its better writing how to do that in the local install so the users can getting there devices updated and dont being frustrated then its not working in Z2M and running for deCONZ (That is the worse they can do for the moment) ;-))

I finding the ZHA solution is great with the possible configuring alternative URL for the OTA json and also using local file (with proposed version override for firmware downgrading).

Edit: Or upload the 4 broke files to the Z2M OTA reptory and making it override the "IKEA OTA" and deleting the 4 files then the problem is solved on the feed.

[MattWestb]

@murjam Test changing the

const url = 'http://fw.ota.homesmart.ikea.net/feed/version_info.json'; 

to

const url = 'http://fw.test.ota.homesmart.ikea.net/feed/version_info.json';

in your zigbee-herdsman-converters/ota/tradfri.js
If its working then its great :-)))

[fr4ncis3]

I have modified as suggested the tradfri.js file

And tested the update for an IKEA control outlet:

IKEA E1603/E1702
Model E1603/E1702
Vendor IKEA
Description TRADFRI control outlet

https://www.zigbee2mqtt.io/devices/E1603_E1702.html

"message":"Update of 'control_outlet' at 98.67%, +- 1 minutes remaining","meta":{"device":"control_outlet","progress":98.67,"status":"update_progress"},"type":"ota_update"}
Client mosqsub|28575-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/bridge/log', ... (90 bytes))
{"message":"announce","meta":{"friendly_name":"control_outlet"},"type":"device_announced"}
Client mosqsub|28575-openhab received PUBLISH (d0, q0, r0, m0, 'zigbee2mqtt/control_outlet', ... (57 bytes))
{"linkquality":31,"state":"OFF","update_available":false}

No more update available now. :-)

Regards,
fr4ncis3

[Routout]

Also problems with Philips HUE 4-Button dimmer switch. I'ts showing upgrade all the time, but won't update. I'm using Zigbee2mqtt Edge branch.

INFO: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":{"id":"0x001788010874da2e"},"meta":{"device":"0x001788010874da2e","from":{"date_code":"20160302","software_build_id":"5.45.1.17846"},"status":"update_succeeded","to":{"date_code":"20160302","software_build_id":"5.45.1.17846"}},"type":"ota_update"}'
INFO: MQTT publish: topic 'zigbee2mqtt/bridge/response/device/ota_update/update', payload '{"data":{"from":{"date_code":"20160302","software_build_id":"5.45.1.17846"},"id":"0x001788010874da2e","to":{"date_code":"20160302","software_build_id":"5.45.1.17846"}},"status":"ok"}'
INFO: MQTT publish: topic 'zigbee2mqtt/0x001788010874da2e', payload '{"battery":100,"brightness":159,"counter":1,"linkquality":70,"update":{"state":"available"},"update_available":true}'
INFO: Update available for '0x001788010874da2e'
INFO: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"Update available for '0x001788010874da2e'","meta":{"device":"0x001788010874da2e","status":"available"},"type":"ota_update"}'