Ikea Trådfri firmware updates

NilsOF · January 18, 2022, 8:49pm

I while back I wrote a crude Python script to extract the final firmware to send as OTA from the Ikea download.
But I did not succeed, the devices did not want my extracted ‘blob’: Later I read there was some file corruption on Ikeas side.

Last week we had a lot of power outages. That made me do a lot of resetting and reregistration with those pesky Ikea devices.

So seeing this: RELEASE NOTES

Solves an issue where product become unreachable after a gateway restart.

I decided to give it a new round.

Posting my notes here for further comments and ideas

The Python script can be found here: GitHub - NilsOF/ikea-fw: Crude Zigbee Firmware extractor for IKEA
It is very simple:

If it get a argument that contains “version_info.json” it just print out the json info in a bit more human readable way.
if it get a argument that contains “.ota” it will look for a file and try to exctract the OTA bit.
when it finds a contained OTA, it extracts the OTA and just append “.ota” to the argument as a file name.
Extraction is done this way: search for 0x0BEEF11E in the orginal Ikea binary download, This is the OTA header identificator and the start of the binary to be extracted.
The length of the OTA binary is then lifted out from the assumed to be valid OTA header.

Get a -json file to work on:

wget http://fw.ota.homesmart.ikea.net/feed/version_info.json

In openhab-console, find the device that you want to update and get the “Image Type ID”:

openhab> zigbee nodes
Total known nodes in network: 2
Network  Addr  IEEE Address      Logical Type  State      EP   Profile                    Device Type                Manufacturer     Model          
      0  0000  000D6F000D3AF64C  COORDINATOR   ONLINE   
  25543  63C7  086BD7FFFE1AFDDC  ROUTER        ONLINE      1  ZIGBEE_HOME_AUTOMATION     ON_OFF_PLUG_IN_UNIT        IKEA of Sweden   TRADFRI control outlet
                                                         242  ZIGBEE_GREEN_POWER         0061                                                       

openhab> zigbee read 25543/1  25 8
Reading endpoint 63C7/1, cluster client cluster Ota Upgrade (0019), attributes Image Type ID
Response for cluster 0x0019
Attribute    8  Image Type ID                                       UNSIGNED_16_BIT_INTEGER         4353

openhab>

run the python script with the .json file as argument:

./ikeafw.py version_info.json

Search the printout to find the ImageTypeID (in this case : 4353)

fw_binary_url : http://fw.ota.homesmart.ikea.net/global/GW1.0/01.17.019/bin/10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed
fw_file_version_LSB : 38449
fw_file_version_MSB : 8968
fw_filesize : 209136
fw_image_type : 4353
fw_manufacturer_id : 4476
fw_type : 2

Download the binary and extract the OTA:

wget http://fw.ota.homesmart.ikea.net/global/GW1.0/01.17.019/bin/10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed

./ikeafw.py 10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed

the resulting file ( 10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed.ota ) can now be copyed to a place where openhab can read it. ( /var/lib/openhab )

in openhab-console see what the binding thinks about the file and start the OTA procedure.

openhab> zigbee otaupgrade file 10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed.ota
OTA File: ZigBeeOtaFile [headerVersion=256, manufacturerCode=4476, imageType=1101, fileVersion=23089631, stackVersion=ZIGBEE_PRO, headerString=GBL GBL_tradfri_control_outlet, imageSize=209136]

openhab> zigbee otaupgrade start 25543/1 10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed.ota
OTA File "10005777-TRADFRI-control-outlet-2.3.089.ota.ota.signed.ota" set.

openhab>

( imageType=1101 is hex for 4353 decimal )

I so far managed to upgrade the firmware on the control outlet in the notes above and 5 of my most anoying pesky bulbs.

What I notice is that “zigbee_datecode” seen in UI is not updated, but firmwareVersion is.

10 more pesky Ikea bulbs to go. It is tedious…

I really wish I could use the “Zigbee OTA Firmware Provider” described in ths thread: Zigbee OTA Firmware Provider

or mybe I can?
Extending my crude python to download, extract and pack up ready for the OTA firmware provider is something I can do, even with my limited programming skils.

m0rgoth · January 20, 2022, 8:09pm

Interesting, I have a bunch of IKEA devices which I’d like to update. Looking at that OTA firmware provider, I think it wouldn’t be too hard to modify it so that it would download firmware from IKEA website, unpack it and start firmware update automatically. I’ll look into that when I have some time.

mstormi · January 20, 2022, 9:16pm

Should you succeed please consider contributing this to openHABian so everybody can benefit. Thanks.

chris · January 20, 2022, 9:25pm

If it’s added to the firmware provider, then openHABian should be able to take this, but it should “just” be another bundle - that’s what the firmware provider is.

NilsOF · January 21, 2022, 9:58am

What is missing around the (current existing) OTA firmware provider to work under OH 3.x?
I guess some work must be done to the UI to show a “update firmware” button?
A way to initiate the firmware provider OTA from the console would be a good start

The script just search for 0x0BEEF11E in the orginal Ikea binary download (This is the OTA header identificator.)
The length of the OTA binary is then lifted out from the hopefully valid OTA header.

Is there a way to calculate the integrity of the extracted OTA?
I see there something that looks like a certificate at the end of the extracted blob.
I have tried to skim the Silabs documents that I can find on the internet, but there is a lot of pages to read and understand.

I have some problems on my “production” system that is causing my outdoor bulbs to ask repeatly for resending the OTA parts its getting. Finally the bulb(s) give up.
I will make another thread for ths as it is not related (I believe).

I did manage to update five bulbs on the “production” system.
For now I will say it is a partially success

NilsOF · January 21, 2022, 1:51pm

Please,
try to update one or two devices and report back
It may reveal if this simple OTA-firmware extraction is enough or not.
My collection of Ikea devices to upgrade is limited.
But, I have a couple of spares that I prefer to have on the shelf in case further experimentation is required down the road.

m0rgoth · January 21, 2022, 3:58pm

That’s the plan, but first I need to rewrite it in java. Is there some more info about that IKEAs signed ota format? I swear I saw some discussion about it - that there’s offset where the embedded ota file begins, somewhere at the beginning of the file after that NGIS constant, but I can’t find it anywhere.

NilsOF · January 21, 2022, 4:53pm

It apears as the OTA binary is at a fixed offset. But as we do not know what the extra bytes are for, it is hard to tell if it is fixed by coincidence or not.
Ikea is using the extra bytes for something. I have not found anything on that, but would love to know more.

m0rgoth · January 21, 2022, 5:29pm

I found some more info here GitHub - dresden-elektronik/deconz-ota-plugin: Server side implementation of the standard Zigbee OTA protocol.. Looks like they do the parsing the same way - just looking for the start of ota file constant.

NilsOF · January 21, 2022, 6:28pm

Nice, I did not know that (parts of?) Deconz is open source.
And it is excellent to have more eyes on this.
I will try to keep up

Just “maybe future usable” info: koenkk at zigbee to mqtt have a collection of zigbee firmware with nice json info here: GitHub - Koenkk/zigbee-OTA: A collection of Zigbee OTA files.
However, It is polite to ask before it is implemented in something OH-related

chris · January 21, 2022, 8:26pm

The way zigbee works is that devices will periodically request their firmware status from the server, so a UI to force the firmware update is not 100% necessary. I’m not sure if the firmware updates are implemented in the OH3 UI as they were in OH2 as I’ve not checked this personally, but it’s not impossible that this is already implemented (it will not show up in the UI unless certain information is available though).

Why Silabs? The Zigbee documents are published by the Zigbee alliance - not Silabs.

The idea is that the end device is meant to check the integrity, although maybe some companies such as Ikea may implement something non standard in addition to this.

There is also a processor for OTA files in the ZSS ZigBee library - we implemented this for Qivicon (Deutsch Telekom) a couple of years back.

I guess here you are talking about Ikea specific files? Otherwise the whole OTA file, with the header, all gets sent to the device. I think Ikea stick something non-standard on the front of their files which needs to be extracted and I guess this is the fixed offset you mean?

Interesting - I’d not come across this. I’m not completely sure how legal this is as they seem to hold firmware that in the past I know was under license…

Anyway, I might take a look at a firmware provider to hook into this, or possibly a fork of this so that we could also add other firmware such as the coordinator firmware that requires more OH specific information.

chris · January 22, 2022, 3:43am

I had a look at this, and it’s a shame that it doesn’t provide the information required to identify the device that the firmware is for . This will make it difficult to use within OH - to present the information to the user to allow the user to select the firmware, we need some way to link firmware and thing, which we don’t have.

There is information that could make it work automatically since the imagetypeid is provided, but we only know this once an OTA starts and the device makes the request. In theory I can hook this in so that when the device makes an OTA request, the firmware provider gets the firmware - this is how zigbee is meant to work, but isn’t quite in line with how OH implements the firmware provider so triggering a manual update may be a problem.

NilsOF · January 22, 2022, 11:32am

I think that some form for user control is needed for firmware rollout.
If I have 10 bulbs of the same model, I probably want to upgrade one or two first just to see how that turn out.

Ok, this is easy to test.

Yes, and I think we have the same information that the device has.
I hoped that a standard and defined way exists to check this.
That way one can eliminate bad firmware files at the download from internet stage.

Yes. There is unknown data before and after the chunk that is the OTA binary.

NilsOF · January 22, 2022, 12:28pm

I think I understand.
What if the “button” is renamed to “Allow firmware upgrade” (for the thing/device)?

It would be nice to have a “this device is looking for fimwareID(s)” field too.
And maybe a indicator if this firmwareID has a OTA file in the db. Perhaps something like “fimwareID provided by filenamexyz”.

It was quite a challenge to find out that this device

IKEA of Sweden TRADFRI bulb E27 WS opal 1000lm

wants firmwareID 16900
But once I had that it was easy to pick

10038562-2.1-TRADFRI-sy5882-bulb-ws-2.0.029.ota.ota.signed

as the file to extract OTA firmware from.

m0rgoth · January 22, 2022, 3:42pm

Ok so small update, I wrote my own OTA file parser, used it to extract the ota file from IKEAs .ota.signed file and updated one of mine sockets using zigbee console.

Now I’m looking into actually implement the FirmwareProvider and I’m struggling with what you guys already mentioned. As I understand it, to pick the right firmware file I need to match manufacturer_id and image_type. But looking at the data that are accessible in the Thing, i can:

check the thing is “zigbee:device”
check the manufacturer is IKEA - 4476 (0x117c), using “zigbee_manufacturercode” property on the Thing

But the image_type does not seem to be available on the Thing. I can get it from zigbee console by running zigbee read 37/1 0x019, which for my test bulb gets me:

Reading endpoint 0025/1, cluster client cluster Ota Upgrade (0019), attributes Upgrade Server ID, File Offset, Current File Version, Image Upgrade Status, Manufacturer ID, Image Type ID
Response for cluster 0x0019
Attribute    0  Upgrade Server ID                                   IEEE_ADDRESS                    000D6F000DBF64B8
Attribute    1  File Offset                                         UNSIGNED_32_BIT_INTEGER         -1
Attribute    2  Current File Version                                UNSIGNED_32_BIT_INTEGER         304182642
Attribute    6  Image Upgrade Status                                ENUMERATION_8_BIT               0
Attribute    7  Manufacturer ID                                     UNSIGNED_16_BIT_INTEGER         4476
Attribute    8  Image Type ID                                       UNSIGNED_16_BIT_INTEGER         8705

So I know I’m looking for image 8705, which looking at the response from IKEAs OTA server should be this:

{
  "fw_binary_url": "http://fw.ota.homesmart.ikea.net/global/GW1.0/01.17.019/bin/10035514-2.1-TRADFRI-bulb-ws-2.3.087.ota.ota.signed",
  "fw_file_version_LSB": 30257,
  "fw_file_version_MSB": 8968,
  "fw_filesize": 215596,
  "fw_image_type": 8705,
  "fw_manufacturer_id": 4476,
  "fw_type": 2
}

So for the FirmwareProvider to do this automatically I’d have to find out what image_type the device has. @chris I’m thinking doing it the same way the zigbee console commands work - get ZigBeeNetworkManager from associated bridge’s ZigBeeCoordinatorHandler and then read the attribute like ZigBeeConsoleAttributeReadCommand does it. Do you think it’s the right way to do it?

chris · January 22, 2022, 5:57pm

But this can’t be done through the firmware provider concept. We could provide an option in the parameters to allow upgrade.

No - the device can use different information such as a private key to ensure the firmware integrity and also to ensure it’s source. This is becoming more common.

What is this field? Do you mean the imageType? Is so, this is not available.

As I said above, currently there is no information in this firmware database to link the firmware to the device until after the OTA starts.

The other thing to keep in mind is that a zigbee OTA transfer can contain more than one file. The device will request the file it wants, and it can then request a second one. I don’t think this is too common, but it does happen and I had one device that did this during testing. This is not easy to implement when the user is involved.

And how did you do this? Not from the device.

I also wrote the provider yesterday, but as mentioned it is not really possible to do this as the database does not contain the information we need. The only thing I can currently think to do is to implement this all directly in the binding. This is not too hard, but there will be no standard UI in OH. The other option is to start adding more information to the firmware database to allow it to link to devices.

No - image type is only available AFTER the OTA starts. This is partly because the image types can change, and there can be multiple images as part of a single OTA.

This isn’t possible (at least not in a standard way).

What attribute do you mean? As far as I know there is no attribute available, but maybe a new one has been added that I’m not aware of.

chris · January 22, 2022, 6:01pm

Sorry - I missed this question. No - this definitely should NOT be done in the firmware provider if that’s hat you mean. The firmware provider shouldn’t be involved with ZB transactions as that would slow everything down which is not allowed.

I don’t think this is needed, and again as above, I don’t believe there is a way to read this anyway.

I guess we are both doing the same thing . Foolishly I completed the firmware provider before I worked out that the firmware database didn’t contain the required information to actually link the firmware to a thing

m0rgoth · January 22, 2022, 6:26pm

I’m talking about Image Type ID attribute from OTA cluster (0x0008). Reading specs it says:

11.10.9 Image Type ID Attribute
This attribute SHALL indicate the image type identifier of the file that the client is currently downloading, or a file
that has been completely downloaded but not upgraded to yet. The value of this attribute SHALL be 0xFFFF when
the client is not downloading a file or is not waiting to apply an upgrade.

but for some reason all my zigbee devices actually report it even when not updating (I have buch of tradfri bulbs, 2 sockets and philips hue ceiling light) and they match the value in IKEA json.

NilsOF · January 22, 2022, 6:27pm

openhab> zigbee read 62005/1  25 8
Reading endpoint F235/1, cluster client cluster Ota Upgrade (0019), attributes Image Type ID
Response for cluster 0x0019
Attribute    8  Image Type ID                                       UNSIGNED_16_BIT_INTEGER         16900

openhab>

well, it must come from the device.
Do it hurt anything if a query like this is done once in a while?

I am aware that a device can request more OTA files. It is a possibility there be more than one OTA file in the blob downloaded from the Ikea server too.

NilsOF · January 22, 2022, 6:36pm

I just checked one of my 5 Ikea bulbs that i did upgrade, and it reports 0xFFFF.
They are now upgraded to zigbee_applicationVersion 33
So this is not a reliable way to do it.