Zigbee2mqtt revisited: No more ugly transformations

No, it is not. All of my wireless battery powered xiaomi sensors (smoke, temp, motion) do send an update on regulary bases.

You shoud check the zigbee2mqtt device documentation. Sometimes sensore behave a little weird

Hi, tell me.I have configured the zigbeetomqtt binding there is one problem.after the reboot, the sensors that I connected go offline. at the same time, the broker itself is in the online mode.when I go to the broker’s thing and click on save, the sensors connect to the broker.how do I make sure that the sensors are online after a reboot?

Some sensors do only on “activity” or on a set timeframe.
Example:
My Xiaomi motion sensor only updates (all) values (i.e. last seen) when motion is detected.
On the other hand, my Xiaomi smoke detector only updates every hour (when there is no alarm).
You can check the z2mqtt documentation, whereas some of this behavior is not necessarily documented there.

Hello Guys,

first thanks for this topic this helps a lot. Today i were able to “integrate” my Ikea LED1624G9 bulb.
I added channels for Power ON/OFF and for Brightness these work fine,

Now i want to add color but i never get “Sliders for color” i get always NULL

Here are my stats from MQTT Explorer:
Bildschirmfoto 2021-05-14 um 20.01.00

i tried other ColorModes but i never get “Silders”


I think maybe i am doing something wrong here.


found the “solution” i removed the Device complete and added it again now i have a silder (after 3hours :wink: )

@derhelge

I just try to achieve the same.
However Iam at a different point. Could you show me your outgoing transformation ? For some reason the colorpicker gives me a RGB value and not xy color mode …

Hey,

do you mean this?

1 Like

Thx ! This exactly.
I was looking at it in MQTT Explorer and I was sure the bulb expects the color information in CIE xyY. Neither I would have tried to send it out formated as RGB nor referring to a value which even does not show :wink:

1 Like

While this solution works to set the RGB value I was a little bit unhappy that within the color picker the slider below does not set the brightness nicely.

So I tried something else and found a way to address the color with CIE XYY:

  - id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%s","y":"%s"},"brightness":"%s"}'
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

Only issue left: The last %s value for the brightness is a percentage (0-100) where the value expected is 1-254…
Any ideas in that ?

2 Likes

I solved this via min/max-Definition in the channel-config.

         Thing topic light_tradfri "light_tradfri" {
             Channels:
                   Type switch : state           "state" [ stateTopic = "zigbee2mqtt/light_tradfri/state", commandTopic = "zigbee2mqtt/light_tradfri/set/state", on = "ON", off = "OFF" ]
                   Type dimmer : brightness      "brightness" [ stateTopic = "zigbee2mqtt/light_tradfri/brightness", commandTopic = "zigbee2mqtt/light_tradfri/set/brightness", min = 0, max = 254 ]
                   Type dimmer : color_temp      "color_temp" [ stateTopic = "zigbee2mqtt/light_tradfri/color_temp", commandTopic = "zigbee2mqtt/light_tradfri/set/color_temp", min = 250, max = 454 ]
                   Type string : effect          "effect" [ stateTopic = "zigbee2mqtt/light_tradfri/effect" ]
                   Type string : power_on_behavior "power_on_behavior" [ stateTopic = "zigbee2mqtt/light_tradfri/power_on_behavior" ]
                   Type number : linkquality     "linkquality" [ stateTopic = "zigbee2mqtt/light_tradfri/linkquality" ]
         }

BTW, I’v made a groovy script to convert zigbee2mqtt-devices to Things. (see attachement)
mqtt2things.groovy (4.3 KB)

I switched from deconz to zigbee2mqtt today and i didn’t want to waste my time with copy/paste and repetetive typing :wink:

  1. rename it to mqtt2things.groovy
  2. pipe topic zigbee2mqtt/bridge/devices in to it. This generates above output.
mosquitto_sub -t zigbee2mqtt/bridge/devices -C 1 |  groovy ./mqtt2things.groovy  -n light_tradfri

Then -n parameter is optional. It restricts procession to the given device-name

Hi,
did you find a solution for the brightness problem?

@p_schlarb
I solved that by using an additional JS script for the necessary transformation.

My channel using the XYY format for the color and brightness looks as follows:


- id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      postCommand: true
      colorMode: XYY
      transformationPatternOut: JS:mqtt_brightness.js
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

The mqtt_brightness.js script is quite simple and just returns the correct formated string with the correct value for the brightness:

(function (x) {
        var tmp = x.split(',');
        // The brightness value is received as percentage but needed as integer 1-254
        tmp[2] = tmp[2] * 2.54;

        return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

I hope that helps.

Yes, that helps a lot und pointed me into the right direction! I wrote a second script to also transform the incoming value. To summarize for other users, here is my complete solution (looks little different cause i use config files:

Thing:

Thing mqtt:topic:zbltbed01 "Zigbee - Light Schlafzimmer 01" (mqtt:broker:mosquitto) @ "Bedroom"
 {
     Type color  : color         "Farbe" [
        stateTopic="zigbee/light_bedroom_01",
        transformationPattern="JS:tradfri_brightness_in.js",
        colorMode="XYY",
        commandTopic="zigbee/light_bedroom_01/set",
        transformationPatternOut="JS:tradfri_brightness_out.js"
    ]
 }

tradfri_brightness_in.js

(function (x) {
    var tmp = JSON.parse(x);
    // The brightness value is received as integer 1-254 but needed as percentage
    var brightness = Math.round(tmp.brightness / 254 * 100);

    return tmp.color.x + ',' + tmp.color.y + ',' + brightness
})(input)

tradfri_brightness_out.js

(function (x) {
    var tmp = x.split(',');
    // The brightness value is received as percentage but needed as integer 1-254
    tmp[2] = tmp[2] * 2.54;

    return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

Thanks for your help

3 Likes

Hi Helge,

I have the same Problem with the slider. Did you remove the bulb only in OH or in zigbee2mqtt configuation.yml? THX
Mark

Hi everyone,
I’ve got troubles trying to make work a Sonoff SNZB-02 Sensor with zigbee2mqtt/openhab even following the instructions in the first message of this topic.

My thing has an UNKNOWN status…
My Zigbee coordinator is up to date, my zigbee2mqtt also (1.25.2).
Zigbee2mqtt configuration has these parameters:

homeassistant: true
advanced:
  output: 'attribute_and_json'

It seems just setting output to attribute is no longer supported as I had an error while starting z2m (“Error: Home Assistant integration is not possible with attribute output!”).

I guess activating both JSON and Attribute can’t hurt.

My item is discovered by OH (3.2 then tried with 3.3.0-M7).
But its states remains ‘UNKNOWN’ and I’m unable to grab any value from the sensor (even if they are available/published in MQTT).

Example:
zigbee2mqtt/0x00124b00251ce9eb/temperature=23.23
and from z2m log file:

info  2022-06-21 18:57:33: MQTT publish: topic 'zigbee2mqtt/0x00124b00251ce9eb/temperature', payload '23.23'

NOTE: I’m not sure experimental/output is still used (unable to find an official doc about this attribute). I think advanced is now the same, but using a different name (unable to find a confirmation either…).

Any clue?

EDIT: obviously, if I set homeassistant to “false”, the item becomes ONLINE… (even if for an unknown reason, temperature is still NULL whereas voltage, battery, humidity have values…)
BUT if homeassistant if set to “false”, the item is not discoverable…

Hi all,
I know this is an old one but i think this is the best topic for my problem:

I’ve switched from deconz to zigbee2mqtt and most devices are working fine.

The one im struggling with is a LED Strip Controller Miboxer FUT038Z which i cant get to work.

Error Entry from openhab.log

 [ab.binding.mqtt.generic.ChannelState] - Command '{hue=123, saturation=100, x=0.1704, y=0.709}' from channel 'mqtt:topic:cf94322e0a:0c66bc9c5d:MyDeviceColorXY' not supported by type 'ColorValue': {hue=123, saturation=100, x=0.1704, y=0.709} is not a valid string syntax

I’ll guess that the channel want only x, y but i cant figure out how to transform this into the correct format.

My Channel Config:

  - id: LEDMyDeviceColorXY
    channelTypeUID: mqtt:color
    label: LEDMyDeviceColorXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDMyDevice/set
      postCommand: true
      colorMode: XYY
      stateTopic: zigbee2mqtt/LEDMyDevice
      transformationPattern: JSONPATH:$.color

And here is a MQTT Explorer Value copy of that topic:

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

So if anyone could give me a hint whats going on and how could i fix this would be really appreciated :slight_smile:

Reason for your issue is, that the mqtt message from z2m is not in the format OH will expect.

From my understanding you have 2 options:

  1. based on rules or transformation you can parse the message to get the correct state in the right format

  2. if it’s only OH that’s working with z2m together and you do not have any other software changing the state of your light directly via mqtt, than you maybe don’t need to read the current state, as OH will always store the state within the item by itself and it is sufficient to only send a command to z2m

1 Like

To elaborate on option 1:

You would have to transform

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

into

0.1704,0.709,78.43

Assumptions:

  • (x, y) is a point in CIE 1931 color space.
  • Brightness max is 255 (200/255=78.43…).

REGEX:s/\{"brightness":(.*?),.*"x":(.*?),"y":(.*?)\}.*/$2,$3,$1/g
(as Incoming Value Transformation for the Color Channel; untested) puts the parameters in the right order, but it cannot normalize the brightness to [0…100], so you would have to use a JS Transformation.

For HSB color space the REGEX would look like
REGEX:s/\{"brightness":(.*?),.*"hue":(.*?),"saturation":(.*?),.*/$2,$3,$1/g
(untested; again without normalization of brightness to [0…100]).

IIUC, a MQTT Color Channel in CIE mode accepts xyY values, but it converts the CIE 1931 color point to HSB (after application of the Incoming Value Transformation).

Use an analog Outgoing Value Transformation to transform the three HSB values into JSON (don’t forget to de-normalize the brightness) for the MQTT Command Topic.

tl;dr
Use option 2. :slight_smile:

1 Like

Nice! Thanks for pointing me in the right direction. I hope I can figure this out how this works.

I just cp your example in the transformation field but got still the same error about the colorvalue… i know this cloud be not working but at least i didnt get an error about the regex :wink:

I will remove and readd the channel as I tried a lot already. Maybe I messed something up.

Will report back later today.

But thanks again!

I’ve got the first part working!

So here is the channel with the regex transformation

configuration: {}
bridgeUID: mqtt:broker:cf94322e0a
channels:
  - id: LEDKochinselHSB
    channelTypeUID: mqtt:color
    label: LEDKochinselHSB
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: "%s"
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g

I did need to extend the regex but extract only brightness ($1) x($4) and y ($5).

Next I need to get the outgoing stuff working and add the brightness conversion … Man is that difficult to get this stuff working.

I will create an issue on the zigbee2mqtt git as I think they expose the wrong values or at least mix of xyY and HSB…

Update: DONE!
So here are the working channels:

channels:
  - id: LEDKochinselXY
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%1$s","y":"%2$s"}}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g
  - id: LEDKochinselBrightness
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselBrightness
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 0
      formatBeforePublish: '{"brightness":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.brightness
      max: 254
  - id: LEDKochinselColorTemp
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselColorTemp
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 153
      formatBeforePublish: '{"color_temp":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.color_temp
      max: 500

Works for my liking. The only thing that is bothering me is that i cant get the brightness contol in the color picker working as it is a diffrent channel - but thats for future me i guess :wink:

1 Like