Zigbee2mqtt revisited: No more ugly transformations

Hello Guys,

first thanks for this topic this helps a lot. Today i were able to “integrate” my Ikea LED1624G9 bulb.
I added channels for Power ON/OFF and for Brightness these work fine,

Now i want to add color but i never get “Sliders for color” i get always NULL

Here are my stats from MQTT Explorer:
Bildschirmfoto 2021-05-14 um 20.01.00

i tried other ColorModes but i never get “Silders”


I think maybe i am doing something wrong here.


found the “solution” i removed the Device complete and added it again now i have a silder (after 3hours :wink: )

@derhelge

I just try to achieve the same.
However Iam at a different point. Could you show me your outgoing transformation ? For some reason the colorpicker gives me a RGB value and not xy color mode …

Hey,

do you mean this?

1 Like

Thx ! This exactly.
I was looking at it in MQTT Explorer and I was sure the bulb expects the color information in CIE xyY. Neither I would have tried to send it out formated as RGB nor referring to a value which even does not show :wink:

1 Like

While this solution works to set the RGB value I was a little bit unhappy that within the color picker the slider below does not set the brightness nicely.

So I tried something else and found a way to address the color with CIE XYY:

  - id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%s","y":"%s"},"brightness":"%s"}'
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

Only issue left: The last %s value for the brightness is a percentage (0-100) where the value expected is 1-254…
Any ideas in that ?

2 Likes

I solved this via min/max-Definition in the channel-config.

         Thing topic light_tradfri "light_tradfri" {
             Channels:
                   Type switch : state           "state" [ stateTopic = "zigbee2mqtt/light_tradfri/state", commandTopic = "zigbee2mqtt/light_tradfri/set/state", on = "ON", off = "OFF" ]
                   Type dimmer : brightness      "brightness" [ stateTopic = "zigbee2mqtt/light_tradfri/brightness", commandTopic = "zigbee2mqtt/light_tradfri/set/brightness", min = 0, max = 254 ]
                   Type dimmer : color_temp      "color_temp" [ stateTopic = "zigbee2mqtt/light_tradfri/color_temp", commandTopic = "zigbee2mqtt/light_tradfri/set/color_temp", min = 250, max = 454 ]
                   Type string : effect          "effect" [ stateTopic = "zigbee2mqtt/light_tradfri/effect" ]
                   Type string : power_on_behavior "power_on_behavior" [ stateTopic = "zigbee2mqtt/light_tradfri/power_on_behavior" ]
                   Type number : linkquality     "linkquality" [ stateTopic = "zigbee2mqtt/light_tradfri/linkquality" ]
         }

BTW, I’v made a groovy script to convert zigbee2mqtt-devices to Things. (see attachement)
mqtt2things.groovy (4.3 KB)

I switched from deconz to zigbee2mqtt today and i didn’t want to waste my time with copy/paste and repetetive typing :wink:

  1. rename it to mqtt2things.groovy
  2. pipe topic zigbee2mqtt/bridge/devices in to it. This generates above output.
mosquitto_sub -t zigbee2mqtt/bridge/devices -C 1 |  groovy ./mqtt2things.groovy  -n light_tradfri

Then -n parameter is optional. It restricts procession to the given device-name

Hi,
did you find a solution for the brightness problem?

@p_schlarb
I solved that by using an additional JS script for the necessary transformation.

My channel using the XYY format for the color and brightness looks as follows:


- id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      postCommand: true
      colorMode: XYY
      transformationPatternOut: JS:mqtt_brightness.js
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

The mqtt_brightness.js script is quite simple and just returns the correct formated string with the correct value for the brightness:

(function (x) {
        var tmp = x.split(',');
        // The brightness value is received as percentage but needed as integer 1-254
        tmp[2] = tmp[2] * 2.54;

        return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

I hope that helps.

Yes, that helps a lot und pointed me into the right direction! I wrote a second script to also transform the incoming value. To summarize for other users, here is my complete solution (looks little different cause i use config files:

Thing:

Thing mqtt:topic:zbltbed01 "Zigbee - Light Schlafzimmer 01" (mqtt:broker:mosquitto) @ "Bedroom"
 {
     Type color  : color         "Farbe" [
        stateTopic="zigbee/light_bedroom_01",
        transformationPattern="JS:tradfri_brightness_in.js",
        colorMode="XYY",
        commandTopic="zigbee/light_bedroom_01/set",
        transformationPatternOut="JS:tradfri_brightness_out.js"
    ]
 }

tradfri_brightness_in.js

(function (x) {
    var tmp = JSON.parse(x);
    // The brightness value is received as integer 1-254 but needed as percentage
    var brightness = Math.round(tmp.brightness / 254 * 100);

    return tmp.color.x + ',' + tmp.color.y + ',' + brightness
})(input)

tradfri_brightness_out.js

(function (x) {
    var tmp = x.split(',');
    // The brightness value is received as percentage but needed as integer 1-254
    tmp[2] = tmp[2] * 2.54;

    return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

Thanks for your help

3 Likes

Hi Helge,

I have the same Problem with the slider. Did you remove the bulb only in OH or in zigbee2mqtt configuation.yml? THX
Mark

Hi everyone,
I’ve got troubles trying to make work a Sonoff SNZB-02 Sensor with zigbee2mqtt/openhab even following the instructions in the first message of this topic.

My thing has an UNKNOWN status…
My Zigbee coordinator is up to date, my zigbee2mqtt also (1.25.2).
Zigbee2mqtt configuration has these parameters:

homeassistant: true
advanced:
  output: 'attribute_and_json'

It seems just setting output to attribute is no longer supported as I had an error while starting z2m (“Error: Home Assistant integration is not possible with attribute output!”).

I guess activating both JSON and Attribute can’t hurt.

My item is discovered by OH (3.2 then tried with 3.3.0-M7).
But its states remains ‘UNKNOWN’ and I’m unable to grab any value from the sensor (even if they are available/published in MQTT).

Example:
zigbee2mqtt/0x00124b00251ce9eb/temperature=23.23
and from z2m log file:

info  2022-06-21 18:57:33: MQTT publish: topic 'zigbee2mqtt/0x00124b00251ce9eb/temperature', payload '23.23'

NOTE: I’m not sure experimental/output is still used (unable to find an official doc about this attribute). I think advanced is now the same, but using a different name (unable to find a confirmation either…).

Any clue?

EDIT: obviously, if I set homeassistant to “false”, the item becomes ONLINE… (even if for an unknown reason, temperature is still NULL whereas voltage, battery, humidity have values…)
BUT if homeassistant if set to “false”, the item is not discoverable…

Hi all,
I know this is an old one but i think this is the best topic for my problem:

I’ve switched from deconz to zigbee2mqtt and most devices are working fine.

The one im struggling with is a LED Strip Controller Miboxer FUT038Z which i cant get to work.

Error Entry from openhab.log

 [ab.binding.mqtt.generic.ChannelState] - Command '{hue=123, saturation=100, x=0.1704, y=0.709}' from channel 'mqtt:topic:cf94322e0a:0c66bc9c5d:MyDeviceColorXY' not supported by type 'ColorValue': {hue=123, saturation=100, x=0.1704, y=0.709} is not a valid string syntax

I’ll guess that the channel want only x, y but i cant figure out how to transform this into the correct format.

My Channel Config:

  - id: LEDMyDeviceColorXY
    channelTypeUID: mqtt:color
    label: LEDMyDeviceColorXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDMyDevice/set
      postCommand: true
      colorMode: XYY
      stateTopic: zigbee2mqtt/LEDMyDevice
      transformationPattern: JSONPATH:$.color

And here is a MQTT Explorer Value copy of that topic:

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

So if anyone could give me a hint whats going on and how could i fix this would be really appreciated :slight_smile:

Reason for your issue is, that the mqtt message from z2m is not in the format OH will expect.

From my understanding you have 2 options:

  1. based on rules or transformation you can parse the message to get the correct state in the right format

  2. if it’s only OH that’s working with z2m together and you do not have any other software changing the state of your light directly via mqtt, than you maybe don’t need to read the current state, as OH will always store the state within the item by itself and it is sufficient to only send a command to z2m

1 Like

To elaborate on option 1:

You would have to transform

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

into

0.1704,0.709,78.43

Assumptions:

  • (x, y) is a point in CIE 1931 color space.
  • Brightness max is 255 (200/255=78.43…).

REGEX:s/\{"brightness":(.*?),.*"x":(.*?),"y":(.*?)\}.*/$2,$3,$1/g
(as Incoming Value Transformation for the Color Channel; untested) puts the parameters in the right order, but it cannot normalize the brightness to [0…100], so you would have to use a JS Transformation.

For HSB color space the REGEX would look like
REGEX:s/\{"brightness":(.*?),.*"hue":(.*?),"saturation":(.*?),.*/$2,$3,$1/g
(untested; again without normalization of brightness to [0…100]).

IIUC, a MQTT Color Channel in CIE mode accepts xyY values, but it converts the CIE 1931 color point to HSB (after application of the Incoming Value Transformation).

Use an analog Outgoing Value Transformation to transform the three HSB values into JSON (don’t forget to de-normalize the brightness) for the MQTT Command Topic.

tl;dr
Use option 2. :slight_smile:

1 Like

Nice! Thanks for pointing me in the right direction. I hope I can figure this out how this works.

I just cp your example in the transformation field but got still the same error about the colorvalue… i know this cloud be not working but at least i didnt get an error about the regex :wink:

I will remove and readd the channel as I tried a lot already. Maybe I messed something up.

Will report back later today.

But thanks again!

I’ve got the first part working!

So here is the channel with the regex transformation

configuration: {}
bridgeUID: mqtt:broker:cf94322e0a
channels:
  - id: LEDKochinselHSB
    channelTypeUID: mqtt:color
    label: LEDKochinselHSB
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: "%s"
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g

I did need to extend the regex but extract only brightness ($1) x($4) and y ($5).

Next I need to get the outgoing stuff working and add the brightness conversion … Man is that difficult to get this stuff working.

I will create an issue on the zigbee2mqtt git as I think they expose the wrong values or at least mix of xyY and HSB…

Update: DONE!
So here are the working channels:

channels:
  - id: LEDKochinselXY
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%1$s","y":"%2$s"}}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g
  - id: LEDKochinselBrightness
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselBrightness
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 0
      formatBeforePublish: '{"brightness":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.brightness
      max: 254
  - id: LEDKochinselColorTemp
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselColorTemp
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 153
      formatBeforePublish: '{"color_temp":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.color_temp
      max: 500

Works for my liking. The only thing that is bothering me is that i cant get the brightness contol in the color picker working as it is a diffrent channel - but thats for future me i guess :wink:

1 Like

Well my previous “solution” was bad as it does not convert the brightness part to the correct values as @anon71759204 pointed out earlier :wink: So here is what i came up with which now everything is working as expected: (Also most parts are c + p from other community members which i’m really glad that they share thier findings with us.)

openHAB channel config:

channels:
  - id: LEDKochinselXY
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      transformationPatternOut: JS:zigbeeColorOUT.js
      formatBeforePublish: "%"
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JS:zigbeeColorIN.js

transformation scripts:

zigbeeColorOUT.js

(function(color){
	// convert 100% to 254 Steps of brightness
	var bright = Number(color.split(/[,:]/)[2]) * 2.54;
	// example output to mqtt: {"brightness":239,"color":{"x":0.601297,"y":0.311017}}
	return '{"brightness":' + bright.toFixed(0) + ',"color":{"x":' + color.split(/[,:]/)[0] + ',"y":' + color.split(/[,:]/)[1] + '}}';
})(input)

zigbeeColorIN.js

// example input: {"brightness":50,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158}
(function(color){
	// convert 254 Steps to 100% of brightness
	var bright = Number(color.split(/[,:]/)[1]) / 2.54;
	// example output to openHAB: x,y,brightness - f.e.: 0.6942,0.2963,50
	return color.split(/[,:]/)[8] + "," + color.split(/[,:}]/)[10] + "," + bright.toFixed(0);
})(input)
7 Likes

Hi,
i tried to integrate a Hue color bulb connected with Zigbee2Mqtt into OH3 and had no luck so far.
I have the channel config and the .js scripts from the link in this post. The .js files are in the conf/transform folder. I still dont get any color information in my Channel/item.
Through the UI i created the channel and the item.
channels:

  • id: MQQTTZ2MColorCh1
    channelTypeUID: mqtt:color
    label: MQQTTZ2MColorCh1
    description: “”
    configuration:
    commandTopic: zigbee2mqtt/0x0017880106788d39/set
    colorMode: XYY
    transformationPatternOut: JS:zigbeeColorOUT.js
    formatBeforePublish: “%”
    stateTopic: zigbee2mqtt/0x0017880106788d39
    transformationPattern: JS:zigbeeColorIN.js

I created an item type: color.
I dont get any Info on that channel/item:

I also tried to text config the channel and the item:

    Type color : farbe "MQTT Hue WhiteColor1 Color" [ stateTopic="zigbee2mqtt/0x0017880106788d39/color", commandTopic="zigbee2mqtt/0x0017880106788d39/set/color", transformationPattern="JS:zigbeeColorIN.js", transformationPatternOut="JS:zigbeeColorOUT.js"]
Color Farbe "Mqtt hue white color1 color" {channel="mqtt:topic:MyMQTTBroker:MQTTHueWhiteColor1:farbe"}

With the Text File configuration i get this result:

Does anyone have a tip for me? I think there can only be something small i am missing.
Thnx in advance

Hey @Airmr,

We need a bit more Infos to try to figure out what’s wrong. So here are some steps your cloud do:

  • make sure zgbee2mqtt is sending messages to openhab (i.e. broker config). Use the WebUI from zigbee2Mqtt and then use the expose feature (device > expose) to change the color or turn on the bulb.

  • Use f.e. MQTT Explorer to see the messages that are received and send via your broker

  • check your syntax like 10 times. And then again :slight_smile: It could be just a space or so to render everything not working

  • check openhab.log : maybe enable debug log for mqtt

  • check if all add-ons and services you want to use are installed and working

  • post the context of your JavaScript transform files

  • also check the device support site on zigbee2mqtt - there might be some useful hints on how everything needs to be formatted, which color type use have to use and so on.

  • also check if you enabled the attribute_json output in zigbee2mqtt. Not sure if really needed but I have it turned on :wink:

Good luck :crossed_fingers:

Hi Andre, thnx for the quick answer.
Z2M is sending messages out.

MQTT publish: topic 'zigbee2mqtt/0x0017880106788d39', payload '{"brightness":53,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158,"color_temp_startup":341,"linkquality":61,"power_on_behavior":"off","state":"ON","update":{"state":"available"}}'

MQTT Explorer:

{"brightness":53,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158,"color_temp_startup":341,"linkquality":61,"power_on_behavior":"off","state":"ON","update":{"state":"available"}}

Syntax is just copied from your post with the friendly name changed. Checked the Syntax many times.
I dont really know how to enable a debug log for mqtt
The JavaScript transform files are the ones you posted saved in the transform directory.

I couldnt find any working examples on the Z2M device site.
And i couldnt find the attribute_json output in Z2M. I use the Frontend and am not editing the yaml file.

I think there is something not right in the way i created the Thing in the .things file…but i cant figure it out…
Shouldn’t the UI created Thing at least work when using your code you posted with my friendly name?

Thnx