Zigbee2mqtt revisited: No more ugly transformations

Thx ! This exactly.
I was looking at it in MQTT Explorer and I was sure the bulb expects the color information in CIE xyY. Neither I would have tried to send it out formated as RGB nor referring to a value which even does not show :wink:

1 Like

While this solution works to set the RGB value I was a little bit unhappy that within the color picker the slider below does not set the brightness nicely.

So I tried something else and found a way to address the color with CIE XYY:

  - id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%s","y":"%s"},"brightness":"%s"}'
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

Only issue left: The last %s value for the brightness is a percentage (0-100) where the value expected is 1-254…
Any ideas in that ?

2 Likes

I solved this via min/max-Definition in the channel-config.

         Thing topic light_tradfri "light_tradfri" {
             Channels:
                   Type switch : state           "state" [ stateTopic = "zigbee2mqtt/light_tradfri/state", commandTopic = "zigbee2mqtt/light_tradfri/set/state", on = "ON", off = "OFF" ]
                   Type dimmer : brightness      "brightness" [ stateTopic = "zigbee2mqtt/light_tradfri/brightness", commandTopic = "zigbee2mqtt/light_tradfri/set/brightness", min = 0, max = 254 ]
                   Type dimmer : color_temp      "color_temp" [ stateTopic = "zigbee2mqtt/light_tradfri/color_temp", commandTopic = "zigbee2mqtt/light_tradfri/set/color_temp", min = 250, max = 454 ]
                   Type string : effect          "effect" [ stateTopic = "zigbee2mqtt/light_tradfri/effect" ]
                   Type string : power_on_behavior "power_on_behavior" [ stateTopic = "zigbee2mqtt/light_tradfri/power_on_behavior" ]
                   Type number : linkquality     "linkquality" [ stateTopic = "zigbee2mqtt/light_tradfri/linkquality" ]
         }

BTW, I’v made a groovy script to convert zigbee2mqtt-devices to Things. (see attachement)
mqtt2things.groovy (4.3 KB)

I switched from deconz to zigbee2mqtt today and i didn’t want to waste my time with copy/paste and repetetive typing :wink:

  1. rename it to mqtt2things.groovy
  2. pipe topic zigbee2mqtt/bridge/devices in to it. This generates above output.
mosquitto_sub -t zigbee2mqtt/bridge/devices -C 1 |  groovy ./mqtt2things.groovy  -n light_tradfri

Then -n parameter is optional. It restricts procession to the given device-name

Hi,
did you find a solution for the brightness problem?

@p_schlarb
I solved that by using an additional JS script for the necessary transformation.

My channel using the XYY format for the color and brightness looks as follows:


- id: Color
    channelTypeUID: mqtt:color
    label: Farbe
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/Livingroom_Floorlamp/set
      postCommand: true
      colorMode: XYY
      transformationPatternOut: JS:mqtt_brightness.js
      stateTopic: zigbee2mqtt/Livingroom_Floorlamp
      transformationPattern: JSONPATH:$.color

The mqtt_brightness.js script is quite simple and just returns the correct formated string with the correct value for the brightness:

(function (x) {
        var tmp = x.split(',');
        // The brightness value is received as percentage but needed as integer 1-254
        tmp[2] = tmp[2] * 2.54;

        return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

I hope that helps.

Yes, that helps a lot und pointed me into the right direction! I wrote a second script to also transform the incoming value. To summarize for other users, here is my complete solution (looks little different cause i use config files:

Thing:

Thing mqtt:topic:zbltbed01 "Zigbee - Light Schlafzimmer 01" (mqtt:broker:mosquitto) @ "Bedroom"
 {
     Type color  : color         "Farbe" [
        stateTopic="zigbee/light_bedroom_01",
        transformationPattern="JS:tradfri_brightness_in.js",
        colorMode="XYY",
        commandTopic="zigbee/light_bedroom_01/set",
        transformationPatternOut="JS:tradfri_brightness_out.js"
    ]
 }

tradfri_brightness_in.js

(function (x) {
    var tmp = JSON.parse(x);
    // The brightness value is received as integer 1-254 but needed as percentage
    var brightness = Math.round(tmp.brightness / 254 * 100);

    return tmp.color.x + ',' + tmp.color.y + ',' + brightness
})(input)

tradfri_brightness_out.js

(function (x) {
    var tmp = x.split(',');
    // The brightness value is received as percentage but needed as integer 1-254
    tmp[2] = tmp[2] * 2.54;

    return '{"color":{"x":"' + tmp[0] + '","y":"' + tmp[1] + '"},"brightness":"' + tmp[2] + '"}'
})(input)

Thanks for your help

3 Likes

Hi Helge,

I have the same Problem with the slider. Did you remove the bulb only in OH or in zigbee2mqtt configuation.yml? THX
Mark

Hi everyone,
I’ve got troubles trying to make work a Sonoff SNZB-02 Sensor with zigbee2mqtt/openhab even following the instructions in the first message of this topic.

My thing has an UNKNOWN status…
My Zigbee coordinator is up to date, my zigbee2mqtt also (1.25.2).
Zigbee2mqtt configuration has these parameters:

homeassistant: true
advanced:
  output: 'attribute_and_json'

It seems just setting output to attribute is no longer supported as I had an error while starting z2m (“Error: Home Assistant integration is not possible with attribute output!”).

I guess activating both JSON and Attribute can’t hurt.

My item is discovered by OH (3.2 then tried with 3.3.0-M7).
But its states remains ‘UNKNOWN’ and I’m unable to grab any value from the sensor (even if they are available/published in MQTT).

Example:
zigbee2mqtt/0x00124b00251ce9eb/temperature=23.23
and from z2m log file:

info  2022-06-21 18:57:33: MQTT publish: topic 'zigbee2mqtt/0x00124b00251ce9eb/temperature', payload '23.23'

NOTE: I’m not sure experimental/output is still used (unable to find an official doc about this attribute). I think advanced is now the same, but using a different name (unable to find a confirmation either…).

Any clue?

EDIT: obviously, if I set homeassistant to “false”, the item becomes ONLINE… (even if for an unknown reason, temperature is still NULL whereas voltage, battery, humidity have values…)
BUT if homeassistant if set to “false”, the item is not discoverable…

Hi all,
I know this is an old one but i think this is the best topic for my problem:

I’ve switched from deconz to zigbee2mqtt and most devices are working fine.

The one im struggling with is a LED Strip Controller Miboxer FUT038Z which i cant get to work.

Error Entry from openhab.log

 [ab.binding.mqtt.generic.ChannelState] - Command '{hue=123, saturation=100, x=0.1704, y=0.709}' from channel 'mqtt:topic:cf94322e0a:0c66bc9c5d:MyDeviceColorXY' not supported by type 'ColorValue': {hue=123, saturation=100, x=0.1704, y=0.709} is not a valid string syntax

I’ll guess that the channel want only x, y but i cant figure out how to transform this into the correct format.

My Channel Config:

  - id: LEDMyDeviceColorXY
    channelTypeUID: mqtt:color
    label: LEDMyDeviceColorXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDMyDevice/set
      postCommand: true
      colorMode: XYY
      stateTopic: zigbee2mqtt/LEDMyDevice
      transformationPattern: JSONPATH:$.color

And here is a MQTT Explorer Value copy of that topic:

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

So if anyone could give me a hint whats going on and how could i fix this would be really appreciated :slight_smile:

Reason for your issue is, that the mqtt message from z2m is not in the format OH will expect.

From my understanding you have 2 options:

  1. based on rules or transformation you can parse the message to get the correct state in the right format

  2. if it’s only OH that’s working with z2m together and you do not have any other software changing the state of your light directly via mqtt, than you maybe don’t need to read the current state, as OH will always store the state within the item by itself and it is sufficient to only send a command to z2m

1 Like

To elaborate on option 1:

You would have to transform

{"brightness":200,"color":{"hue":123,"saturation":100,"x":0.1704,"y":0.709},"color_mode":"xy","color_temp":153,"linkquality":220,"state":"ON"}

into

0.1704,0.709,78.43

Assumptions:

  • (x, y) is a point in CIE 1931 color space.
  • Brightness max is 255 (200/255=78.43…).

REGEX:s/\{"brightness":(.*?),.*"x":(.*?),"y":(.*?)\}.*/$2,$3,$1/g
(as Incoming Value Transformation for the Color Channel; untested) puts the parameters in the right order, but it cannot normalize the brightness to [0…100], so you would have to use a JS Transformation.

For HSB color space the REGEX would look like
REGEX:s/\{"brightness":(.*?),.*"hue":(.*?),"saturation":(.*?),.*/$2,$3,$1/g
(untested; again without normalization of brightness to [0…100]).

IIUC, a MQTT Color Channel in CIE mode accepts xyY values, but it converts the CIE 1931 color point to HSB (after application of the Incoming Value Transformation).

Use an analog Outgoing Value Transformation to transform the three HSB values into JSON (don’t forget to de-normalize the brightness) for the MQTT Command Topic.

tl;dr
Use option 2. :slight_smile:

1 Like

Nice! Thanks for pointing me in the right direction. I hope I can figure this out how this works.

I just cp your example in the transformation field but got still the same error about the colorvalue… i know this cloud be not working but at least i didnt get an error about the regex :wink:

I will remove and readd the channel as I tried a lot already. Maybe I messed something up.

Will report back later today.

But thanks again!

I’ve got the first part working!

So here is the channel with the regex transformation

configuration: {}
bridgeUID: mqtt:broker:cf94322e0a
channels:
  - id: LEDKochinselHSB
    channelTypeUID: mqtt:color
    label: LEDKochinselHSB
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: "%s"
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g

I did need to extend the regex but extract only brightness ($1) x($4) and y ($5).

Next I need to get the outgoing stuff working and add the brightness conversion … Man is that difficult to get this stuff working.

I will create an issue on the zigbee2mqtt git as I think they expose the wrong values or at least mix of xyY and HSB…

Update: DONE!
So here are the working channels:

channels:
  - id: LEDKochinselXY
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      formatBeforePublish: '{"color":{"x":"%1$s","y":"%2$s"}}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: REGEX:s/\{"brightness":(.*?),"color":\{(?<=)"hue":(.*?),"saturation":(.*?)(?=),"x":(.*?),"y":(.*?)\},.*\}/$4,$5,$1/g
  - id: LEDKochinselBrightness
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselBrightness
    description: ""
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 0
      formatBeforePublish: '{"brightness":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.brightness
      max: 254
  - id: LEDKochinselColorTemp
    channelTypeUID: mqtt:dimmer
    label: LEDKochinselColorTemp
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      min: 153
      formatBeforePublish: '{"color_temp":"%s"}'
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JSONPATH:$.color_temp
      max: 500

Works for my liking. The only thing that is bothering me is that i cant get the brightness contol in the color picker working as it is a diffrent channel - but thats for future me i guess :wink:

1 Like

Well my previous “solution” was bad as it does not convert the brightness part to the correct values as @anon71759204 pointed out earlier :wink: So here is what i came up with which now everything is working as expected: (Also most parts are c + p from other community members which i’m really glad that they share thier findings with us.)

openHAB channel config:

channels:
  - id: LEDKochinselXY
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
      commandTopic: zigbee2mqtt/LEDKochinsel/set
      colorMode: XYY
      transformationPatternOut: JS:zigbeeColorOUT.js
      formatBeforePublish: "%"
      stateTopic: zigbee2mqtt/LEDKochinsel
      transformationPattern: JS:zigbeeColorIN.js

transformation scripts:

zigbeeColorOUT.js

(function(color){
	// convert 100% to 254 Steps of brightness
	var bright = Number(color.split(/[,:]/)[2]) * 2.54;
	// example output to mqtt: {"brightness":239,"color":{"x":0.601297,"y":0.311017}}
	return '{"brightness":' + bright.toFixed(0) + ',"color":{"x":' + color.split(/[,:]/)[0] + ',"y":' + color.split(/[,:]/)[1] + '}}';
})(input)

zigbeeColorIN.js

// example input: {"brightness":50,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158}
(function(color){
	// convert 254 Steps to 100% of brightness
	var bright = Number(color.split(/[,:]/)[1]) / 2.54;
	// example output to openHAB: x,y,brightness - f.e.: 0.6942,0.2963,50
	return color.split(/[,:]/)[8] + "," + color.split(/[,:}]/)[10] + "," + bright.toFixed(0);
})(input)
7 Likes

Hi,
i tried to integrate a Hue color bulb connected with Zigbee2Mqtt into OH3 and had no luck so far.
I have the channel config and the .js scripts from the link in this post. The .js files are in the conf/transform folder. I still dont get any color information in my Channel/item.
Through the UI i created the channel and the item.
channels:

  • id: MQQTTZ2MColorCh1
    channelTypeUID: mqtt:color
    label: MQQTTZ2MColorCh1
    description: “”
    configuration:
    commandTopic: zigbee2mqtt/0x0017880106788d39/set
    colorMode: XYY
    transformationPatternOut: JS:zigbeeColorOUT.js
    formatBeforePublish: “%”
    stateTopic: zigbee2mqtt/0x0017880106788d39
    transformationPattern: JS:zigbeeColorIN.js

I created an item type: color.
I dont get any Info on that channel/item:

I also tried to text config the channel and the item:

    Type color : farbe "MQTT Hue WhiteColor1 Color" [ stateTopic="zigbee2mqtt/0x0017880106788d39/color", commandTopic="zigbee2mqtt/0x0017880106788d39/set/color", transformationPattern="JS:zigbeeColorIN.js", transformationPatternOut="JS:zigbeeColorOUT.js"]
Color Farbe "Mqtt hue white color1 color" {channel="mqtt:topic:MyMQTTBroker:MQTTHueWhiteColor1:farbe"}

With the Text File configuration i get this result:

Does anyone have a tip for me? I think there can only be something small i am missing.
Thnx in advance

Hey @Airmr,

We need a bit more Infos to try to figure out what’s wrong. So here are some steps your cloud do:

  • make sure zgbee2mqtt is sending messages to openhab (i.e. broker config). Use the WebUI from zigbee2Mqtt and then use the expose feature (device > expose) to change the color or turn on the bulb.

  • Use f.e. MQTT Explorer to see the messages that are received and send via your broker

  • check your syntax like 10 times. And then again :slight_smile: It could be just a space or so to render everything not working

  • check openhab.log : maybe enable debug log for mqtt

  • check if all add-ons and services you want to use are installed and working

  • post the context of your JavaScript transform files

  • also check the device support site on zigbee2mqtt - there might be some useful hints on how everything needs to be formatted, which color type use have to use and so on.

  • also check if you enabled the attribute_json output in zigbee2mqtt. Not sure if really needed but I have it turned on :wink:

Good luck :crossed_fingers:

Hi Andre, thnx for the quick answer.
Z2M is sending messages out.

MQTT publish: topic 'zigbee2mqtt/0x0017880106788d39', payload '{"brightness":53,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158,"color_temp_startup":341,"linkquality":61,"power_on_behavior":"off","state":"ON","update":{"state":"available"}}'

MQTT Explorer:

{"brightness":53,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158,"color_temp_startup":341,"linkquality":61,"power_on_behavior":"off","state":"ON","update":{"state":"available"}}

Syntax is just copied from your post with the friendly name changed. Checked the Syntax many times.
I dont really know how to enable a debug log for mqtt
The JavaScript transform files are the ones you posted saved in the transform directory.

I couldnt find any working examples on the Z2M device site.
And i couldnt find the attribute_json output in Z2M. I use the Frontend and am not editing the yaml file.

I think there is something not right in the way i created the Thing in the .things file…but i cant figure it out…
Shouldn’t the UI created Thing at least work when using your code you posted with my friendly name?

Thnx

Hm, that wired. You could enable debug by following How to debug MQTT2 Binding issues? f.e.

Could you post the manufacturer and model name of the device?
Also the device should show up as supported and there should be a link behind the model like this:

Also the Output Setting is configurable via the webGUI:

Well, I created everything in openHAB via the Webinterface (I don’t like Textfiles anymore :slight_smile: ) so this should be fine.

Maybe try to give the device in z2m a friendly name which is shorter…

Hi Andre, thnx for the time your spending on this.
Here is a Screenshot from the device:

I also changed the MQTT output type.

I made 2 other channels with this bulb and everything is working correct:


And i made a String item:


The result shown is the payload:

{"brightness":153,"color":"hue":308,"saturation":100,"x":0.4066,"y":0.1643},"color_mode":"xy","color_temp":153,"color_temp_startup":341,"linkquality":47,"power_on_behavior":"off","state":"ON","update":{"state":"available"}} 

The only thing i can think of is the .js Scripts. I checked them 10 times and they are just as yours are.
The example you give in the zigbeeColorIN.js :

example input: {"brightness":50,"color":{"hue":359,"saturation":100,"x":0.6942,"y":0.2963},"color_mode":"xy","color_temp":158}

differs from the text i read in Licht Büro Text, thats the only difference i see.
I Have the Java Script, Regex and Json Path transformation installed.

The Channel:

  • id: MQQTTZ2MColorCh1Test
    channelTypeUID: mqtt:color
    label: LEDKochinselXY
    description: null
    configuration:
    commandTopic: zigbee2mqtt/0x0017880106788d39/set
    colorMode: XYY
    transformationPatternOut: JS:zigbeeColorOUT.js
    formatBeforePublish: “%”
    stateTopic: zigbee2mqtt/0x0017880106788d39
    transformationPattern: JS:zigbeeColorIN.js

I created an item with no profile:

Anything i missed ? I am going crazy :slight_smile:

MUHAHAHAHA I got it working :slight_smile:

Well at least i can control the color etc. from the Thing/item in the UI.
I still have to try to get it going in the Openhab UI. But that is another adventure.

After cleaning up all the Channels and items i have tried with and adding the “attribute and json” Output type it works.
I will test if it really works with the “attribute and json” option or if i just messed up everything, even with checking 100 times there must be something that was still wrong.
At the end i learned a lot.

Thank you Andre for helping me out here and spending your time on my issue.!! Big Thank You!!!
If you ever happen to be near Frankfurt/Germany give me a buzz and i will pay you a coffee or beer.
If my investigation turns out that i really missed something even though i checked many times, it will be more than 1 beer :slight_smile:

1 Like