Jinja transformation failed - wallplug with auto discover homeassistan

Hi, see this i think it comes from homeassistant autodiscover from zigbe2mqqt but not sure which device is posting such json (in zigbee2mqtt i don’t see such device how can i trace it ?

Exception: Unknown token found: value_json.energy
2022-09-28 23:08:59.252 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json[‘update’][‘state’]
2022-09-28 23:08:59.662 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.energy
2022-09-28 23:08:59.667 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json[‘update’][‘state’]
2022-09-28 23:09:05.576 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.energy
2022-09-28 23:09:05.583 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json[‘update’][‘state’]
2022-09-28 23:09:07.941 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.energy
2022-09-28 23:09:07.950 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json[‘update’][‘state’]
2022-09-28 23:09:19.977 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json.energy
2022-09-28 23:09:19.988 [WARN ] [t.generic.ChannelStateTransformation] - Executing the JINJA-transformation failed: An error occurred while transformation. UnknownTokenException: Unknown token found: value_json[‘update’][‘state’]

I believe you have to separately install the Jinja Transformation add-on to do HA discovery with the MQTT binding.

i did, it is installed. but i can’t see any device posting something like update state so don’t know what is triggering this automation to run

Something must be posting. The MQTT binding doesn’t get triggered unless it receives a message somehow. The only time it’ll receive a message that wasn’t just posted by the device is if the retained bit was set, in which case OH will get the message when the binding first connects to the broker no matter when the message was originally sent.

The warning is coming from a ChannelStateTransformation so I’m pretty sure this is the binding attempting to apply the transformation to some message (as opposed to the transform profile or something like that).

I do not fully know and understand the HA spec for MQTT but I suspect it might be kind of chatty. For certain it publishes a bunch of retained messages to special topics which is what allows for the auto discovery in the first place.

If you put the binding into debug mode, you might see what messages on what topics it’s trying to process to generate these warnings.

ok found it.
Transformation ‘{“child_lock”:“UNLOCK”,“current”:0,“energy”:null,“indicator_mode”:“off/on”,“last_seen”:“2022-09-29T00:10:51+02:00”,“linkquality”:0,“power”:0,“power_outage_memory”:“restore”,“state”:“OFF”,“update”:{“state”:null},“update_available”:null,“voltage”:231}’ returned null on ‘JINJA’, discarding message
device json :

{
    "child_lock": "UNLOCK",
    "current": 0,
    "indicator_mode": "off/on",
    "last_seen": "2022-09-29T00:10:51+02:00",
    "linkquality": 0,
    "power": 0,
    "power_outage_memory": "restore",
    "state": "OFF",
    "voltage": 231,
    "energy": null,
    "update": {
        "state": null
    },
    "update_available": null
}

but how to solve this issue? this thing is auto-generated by discovery :

{“availability”:[{“topic”:“zigbee2mqtt/bridge/state”,“value_template”:“{{ value_json.state }}”},{“topic”:“zigbee2mqtt/WallPlug2/availability”,“value_template”:“{{ value_json.state }}”}],“availability_mode”:“all”,“device”:{“identifiers”:[“zigbee2mqtt_0xa4c138764bf197d1”],“manufacturer”:“TuYa”,“model”:“Smart plug (with power monitoring) (TS011F_plug_1)”,“name”:“WallPlug2”},“enabled_by_default”:false,“entity_category”:“diagnostic”,“icon”:“mdi:update”,“json_attributes_topic”:“zigbee2mqtt/WallPlug2”,“name”:“WallPlug2 update state”,“state_topic”:“zigbee2mqtt/WallPlug2”,“unique_id”:“0xa4c138764bf197d1_update_state_zigbee2mqtt”,“value_template”:“{{ value_json[‘update’][‘state’] }}”}

Probably choking on the null values? Seems to correspond with the complaints.

First identify if nulls are allowed in Homeassistant discovery protocol - if not, the device needs fixing.
If nulls are permitted, looks like the HA section of the openHAB MQTT binding needs fixing.

1 Like

While it’s interesting to note that Homeassistant has troubles with JINJA handling null values, it doesn’t seem to move you forward.
This is openHAB.
JINJA is just used as a tool here to interpret the MQTT payload.

Let’s try step one;

The HA Discovery protocol is expressed in JSON.
So it must be valid JSON.
Does JSON allow null values?
Yes it does.

So turning to HA Discovery protocol proper.
Does that allow null values?

I can’t see anywhere it is expressly forbidden.

Neither can I see any reason that you’d want to send null values.
If your device has no ‘energy’ sensing for example, you simply would not send an ‘energy’ keyword in the discovery announcement.
Sending "energy": null is announcing that you don’t know anything about “energy”, you are not able to describe it in any way, the same as saying nothing at all.

So to me, it appears to be a mad thing to do, and may or may not be actually “allowed” in HA discovery.

I’d missed out before, that it would be sensible to find out if zigbee2mqtt is supposed to be sending this null stuff at all.

Either way, it seems to me that openHAB ‘HA Discovery’ is doing exactly the right thing - generating a WARN that a nonsense payload has been received.
I am assuming here that your device/Thing etc. is all working properly otherwise (now that you know which one it is) - you didn’t say?

I suppose the WARN could be enhanced to give the suspect MQTT topic as well, to aid diagnosis.

Ah wait - I’ve misinterpreted.

The payload causing the WARN is not the discovery part all. Whether that described e.g. ‘energy’ or not is irrelevant.

This a working payload, a “normal update”. It’s just JSON, so null is valid.

The channel (that was previously configured by discovery) cannot handle the null.
People encounter this issue in regular non-HA MQTT usage as well, and have to configure their channels to work around it - there’s a choice, because you might want to treat as an error and report it; or you might want to just ignore and not update the linked Item, or you might want to treat it as meaningful and update the Item with UNDEF or similar.

The problem here is that the channel was auto-configured so you don’t get to manipulate how it responds.

I don’t think there’s any HA rule about how to treat null values, which would be sensible. If you’ve nothing to say, don’t say it.
So I still arrive at the conclusion the binding is doing the correct thing - reporting WARN about nonsense received (and I guess issuing no update at all to linked Item?)
And would still question if zigbee2mqtt is doing the wrong thing sending nulls in the first place.