Mqtt statetopic filter by field (milesight payload)

[sfoglia]

Dear Sirs,

I’m trying to integrate multiple Milesight Iot devices into openHAB 5 using MQTT, but I’m stuck on a single device while I need to manage 5 and I’d like to know the recommended approach.

I am using a Milesight LoRaWAN Gateway, which forwards all uplinks to a single MQTT topic.

All devices publish to the same topic:

mqtt/uplinkdata

Each message includes a devEUIor deviceName field to identify the device I can replicate original beahvior with:

mosquitto_pub -h mybroker -t ug/uplinkdata -m ‘{“applicationID”:1,“devEUI”:“HIDDEN”,“deviceName”:“YV01”,“gatewayTime”:“2026-05-10T10:19:25+02:00”,“valve_1_task”:{“pulse_rule_enable”:0,“sequence_id”:0,“time_rule_enable”:0,“valve_status”:0}}’

In openHAB I need to:

• Handle multiple devices (YV01–YV05)

• All sharing the same MQTT topic (ug/uplinkdata)

• Filter messages per device using devEUI

However:

• The UG56 does not allow changing payload format or topic structure

• Everything is forced into a single topic

• So I cannot split topics at the gateway level

I would like to know the best practice in openHAB for:

1. Handling multiple devices on the same MQTT topic

2. Filtering messages by devEUI or name

3. Mapping each device to separate Things/items

4. Avoiding duplicated or broken channel transformations

Here the thing:

things:
mqtt:topic:xx:yy:
bridge: mqtt:broker:base
label: valve controllers iot milesight
channels:
raw_1:
type: string
label: YV01 Raw
config:
stateTopic: ug/uplinkdata
valve_1:
type: number
label: YV01
config:
stateTopic: ug/uplinkdata
commandTopic: ug/downlinkdata
transformationPattern:
- JSONPATH:$.valve_1_task.valve_status
formatBeforePublish: ‘%s’
“off”: “{"devEUI":"xx","fPort":85,"confirmed":false,"data":"ff1d0002"}”
“on”: “{"devEUI":"xx","fPort":85,"confirmed":false,"data":"ff1d2001"}”

  gpio_1:
    type: number
    label: GPIO 1
    config:
      stateTopic: ug/uplinkdata
      transformationPattern:
        - JSONPATH:$.gpio_1
  pulse_rule_enable:
    type: number
    label: Pulse rule enable
    config:
      stateTopic: ug/uplinkdata
      transformationPattern:
        - JSONPATH:$.valve_1_task.pulse_rule_enable
  time_rule_enable:
    type: number
    label: Time rule enable
    config:
      stateTopic: ug/uplinkdata
      transformationPattern:
        - JSONPATH:$.valve_1_task.time_rule_enable
  valve_2345:
    type: number
    label: YV02
    config:
      stateTopic: ug/uplinkdata
      commandTopic: ug/downlinkdata
      transformationPattern:
        - JSONPATH:$.valve_1_task.valve_status
      formatBeforePublish: '%s'
      "off": "{\"devEUI\":\"x\",\"fPort\":85,\"confirmed\":false,\"data\":\"ff1d0002\"}"
      "on": "{\"devEUI\":\"x\",\"fPort\":85,\"confirmed\":false,\"data\":\"ff1d2001\"}"

Thanks in advance, best regards

[Tschetan]

Hi,
if I understand correctly, you are looking for a filter for reception. JsonPath can do this for you. Try it out
transformationPattern: - "JSONPATH:$[?(@.devEUI== 'HIDDEN')].valve_1_task.valve_status"

[rlkoshak (Rich Koshak)]

@Tschetan’s answer is the best answer for this situation. But I want to mention other approaches for users who end up in a similar but not exactly the same situation.

Another approach can be to use a REGEX transformation chained to the JSONPATH if you ever hit a situation where just the JSONPATH is not sufficient (e.g. you have to do a fuzzyy match).

REGEX:(.*"devEUI":"HIDDEN")∩JSONPATH:$.valve_1_task_valve_status

If the REGEX fails the message will be silently dropped.

And just to make it clear for future readers:

1. and 2. When you have to do this, use REGEX filter or JSONPATH matching to idenitfy the messages for a given device and ignore the rest.

2. Using the filtering transformations you would create separate Thingswith filters to only process the messages for that given device. Once you have one working, you can duplicate the Thing and just change the transformations for each one.

3. This is the interesting one. through the UI I don’t know that there’s a whole lot you can do short of creating a SCRIPT transformation. But that starts to get more complicated and the benefits may not be worth it.

   The approach is you’d create one SCRIPT transformation for each of the fields you want to extact. You can do this under MainUI → Settings → Transformations. The cool thing about SCRIPT transformations is you can pass arguments into them so you could pass that devEUI into the transformation to tell it which messages to match on. Inside the transformation you would look to see if it’s a message that should be processed and extract the one field you care about and return that. If it’s not a message that should be processed I think you can return null.

   You gain the advantage of only needing to implement the transformation once but I’m not sure it really buys you anything long term in maintainability.

Finaly, I’m not 10% sure if his got me yet or not. Based on the docs it sems not yet. But eventually the new YAML file formats will let you create templates. When that is available you could create a template of your Thing with just the parts that need to be different (e.g. the devEUI) needing to be defined for each instance.

[sfoglia]

Thank you for your answers, I’ve created channels, but it seems that json received is not an array so the filter predictate seems to be ignored. I’m actually stuck with this error from log viewer that is repeated for every channel:

Command ‘NULL’ from channel ‘mqtt:topic:9c527dd867:YV01:YV03_time’ not supported by type ‘NumberValue’: Character N is neither a decimal digit number, decimal point, nor “e” notation exponential mark.

[rlkoshak (Rich Koshak)]

You’ll have to post your specific configuration as it is now and an example message I think if we are to have enough info to help.

[sfoglia]

Thanks @rlkoshak , here the channels config:

version: 1
things:
  mqtt:topic:9c527dd867:YV01:
    bridge: mqtt:broker:9c527dd867
    label: YV01-5 iot ug56 milesight
    channels:
      YV01_valve:
        type: number
        label: YV01 Valve
        config:
          stateTopic: ug/uplinkdata
          transformationPattern:
            - "JSONPATH:$[?(@.deviceName== 'YV01')].valve_1_task.valve_status"
      YV01_gpio:
        type: number
        label: GPIO 1
        config:
          stateTopic: ug/uplinkdata
          transformationPattern:
            - "JSONPATH:$[?(@.deviceName== 'YV01')].valve_1_task.gpio_1"
      YV01_pulse:
        type: number
        label: Pulse rule enable
        config:
          stateTopic: ug/uplinkdata
          transformationPattern:
            - "JSONPATH:$[?(@.deviceName== 'YV01')].valve_1_task.pulse_rule_enable"
      YV01_time:
        type: number
        label: Time rule enable
        config:
          stateTopic: ug/uplinkdata
          transformationPattern:
            - "JSONPATH:$[?(@.deviceName== 'YV01')].valve_1_task.time_rule_enable"
      YV01_raw_1:
        type: string
        label: YV01 Raw
        config:
          stateTopic: lora/splitted/YV01

      YV01_valve_new:
        type: number
        label: YV01 Valve New
        config:
          stateTopic: ug/uplinkdata
          transformationPattern:
            - REGEX:(.*"deviceName":"YV01".*)∩JSONPATH:$.valve_1

Another way could be to split from a single topic to many topics but for this I’ve setup a homeassistant automations too and I don’t like the idea to keep up both.

Anyway now when a payload is received from logviewer I get:

RN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV05_pulse' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.547	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV05_time' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.549	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV02_pulse' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.551	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV04_time' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.553	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV01_valve' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.554	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV03_valve_new' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.556	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV02_valve_new' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.558	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV04_gpio' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.560	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV03_valve' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.561	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV04_valve' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.563	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV05_gpio' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.565	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV03_gpio' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.567	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV01_time' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.568	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV04_pulse' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.570	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV01_gpio' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.571	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV04_valve_new' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.573	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV05_valve' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.575	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV03_pulse' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.577	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV02_time' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.578	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV01_pulse' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
flag 21:50:21.580	WARN	org.openhab.binding.mqtt.generic.ChannelState	Command 'NULL' from channel 'mqtt:topic:9c527dd867:YV01:YV02_gpio' not supported by type 'NumberValue': Character N is neither a decimal digit number, decimal point, nor "e" notation exponential mark.
info_circle 21:50:21.584	INFO	openhab.event.ItemStateChang

I forgot to mention that payload may vary:

{"applicationID":1,"devEUI":"device","deviceName":"YV05","gatewayTime":"2026-05-19T14:41:23+02:00","gpio_1":1,"valve_1":1}'

or 

{"applicationID":1,"devEUI":"device","deviceName":"YV05","gatewayTime":"2026-05-19T18:29:44+02:00","valve_1_task":{"pulse_rule_enable":0,"sequence_id":0,"time_rule_enable":0,"valve_status":0}}'

[rlkoshak (Rich Koshak)]

@sfoglia

{"applicationID":1,"devEUI":"device","deviceName":"YV05","gatewayTime":"2026-05-19T14:41:23+02:00","gpio_1":1,"valve_1":1}'

This message never has a valve_1_task section so all the Channels you have configured to pull something from that (e.g. YV01_valve) is going to fail with a warning. It avoid the warning you need to apply the REGEX prefilter so the clannel doesn’t try to process messages that do not have those fields similar to the way you do on the Valve New Channel.

@sfoglia

Another way could be to split from a single topic to many topics but for this

That is the more standard MQTT approach. If one is strictly following the MQTT guiding principals one would use JSON either. One of the MQTT guiding principals is to not impose work on the clients, meaning the service that publishes the MQTT messages should make it as easy and as little work on the clients as possible to consume those messages. Needing to parse out the data is an imposition on the client.

But I find as many tools out there disreguard this as follow it.

[sfoglia]

Thanks for the reply! Regarding design, I prefer one topic with many consumers and client-side filtering:

one topic → many consumers, client-side filtering

I managed this in Home Assistant, but it has a different philosophy template-centric where filtering and transformation happen in Jinja2 templates:

- name: "YV01 Status"
  unique_id: yv01_status
  state_topic: "ug/uplink"

  value_template: >
    {% set dev = states('input_text.valvola_yv01') %}

    {% if value_json.devEUI != dev %}
      {{ this.state }}

    {% elif value_json.valve_1 is defined %}
      {{ 'APERTA' if value_json.valve_1 == 1 else 'CHIUSA' }}

    {% elif value_json.valve_1_task is defined %}
      {{ 'APERTA' if value_json.valve_1_task.valve_status == 1 else 'CHIUSA' }}

    {% else %}
      {{ this.state }}
    {% endif %}

OpenHAB instead seems transformation-centric — chaining JSONPATH expressions directly in the channel definition achieves the same result more concisely:

  YV01_valve:
    type: number
    label: YV01 Valve
    config:
      stateTopic: ug/uplinkdata
      transformationPattern:
        - "JSONPATH:$[?(@.deviceName == 'YV01')].valve_1"
        - "JSONPATH:$[?(@.deviceName == 'YV01')].valve_1_task.valve_status"
      "off": "0"
      "on": "1"

Both approaches solve the problem elegantly, just with different paradigms. openHAB feels more declarative and concise here, while HA gives more flexibility for complex stateful logic.

Thanks, BR

[Tschetan]

@sfoglia

while HA gives more flexibility for complex stateful logic.

Hi,
Many roads lead to Rome
I like your solution with the double JSONPATH.
The solution approach from HA can also be recreated in OH. Examples Javascript

transformationPattern:
	- "JS(|(function(i){var d=JSON.parse(i);if(d.deviceName!=='YV01')return null;if(d.valve_1!=null)return d.valve_1===1?'APERTA':'CHIUSA';if(d.valve_1_task?.valve_status!=null)return d.valve_1_task.valve_status===1?'APERTA':'CHIUSA';return null;})(input))"

transformationPattern:
	- JS(|var d=JSON.parse(input); d.deviceName!=="YV01"?null:d.valve_1!=null? d.valve_1 :d.valve_1_task?.valve_status!=null?d.valve_1_task.valve_status:null)