JSONPATH intermittently stops working for some 'Thing' channels

After an apparently random amount of time (hours-days), Openhab appears to stop updating a specific Thing channel that uses JSONPATH

###############################################################################
## Release = Raspbian GNU/Linux 11 (bullseye)
## Kernel = Linux 5.15.61-v7l+
## Platform = Raspberry Pi 4 Model B Rev 1.5
## Uptime = 21 day(s). 19:20:3
## CPU Usage = 1.51% avg over 4 cpu(s) (4 core(s) x 1 socket(s))
## CPU Load = 1m: 0.08, 5m: 0.07, 15m: 0.07
## Memory = Free: 0.13GB (7%), Used: 1.74GB (93%), Total: 1.87GB
## Swap = Free: 2.85GB (99%), Used: 0.03GB (1%), Total: 2.89GB
## Root = Free: 1.89GB (28%), Used: 4.80GB (72%), Total: 7.01GB
## Updates = 131 apt updates available.
## Sessions = 1 session(s)
## Processes = 140 running processes of 32768 maximum processes
###############################################################################
openHAB 3.3.0 - Release Build
openjdk version “11.0.16” 2022-07-19
OpenJDK Runtime Environment (build 11.0.16+8-post-Raspbian-1deb11u1)
OpenJDK Server VM (build 11.0.16+8-post-Raspbian-1deb11u1, mixed mode)
mosquitto version 2.0.11

I have a weight sensor that measures the weight of my Water Softener in order to calculate how much salt is present.
The weight sensor connects to wifi and sends a JSON to an MQTT topic.
An MQTT Thing creates channels that are linked to Items.
Example JSON - { “IPaddress”:“192.168.1.231”, “SSID”:“Wedding2”, “RSSI”:52, “Salt”:51.0 }

The Items, after some time, stop being updated.
Using an MQTT tool, I can see the JSON is still being published.
Restarting the openhab.service gets it working again.

I modified the Weight Sensor to publish the JSON and a discrete number to a different topic.
With an extra Thing channel and extra Item I get both JSON Weight and Discrete Weight.
Persisted them to an Influx database - graphed with Grafana.
Now I can see the exact time the JSON weight stopped being updated.
With the exact time, I wade through the log files looking for errors or warnings or something strange.
Nothing!
Below is the Thing file.
The same Thing has a channel for the JSON weight and a channel for discrete weight.
The only difference is that one uses JSONPATH and the other doesn’t. So I assume it is JSONPATH that has failed.
What can I do to discover why JSONPATH stops?

For the weight sensor, I have side stepped the problem with the discrete value because I also own the code in the sensor. Without a root cause analysis, I’m concerned that JSONPATH may stop for a sensor where I can’t change the MQTT publishing. e.g. a Shelly switch or Tasmota socket!
I have many sensors that publish JSONs without issue.

The chart belows shows discrete weight in Green and JSON weight in Yellow. At the start JSONPATH is failed. Then an OpenHab restart. Then a few days later, JSONPATH failure.
The two lines should track exactly together

// Water Softener Scales
    Thing mqtt:topic:waterSoftener "Water Softener 2 MQTT" (mqtt:broker:MyMqttBroker)  {
      Channels:
		  Type number : saltJSON "Weight JSON" [
			stateTopic="waterSoftener/status",
			transformationPattern="JSONPATH:$.Salt",
			postCommand=true      // Sends a command as well as an update
		]
		  Type number : salt "Weight" [
			stateTopic="waterSoftener/weight",
			postCommand=true      // Sends a command as well as an update
		]		
		  Type string : weightSSID "SSID" [
			stateTopic="waterSoftener/status",
			transformationPattern="JSONPATH:$.SSID"
		]		
		  Type number : weightRSSI "RSSI" [
			stateTopic="waterSoftener/status",
			transformationPattern="JSONPATH:$.RSSI",
			postCommand=true      // Sends a command as well as an update
		]
		  Type string : weightAddress "IP Address" [
			stateTopic="waterSoftener/status",
			transformationPattern="JSONPATH:$.IPaddress"
		]
		  Type string : debug "Debug Message" [
			stateTopic="waterSoftener/debug"
		]
    }

As a general rule, the philosophy of MQTT is that publishers should not impose work on their subscribers. As such, it is somewhat of an anti-pattern to publish JSON over MQTT in the first place as that imposes a cost to parse that JSON on the subscribers. The data should be split into multiple topics using the topic hierarchy so that subscribers can just subscribe to the data they really want and not have to parse it.

Of course there are lots of other considerations and this philosophy is often not followed. But that is what it states in it’s “manifesto”.

Given that, your work around might be the more correct approach anyway. But that doesn’t address the problem you are seeing.

I’ve never seen any reports of the JSONPATH transformation failing. You can verify that is indeed the source of the problem by moving the transformation from the Channel config to a transform profile. At the same time add a String Channel without any transform. From there you can see if the problem is in the binding or in the JSONPATH. If the String Item also stops updating you know it’s the binding that’s the problem and JSONPATH isn’t. If just the one with the JSONPATH stops then you know that it’s the JSONPATH.

I’m not sure what else you can do to identify the problem.

Hi Rich,

Thanks for the advice.
I have added a string channel for the complete JSON and a matching item.
I have also added an item transformed from the JSON string via a profile.
Everything piped into InfluxDB so I can see what happens.
Now I just have to wait for the failure!

Interesting snippet - the failed JSONPATH is fixed when the .thing file is saved

Ray

Update:
Weight from weight topic still working
Weight from JSON with transformation in Thing failed
Weight from JSON with transformation via profile failed
Complete JSON string failed

So it is not the JSONPATH failing but the MQTT Thing has stopped updating even though valid JSONs are still being published.
The MQTT Thing is not completely broken; the Weight from Topic channel still works but all the other channels have stopped updating.

I can get all channels working again by pausing the Thing in the Settings-Thing UI. Then resuming.

What can I do to understand why the Thing stops?

Ray

The best I can offer is to put the binding into trace level logging and see if there is any sort of clue when those channels stop working.

Hi Rick,

Trace level logging is very informative!
It showed me that the binding unsubscribes and resubscribes to the MQTT broker when the Thing file is saved.
Which gave me an idea! Maybe the binding was all good but the MQTT broker was not publishing that topic to openhab anymore!
I waited a few days for the failure to occur, then I restarted the Mosquitto service. The binding came back to life!!!
So the evidence is pointing at mosquitto now.
I don’t know what can cause mosquitto to stop publishing. I will wade through the Mosquitto logs to see what I can find
Ray