Ups, sorry for this false statement. However, I would suggest to spare an item for each value.
It is hard to diagnose problems when you keep both the nature of the patient and the symptoms a secret, that’s all.
no no no
i was saying"why is it hard" or an,“odd way” in reply to a previous post, claiming in taking the hard way, which i am not.
i simply asked, what is wrong with the JSON transformation syntax, that’s all, please, i waa not asking about design decisions, just and simply, what is wrong with my JSON syntax on the label. To me it appears reasonable and sitemap labels can have this attribute, I’m sure…looking at doc?
It’s the hard way to go about it because openHAB Items are intended to hold a single piece of information - temperature, on/off, message.
You’ve chosen to embed multiple bits of information in a single Item, and now want to fish just one bit out.
It’s unconventional.
I don’t know. Your error message is still secret. Your Item type is still secret. Your Item state is secret.
If you want a guess, try escaping the single quotemarks.
1 Like
I’m on a mobile, in the field, working … its nigh impossible to get the info, until I’m on my laptop later at home on my LAN.
and my item does contains one piece of data, a JSON block from an MQTT query. no.prob.
the syntax works on JSON check tool online, that’s the point (and focus) of the OP, the JSON syntax on a label.
never mind,work it out myself when I’m home.
OH
and thankyou for the escape option, l shall try that too.
found this,
so i think its the use of two square brackets/parentheses pairs, in the transform inside the string on the label…commencing at “Diag [”
I shall investigate…
I caution you that the openHAB “JSONPATH Transformation Service” is not JSONPATH. For a start, it’d be no use returning lists for an Item label transform…
Such a test is not entirely conclusive.
ok, apologies for the delay, I was home late yesterday and did not switch on the laptop.
Ok, so here is the item:(I did away with the transform on a site label , as you all recommended):from the .items file
/* Diagnostics */
String i_TRV_Lounge_Diag1 { channel="mqtt:topic:b_MQTT_Broker:t_TRV_Lounge:c_Diagnostics" [profile="transform:JSONPATH", function="<jsonPath>", sourceFormat="$.[\'Motor current below expectation\']"] }
…the item contents and the transformation error (both from openhab.log,file:
2020-01-13 10:44:19.319 [INFO ] [el.core.internal.ModelRepositoryImpl] - Loading model 'TRV.items'
2020-01-13 10:44:19.523 [WARN ] [ofiles.JSonPathTransformationProfile] - Could not transform state '{
"Motor current below expectation": false,
"Motor current always high": false,
"Motor taking too long": false,
"discrepancy between air and pipe sensors": false,
"air sensor out of expected range": false,
"pipe sensor out of expected range": false,
"low power mode is enabled": false,
"no target temperature has been set by host": false,
"valve may be sticking": false,
"valve exercise was successful": false,
"valve exercise was unsuccessful": false,
"driver micro has suffered a watchdog reset and needs data refresh": false,
"driver micro has suffered a noise reset and needs data refresh": false,
"battery voltage has fallen below 2p2V and valve has been opened": false,
"request for heat messaging is enabled": false,
"request for heat": false
}' with function '<jsonPath>' and format '['Motor current below expectation']'
I have not linked the item to a sitemap label as yet because I’m just trying to get the item definition working.
Other item definitions of the transform JSON source strain I tried were:
** No period/dot after the $
String i_TRV_Lounge_Diag1 { channel="mqtt:topic:b_MQTT_Broker:t_TRV_Lounge:c_Diagnostics" [profile="transform:JSONPATH", function="<jsonPath>", sourceFormat="$[\'Motor current below expectation\']"] }
and
** Use double quotes instead of single
String i_TRV_Lounge_Diag1 { channel="mqtt:topic:b_MQTT_Broker:t_TRV_Lounge:c_Diagnostics" [profile="transform:JSONPATH", function="<jsonPath>", sourceFormat="$.[\"Motor current below expectation\"]"] }
and
** Remove period/dot and use double quotes
String i_TRV_Lounge_Diag1 { channel="mqtt:topic:b_MQTT_Broker:t_TRV_Lounge:c_Diagnostics" [profile="transform:JSONPATH", function="<jsonPath>", sourceFormat="$[\"Motor current below expectation\']"] }
and
** Remove period/dot and use single quotes
String i_TRV_Lounge_Diag1 { channel="mqtt:topic:b_MQTT_Broker:t_TRV_Lounge:c_Diagnostics" [profile="transform:JSONPATH", function="<jsonPath>", sourceFormat="$[\"Motor current below expectation\']"] }
Rather bizzrely, the last definition above “works” as in throws no error, but yields a strange string…
M. { "Motor current below expectation": false, "Motor current always high"
mmm???
I would say there are newline characters in your source JSON
Yes, that’s a good observation, I thought the same after reading the link in the OP which had newlines in so. I tried things like \n and 0x20 and %20 all to no avail. Perhaps OH JSON doesnt have newline escapes?
Do you think I could get away with REGEX instead, I’m ok with REGEX strings ??
I dunno. “whitespace” between JSON tokens is allowed, and that includes newlines. People get in the cart with newlines embedded in the data as a rule, not what is happening here.
Having said that, JSON’s idea of “whitespace” is fairly limited and there are a bunch of invisible characters that can ruin the show.
I’d be inclined to pull the raw JSON into a rule for analysis, but if you just want to mend it, press on.
In this case you know the “data” is limited to true/flase which should simplify any matching.
Tried with regex , which does not throw an error in the online checker (when using \s instead of the OH escape \s, but does throw an error in OH.:
rule
rule "Parse Diagnostics"
when
Item i_TRV_Lounge_Get_Diagnostics received update
then
i_TRV_Lounge_Diag1 = transform("REGEX", "(?:\"Motor current below expectation\":\\s+)(true|false)", i_TRV_Lounge_Get_Diagnostics.state.toString)
logInfo("TRV_Refresh.rules", "!! Diag updated.")
end
i_TRV_Diag1 is now just an String item with no other attributes:
String i_TRV_Diag1
The regex ues two groups , one that is non-capture and one that is … which should contain either true or false
{
"Motor current below expectation": false,
"Motor current always high": false,
"Motor taking too long": false,
"discrepancy between air and pipe sensors": false,
"air sensor out of expected range": false,
"pipe sensor out of expected range": false,
"low power mode is enabled": false,
"no target temperature has been set by host": false,
"valve may be sticking": false,
"valve exercise was successful": false,
"valve exercise was unsuccessful": false,
"driver micro has suffered a watchdog reset and needs data refresh": false,
"driver micro has suffered a noise reset and needs data refresh": false,
"battery voltage has fallen below 2p2V and valve has been opened": false,
"request for heat messaging is enabled": false,
"request for heat": false
}
The item i_TRV_Lounge_Get_Diagnostics correctly contains the above structure.
2020-01-13 14:27:59.821 [ERROR] [ntime.internal.engine.RuleEngineImpl] - Rule 'Parse Diagnostics': An error occurred during the script execution: Cannot assign a value in null context.
I’m really stuck here, cant use a JSONPATH nor can I use a regex.
If that is an Item, you don’t send states to Items in that way.
Why not take baby steps for now
var mywork = transform( ...
I couldn’t comment on your REGEX, but will note that like JSONPATH, REGEX transformation service is not REGEX. Rich @rlkoshak is fond of saying
“In openHAB, your REGEX expression must match the full message and the part that get’s returned is in the first capture group (first set of parens)”
which seems important but means little to me.
yeah, should have mentioned, I tried using a var but that didnt work either, so I tried the sendCommand which didnt work either.
I debugged and it is the string in the transform pattern that is the problem, but I do not know why because it works perfectly in https://regex101.com/ - provided I remove the escape on the \s+
Making it more complicated is unlikely to help. For now, concentrate on just getting your extraction into a string variable.
At the moment, you don’t know if your source data is iffy or your technique is wrong.
Try to get this working first, extract fred value -
val fakedata = "{ \"fred\" : false }"
Then build on it, e.g. "fred with spaces", then add a "mary":true. When you know the tools work, apply to your real data.
The source data is okay. A label on the sitemap is showing the block as per OP.
The regexp works in regex101 with the data copied from my sitemap output, for the source.
The only difference is I’m placing the regex in a rule and adding an escape “” for the greedy whitespace “\s+”
Ive used linux and programmed in it for 30 years mate, including regex .
It is OH that is the issue (or my use of it) and the fact that the functionality of OH is through complimentary code - so they may not be a 1:1 match
Again, you are using openHAB REGEX transformation service. Not REGEX.
So, that’s been processed by the sitemap and then by your browser and then copy pasted into another browser for further processing. It is not your raw source. I’m still afeared of hidden control characters.
yes, sorry, I was just editing my post before yours to concur with you.
I’ve read the OH regex docs , I shall re-read
EDIT: The OH docs even refer to the online validator - the very one I used with success. So it has to be the source, as you say possibly containing ctrl chars. I will rework the regex pattern.
I wouldn’t say I like saying it. I’m forced to say it often because for some reason OH decided to make REGEX work differently from normal so you will find lots of users say things like “it works on regex101.com but not in openHAB.”
Usually the first three or four lines of the exception is sufficient.
I don’t use JSONPATH that much but what you really want is to extract the value 20, not create a new JSON string { 'Motor current below expectation' : 20 }. When you use the square brackets in the way you are, you get back the latter. to get just the value you would use $.Motor current below expectation. You might need to escape the spaces.
For details see the JsonPath Transform docs and in particular click on the links.
No, it contains a JSON string that encodes two different values, “Motor current below expectation” and “Motor voltage”. I like rossko57’s term. That is unconventional in openHAB. Typically you would be splitting those two values into two separate Items at the MQTT Channel level, not leaving the values embedded into a single Item and splitting the values out at the Sitemap level. That is the part that is unconventional.
But unconventional doesn’t mean won’t work. It just unusual and ultimately if you want to use these values anywhere else in openHAB, it will be significantly more work in the long run because you will have to write rules to split the values out of the one Item each and every time.
As rossko57 indicated. REGEX in OH is slightly different from normal REGEX. At least when used from the transformation service. In that case your expression must match the entire string and only the first group is what gets returned by the transformation. And by entire String I mean the whole thing, not just the line. If your String has multiple lines you must match all of them.
Thanks muchly Ritch. The square brackets are needed in JSON to reference a key with spaces in it.
However, I have moved away from using JSON to parse my data because it contains control chars , certainly newlines.
Notwithstanding design decisions as to what where things “should” go, let us focus on the very simple regexp - please - and understand if we can please, why it is not working. It shouldnt matter where the expr is , so lets just use this (bad) example eh, once working I will correct the design.
So I moved to REGEX with a very simple pattern…
(?:“Motor current below expectation”[^a-z]*?)(true|false)
Meaning:
- Non-capturing group (?:“Motor current below expectation”[^a-z]*?)
- “Motor current below expectation” matches the characters “Motor current below expectation” literally (case sensitive)
- Match a single character not present in the list below [^a-z]*?
- *? Quantifier — Matches between zero and unlimited times, as few times as possible, expanding as needed (lazy)
- a-z a single character in the range between a (index 97) and z (index 122) (case sensitive)
- 1st Capturing Group (true|false)
- 1st Alternative true
- true matches the characters true literally (case sensitive)
- 2nd Alternative false
- false matches the characters false literally (case sensitive)
Using the text in the OP , and including newlines and tabs, this expression correctly parses the text.
So the question is, and help needed, why is the regexp not working in OH, because the capture group 1 correctly yields “false”. Whilst the regexpr can be improved to better guard against bad data, it is very simple and works , question is why not in OH? What exactly is OH refusing to parse in the very simple regexp?.
p.s. Ritch,
I do not understand what you mean by:
In that case your expression must match the entire string and only the first group is what gets returned by the transformation. And by entire String I mean the whole thing, not just the line. If your String has multiple lines you must match all of them
`
What is the point of a regexp that must match the entire data, rather than just a string. And what is the “first group” in OH terms?
Regex appears to much different than how it should work, so I will go back to JSON, I’m almost there , because it recognised the structure , from openhab.log
2020-01-13 16:40:56.348 [WARN ] [ofiles.JSonPathTransformationProfile] - Could not transform state '{
"Motor current below expectation": false,
"Motor current always high": false,
"Motor taking too long": false,
"discrepancy between air and pipe sensors": false,
"air sensor out of expected range": false,
"pipe sensor out of expected range": false,
"low power mode is enabled": false,
"no target temperature has been set by host": false,
"valve may be sticking": false,
"valve exercise was successful": false,
"valve exercise was unsuccessful": false,
"driver micro has suffered a watchdog reset and needs data refresh": false,
"driver micro has suffered a noise reset and needs data refresh": false,
"battery voltage has fallen below 2p2V and valve has been opened": false,
"request for heat messaging is enabled": false,
"request for heat": false
}' with function '$.<jsonPath>' and format '<valueFormat>'
Reading this, Problem with spaces in json keys , it appears I need a rule to strip out newlines etc. Blimey, hard work this OH. I will go back to regexpr