[SOLVED] How to escape double quotes from jsonpath string?

Hi Rich,

thanks a lot!

I slightly modified it to
transform("REGEX", ".*\"status\":\"(..).*", json)
This works like desired.
Thanks for your help (again :slight_smile:

Or you could probably use the .replace("\"'","") method after the JSON transform e,g.

val String response = transform("JSONPATH", "$.status", json).replace("\"","")

1 Like

Thanks, I will play around with this as well :slight_smile:

If you need to strip the quotes out before it gets assigned to an Item (e.g. a Number Item) you have to use the REGEX or JS transform. the double quotes will prevent OH from being able to parse the value into a Number. In this case though, since we are working with a String, the rule approach works just fine.

I hadn’t thought about that, but yes assigning to a non-string may be a problem with this approach. I haven’t tried it but am wondering whether it would be possible to cast to the correct type in such situations?

It already has to be of the right type before you can cast to it. Casting doesn’t really convert objects into another object, it is just a way to look at an existing object in a different way. For example, if I have the following class hierarchy:

                 Object
                 /    \
              Cat     Dog
              /  \    /  \
       Siamese Tabby Corgi Spaniel

I can cast a Corgi to a Dog or I can cast a Corgi to an Object. I cannot cast a Corgi to a Spaniel nor can I cast a Corgi to a Tabby.

If I have a Dog, I can cast the Dog to either a Corgi or a Spaniel if and only if the Object already is of type Corgi or Spaniel.

You can test this with the instanceof operator. If you have a Corgi Object but you are looking at it as if it were a Dog you can do something like if(myDog instance of Corgi) val Corgi = myDog as Corgi

So, if you were to look at the class hierarchy of String and Number Items you would see:

                    Object
                       |
               Some stuff I'm sure
                       |
                     Item
                       |
                  GenericItem
                  /          \
             StringItem    NumberItem

So you can cast a NumberItem to a GenericItem but not to a StringItem.

When you are working with Rules, by default it gives you your items as Type Item. But you know because you created the Items that MyNumberItem is of type NumberItem so you can cast it to a NumberItem.

You also know that because you created the Item, plus a little research in the Docs, that a NumberItem carries its state as DecimalType so you can cast the state to DecimalType. A little more research will tell you that DecimalType is itself of type Number so MyNumberItem.state as Number is valid. But a StringItem carries its state as a String so MyStringItem.state as Number will generate errors.

Make sense?

1 Like

Thanks and yes that sounds completely logical!

I’m sorry, but this thread is baffling me so much that I had to post! It looks like the OP was trying to parse the data on an external website instead of using openHAB’s JSONPATH transformation service, and that’s why the double quotes needed to be escaped. Why wouldn’t JSONPATH work, just as the OP had it in his original post and used in the example rule below? Am I missing something obvious? :upside_down_face:

rule "Test"
when
    Item Virtual_Switch_1 changed
then
  
    val String json = '{"status":"OK","statustext":"Switching the screen off"}'
    val String parsedJSON = transform("JSONPATH", "$.status", json)
    logDebug("Rules", "Test: response=[{}]",parsedJSON)
end

No, OP’s problem was they wanted to get the status field without the quotes around it. I.e OK instead of "OK". He is successfully extracting the value.

JSONPATH returns everything to the right of the :, which includes the " around the “OK”. There is no way to tell JSONPATH you only want what is in the quotes and not the quotes themselves.

You need to do additional processing to strip out the quotes, which the JSONPATH transform does not support. So to remove the " you need to use the JS transform to preform that extra step, REGEX transform to exclude the " in the first place, or write a rule that replaces the " with the empty string.

Tl;Dr, if you want “OK” JSONPATH works just fine. If you want OK to need to use something else.

Sorry, I did not want to create an extensive discussion.

My original idea was to use jsonpath, because the format returned from the website (according to it’s programmer) is json and I use this approach in a mqtt message (owntracks) to extract lat / lon from the string.

The only difference was, that the value I wanted to extract in my original post is written with double quotes (“OK”) - which is not the case for the value of lat / lon in the mqtt message ( {“t”:“c”,“tst”:1507033778,“acc”:165,"_type":“location”,“alt”:10,“lon”:xxx,“vac”:29,“lat”:xxx, …).

However, I have a working solution and I am happy.
So no need for a fight. :wink:

No no no! I was just so curious that I had to jump in and ask! No harm intended! I saw a puzzle and the solution was just baffling to me and had to ask. I can put it down… but my brain is still going to try to process why JSONPATH wouldn’t work and how quotes are in any way involved. A string is a string is a string…

Fair enough :slight_smile:
It’s really a huge advantage for guys like myself (with limited programming experience) to have a forum like this.
Otherwise I would never have been able to realize my (almost complete) home automation project.

Thanks to all of you for your patience and help!
And of course to @Kai and all the @maintainers
:slight_smile:

1 Like

Are you still confused?

For example:

    val String json = '{"status":"OK","statustext":"Switching the screen off"}'
    val String parsedJSON = transform("JSONPATH", "$.status", json)

    if(parsedJSON == "OK")     // will evaluate to false
    if(parsedJSON == "\"OK\"") // will evaluate to true

The " is actually part of the String that gets returned by JSONPATH and that was unwanted.

Thank you, Rich! This was still bugging me, and I really don’t like not being able to figure things out. I would love an education on this one.

That is the root of my confusion, as I do not see this behavior with JSONPATH! The following rule logs “Test: response is OK (no quotes)”. I would have MANY rules that would not be working if JSONPATH returned quotes in the strings.

rule "Test"
when
    System started
then
    val String json = '{"status":"OK","statustext":"Switching the screen off"}'
    val String parsedJSON = transform("JSONPATH", "$.status", json)
    logDebug("Rules", "Test: response=[{}]",parsedJSON)
    switch (parsedJSON) {
        case "OK" : logDebug("Rules", "Test: response is OK (no quotes)")
        case "\"OK\"" : logDebug("Rules", "Test: response is \\\"OK\\\" (with quotes)")
    }
end

Well, I can’t answer that. It might have something to do with the way you define the String in your rule above. It might have something to do with the format of the Strings your other JSONs return.

All I know is OP was getting the quotes as well in his JSONPATH and I have seen similar behavior in the past (all my JSONPATHs are Numbers now so I don’t see that behavior).

I get the impression that the JSONPATH library used by OH is not the most consistent nor capable (i.e. I’m pretty sure it doesn’t support bracket notation)

Maybe it is a bug that has been fixed in your version and OP was using an older version of OH.

I don’t know.

The OP was using curl to do a JSONPATH transform, instead of using OH’s transformation service. At least that is how I read the first post. Your solution addressed the immediate issue, where the value returned from the website was a string with escaped quotes, but the OP could have just used the transformation service. This was another big factor in my confusion. But if you’ve seen this behavior before, then the pieces are falling into place!

I’ve actually seen the stability and functionality of JSONPATH improve since starting to use it in OH1.8. A big fix for me was that it no longer throws a NPE when there is no value to return. It does currently support bracket notation too (I’m using 2.2 snapshot 1054).

Thank you again. You’re experience and expertise are very valuable and appreciated!

I don’t think that is correct because that doesn’t make sense to me. I don’t think you CAN do JSONPATH with curl. If you can it is news to me. But even so, that isn’t what NCO was doing. In the original posting he used curl to show what was being returned by the website.

It is unclear from this posting but he might be getting the website using a curl command using executeCommandLine to get the JSON data but he does state he is using the OH transformation with the line:

val String response = transform("JSONPATH", "$.status", json)

This is the part where I get a little lost. The term “escaped quotes” really only makes sense when you are trying to populate a String that includes quotes (e.g. val myString = "\"foo\""). Once you have a String that includes quotes, you just happen to have a String that include " characters. To continue the example, myString doesn’t include escaped quotes, it just includes quotes. We only have to deal with escaping them because the quotes have a special meaning to the programming language and you need to tell it that you really do mean just the character " and you are not trying to start or end a String.

NCO did use the transformation service.

I have which is largely why I just tried to address his immediate concern. As far as I know there is no underlying root cause that we can do anything about to address the problem. Also, there may be other situations where one wants to do similar string manipulations on the result from a JSONPATH (e.g strip the % off of a humidity reading) which, too, is not possible and would require using a REGEX or JS transform instead.

That is good to hear. I’ve only done some simple things with it but I know a lot of people are using it for far more complicated JSON than I’ve ever seen.

A big question I was unable to help someone with on another thread, does it support array indexes? For example:

{ "foo": ["one", "two", "three"] }

I’ve never been able to get something like $.foo[1] to get at the second element of the array to work. Luckily I’m usually able to figure something out with REGEX instead or know I can do it in JS if I have to.

1 Like

I read it as the OP was using curl to get a response from the JSONPATH Expression Tester website that he’d linked, and the response had quotes in it. If that is not the case, then everything makes a LOT more sense. But potentially identifies a bug in the JSONPATH transform, which you’ve stated that you’ve also experience. I’ve never seen it reported or seen it first-hand, and is just another point layer of befuddlement I had with reading this thread.

This makes me think of something though. In a recent post, I commented on Smart Quotes being a source of potential problems, which I have seen in my own rules. If the input to JSONPATH has smart quotes around the value, they will be returned in the result. For example, this will log Test: smartKeyValue=[null], smartValue=[“OK“], nosmart=[OK], arrayIndex=[two]. Maybe related. Maybe just an oddity.

rule "Test"
when
    System started
then
    val String json = '{“status1“:“OK“,"status2":“OK“,"status3":"OK","foo": ["one", "two", "three"]}'
    val String smartKeyValue = transform("JSONPATH", "$.status1", json)
    val String smartValue = transform("JSONPATH", "$.status2", json)
    val String nosmart = transform("JSONPATH", "$.status3", json)
    val String arrayIndex = transform("JSONPATH", "$.foo[1]", json)
    logDebug("Rules", "Test: smartKeyValue=[{}], smartValue=[{}], nosmart=[{}], arrayIndex=[{}]",smartKeyValue,smartValue,nosmart,arrayIndex)
end

Yes, array indexing works. I used them a while ago parsing data from a Bloomsky weather station and one of their older APIs.

1 Like

Hey Rich, I’ve now seen this issue first-hand, but I don’t know why it doesn’t affect my rules. Still investigating and testing some things, but I plan on submitting a PR when I understand it a little better.

@rlkoshak, it all makes sense now!