Http doesn't send out http request when using transformation in things

tdlr · August 26, 2023, 5:31pm

Openhab Configuration

##   Release = Debian GNU/Linux 12 (bookworm)
##    Kernel = Linux 6.1.0-10-amd64
##  Platform = VMware Virtual Platform/440BX Desktop Reference Platform
## CPU Usage = 1.49% avg over 2 cpu(s) (1 core(s) x 2 socket(s))
##  CPU Load = 1m: 0.08, 5m: 0.02, 15m: 0.01
##    Memory = Free: 0.88GB (23%), Used: 3.16GB (77%), Total: 3.82GB
##      Swap = Free: 0.95GB (100%), Used: 0.00GB (0%), Total: 0.95GB
##      Root = Free: 10.20GB (73%), Used: 3.73GB (27%), Total: 14.70GB

Problem Statement
I’m using the http binding to query and set a media device. It uses XML which has been a challenge for me. I’ve been able to create a time-based js script that turns on the radio, sets the tuner frequency, and the default volume. With the http binding, I’m able to query the settings on the radio, parse the xml data, and update items, The challenge is when trying to control the radio via Openhab. To configure it, I must be able to send the appropriate request payload. I could use a dummy item and trigger the change with scripts but I wish to learn how to use the transformation functions.

Steps Tried

Created a http channel thing for turning on/off the device. Parsed the state with XPATH expression
Linked it to an item without using any profile transformation
Verified with Wireshark that request went out on the wire with the respective ON/OFF value
Created a js tranformation via the GUI
Went back into http channel and click on existing item and change the transformation to js transformation. Assigned script.
Enabled TRACE on http binding and verified no errors.
Turn switch ON and OFF and verified incoming data and returned value from js script transformation on console.
Verified with Wireshark that no request packet was sent out on the wire.

Questions

Why is the item profile display only when I access the item from the thing channel?
What is being passed to the profile transformation? Is it an object or string? I presume it is the latter because I’m able to successfully do a comparison.
Can the returnValue be a long string? I’m suspecting that somehow, the binding is treating it like a null.
Is there anymore diagnostic data available from the http binding?

Configuration items are shown below.

http Thing Configuration

UID: http:url:Yamaha_CRX-N560
label: Yamaha CRX-N560
thingTypeUID: http:url
configuration:
  authMode: BASIC
  headers:
    - content-type=application/xml
  ignoreSSLErrors: false
  baseURL: http://ipaddress/YamahaRemoteControl/ctrl
  delay: 5000
  stateMethod: POST
  refresh: 30
  commandMethod: POST
  contentType: text/xml
  timeout: 30000
  bufferSize: 2048
location: Kitchen
channels:
  - id: PowerStatus
    channelTypeUID: http:string
    label: Power Status
    description: ""
    configuration:
      mode: READONLY
      stateTransformation: XPATH:/YAMAHA_AV/System/Power_Control/Power
      stateContent: <?xml version="1.0" encoding="utf-8"?><YAMAHA_AV
        cmd="GET"><System><Power_Control><Power>GetParam</Power></Power_Control></System></YAMAHA_AV>
  - id: PowerControl
    channelTypeUID: http:switch
    label: Power Control
    description: ""
    configuration:
      mode: READWRITE
      onValue: On
      stateContent: <?xml version="1.0" encoding="utf-8"?><YAMAHA_AV
        cmd="GET"><System><Power_Control><Power>GetParam</Power></Power_Control></System></YAMAHA_AV>
      offValue: Standby
      stateTransformation: XPATH:/YAMAHA_AV/System/Power_Control/Power
  - id: InputSel
    channelTypeUID: http:string
    label: Input Select
    description: ""
    configuration:
      mode: READONLY
      stateTransformation: XPATH:/YAMAHA_AV/System/Basic_Status/Input/Input_Sel
      stateContent: <?xml version="1.0" encoding="utf-8"?><YAMAHA_AV
        cmd="GET"><System><Basic_Status>GetParam</Basic_Status></System></YAMAHA_AV>

JS script transformation created via GUI

(function(data) {
  PowerSet = '<?xml version="1.0" encoding="utf-8"?><YAMAHA_AV cmd="PUT"><System><Power_Control><Power>data</Power></Power_Control></System></YAMAHA_AV>';
  console.log("Incoming data is:", data);
  if (data == "ON") {DesiredPower = "On"} else (DesiredPower = "Standby");
  returnValue = PowerSet.replace("data", DesiredPower);
  console.log("Transformation is:", returnValue);
  return returnValue
})(input)

Console log when js transformation script is used. Incoming data is OFF and a return string is returned.

==> /var/log/openhab/openhab.log <==
2023-08-26 09:48:09.975 [TRACE] [.HttpDynamicStateDescriptionProvider] - returning new stateDescription for http:url:Yamaha_CRX-N560:PowerControl
2023-08-26 09:48:09.975 [INFO ] [org.openhab.automation.script       ] - Incoming data is: OFF
==> /var/log/openhab/events.log <==
2023-08-26 09:48:09.975 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'Power_CRX_N560' updated to OFF
2023-08-26 09:48:09.975 [INFO ] [openhab.event.ItemStateChangedEvent ] - Item 'Power_CRX_N560' changed from ON to OFF
==> /var/log/openhab/openhab.log <==
2023-08-26 09:48:09.975 [INFO ] [org.openhab.automation.script       ] - Transformation is: <?xml version="1.0" encoding="utf-8"?><YAMAHA_AV cmd="PUT"><System><Power_Control><Power>Standby</Power></Power_Control></System></YAMAHA_AV>

Wireshark Capture on Openhab installation shows outgoing request with no transformation. No request goes out if a js transformation is used.

Hypertext Transfer Protocol
    POST /YamahaRemoteControl/ctrl HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): POST /YamahaRemoteControl/ctrl HTTP/1.1\r\n]
            [POST /YamahaRemoteControl/ctrl HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: POST
        Request URI: /YamahaRemoteControl/ctrl
        Request Version: HTTP/1.1
    Accept-Encoding: gzip\r\n
    User-Agent: Jetty/9.4.50.v20221201\r\n
    Content-Type: text/xml\r\n
    Content-Type: application/xml\r\n
    Host: kitchen.radio.ip\r\n
    Content-Length: 2\r\n
        [Content length: 2]
    \r\n
    [Full request URI: http://kitchen.radio.ip/YamahaRemoteControl/ctrl]
    [HTTP request 1/1]
    File Data: 2 bytes
eXtensible Markup Language
    On

tdlr · August 27, 2023, 9:32pm

Some additional observations

I verified that a string is being passed in the js transform using the typeof() function.
I created a very simply transform where I set the data (input) equal to the returnValue (output). That got sent on the wire.
Any other string manipulation does not get sent on the wire.

GUI JS Simple Tranformation

(function(data) {
  console.log("Type of variable: ", typeof(data));
  //returnValue = data;                // Works.  Http Request is seen in Wireshark.
  returnValue = data + " string";      // HTTP Request not seen in Wireshark.  Openhab reports no errors.
  console.log("Type of variable: ", typeof(returnValue));
 return returnValue
})(input)

rlkoshak · August 28, 2023, 2:42pm

You are not accessing the Item. You are accessing the Link. If you are on the Item’s page and click on the “Channel Link” you will get to the same place. The Link is separate from and independent from both the Item and the Channel.

The transforms always receive and are expected to produce a String.

There is no limit to the length of the result from a transform unless the binding itself has one. Nothing in OH core limits the length.

TRACE level logging is all. But if the problem is the profile that is independent from the binding.

I don’t see any commandTransformation entries. That is what gets applied when commanding the Item. There are two different transformation fields, one to transform the data coming from the device and another different one to transform the command on the Item to a message that the device can understand.

With the config you posted, there is nothing there that is actually calling your JS transformation. They are all using the XPATH on the incoming data and nothing on the outgoing data, assuming no profiles are configured. So either this Thing presented is not complete or not up to date or you’ve set up a profile to call the JS transformation.

tdlr · August 28, 2023, 4:28pm

Thanks @rlkoshak for answering most of my questions.

rlkoshak
I don’t see any commandTransformation entries. That is what gets applied when commanding the Item. There are two different transformation fields, one to transform the data coming from the device and another different one to transform the command on the Item to a message that the device can understand.

I believe the XPATH transformation works only on the stateTransformation entries. From what I read in the forums, I could not find an example of a XPATH command transformation that I can use to pass the entire string or to properly create a command string

With the config you posted, there is nothing there that is actually calling your JS transformation. They are all using the XPATH on the incoming data and nothing on the outgoing data, assuming no profiles are configured. So either this Thing presented is not complete or not up to date or you’ve set up a profile to call the JS transformation.

Yes, I left the commandTransformation entry blank in the channel entry but I applied the js script transformation in the channel link. I presumed that it would just pass that string in the request payload.

rlkoshak · August 28, 2023, 4:35pm

Correct. I don’t expect to see an XPath for the outgoing transformation. But I don’t see any outgoing transformations to include your JS one. At least not on the Thing and you didn’t show it being used on a Profile.

Why configure it two different ways? I’d expect either both to be defined as a profile or both to be defined on the Thing (I’m partial to moving such things as close to the data as possible meaning on the Thing).

tdlr · August 28, 2023, 5:13pm

Are you suggesting that I call out the js script in the command transformation entry? In 4.1, we have two ways of creating transformations. One in the GUI and one in cli. The transformation I’m using was created in the GUI. I tried entering “config:js:yamahaSetPower” and “js:yamahaSetPower” in the Command Transformation entry and neither worked.

Since this is a switch channel, I received an error in the http thing if I don’t supply the ON/OFF optional values. The State Content entry requests the XML data and the State Transformation entry parses the data into ON/OFF state. I agree with you that I would like to keep it close the thing but I’m not aware of any support for outgoing XPATH transformations.

tdlr · August 28, 2023, 5:21pm

By the way, I did call out the JS script transformation in the Item To Thing Transformation entry of the channel link

rlkoshak · August 28, 2023, 5:51pm

Yes.

Irrelevant. A transform is a transform and can be used anywhere a transform can be used.

You still have to tell it which transform service to use. JS:config:js:yamahaSetPower. I know it’s redundant but that’s the way it is. You gotta have that JS in front. After that is either the name of the file or the UID of the managed transform.

But that’s what the transform is for. To transform that ON/OFF to what ever it needs to be used. In the Thing to Item direction you’ve the xpath transform to convert the XML to ON/OFF. In the Item to Thing direction, you need something to transform the ON/OFF to XML that the device understands.

You wouldn’t use xpath for the outgoing transformation. You could use Jinja, or one of the script transformations depending on how transformed the data needs to be. Jinja is good if the XML is relatively short and you just need to drop the Item state unmodified into a certain spot.

The ON/OFF optional values get evaluated after the transformations for incoming and before the transformation outgoing. I don’t know why they are not optional but they are not. But if the result of your transform is already ON/OFF just put those in for the mapping.

You didn’t until I asked for it which lead to this whole conversation in the first place

tdlr · August 28, 2023, 6:20pm

Yipee! That worked. I knew that I had to preface the name of the transform with the type of transform but what threw me off was the lowercase js prepended to the name of the managed transform. Sometimes, I’m staring at the problem too long that after awhile I forget what worked or failed. You came through for me again.

In summary, I agree to keep the state and command transformation together, preferably in the thing channel. I did pursue having the transformations in the channel link profile but the mandatory ON/OFF optional values prevented me from doing the state transformation on the channel link profile.

I still think that the channel link profile needs some refinement or the transformation documentation needs clarification since my script failed at the channel link profile but not when run from the command transformation on the thing channel. In particular, should users avoid mixing transformations between the thing channel and the channel link profile?

Regardless, I appreciate you taking the time to help me with my issue. This has been long challenge for me and this transformation example will definitely open up opportunities. Thank you again!

rlkoshak · August 28, 2023, 7:17pm

No doubt.

The transformations might be happening in a different oder for those two cases. With the profile the transformation happens before it gets to the Thing. Then the Thing tries to do that ON/OFF mapping and of course fails because that doesn’t make sense for a full XML document. When the transform is a part of the Channel, the ON/OFF mapping happens first and then that mapped value is transformed.

Understanding the order of the operations is important.

Unfortunately the HTTP binding is one of those low level bindings that can be challenging to set up because they are so low level.

There is nothing technically wrong with doing it that way but users should indeed avoid doing so as it vastly increases complexity and reduces the feedback. Generally when a transform fails on a Channel you get pretty good error messages. At the Profile level not as much.