Profile transformation into other types then String

[Mamut (Peter Prochazka)]

Hi,

I get error while i try to use channel profiles for transforming incoming channel values (XML) into Items.
Main issu is the prifiles genereate only Strings.

Number <itemName> { channel="<channelUID>"[profile="transform:XPATH", function="<xpath>", sourceFormat="<valueFormat>"]}

I propose to upgrade to introduceability to define “targetType” ability

Number <itemName> { channel="<channelUID>"[profile="transform:XPATH", function="<xpath>", sourceFormat="<valueFormat>"  targetType="Number" ]}

But not uset only for Number but also for DateTime/Contact/Switch too.

DateTime <itemName> { channel="<channelUID>"[profile="transform:XPATH", function="<xpath>", sourceFormat="<valueFormat>" targetType="DateTime" targetFormat="yyyy-MM-dd kk:mm:ss"]}

DateTime <itemName> { channel="<channelUID>"[profile="transform:XPATH", function="<xpath>", sourceFormat="<valueFormat>" targetType="DateTime" targetFormat="UNIXTIME"]}

I all ready implement it yet (in my private openhab, based on 2.5.1 git sources), but I want to open discusion about it.

It can be easily ported also into JSONPATH a JAVASCRIPT or other profiles too.

[rossko57]

I would like to see some intelligence applied here; if the profile can identify the type of the target Item, then it can automatically decide typing.

Don’t forget UoM in this process, e.g. Number:Temperature
I’d like to see a profile that allowed adding a specified unit to a number channel to make a quantity type.

See also -

[transform profiles] channel profiles only support String types

[rlkoshak (Rich Koshak)]

I would definitely like to see Profiles work with any Item type. In particular I’d love if it were able to be used to convert, for example, something the Channel requires to be a Switch to a Contact. But is that possible? Or does the requirements of the Channel take precedence and if the Channel requires a Switch the Item must be a Switch.

@Mamut

It can be easily ported also into JSONPATH a JAVASCRIPT or other profiles too.

If it were feasible, I’d like to see it made generic to the transform profile and not needing to be implemented in each and every transform type individually. Than, for example, I could use the MAP transform in my example above.

I think giving us more power to transform things into other types would be very useful over all.

[Mamut (Peter Prochazka)]

Now I try to deeple understand the implemetation a think how it can be done.

1. It must can be done in core part of openhab look like in class ProfileCallbackImpl.
   but you loose ability to put there any configuration like date paetrn or any pattern about data like date format …, igger rework in core.

2. Expose item type some how into ad-on (ad some new methods used for send information about item)
   and implment it again and in all transform add-ons.

For me both of way have some advantages and disadvatagse.

#1. is nice but not allow to also do “formating” and only fixed string will work. It OK with JS but imposible ti XPATH/JSONPATH because you need to prepara data to “proper format” you can’t use it into date parsing and onli one format is supported. You have nicer code bud ugly configuration becau foreach for example unixtime stamp you need to implement virtual parameter and rules.
I think is nog good approcch to expose part of add-on configuration into core.

#2. You need to implement it again and again in transformation add-on but you get simple configuration and and minimal impact on core (just exposing information about item type via core).
again and again implmentation can by elminated viaHelper function in core like comon part reusable by all add-ons (sample of this purpose - TransformationHelper).

Which one you think is better ?
After this philosophy result we may discuss more details

[Mamut (Peter Prochazka)]

There is question if we want to enable chaing of transforamtion profiles ?

#1 XML source -> XPATH-> javascript -> datType transform

In generic case or we is in enought to have

#2 XML Source -> XPATH(resultToType)

Too diffrent was how to go.
if there is demand about this kind of features like #1.
If we check multiple question/bugs discustion I think user want/miss simpler version 2.

[rossko57]

I expect you are finding out why the existing profiles have this limitation. Not so easy as it looks.

For comments about the inner workings of OH core you might want to change this thread’s tags to development to attract the right attention.
(It’s all beyond me)

[Mamut (Peter Prochazka)]

Limitation it self is easy output of function is String no retyping. It look like somebody was lazy.

StringType resultType = new StringType(result);
logger.debug("Transformed '{}' into '{}' ", state, resultType);
return resultType;

In end of all transformation in method transform.

[rlkoshak (Rich Koshak)]

I’m not an OH developer so details based on where and how to implement it is above me. But I do a lot of support on the forum so I do have opinions on how it would be used.

I would treat chaining of transformation profiles as something else and not lump it in with the changes you are proposing. I like the idea and use it heavily in MQTT but there are a lot of unanswered questions as to whether it’s appropriate for the profiles or not.

[Mamut (Peter Prochazka)]

@rlkoshak

on the forum so I do have opinions on how it would be use

I think this is the bast way not start with deep technology (devel) discusion but start what realy user wants.
In initial part of this topic I propose some way how it can be used. I think It will be way how to detect what item wants and then not to define exactly the type and define only hint for transforamtion.

Number temp0_value { channel=whatever:output [profile="transform:XXX", function="foo1"]}
DateTime temp0_time { channel=whatever:output [profile="transform:XXX", function="foo2" , targetFormat="yyyy-MM-dd hh:mm"]}'
Contact temp0_status {channel=whatever:output [profile="transform:XXX", function="foo3", targetFormat="true"]}
Switch temp0_power {channel=whatever:output [profile="transform:XXX", function="foo4", targetFormat="powered"]}

This is we I think is wanted ??? I think it is very simple and use full
Documetation
Item DateTime - then targetFormat must be:

1. format of provided timestamp like- > (DateTimeFormat (GWT Javadoc))
2. key word UNIXTIME - if timestamp is unix time epoch number of sec since 1.1.1970
3. key word UNIXTIMEMILLIS - if timestamp is unix time epoch number of millis since 1.1.1970
4. empty/not provided → like #1 “yyyy-MM-dd kk:mm:ss X”

Item Switch - then targetFormat must be hint for state ON:

1. format defines value for state ON
2. only exact mathch of this strin means ON any other mean OFF

Item Contact - then targetFormat must be hint for state OPEN:

1. format defines value for state OPEN
2. only exact mathch of this strin means ON any other mean CLOSED

Or do you have other proposal ?

[rossko57]

@Mamut

This is we I think is wanted ???

Not me.
I want -

Number tempN { channel=whatever:output [profile="transform:XXN", function="foo1"]}
Number:Temperature tempNT { channel=whatever:output [profile="transform:XXT", function="foo1"]}
DateTime tempDT { channel=whatever:output [profile="transform:XXD", function="foo2"]}'
Contact tempC {channel=whatever:output [profile="transform:XXC", function="foo3"]}

In other words, just do it.
I already know the type of my target Item, I don’t need a special parameter to instruct the transform.
I already have to write the transform to output a suitable format for that kind of Item. (and will get an error if I don’t)
example, I don’t expect “22.2” to ever be suitable for a DateTime no matter how hard I tell it to be

This would be directly analogous to
myContactItem.postUpdate("OPEN") etc.
myNumberTempItem.postUpdate("21.5 | °C") etc.
The framework can parse that string into the “real” Item state.
It’s up to me (my rule, UI or transform etc.) to supply an appropriate string there of course.

This doesn’t work now in profiles because -
Transform outputs are always strings. This is not the problem, that can be made to work fine in bindings and labels and other places.
But - the profile service cannot parse the string to any other Item type. It does not have the capabilities that postUpdate has.

[Mamut (Peter Prochazka)]

apropos “tragetFormat” is just optional hinthint for retyping to change defult behaviur it is optional if you accept defautl format and fit to you it is not needed.

as I mention in example and part documentation.
if your are OK with default then you not need to specify hint.

Example:
IF we agree the default beahiour for switch is if string is equal to “on” then output of profile is ON STATE of switch.
then you not need to specifie anything.

But if your thing no give respone about switch status as string "open’ but it produce string “powered”. Then you need to provide hit to transforamtion porfile wath does it mean powered.
please forget map transformation in this case, try to keep it simple and generic.

[rossko57]

@Mamut

if your are OK with default then you not need to specify hint.

I don’t understand what possible use it has. A Number Item accepts numbers, what would a non-default format/hint look like?

@Mamut

IF we agree the default beahiour for switch is if string is equal to “on” then output of profile is ON

But that’s what the transform already does, transforms “banana” or whatever the raw data is into “ON”. There is no existing problem at all with doing this part with the existing transform profile, whether we use a JS or a MAP or whatever.

The current problem is that we cannot send a returned “ON” to a Switch type Item.

[rlkoshak (Rich Koshak)]

What is targetFormat? Is this the format you want to send to the DateTime or the format you are expecting from the result of the transform?

DateTime Items already have a way to define it’s output format. All Items already have a way to control their output format. So defining the targetFormat in the profile doesn’t make sense.

If this is the format of the result of the transform, shouldn’t the transform output the format needed by the Item type in the first place? If this is what you mean than targetFormat is redundant.

I just don’t see the point of it. It seems like you are trying to reproduce functionality that already exists.

@rossko57

example, I don’t expect “22.2” to ever be suitable for a DateTime no matter how hard I tell it to be

And we should expect an error in the log if you try.

@rossko57

But - the profile service cannot parse the string to any other Item type. It does not have the capabilities that postUpdate has.

That really is the root of the problem. I never really understood why that is the case, but I don’t look at the code so I assumed there is some technical limitation. But from a usability perspective, I at least expected Profiles to work like postUpdate or like using a transform in a Channel config (e.g. see MQTT, Exec, etc.). Yes, the transform only outputs a String but we can then pass that String to the Item that tries to parse it and apply it. And if it fails generates an error.

@Mamut

as I mention in example and part documentation.
if your are OK with default then you not need to specify hint.

I still don’t see what the hint does or why it is needed. What is it providing the hint to? To the transform? What exactly would targetFormat="yyyy-MM-dd hh:mm" mean to a JSONPATH transform? What would targetFormat="true" mean to a MAP transform?

On the other hand, if you are trying to change what MyItem.state.toString outputs or how the Item appears on the sitemap, well we already have a way to format that. Two ways actually if you consider the fact that you can set the label on both the Item and in the sitemap. So this hint is not only duplicative of existing functionality, it is wholly inconsistent with how that functionality is already implemented.

If my complaint is that Profiles work against expectations based on how other parts of OH already work, I have to complain about this targetFormat too. It’s inconsistent.

@Mamut

But if your thing no give respone about switch status as string "open’ but it produce string “powered”. Then you need to provide hit to transforamtion porfile wath does it mean powered.
please forget map transformation in this case, try to keep it simple and generic.

I can’t forget about the map transform in this case because:

1. That’s what I put into profile="transform:XXC"
2. That’s what MAP transformation was created for
3. That what the MAP transformation is used for everywhere else in OH

The whole point of this exercise is to add the ability for the transform Profile to work with more Item types beyond just String Items. It isn’t to replace the existing transformations. If you want to create a separate and brand new Profile that does this sort of thing than maybe I could be on board. But from a usability perspective, it’s confusing and inconsistent and duplicative. It’s not an improvement IMHO.

[rossko57]

@rlkoshak

I assumed there is some technical limitation.

I’m sure I read in a PR somewhere that is difficult for a profile to determine what Item type is involved.
Seems reasonable - the channel type may be more easily determined, but one of the cases we’re trying to cover here is matching say a string channel to a DateTime Item.

[Mamut (Peter Prochazka)]

@rlkoshak

That what the MAP transformation is used for everywhere else in OH

Nice but go for exact examples:
we have thing based on exec addon

Thing exec:command:statusReport      [ command="/usr/bin/reportArrayStatus", interval=60, timeout=5 ]`

This command produce for example XML output

<?xml version="1.0"?>
<SmartStats controllerID="0" time="1578564101"  serialNumber="5A0613AB28A">
  <PhysicalDriveSmartStats channel="0" id="1" nonSpinning="false">
    <Attribute id="0x01" name="Read Error Rate" rawValue="1"/>
    <Attribute id="0x03" name="Spin-Up Time in millisecs" rawValue="8700"/>
    <Attribute id="0xC2" name="Current Internal Temperature" rawValue="51"/>    
  </PhysicalDriveSmartStats>
  <PhysicalDriveSmartStats channel="0" id="2" nonSpinning="true">
    <Attribute id="0x01" name="Read Error Rate" rawValue="0"/>
    <Attribute id="0x03" name="Spin-Up Time in millisecs" rawValue="7366"/>
    <Attribute id="0xC2" name="Current Internal Temperature" rawValue="43"/>    
  </PhysicalDriveSmartStats>
  <PhysicalDriveSmartStats channel="0" id="0" nonSpinning="false">
    <Attribute id="0x01" name="Read Error Rate" rawValue="0"/>
    <Attribute id="0x03" name="Spin-Up Time in millisecs" normalizedCurrent="193" normalizedWorst="165" rawValue="9333"/>
    <Attribute id="0x04" name="Start/Stop Count"rawValue="51"/>
    <Attribute id="0xC2" name="Current Internal Temperature" rawValue="49"/>    
  </PhysicalDriveSmartStats>
  <PhysicalDriveSmartStats channel="0" id="3" nonSpinning="true">
    <Attribute id="0x01" name="Read Error Rate" rawValue="0"/>
    <Attribute id="0x03" name="Spin-Up Time in millisecs" rawValue="7700"/>
    <Attribute id="0xC2" name="Current Internal Temperature" rawValue="39"/>    
  </PhysicalDriveSmartStats>
</SmartStats>

And you want to monitor/control via OH
Time → when the data was get ( parameter time in tag SmartStats)
temperature → of each disk (and want to store in rrd)
spinnig status → of each disk
.
how you can do this cmd thing have channel:
output - String - Output of the last execution of the command

Now you have 2 way how to do this
#1

     DateTime RAID_lastupdate             "Status [%s]" {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/@time" ] }

    Switch RAID_diskSpin_c1_d0 "Disk spin 1/0 [%s]"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='1']/@nonSpinning" ] }
    Switch RAID_diskSpin_c1_d1 "Disk spin 1/1 [%s]"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='1']/@nonSpinning" ] }
    Switch RAID_diskSpin_c1_d2 "Disk spin 1/2 [%s]"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='2']/@nonSpinning" ] }
    Switch RAID_diskSpin_c1_d3 "Disk spin 1/3 [%s]"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='3']/@nonSpinning" ] }

    Number RAID_diskTemp_c1_d0 "Disk temp 1/0 [%s]C"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='1']/Attribute[@id='0xC2']/@rawValue" ] }
    Number RAID_diskTemp_c1_d1 "Disk temp 1/1 [%s]C"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='1']/Attribute[@id='0xC2']/@rawValue" ] }
    Number RAID_diskTemp_c1_d2 "Disk temp 1/2 [%s]C"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='2']/Attribute[@id='0xC2']/@rawValue" ] }
    Number RAID_diskTemp_c1_d3 "Disk temp 1/3 [%s]C"  {channel="exec:command:statusReport:output" [ profile="transform:XPATH", function="/SmartStats/PhysicalDriveSmartStats[@id='3']/Attribute[@id='0xC2']/@rawValue" ] }

#2
Create rule trigered by change of channel output then post update each itemstatus in rule.

What is more confortable for user or better OH approach?

[splatch (Łukasz Dywicki)]

Maybe there is a way to go with org.eclipse.smarthome.core.thing.profiles.ProfileAdvisor? It knows the channel (thus its type) as well as item type and can recommend best profile to be used for given link.

[rossko57]

@Mamut

we have thing based on exec addon

Yes, that’s a good use of profile, splitting a single channel into several Items.

The missing piece is matching that string channel to say, a Number Item.
I think we all agree on that.

[rlkoshak (Rich Koshak)]

@Mamut

What is more confortable for user or better OH approach?

Either one. Some people are afraid of Rules for some reason and will avoid them at all costs. Therefore the Profile approach would be preferable. In other cases a user will prefer to parse out the XML and update Items from a Rule.

There is acknowledged and deliberate overlap between Profiles and Rules in this respect.

But like rossko57 points out, we can’t use Profiles to get at Switch/Contact/Color/Location/DateTime/Number/etc. types. We have to use a Rule for that.

That’s what I thought this whole discussion was about.

[Mamut (Peter Prochazka)]

Yes whole discusion is about it if there is need to have profile like approch for simple thing
NUmber,DateTime,Switch,Contact (Location) . I think we need to avoid use for complex types like Dimmer and Color… a are more complex and still need to use rules, because this transforamtion can be complex (may be) or can be implemened but with some simplification (like for dimmer 0=off any other is on with value and so on).

Or not If I may start to analyze details how to implementit or if it just waste of my time.
If I start then I try know how others see this feature.

I don’t know how to dermine what is need or not in OH. I found some issue which irritate and confuse me and when I check google and forums then I see same confusion from other users.

I try to collect the “wanted” specification or any nearly version.

At this time non automatic but defined version “targetFormat” is working for me, but it just simple hack.
It look like nameing of this configuration also do some disconfort, which mean I chose wrong name.
I mean time (if I have some with 3 childs) i try to fined more generic and easy way with auto detection of item type.

[rlkoshak (Rich Koshak)]

@Mamut

I think we need to avoid use for complex types like Dimmer and Color

Why? If you can transform what ever is coming in to the format needed by those types, it’s just more of the same. A Color is no more complicated than a DateTime. And if you consider Units of Measurement, Number Items are perhaps the most complicated of all.

@Mamut

because this transforamtion can be complex (may be)

Maybe this is a source of the disconnect between rossko57 and I and you. It’s not really your concern whether a transformation is hard or not. That’s not your job (in the scope of this conversation). That’s the job of the actual transformation and it’s the user’s job to figure out how to take what is coming in from the device and transforming it to a format that the Item can understand.

Your job is to make it so I can send whatever comes out of the transformation to any Item type, not just String types.

Just assume that it is possible to transform whatever comes in into a String that is suitable for any of the OH Item types. I say assume because there will always be cases where this is possible. If I have a device that sends me { hsv="250,67,75" } than JSONPATH:$.hsv will give me a String that I can apply, unchanged, to a Color Item, at least if I were using postUpdate from a Rule.

So, if we assume that it can be possible some times to transform whatever is coming from a device into a String that is suitable to populate any one of the Item types, you should support them all.