How to optimize gas station prices processing (Tankerkoenig)

Tags: #<Tag:0x00007faed5caff90>

Yes

Now i tried it with the expire binding approach and get similar results in the log.

rule "Trigger Gas Station Price Processing"
when
    
    Member of gGasStationPrices received update

then
    val String ruleIdentifier = ruleIdentification+"TriggerGasStationPriceProcessing"

    if (iSL_OH_GasStationItems_Timer.state != ON) {
        
        iSL_OH_GasStationItems_Timer.sendCommand(ON)
        logInfo(ruleIdentifier, "Gas station price sorting is triggered by {}. Delay started.", transform("MAP", "i18n_gasstation.map", triggeringItem.name.substring(1, triggeringItem.name.lastIndexOf('_'))))

    } else {

        logInfo(ruleIdentifier, "Gas station price sorting already triggered, ignoring additional request.", transform("MAP", "i18n_gasstation.map", triggeringItem.name.substring(1, triggeringItem.name.lastIndexOf('_'))))

    }

end
2019-07-13 19:03:58.142 [INFO ] [ule.TriggerGasStationPriceProcessing] - Gas station price sorting is triggered by HEM (Elmshorn). Delay started.
2019-07-13 19:03:58.143 [INFO ] [ule.TriggerGasStationPriceProcessing] - Gas station price sorting is triggered by Nordoel (Elmshorn). Delay started.
2019-07-13 19:03:58.145 [INFO ] [ule.TriggerGasStationPriceProcessing] - Gas station price sorting is triggered by Star (Elmshorn). Delay started.
2019-07-13 19:03:58.154 [INFO ] [ule.TriggerGasStationPriceProcessing] - Gas station price sorting already triggered, ignoring additional request.
2019-07-13 19:03:58.158 [INFO ] [ule.TriggerGasStationPriceProcessing] - Gas station price sorting already triggered, ignoring additional request.

Perhaps they are happening too close together and they all make it past the if statement before the first time is created.

I’d change the rule trigger to use changed instead of update. That will eliminate some of the triggers when there is no change.

Since they happen so close together (less then 10 msec) a lock may be the only option. Since that’s the case you have to ask whether the danger of using a lock, which has all sorts of ways things can go wrong, is worth the benefit of having a rule not run repeatedly unnecessarily, which isn’t causing any real problems.

did you initialise delayTimer to null?

var Timer delayTimer = null

@JimT Yes it is initialised to null.

@rlkoshak I think the events are to close to gether and the “problem” will increase if i use faster hardware. So i have to stick with my initial solution or build something new. The hint to use “changed” as a trigger is obvious. I used the “update” trigger only because is triggers regulary every 15 minutes.

Thanks for your ideas

I’m curious. Can you try this:

rule "Process Gas Station Prices"
when
    
    Member of gGasStationPrices received update

then
    
    val String ruleIdentifier = ruleIdentification+"ProcessGasStationPrices"

    if (delayTimer !== null) {
        logInfo(ruleIdentifier, "Gas station price sorting already triggered, ignoring additional request from {}.", triggeringItem.name.toString)
        return;
    }

    logInfo(ruleIdentifier, "Gas station price sorting is triggered by {}.", triggeringItem.name.toString)
    if (delayTimer === null) {
        delayTimer = 1 // not sure if this will work due to type issue, but presumably this should execute very quickly compared to createTimer()
        delayTimer = createTimer(now.plusSeconds(delayValue), [ |
            logInfo(ruleIdentifier, "Gas Station prices are sorted.")
            delayTimer = null
        ])
    }
end

Just realize that you are “fixing” something that isn’t a problem with something that has the potential to stop all of your rules from executing. For example, if one instance of the rule crashes in a way where the lock never gets unlocked (which can happen even with a try/catch/finally) all the other instances will be stuck waiting for the lock and keep any other rule from running.

That seems to be a pretty severe risk to solve a pretty benign annoyance.

Is there any way you can move this further up the chain? Maybe vary the poll times so only one it to changes at a time? Or you get all all the values in one message that you then parse in a single rule?

If not I’d say it’s safer and better to leave well enough alone.

If have gone back to my initial solution with your sorting optimisation. And it works well.

but maybe there is another solution without any risk. My initial approach was to display the gas sation prices in a sorted order so that i see the cheapest first. So i need to see the station name and the price.

The binding polls a webservice and updates the items after each poll (every 15 Minutes). Another way is ti poll the service on my own, but i think that is not the solution.

What about splitting the rule into two (based on the expire binding approach):
1.

rule "Timer Gas Station Price Processing"
when
    
    Member of gGasStationPrices changed

then
    postUpdate.iSL_OH_GasStationItems_Timer(ON)
end
rule "Timer Gas Station Price Processing"
when
    Item iSL_OH_GasStationItems_Timer changed to ON
then
    //sort gastation logic
end

Using the binding updates will come only at the scheduled times. I would use a crown triggered rule iot update the sorting. Look into the log at what time the poll by the binding is done and schedule the rule to trigger some seconds after.
Yes, that way the cron has to be reset each time the binding schedule is changed.

@Syn My first approach with two rules did not succeed, but i wil give it a try. The main difference i see is that you dont care about retriggering the expire timer. That might be the solution.

@opus Thats a solution, but i do not want to change the cron statement every time i change something.

Totally understand the reasoning!
What you WOULD need is a single event raised by the binding each time the prices are polled. Such would be possible (although, as of now,I do’t know how to code it), however that would add an additional event (besides the update/changed event of each price-channel) just for this purpose.

There is no way to ask the binding to use a different polling time for each station? Anything at all to stagger the updates to the Items so they are less likely to all get updates at the same time.

Polling the service yourself could be a solution.

Given how minor the issue is with the Rule running more often then necessary and how severe the potential problems that can be caused by using a lock, I cannot recommend that approach. The risks from the lock far outweigh the original problem.

If you want to fix this minor problem, the only way I see is to move back further in the chain so you can either try to keep the Items from all updating at the same time or get the raw data and update your sorted Items from that.

That could work. The Timer will get updated to ON a bunch of times but then X seconds after the last update it will sort the values. This is a good solution as long as the number of seconds the Expire is set to is shorter than the frequency that the Items are updated. You want to make sure that the sorting Rule is done before the next update or else you will be sitting there waiting forever and the Expire timer never goes off.

I like this approach.

I thought of the cron triggered approach too and was going to post that this morning. I think I like Christoph’s approach a little better but cron would be a good approach too.

That is the key difference. Instead of ignoring all the events after the first one, you ignore all the events until the last one. It’s a subtle difference that I think will make the approach viable.

Yes there is a way. Use a separate Webservice for each fuel station. However, such would increase the load on the Tankerkoenig server, which they asked not to do!
The approach posted by @Syn sounds best to me as well.
Since all updates do come from a single poll the sorting should be triggered after any poll that made a change. Since the poll is done in seconds an exipre time of about 10 seconds should be long enough.

@opus @rlkoshak

Tankerkoenig’s API offers two way to query a gas station. First is to query a single station and second is query upto 10 stations at once. Tankerkoenig request to use the second solution to keep the load of the server low.

I tried @ Syn solution now an with all the optimisations it is working now. Next step is to bring the display to Habpanel. I will post the items and rules later.

1 Like

Correct, but we were referring to the binding!

Yes, thats true, but the binding uses from my point of view the second option (up to 10). It is written in the documentation that a second webservice is required if more than 10 stations should be queried. Possible modification for the binding are:

1.) Use the single station api and wait between two stations a to be defined amount of time. Drops the 10 station limit, but increases the server load

2.) Use the multi station api and wait between the item update a to be dfined amount of time. That’s implementation work and the 10 station limit is still there

3.) Create a new channel that triggers if all items are updated after data is polled. Again implementation work and this did not overcome the 10 station limit.

So from my point of view my current solution is suitable for me and i overcomes the 10 station limit.

Exactly the binding does use this second option and the limit is set to 10 stations (as agreed with tankerkoenig). My posted solution (using a webservice for each station) was showing a possible way with the current binding, not the suggested way! Yes that way the api-call for several stations would be used for a single one.

I’d rather not change the binding for this one and if I’d change I suggest the the binding would raise an event after each time the prices (plural) got polled,.

However that’s all talks about what you do not need, since you got a good solution (I think).

Your are absolutly right. The solution is working and i am happy.

To finish this thread, please find below all related rules, items, sitemaps, etc.

Gas station things

Bridge tankerkoenig:webservice:xxxxxxxxxxx "Bridge_Tankerkoenig Tankerkönig Webservice" [ apikey="xxxxxxxxxxxxxxx", refresh= 15, modeOpeningTime =true ] {

    Thing station station01 "SL_IN_GasStation01 Nordoel (Elmshorn)" @ "Internet"[ locationid = "1ceea95b-d46e-4159-91f6-71f57d612bdb" ]
    Thing station station02 "SL_IN_GasStation02 Nordoel (Uetersen)" @ "Internet"[ locationid = "3d8c2dfb-4612-4aff-8ae7-96e3f88eafe6" ]
    Thing station station03 "SL_IN_GasStation03 HEM (Elmshorn)" @ "Internet"[ locationid = "e1a15081-24e2-9107-e040-0b0a3dfe563c" ]
    Thing station station04 "SL_IN_GasStation04 Star (Elmshorn)" @ "Internet"[ locationid = "005056ba-7cb6-1ed2-bceb-bc577a5e6d4e" ]
    Thing station station05 "SL_IN_GasStation05 HEM (Wedel)" @ "Internet"[ locationid = "e1a15081-254f-9107-e040-0b0a3dfe563c" ]
    Thing station station06 "SL_IN_GasStation06 SB (Wedel)" @ "Internet"[ locationid = "65c348cf-5ce7-4820-a24a-da0dbf816204" ]
}

and all the items that i use

/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: All additional items to prcoess gas station data

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_OH_GasStationItems "Gas station realted items" <scenegasstation> (gSL_openHAB)
    DateTime                    iSL_OH_GasStationItems_LastUpdate
                                "Gas station: Last update [%1$td.%1$tm.%1$ty %1$tH:%1$tM]"
                                <timeclock>
                                (tSL_OH_GasStationItems)

    Switch                      iSL_OH_GasStationItems_Timer
                                "Gas station: Timer [MAP(switch.map):%s]"
                                <timesandglass>
                                (tSL_OH_GasStationItems)
                                { expire="2m,state=OFF" }

    String                      iSL_OH_GasStationItems_Display01
                                "Gas station: Display 01 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

    String                      iSL_OH_GasStationItems_Display02
                                "Gas station: Display 02 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

    String                      iSL_OH_GasStationItems_Display03
                                "Gas station: Display 03 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

    String                      iSL_OH_GasStationItems_Display04
                                "Gas station: Display 04 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

    String                      iSL_OH_GasStationItems_Display05
                                "Gas station: Display 05 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

    String                      iSL_OH_GasStationItems_Display06
                                "Gas station: Display 06 [%s]"
                                <scenegasstation>
                                (tSL_OH_GasStationItems)

/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation01 - Nordoel (Elmshorn)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation01 "Gas station 01: Nordoel (Elmshorn)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation01_Diesel
                                "Gas station 01: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation01, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station01:diesel" }

    Contact                     iSL_IN_GasStation01_Open
                                "Gas station 01: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation01)
                                { channel="tankerkoenig:station:100720191523:station01:station_open" }


/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation02 - Nordoel (Uetersen)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation02 "Gas station 02: Nordoel (Uetersen)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation02_Diesel
                                "Gas station 02: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation02, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station02:diesel" }

    Contact                     iSL_IN_GasStation02_Open
                                "Gas station 02: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation02)
                                { channel="tankerkoenig:station:100720191523:station02:station_open" }


/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation03 - HEM (Elmshorn)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation03 "Gas station 03: HEM (Elmshorn)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation03_Diesel
                                "Gas station 03: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation03, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station03:diesel" }

    Contact                     iSL_IN_GasStation03_Open
                                "Gas station 03: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation03)
                                { channel="tankerkoenig:station:100720191523:station03:station_open" }


/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation04 - Star (Elmshorn)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation04 "Gas station 04: Star (Elmshorn)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation04_Diesel
                                "Gas station 04: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation04, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station04:diesel" }

    Contact                     iSL_IN_GasStation04_Open
                                "Gas station 04: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation04)
                                { channel="tankerkoenig:station:100720191523:station04:station_open" }


/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation05 - HEM (Wedel)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation05 "Gas station 05: HEM (Wedel)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation05_Diesel
                                "Gas station 05: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation05, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station05:diesel" }

    Contact                     iSL_IN_GasStation05_Open
                                "Gas station 05: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation05)
                                { channel="tankerkoenig:station:100720191523:station05:station_open" }


/*---------------------------------------------------------------------------------------------------------------------
  
    Thing: SL_IN_GasStation06 - SB (Wedel)

---------------------------------------------------------------------------------------------------------------------*/
Group tSL_IN_GasStation06 "Gas station 06: SB (Wedel)" <scenegasstation> (gSL_Internet)
    Number                      iSL_IN_GasStation06_Diesel
                                "Gas station 06: Diesel [%.3f €]"
                                <scenegasstation>
                                (tSL_IN_GasStation06, gGasStationPrices, gStandardTimeline)
                                { channel="tankerkoenig:station:100720191523:station06:diesel" }

    Contact                     iSL_IN_GasStation06_Open
                                "Gas station 06: Status [MAP(contact.map):%s]"
                                <scenegasstation>
                                (tSL_IN_GasStation06)
                                { channel="tankerkoenig:station:100720191523:station06:station_open" }

and the rules to sort the prices

/*---------------------------------------------------------------------------------------------------------------------
  
    Imports

---------------------------------------------------------------------------------------------------------------------*/
import org.eclipse.smarthome.model.script.ScriptServiceUtil


/*---------------------------------------------------------------------------------------------------------------------
  
    Global variables and constants

---------------------------------------------------------------------------------------------------------------------*/
val String ruleIdentification = "user.gasstationrule."


/*---------------------------------------------------------------------------------------------------------------------
  
    Rule: Gas Station Price Change Trigger

    Description:    This rule triggers the trigger item which is connected to the expire binding. After 2 minutes the
                    switch is reset to Off and the price sort rule is executed.

---------------------------------------------------------------------------------------------------------------------*/
rule "Gas Station Price Changed Trigger"
when
    
    Member of gGasStationPrices changed

then
    
    val String ruleIdentifier = ruleIdentification+"GasStationPriceChangedTrigger"

    logInfo(ruleIdentifier, "Gas station prices have changed. Trigger is {}.", transform("MAP", "i18n_gasstation.map", triggeringItem.name.substring(1, triggeringItem.name.lastIndexOf('_'))))
    iSL_OH_GasStationItems_Timer.postUpdate(ON)

end


/*---------------------------------------------------------------------------------------------------------------------
  
    Rule: Process Gas Station Prices

    Description: Sort gas station prices and setup the diaplay items in the right order.

---------------------------------------------------------------------------------------------------------------------*/
rule "Gas Station Price Sorter"
when

    Item iSL_OH_GasStationItems_Timer changed to OFF     

then
    
    val String ruleIdentifier = ruleIdentification+"GasStationPriceSorter"

    var Double actualPrice
    var Double previousPrice
    var String indicator

    logInfo(ruleIdentifier, "Gas station price sorting has been triggered.")

    gGasStationPrices.members.filter[ price | (price != UNDEF) && (price != NULL) ].sortBy[ state as DecimalType ].forEach[ gasPrice, index |
        
        val stationOpen = ScriptServiceUtil.getItemRegistry.getItem("i"+gasPrice.name.substring(1, gasPrice.name.lastIndexOf('_'))+"_Open") as ContactItem
        val stationDisplay = ScriptServiceUtil.getItemRegistry.getItem("iSL_OH_GasStationItems_Display"+String::format("%02d", index+1)) as StringItem

        stationDisplay.setLabel(transform("MAP", "i18n_gasstation.map", gasPrice.name.substring(1, gasPrice.name.lastIndexOf('_'))))

        if (stationOpen.state == OPEN) {
        
            actualPrice = (gasPrice.state as DecimalType).doubleValue()
            previousPrice = (gasPrice.previousState(true).state as DecimalType).doubleValue()
            indicator = if (actualPrice > previousPrice) "↑" else if (actualPrice < previousPrice) "↓" else "→"
            
            stationDisplay.postUpdate(String::format("%s %.3f €", indicator, actualPrice))
        
        } else {
        
            stationDisplay.postUpdate(transform("MAP", "i18n_contact.map", stationOpen.state.toString))
        
        }
    
    ]

    iSL_OH_GasStationItems_LastUpdate.postUpdate(now.toString)
    logInfo(ruleIdentifier, "Gas station prices are sorted now")

end

and last but not least the sitemap to display the items

sitemap smarthome label="Smart Home" {

    Text label="Information" icon="messageinfo" {

        Frame label="Dieselpreise" {
            Text item=iSL_OH_GasStationItems_LastUpdate label="Letzte Abfrage [%1$tH:%1$tM]" icon="timeclock"
            Text item=iSL_OH_GasStationItems_Display01
            Text item=iSL_OH_GasStationItems_Display02
            Text item=iSL_OH_GasStationItems_Display03
            Text item=iSL_OH_GasStationItems_Display04
            Text item=iSL_OH_GasStationItems_Display05
            Text item=iSL_OH_GasStationItems_Display06
        }

    }

}

and my first try with habpanel based on the matrix theme.

<div class="section">

  <div class="sectionIconContainer">
    <div class="sectionIcon"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
  </div>

  <div class="title">Dieselpreise {{itemValue('iSL_OH_GasStationItems_LastUpdate') | date:'HH:mm'}}</div>

	<div class="controls">

		<div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">1. {{getItem('iSL_OH_GasStationItems_Display01').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display01')}}</div></div>			
		</div>

		<div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">2. {{getItem('iSL_OH_GasStationItems_Display02').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display02')}}</div></div>			
		</div>

		<div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">3. {{getItem('iSL_OH_GasStationItems_Display03').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display03')}}</div></div>			
		</div>

		<div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">4. {{getItem('iSL_OH_GasStationItems_Display04').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display04')}}</div></div>			
		</div>

		<div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">5. {{getItem('iSL_OH_GasStationItems_Display05').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display05')}}</div></div>			
		</div>

    <div class="widget">
			<div class="icon off"><svg viewBox="0 0 361 361"><use xlink:href="/static/matrix-theme/knx-uf_icons.svg#scenegasstation"></use></svg></div>
			<div class="name">6. {{getItem('iSL_OH_GasStationItems_Display06').label}}</div>
			<div class="valueGroup"><div class="value">{{itemValue('iSL_OH_GasStationItems_Display06')}}</div></div>			
		</div>

	</div>
  </div>
3 Likes