[http] refresh and delay

(OH3 3.1, HTTP binding)

Is someone able to clarify the expected behaviour:

  • refresh - this is the time between activation of a new set of requests for an entire Thing.
  • delay - this is the time between requests between each Channel within a single Thing.

Is that correct? If not, what is?

Background: I have an HTTP Thing with ~10 Channels, but every once in a while the device that is being interrogated has a panic and the request times out. I assumed this was because openHAB was requesting all Thing Channels (with sightly different stateExtension URLs) at the same time, freaking it out. In the documentation, delay has the description:

Delay between two requests in ms (advanced parameter).

which is a little ambiguous. Which requests? A Thing? (Though that would then be the same as refresh, wouldnā€™t it). Or between the Channel requests within a Thing?

Delay: Between two requests within the same thing. Refresh: between refreshing the same channel.

Refresh: 300s, delay 1000ms:

t=0: request channel 1
t=1: request channel 2
t=2: request channel 3
t=300: request channel 1
t=301: request channel 2
t=302: request channel 3

And so on.

Please note that channels with the same stateExtensiob are refreshed with one request.

Cheers @J-N-K , that example is clear.

Does the request fire irrespective of the state of the previous request? i.e. does the timeout come into this at all?

For example t=0 takes longer than a second for Channel 1 to receive data; does Channel 2 wait, or go ahead at t=1 regardless?

The result of the previous request is not considered. Rate-Limiting just delays the sending of the second request by the delay time.

Hi, although this is a rather mature topic, I would like to come up with a question about your statement

Please note that channels with the same stateExtensiob are refreshed with one request.

I am using openHABian on a RPI4 / 8GB; openHAB version 4.0.3. The HTTP binding is used to get data from a Rademacher HomePilot with really many (200+) devices.

I was expecting timeouts of some requests, but they came way sooner than expected. Anyhow, I used the ā€œdelayā€ setting for rate limiting, which solved the issue of timeouts so far. Update rate was 5 seconds, delay 100ms, which seems to make sense to me for something around 10 different stateExtensions.

Adding more devices, I ended up with several hundred different channels all using the same stateExtension.
And then, I found this warning in the log:

2023-09-20 21:26:48.645 [WARN ] [nding.http.internal.HttpThingHandler] - 133 channels in thing http:url:HomePilot with a delay of 100 incompatible with the configured refresh time. Refresh-Time increased to the minimum of 14

For me, this looks like every channel including those with identical stateExtensions query the same address allover and over again instead of the expected auto-caching (and then parsing the answer for the different channels. Which would also explain my HomePilot device heating up unexpectedly :wink:

Or is there something I have to configure to make the HTTP binding explicitely aware which channels are actually using the same extension?

The Thing config looks as such:

Thing http:url:HomePilot "HomePilot" [
    baseURL="http://<myIP>/",  //your HomePilot IP
    authMode="BASIC",
    ignoreSSLErrors="true",
    refresh=5,
    commandMethod="PUT",
    stateMethod="GET",
    contentType="application/json",
    timeout=3000,
    bufferSize=2048,
    delay=100
] 

the channel definitions look like this:

// -----------------------------------------------
        Type string : Schaltkontakt_Steckdose_Balkon "Schaltkontakt Steckdose Balkon" [
        stateExtension="/v4/devices?devtype=Sensor",
        stateTransformation="JSONPATH:$.meters[?(@.did==<myDID1>)].readings.contact_state"
        ]
// -----------------------------------------------
        Type string : Schaltkontakt_Licht_Balkon "Schaltkontakt Licht Balkon" [
        stateExtension="/v4/devices?devtype=Sensor",
        stateTransformation="JSONPATH:$.meters[?(@.did==<myDID2>)].readings.contact_state"
        ]
// -----------------------------------------------
        Type string : Schaltkontakt_Steckdose_Terrasse_SchlafZi "Schaltkontakt Steckdose Terrasse im Schlafzimmer" [
        stateExtension="/v4/devices?devtype=Sensor",
        stateTransformation="JSONPATH:$.meters[?(@.did==<myDID3>)].readings.contact_state"
        ]
// -----------------------------------------------

as I mentioned, there are 100+ definitions which look like this or similar, and they all share the stateExtension="/v4/devices?devtype=Sensor", which returns roundabout 26kB of JSON data containing all required information to do parsing for all channels with this extension.

The warning is printed without checking the state extensions, just the number of channels is checked.

Okay, good to know in general.
If it was just a warning, I could just go on ignoring it with this background :wink:

But the actual issue is that (as the warning tells me) this actively prevents me from setting the desired sampling intervalā€¦ It overwrites the 5 seconds I set with the 14 seconds it computes.

Of course this feature makes the setup more foolproof in the general case of each channel querying a different stateExtension. But in my case, I have something between three or four different extensions, each used by hundreds of channels (huge JSON each).

So the correct action would be querying each of these few extension once every five seconds, waiting 100ms between these queries, and then use the cached replies for parsing through all the many different channels within one sampling period. Which actually was my expectation to be the already implemented functionality, from all I read so far.

Is there a way to:

  • prevent the checker which produces the warning from overwriting my parameters?
  • see if the caching works as expected?

Thanks!

Please, have a closer look at this.

I could workaround it somewhat for the time by having hundreds of channels only using two (!) status URLs which reply with large JSONs. The HomePilot can cope with these two requests at a time most of the time. But not always.

My ideal status would be reading the first URL, waiting some 500ms, then read the second. The rest is done by cache (as it is already now). All of this with a refresh period of 2 seconds.

The automatic increase of the sampling interval actively blocks me from reaching this state. If I set delay to 500ms, it enforces a sampling interval larger than one minute. How can I turn this off?

This is a bug. The actual channel count is used for the calculation, not the number of different caches (which is the same as the number of requests needed).

Okay, thanks for the info!

Just to make sure: The issue is not the real number of caches and requests, this looks fine so far.

The core issue is that there is a checking function which just counts the number of channels regardless if they have mutual, chacheable extensions, multiplies this count with the delay parameter and hard-overwrites the refresh parameter in case the delay * numChannels would yield a longer timespan.

How to go on there? Did you already create some GIT issue or the like?

If you have a preliminary version Iā€™d be glad to test!

Unfortunately we could not reach an agreement in [http] Improve binding by J-N-K Ā· Pull Request #15376 Ā· openhab/openhab-addons Ā· GitHub so I donā€™t think there will be a quick solution for the openHAB version of the HTTP binding.

The discussion on GIT looks like being about several topics and just with discussion ongoing - isnā€™t it?

So ā€œmyā€ topic not going to be fixed within days, but will be included in some future update?

So far, with my current configuration I can go on with this without a big deal, as long as the topic wonā€™t be lost an the long run - is this something I can hope for? :wink:

Thanks so far!

No. The reviewer asked for a change in channel-type-naming, that would be a breaking change for users of the forked binding. Since my explanation was rejected, I closed the PR, because I donā€™t support unnecessary breaking changes for users.

1 Like

Totally agree on ā€œbreaking changes are meeehā€ :wink:

If itā€™s not going to be fixed even in the mid-run, is there any kind of workaround?
Like some setting which completely disables the check, or which at least prevents it from overwriting the refresh parameter?

Maybe even only a line of code I can comment out just on my computer? At my very own risk (misconfiguration, may be overwritten by an update etc)?

Is there an actual script file which I could find this? Would be happy for a hint :wink:

1 Like

Hi, I took a closer look and now Iā€™m unsure what this meansā€¦

  • It looks like you actually merged the fix into some repo - so itā€™s going to be solved despite the discussion about the HTTP binding?
  • In any case - if Iā€™m not completely wrong, the whole code is packed in some .kar file which comes bundled with OpenHABianā€¦ which is way too much down into the system core for me. How to bring it onto my computer?

Thanks in advance!!! :slight_smile:

@J-N-K is the maintainer of the SmartHome/J GitHub repository.

You can actually add this repository as a 3rd party supplier of openHAB add-ons, enabling you to install these add-ons from within openHAB too.

Many people are using the SmartHome/J version of the http binding, instead of the native version. The fix has been applied to the SmartHome/J version, but (as per the previous links) will not be applied to the native version.

1 Like

Okay, meanwhile I set up OH as far as I understood:

  • add https://download.smarthomej.org/addons.json to 3rd Party Addons
  • Removed the original openHAB HTTP binding
  • Installed the Smarthome/J HTTP binding - the one with the same Logo, version 4.0.0 (not the seemingly way older one called ā€œHTTP Binding (SmartHome/J Version)ā€ with last update 2021).
  • Reboot

The actual HomePilot binding is configured with a .things file, using Thing http:url:HomePilot "HomePilot" as definition for the binding. As is the Smarthome/J version from what I saw. Seems to be successful, as everything immediately came up as usual and working so far right from the start.

Then I tried my luck and set delay = 100. And then the error came backā€¦

2023-10-17 14:33:52.347 [WARN ] [nding.http.internal.HttpThingHandler] - 148 channels in thing http:url:HomePilot with a delay of 100 incompatible with the configured refresh time. Refresh-Time increased to the minimum of 15

Just to make sure:

  • I have exactly 146 different channels in my config at the moment.
  • Possible bug? There were two new channels added automatically ā€œLast Failureā€ and ā€œLast Successā€ when I installed the new HTTP binding - seems like these two channels are accounted for in the delay computation as well, as the warning has a channel count of 148!
  • Almost every channel has a stateExtension, but there are only two different stateExtensions existing, in this sense ā€œvery cacheableā€.
  • Quite in contrast, there is a lot of different commandExtensions present, but in my understanding these should not have any cross effects with the caching and delay setting.

Besides this, did I miss anything / did anything wrong?

Thanks in advance!

The fix has been merged to the repo, but a new release is needed. I can try to do that this weekend.