[EBUS2] Send queue is full

Hi folks!

I’m experiencing troubles with my ebus binding.
At some time, I can see in the logs

2020-04-06 10:54:03.060 [ERROR] [de.csdev.ebus.core.EBusQueue        ] - Send queue is full! The eBUS service will reset the queue to ensure proper operation.

Sometimes, but not always, I get a bit later

 [WARN ] [.csdev.ebus.core.EBusEbusdController] - Stop sender thread!

What could i do to get rid of this problem?

Thanks,
Markus

Another log

2020-04-06 11:08:51.883 [ERROR] [de.csdev.ebus.core.EBusQueue        ] - Send queue is full! The eBUS service will reset the queue to ensure proper operation.
2020-04-06 11:08:52.991 [ERROR] [.csdev.ebus.core.EBusEbusdController] - error!
java.lang.NullPointerException: null
        at de.csdev.ebus.core.EBusEbusdController.parseLine(EBusEbusdController.java:179) ~[bundleFile:?]
        at de.csdev.ebus.core.EBusEbusdController.run(EBusEbusdController.java:262) [bundleFile:?]
2020-04-06 11:08:52.995 [WARN ] [.csdev.ebus.core.EBusEbusdController] - Stop sender thread!

I don’t use this binding but you can try stopping OH and cleaning the cache to see if that helps.

Thanks for this idea - sadly not better afterwards :neutral_face:

Any other ideas i could try?

Is there any possibility to monitor if sending happens and restart the ebus binding if not?
Or another relating question: What is the right way to restart the sending? Restarting openHAB service is not the right way - in my opinion?!?

Thanks a lot

You can use the console to restart a single binding and set the log level for EBUS binding to DEBUG, see if you get any additional info that will help.

Thanks for this.

bundle:restart org.openhab.binding.ebus

command in openhab console restarts the binding. But often i get the same problem directly after the restart again - tried today, the 4th restart brought back the binding to work.

Having to restart seems to be needed for whatever reason. I hope this gets fixed in OH 3.0. If the issue is resolved, please click the square box on the post that provided the solution to mark topic as solved.

Thanks

@mwildbolz I think i can see a scenario in the code that leads to that crash when the queue is reset but i dont understand it very well to understand how to fix it or what leads to the queue being full.
Are you connecting in raw mode to the ebus interface?
I am running mine via ebusd which is now reliable for me so one option may be to install and run ebusd and let that talk to the device and configure the binding for that mode, if you arent running that way at the moment.

sounds interesting.
I’m running the binding also via ebusd.

In my opinion, there is quite much traffic while starting the binding due to poll requests being sent. Maybe this leads to the problems i saw while restarting.

The other side is, that sometimes the binding stops work not correlated to a restart. In this case, I often see a send error - error sending telegramm, max. retry count reached (or something similar).

Any ideas?

For the moment i installed a rule to restart the binding, every time the bridge thing leaves ONLINE state.
But that’s not a “solution” - that’s a work around…

same overall scenario for me
PI3 with stack-on 2.2 FEHM ebus adapter board, ebusd running and configured
Some polls were added in Thing definition

The system will usually work for 3-4 days, then logs will show

2020-05-11 03:02:06.938 [ERROR] [de.csdev.ebus.core.EBusQueue        ] - Send queue is full! The eBUS service will reset the queue to ensure proper operation.

I can link the “send queue full” with ebusd.log no longer showing the

[main notice] direct cmd: 

that are reflecting the polling from OpenHab
Restarting the binding will bring it back to life, but again only for a couple of days.

In contrary to what @mwildbolz mentioned, my bridge and things remain online and still receive updates from the bus, only the items that require polling don’t.
So the connection between Ebus binding and ebusd isn’t severed, just the sending part does not seem to be able to post messages

@mwildbolz would you share your rule setup? I would like to replicate this with a time-based trigger until there is a solution available

I also saw the same problem that you mention - with just the send queue full messages and no polling afterwards. But sometimes i have the described problem with additionally the error message leading to a complete stop of the ebus binding.

I did the restart via the exec binding with the following configuration:

exec.things

Thing exec:command:restartEBUS [command="sshpass -p habopen ssh -p 8101 openhab@localhost bundle:restart org.openhab.binding.ebus", interval=0, timeout=5, autorun=false]

exec.items

Switch		restartEBUS_Run			"Restart ebus Binding"										(gHzg)							{channel="exec:command:restartEBUS:run", autoupdate="false"}
DateTime	restartEBUS_lastExec	"Last ebus restart [%1$td.%1$tm.%1$tY %1$tH:%1$tM]"	<time>	(gHzg, gPersistentChange)		{channel="exec:command:restartEBUS:lastexecution"}

exec.rules

rule "EBUS Restart"
when
    Thing "ebus:bridge:hzg" changed from ONLINE
then 
    if (restartEBUS_Run.state != ON) {
        logInfo("EBusRestart", "Ebus restart initiated.")
        restartEBUS_Run.sendCommand(ON)
    }

end

would be really interesting, if someone could fix this issue (as mentioned by @jpharvey in the code).

@jpharvey: Maybe you could provide some more details what you think is the problem - could make it easier for anyone to fix…

@mwildbolz thanks for sharing… exec-binding… not have thought of that :slight_smile:

the scenario for the crash (NullPointer in the log-files) seems rather interesting, because the error (Exception) is caused by the receiving end of the binding when reading an answer from the bus

from what I could see in the code, the send-queue is reset when full (20 entries +)
problem is, that when the “sender” is not able to send more messages, it will not only clear the sending-queue, but also the receiving queue

[EBusQueue.java, line 160/161]

            outputQueue.clear();           <= does clear the sending queue
            resetSendQueue();              <= does clear the receiving queue

so for me it looks like the “receiver” is then trying to process a message that is no longer there, because it was deleted by the queue-reset initiated by the “sender”
This does also cause the “Stop sender thread”, since this is generated when the binding disconnects from ebusd

if so, a fix for the binding to go offline from time to time when the clear failed would be rather easy, just checking if the item that is to be processed is actually there (classical check for null)

The more interesting topic would be why the sender thread is not able to put it’s messages to the queue… that I’m currently investigating
Might be a blocked thread

That was roughly the conclusion i came to looking at the code as well. I think as you say the bigger problem is why it is getting in that state and i can’t help with that since this is all working fine for me.

Oh, I was not aware of this thread but I think I’ve fixed this in my next release.

I still see lot of these errors in openhab.log:

[ERROR] [de.csdev.ebus.core.EBusQueue ] - Send queue is full! The eBUS service will reset the queue to ensure proper operation.

But i don’t see a direct impact on the functionality.