openHAB crashing with out of memory exception (Sonos, 1.7.1)

In openHAB 2.0 / ESH, the SONOS binding uses jUPnP, which is a fork of cling 2.0.
It might be worth to try refactoring the SONOS 1.x binding to use JUPnP as well.

Best
Hans-Jörg

That’s a good idea. I can investigate that possibility.

I’m still waiting for the issue to happen again so I can hopefully get a thread dump and determine the root cause of the excessive thread creation.

I spend endless hours, trying to workaround the Sonos bugs and was never sucessful. Finally I removed the sonos binding and am controlling the sonos speakers via php scripts that are call via http puts.
But if you are Java programmer, you may have more powerful possibilities.

Since then all my memory overflows are gone. Before that I had a crash every 24-48 hours.

@martin_klimke Yes, it is frustrating. I have two more Sonos devices arriving today. I’m a little concerned it will aggravate the memory/threading issue. What types of capabilities do you have with the external scripts? Can you control volume, get the current track, etc? In general, how do the capabilities compare with the binding (assuming there was no memory leak)?

@hmerk I’ve been able to compile the Sonos binding with the forked jupnp. I’ve started doing some testing and it’s updating my Sonos test item, so that’s encouraging.

Hi Steve, that sounds great.
Best
Hans-Jörg

I can control volume, play back sound files and start a radio station.
I cannot get anything back from the sonos.
This is a disadvantageous because I cannot read out the volume and therefore I cannot really restore the state the sonos speaker had before I had applied a command.

this was my post:

So if you have good results with the forked Jupnp, I would be interested in trying it.

Testing is going well. It appears that the binding is working with the openhab jupnp library fork.

There are still quirks that probably are related to the binding implementation itself.

  • Items lose their states and appear to not be restored if an items file is reloaded (seems like relatively common behavior in openHAB bindings).
  • Sonos players aren’t recognized if they are started after openHAB starts.
  • If the Sonos player goes offline while openHAB is running, there are repeated log warning messages (~1/second) from the UPNP layer. This could be an issue for users who want to turn off their Sonos at certain times.
  • There’s an NPE from the SonosZonePlayer if player availability changes while openHAB is running. I haven’t completely identified the pattern of available that triggers the NPE but I’ve seen it several times.

The good news is that the binding seems to work about as well as it did before the jupnp upgrade and now the OutOfMemoryError should be avoided because the UPNP task queue is bounded. I’m going to update my primary openHAB installation and run with the updated implementation for several days to see what happens.

With the older implementation my openHAB crashed again. Unfortunately, the HeapDumpOnOutOfMemoryError did not work (I’ve seen info online that this happens sometimes). However, the hotspot log file shows that there were 4000+ Cling (UNPNP) threads at the time the JVM crashed. I’ve set another tripwire with a rule that’s monitoring the thread count via JMX and will manually dump the heap if the thread count crosses a threshold. This analysis is somewhat low priority since moving to the new library should fix the memory problem anyway.

1 Like

This is really great news.
Aside the Upnp bug, the sonos binding is buggy as such. I have noticed this as well.
But these bugs are not really show stoppers as it was the case with the old upnp stack.

If you have completed testing will you publish your bug fix?

Good job Steve. I encourage you to share a downloadable jar for others to beta-test in parallel, this way we can have a more solid testing period and hopefully more feedback on potential bugs :wink:

Regards
Frode

Depending on how problematic this it - I have been using a Sonos json api for about one year which works faultlessly. I’ve not tried it using Openhab but I use it with Openremote which is very similar. You can even do text to speech directly from a http post which is quite fun.

I’ll do this once I have a stable set of modifications. I did some initial testing and although the thread creation issue was gone, I saw many NullPointerExceptions in my logs because the binding doesn’t check for nulls if an existing service becomes unavailable. I’m adding guards for those conditions and then I’ll retest. I’m also learning about the details in UPnP. Hopefully I’ll something for others to try by the end of the weekend.

Interesting. It could be helpful to see a different implementation to get ideas for solve problems with the openHAB Java binding.

If someone would like to test the modified Sonos binding, you can get the jar here:

https://drive.google.com/file/d/0B9snsUd83Jk0VGhjMDNqZGJZdUk/view?usp=sharing

I’ve been running it for 2 days now and it seems stable.

HI Steve,
Great, I will test it starting on Friday because I have no access to my computer until then.
If would be great if you post whether the binding is running stable in your installation until this date.

regards Martin

Hi Steve,

I have meanwhile installed the binding.
My experience so far.

  1. at the beginning I receive a bunch of warnings of all the upnp devices in my network. Which is probably ok.
  2. when issuing the sonos command to start a radio station it is principally working but I am getting a German error message which says
    “[Fatal error]:1:1: vorzeitiges Dateiende” which is repeated about 8-10 times.
  3. when issuing the playlist command I am also getting those error messages but the command is not executed.
  4. Volume command is working also the play and transportstates are working as well.
  5. Up to now I did not see the heap overflow. Great! But currently I am using a windows machine which has 4 GB of RAM. So it has some tolerance with respect of memory consumption. Also Openhab is running for about 18 hours. My target machine only has 1 GB of RAM and if this is running more than 2 days, this would be a real improvement. WIth the old binding, OPENHAB crashed latest after 24 hours, sometimes even faster.

regards Martin

Do you have the SystemInfo binding installed? If so, you can monitor the virtual memory usage of the openHAB process. With the old version of binding there would be significant jumps in the virtual memory consumption (because of large numbers of active threads) which eventually would exceed the heap size limits.

Can you translate those fatal error messages into English? :smile: I haven’t seen any fatal messages so far, but I haven’t been exercising the playlist commands much.

Could the playlist behavior be related to this issue? Sonos binding: can’t start a playlist when there is no ‘active’ queue.. I also found that some of the controls didn’t work when the devices were grouped and the command was not sent to the group coordinator.

I ran the German error message though Google translate and it was translated as “Premature end of file”. I’ll look in my logs to see if I find anything that might be related.

Hi Steve,
“vorzeitiges Dateiende” means “premature end of file”.
For sure I did not enter this into any part of the system which is under my control.

With respect of sysinfo: I was never able to operate sysinfo under windows. 2 years ago I tried and it always crashed my machine.
You are probably using Linux as your OS, are you?

Also an additional finding which is probably a kind of legacy error of the old binding.
When changing the radio station with the Sonos App (not via Openhab), this change is not reflected in the related item.

regards

Martin

with respect of the playlist error.
The bug you are pointing to is exactly what is happening to me.

The song in the playlists are entered into the queue but the radio which was used before is played.

regards Martin

The workaround for the later bug is to firstly play a mp3 file with sonos and then switch to the playlist.