I just found that my DoorBird video stream causes a lot of error in the log and causes OH2 Basic UI to stop for a few minutes. This is my item configuration:
Video url="http://user:pw@172.17.4.16/bha-api/video.cgi" encoding="mjpeg"
And here is the complete error log:
2017-02-10 16:17:39.082 [WARN ] [eclipse.jetty.server.ServerConnector] -
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)[:1.8.0_121]
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)[80:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[83:org.eclipse.jetty.util:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[83:org.eclipse.jetty.util:9.2.19.v20160908]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
After a lot of those errors also a lot of these errors occur:
2017-02-10 16:17:39.087 [WARN ] [thome.ui.internal.proxy.ProxyServlet] - Proxy servlet failed to stream content: java.net.SocketException: Too many open files
Does anyone have an idea what could cause this problem? I’ve already checked the open files limit with “ulimit -n” and it is set to 65536.
I have the same issue running on the stable release of openHAB (I use the stable as I suppose it works more reliable in a production setting than the nightlies):
2017-04-14 07:52:24.107 [WARN ] [eclipse.jetty.server.ServerConnector] -
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)[:1.8.0_121]
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
2017-04-14 07:52:24.110 [WARN ] [eclipse.jetty.server.ServerConnector] -
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)[:1.8.0_121]
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
2017-04-14 07:52:24.112 [WARN ] [eclipse.jetty.server.ServerConnector] -
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)[:1.8.0_121]
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)[:1.8.0_121]
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)[81:org.eclipse.jetty.server:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)[84:org.eclipse.jetty.util:9.2.19.v20160908]
at java.lang.Thread.run(Thread.java:745)[:1.8.0_121]
Same issue here on Synology OH2.3 stable. Does it within 24 hours . . .
2018-07-07 07:08:18.378 [ERROR] [me.storage.json.internal.JsonStorage] - Error writing JsonDB to /volume1/@appstore/openHAB/userdata/jsondb/org.eclipse.smarthome.core.thing.Thing.json. Cause /volume1/@appstore/openHAB/userdata/jsondb/org.eclipse.smarthome.core.thing.Thing.json (Too many open files)
Interesting, I just ran into this on a new install of Ubuntu 18.04.
The default per-user limit on ubuntu 18.04 is 1024 (which is very low). I have a program that uses 2000 sockets, so I knew this was a problem, and raised the limit to 999999 (the maximum), however I still ran into this problem.
This should be within my 999999 limit, but no, I’m getting:
2018-11-08 08:48:48.340 [DEBUG] [ding.mios.internal.MiosUnitConnector] - run: Exception Error occurred fetching/processing content: java.net.SocketException: Too many open files,java.util.concurrent.ExecutionException: java.net.SocketException: Too many open files. Total failures (10)
Many, many of these errors, CPU usage also climbs to 300% for OH (I have 8 cores).
Anyone have any suggestions on how to deal with this?
Remember my program that opens 2000 sockets? well it opens them into openhab to stream item status (yes I know, this program was written for OH 1.3 or something), using python requests.
With a new install of Ubuntu 18.04, all my python libraries are also shiny and new, and the latest version of requests has this nugget in it:
If you set stream to True when making a request, Requests cannot release the connection back to the pool unless you consume all the data or call Response.close. This can lead to inefficiency with connections. If you find yourself partially reading request bodies (or not reading them at all) while using stream=True , you should make the request within a with statement to ensure it’s always closed:
So when the OH2 connection times out, the socket is left open. This happens every 5 minutes, so every 5 minutes 2000 extra sockets/file handles are consumed.
Thank you requests!
I should really re-write the program to use mqtt or something.
