Several items from different bindings stop updating after a while

hilbrand · July 19, 2019, 2:46pm

The scheduled tasks of the bindings use a limited shared thread pool. If a binding goes rough and deadlocks on those threads it will deplete the available threads and the other bindings will stall too. Some bindings use own threads for certain tasks, and those tasks won’t be effected. So you might still see some updates on certain bindings. In this case the nuki binding seems too be causing this problem.

Falk · July 19, 2019, 2:47pm

Update: just stopped the nuki binding via “bundle:stop” and that seemed to resolve all hangups of the other bindings as far as I can see. Restarted nuki binding again.
The vitotronic entry remains but the binding seems to work and provides updates on items again.

Falk · July 19, 2019, 2:50pm

ok, thanks @hilbrand.
Focusing on the nuki binding seems to be the right thing to do. Anything I can help with?

Falk · July 19, 2019, 3:11pm

… one more thing. Since system appeared to be not fully right finally, i tried to stop OH by using “system:shutdown”. This throws following messages obviously preventing the system to go down:

17:05:50.984 [WARN ] [se.jetty.util.thread.QueuedThreadPool] - QueuedThreadPool[ESH-httpClient-common]@18474c8{STOPPING,10<=10<=40,i=1,q=4}[org.eclipse.jetty.util.thread.TryExecutor$$Lambda$313/28600488@81af1] Couldn't stop Thread[ESH-httpClient-common-99,5,main]
17:05:50.985 [WARN ] [se.jetty.util.thread.QueuedThreadPool] - QueuedThreadPool[ESH-httpClient-common]@18474c8{STOPPING,10<=10<=40,i=1,q=4}[org.eclipse.jetty.util.thread.TryExecutor$$Lambda$313/28600488@81af1] Couldn't stop Thread[ESH-httpClient-common-101,5,main]
17:06:08.030 [WARN ] [se.jetty.util.thread.QueuedThreadPool] - QueuedThreadPool[WebSocketClient@1911742]@198bf1a{STOPPING,8<=8<=200,i=0,q=1}[org.eclipse.jetty.util.thread.TryExecutor$$Lambda$313/28600488@81af1] Couldn't stop Thread[WebSocketClient@1911742-359761,5,main]
17:08:01.896 [WARN ] [org.jline                            ] - The Parser of class org.apache.karaf.shell.impl.console.ConsoleSessionImpl$$Lambda$1041/19902995 does not support the CompletingParsedLine interface. Completion with escaped or quoted words won't work correctly.
17:08:09.533 [ERROR] [raf.system.internal.SystemServiceImpl] - Halt error
org.osgi.framework.BundleException: Unable to acquire the state change lock for the module: osgi.identity; type="osgi.bundle"; version:Version="3.12.100.v20180210-1608"; osgi.identity="org.eclipse.osgi"; singleton:="true" [id=0] STOPPED [STOPPED]
        at org.eclipse.osgi.container.Module.lockStateChange(Module.java:337) ~[?:?]
        at org.eclipse.osgi.internal.framework.EquinoxBundle$SystemBundle$EquinoxSystemModule.asyncStop(EquinoxBundle.java:156) ~[?:?]
        at org.eclipse.osgi.internal.framework.EquinoxBundle$SystemBundle.stop(EquinoxBundle.java:262) ~[?:?]
        at org.eclipse.osgi.internal.framework.EquinoxBundle$SystemBundle.stop(EquinoxBundle.java:267) ~[?:?]
        at org.apache.karaf.system.internal.SystemServiceImpl.lambda$shutdown$0(SystemServiceImpl.java:71) ~[?:?]
        at java.lang.Thread.run(Unknown Source) [?:?]
Caused by: java.util.concurrent.TimeoutException: Timeout after waiting 5 seconds to acquire the lock.
        at org.eclipse.osgi.container.Module.lockStateChange(Module.java:334) ~[?:?]
        ... 5 more

Lolodomo · September 7, 2019, 2:00pm

In this PR, I now run the commands not in separate threads.
Let us know if it helps.

Falk · October 18, 2019, 2:48pm

@Lolodomo, sorry missed your last post. Have the new version 2.5.0.201909221101 in test since today. Will let you know.

Falk · November 20, 2019, 12:01am

Happy to report no problems with the new version of the nuki binding
Thanks @Lolodomo for your great work!

jelmer1 · August 18, 2022, 3:47pm

org.apache.cxf.interceptor.Fault: unable to create native thread: possibly out of memory or process/resource limits reached

at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162) ~[bundleFile:3.4.5]

at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128) ~[bundleFile:3.4.5]

at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201) ~[bundleFile:3.4.5]

at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104) ~[bundleFile:3.4.5]

at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) ~[bundleFile:3.4.5]

at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) ~[bundleFile:3.4.5]

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298) ~[bundleFile:3.4.5]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:222) ~[bundleFile:3.4.5]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) ~[bundleFile:3.1.0]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273) ~[bundleFile:3.4.5]

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) ~[bundleFile:9.4.46.v20220331]

at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:74) ~[bundleFile:?]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[bundleFile:9.4.46.v20220331]

at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:294) ~[bundleFile:?]

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[bundleFile:9.4.46.v20220331]

at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:90) ~[bundleFile:?]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [bundleFile:9.4.46.v20220331]

at java.lang.Thread.run(Thread.java:829) [?:?]

Caused by: java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached

at java.lang.Thread.start0(Native Method) ~[?:?]

at java.lang.Thread.start(Thread.java:798) ~[?:?]

at org.openhab.core.io.rest.JSONResponse.createResponse(JSONResponse.java:180) ~[?:?]

at org.openhab.core.io.rest.JSONResponse.createResponse(JSONResponse.java:87) ~[?:?]

at org.openhab.core.io.rest.core.internal.persistence.PersistenceResource.getItemHistoryDTO(PersistenceResource.java:235) ~[?:?]

at org.openhab.core.io.rest.core.internal.persistence.PersistenceResource.httpGetPersistenceItemData(PersistenceResource.java:175) ~[?:?]

at jdk.internal.reflect.GeneratedMethodAccessor323.invoke(Unknown Source) ~[?:?]

at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]

at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]

at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179) ~[bundleFile:3.4.5]

at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) ~[bundleFile:3.4.5]

… 50 more

2022-08-18 17:29:02.570 [WARN ] [e.internal.SseItemStatesEventBuilder] - Attempting to send a state update of an item which doesn’t exist: undefined

2022-08-18 17:29:02.591 [WARN ] [org.eclipse.jetty.server.HttpChannel] - /rest/persistence/items/Lichtsterkte_zonnecollectoren

org.apache.cxf.interceptor.Fault: unable to create native thread: possibly out of memory or process/resource limits reached

at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162) ~[?:?]

at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128) ~[?:?]

at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:201) ~[?:?]

at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:104) ~[?:?]

at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) ~[?:?]

at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) ~[?:?]

at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) ~[?:?]

at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) ~[?:?]

at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:265) ~[?:?]

at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) ~[?:?]

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) ~[?:?]

at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) ~[?:?]

at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:225) ~[?:?]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:298) ~[?:?]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:222) ~[?:?]

at javax.servlet.http.HttpServlet.service(HttpServlet.java:687) ~[bundleFile:3.1.0]

at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:273) ~[?:?]

at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799) ~[?:?]

at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:550) ~[?:?]

at org.ops4j.pax.web.service.jetty.internal.HttpServiceServletHandler.doHandle(HttpServiceServletHandler.java:74) ~[?:?]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:600) ~[?:?]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[bundleFile:9.4.46.v20220331]

at org.ops4j.pax.web.service.jetty.internal.HttpServiceContext.doHandle(HttpServiceContext.java:294) ~[?:?]

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501) ~[?:?]

at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[bundleFile:9.4.46.v20220331]

at org.ops4j.pax.web.service.jetty.internal.JettyServerHandlerCollection.handle(JettyServerHandlerCollection.java:90) ~[?:?]

at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[bundleFile:9.4.46.v20220331]

at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) ~[bundleFile:9.4.46.v20220331]