I have just finished a year developing in Spring Boot WebClient.
All i can say is i cannot wait until Project Loom (link to Loom wiki) is GA in a LTS JDK release.
I think it will put a bullet into the brain of “Reactive programming”.
So, as of now in JDK 19, Virtual Threads is in “Preview” and Structured Concurrency is in “Incubation”.
Not exactly GA in a LTS release.
I have read here that long running threads can be a problem for OH, not that I have experienced it.
OH has zero chance of implementing a reactive implementation.
When would the community look to embracing virtual threads and structured concurrency?
I’d like to participate.
It depends on when you read this and in what context. Long running rules could be a problem in OH 3 under certain circumstances since each ru ok e gets only a single thread and you can send up with high latency between event and the rule running. On OH 1 and 2 they could be really bad because all rules shared threads from a thread pool, and there pool was relatively small. So too many long running rules could use up the pool and no rules could run.
But these only had to do with rules writers. I know of no problems with long running threads on add-ons or in core.
I believe the JDK that is targeted for OH 4 is 17. Per policy, OH sucks to the LTS versions of Java. So if these are features only available in later Java versions, it’ll probably be OH 5 before they can be taken advantage of.
Is there any further discussion among openHAB developers to switch from thread pools to JEP 444 Virtual Threads? There seem to be a number of cases where thread pools are being exhausted but the overall system is not hitting any other resource limits. The potential advantages are extremely attractive, with the possibility that there are constraints elsewhere in libraries or codebases. I’m just curious to follow any discussions that might have happened lately. Thanks!
For rules since OH 3 each rule gets its own thread. So there really isn’t a pool involved for those and even if you write a poorly constrained rule at worse you’ll starve out just that one rule, not all rules.
There was some discussion on using virtual threads in add-ons, particularly the Thing Handler but the consensus was that they were not really applicable there because of the way the various add-ons use threading. Virtual threads apparently only really work in certain circumstances, but I have little experience with them so can’t really discuss the technical specifics. When I’m at a computer I’ll try to remember to dig up the issues and PRs where they were discussed.
But there are still some issues where some add-ons block on some IO operations and use a thread from the common pool (add-ons have the option to create their own thread pool) which can starve out other bindings. A couple of add-ons have been fixed in this regard over the past few months (Exec, Network, and Chromecast are the ones that come to mind) and new logging was added to help identify others.
So there are still some areas of concern and there might be other areas of OH where a Virtual Thread approach may be more applicable.
Thanks for the reply, Rich. I’ve read summaries like this one and except in a minority of cases, the semantics of virtual threads and native threads are identical. The resource cost of native threads is much higher than virtual threads, and virtual threads would eliminate the limits of native threads and thread pools. I would like to see any past discussions you can locate so I can better understand what the obstacles might be. Regards
edit: By the way, I read some more about virtual threads to see if they could help out here. But, they have quite a few caveats, and don’t perform well unless the code is written in “blocking style” without locks. That narrows things down quite a lot, and it seems to me like it’s mostly suited to very specialized tasks like running a large webserver. In addition, Oracle’s documentation states that:
As a rule of thumb, if your application never has 10,000 virtual threads or more, it is unlikely to benefit from virtual threads.
I’ve looked at it a little bit, and to me it seems like virtual threads aren’t really very useful for OH. They are basically designed for a very different purpose, they are meant for huge web applications with tens of thousands of threads.
Virtual threads aren’t “real”, it’s an abstraction done in the JVM - where the whole JVM in a way acts as one huge, unlimited thread pool, and virtual threads are just “tasks” in this pool. Once they have work to do, they will be assigned to an actual thread. This has quite a lot of overhead, which is why Oracle says that you’re unlikely to benefit from virtual threads if you have below 10000 threads. Virtual threads also have quite a few limitations, they don’t “actually work” if they access shared resources (using monitors), if they do, they get degraded from virtual to “real” threads - so you get no benefit at all, but you do get the extra cost/overhead.
OH typically run 150-300 threads or so, depending on bindings etc., so, quite far away from this.
I don’t think virtual threads are needed to speed OH up drastically. The thread starvation that makes things slow today, are by design/choice. The argument is that OH is intended to run on very weak hardware, so it shouldn’t consume many resources, RAM in particular. Each thread allocates a stack when it is created, which can be anywhere from 512 KB to 1 MB depending on system architecture and JVM, which makes threads in themselves consume RAM. There are hard limits on most thread pools in OH, which is causing most of the slowness you can see. Until recently, most work across all bindings have been performed by a pool that is limited to 5 threads. This has been increased to 15 in 5.1.
These are “artificial limitations”. I’ve been trying to argue that this should be a choice the user makes, not OH “on behalf of the user”. I’m confident that the typical RPi can handle the RAM consumption that would result from not limiting thread pools at all without problems, thread pools in general only create the threads they need, they reuse threads and shut them down when they have been idle for a while. So, by allowing unlimited thread pools, you would see increased RAM consumption during startup or when a lot of stuff is happening, but it would also release the RAM when it’s no longer needed. I’m convinced that many would find this solution preferable.
I’ve even made an “implementation suggestion” that allows making existing thread pools unlimited with minimal changes, but I haven’t found anybody with a large installation willing to test it. I have tested it myself, and OH doesn’t really consume notably more resources, but things happens a lot quicker.
Others would have to be convinced that this is the right approach for it to go anywhere. Testing on actual installations with real results would probably help a lot. I envision a solution where you can configure OH to either run with “restricted” (as it is today) or “unrestricted” thread pools, so that the benefits of the current solution can still be had where that is preferable (basically trading time from memory).
Thanks for your thoughts, @Nadahar . Your understanding of the overhead and limitations of virtual threads compared to native threads appears, at least superficially, to be at odds with the descriptions I’ve read. Changing the size of thread pools from 5 to 15 is a great example of relaxing arbitrary hard limits to choose new arbitrary limits, whereas virtual threads do not require pools of any size to be pre-determined. The very reason for pools is the high overhead of native threads, eliminated by replacing them with disposable virtual threads.
I agree that empirical testing and hands-on experience would answer a lot of real-world questions. The memory overhead of native threads, usually blocked on I/O, as opposed to the nearly zero overhead of virtual threads, with no semantic differences, would be very interesting to see in practice.
This is of course all academic until a developer wants to find a proper long-term solution to excessive resource usage and starvation. Regards
The point is that virtual threads aren’t “magical” either. They do use real threads underneath, it’s just that while one thread is waiting for I/O, the thread is “disconnected” and set to do another task. You still need “real” threads to do the actual work.
This all sounds very enticing, until you start reading about the caveats. One of these is that virtual threads get “pinned” if any synchronization/locks are in use. “Pinning” simply means that the virtual thread gets permanently attached to one real thread, so it can no longer be released if the code enters blocking I/O.
In an application like OH, where you deal with shared data pretty much all the time (threads don’t work in isolation, they do something with other parts of the system), most threads will be “pinned” most of the time, and the whole “benefit” goes up in smoke.
Virtual threads are really mostly meant for a high number of concurrent tasks that are independent of each other, typically a web server. These tasks are quite different in nature from the tasks performed in OH.
You don’t need virtual threads to not have arbitrary limits, this is purely a choice that has been made in OH. So, don’t mix one with the other.
Having a “blocked” platform thread is wasting a scarce resource. “Fixing” that wasted resource is what the “reactive async” programming model was created to “fix”.
Reactive Asyc programming does increase the throughput, “fixing” the wasted machine resources kinda inherent to “imperative programming” but OMG… It was hard to code! hard to test! hard to debug!
A “blocked” virtual thread is a near zero cost making efficient use of machine resources with the immensely simpler imperative programming model, giving the all of the extra throughput of the reactive programming model.
Notice I said “throughput”, not “speed”.
Now OH is a very old code base that had many cooks making the broth.
Just thinking out loud, OH IS a candidate to refactor to the usage of virtual threads, structured concurrency and scoped values, the main features of Project Loom, which was where these JDK features were developed. That requires VOLUNTEER devs working in the morass of technical debt that ALL projects accumulate.
I have grand thoughts about tackling that challenge, then I feel a pressing need for a snack and a nap.
I read the official spec in a way opposite to your understanding. You might want to verify your understanding of what virtual threads limits are against the official documentation.
I read the short section about virtual threads in your link, and it doesn’t say anything new or different from what I’m saying that I can see.
That said, I’m really not that interested in having an “ideological discussion” about some “fantastic feature” that will solve everything. I’m not saying that they aren’t useful, I believe they can make a big difference for large servers that serve a large number of clients. That isn’t a scenario that fits OH.
How the whole thing works is relatively well explained by Oracle, in particular notice this paragraph:
As a rule of thumb, if your application never has 10,000 virtual threads or more, it is unlikely to benefit from virtual threads. Either it experiences too light a load to need better throughput, or you have not represented sufficiently many tasks to virtual threads.
If that’s true, it might change the picture, but I’d need to know how they have technically “solved” that to judge. Since we aren’t on Java 25 now, that can wait.
But, the point I was trying to make above is being missed completely: Using virtual threads means that you don’t impose limits on the number of threads. The JVM will serve you threads from an “unlimited thread pool”. Doing that means switching away from the current philosophy if strictly limiting available threads for OH.
I’m quite confident that a huge performance benefit coud be had by simpy not limiting the threads, “virtual” or not. By going with virtual threads, you no longer have any choice, you’re forced to accept the unlimited pool. So, comparing the current limiting strategy with using virtual, and thus unlimited, threads, would be very “unfair”, as the performance benefit would come from the fact that you don’t throttle threads, not from them being “virtual”.
A webserver typically has one thread serve one connection, which can be many per browser browsing the site. They spend the majority of their time waiting for input from the browser, aka “blocking”. Having these (thousands of) threads released from waiting, being able to do other tasks instead is obviously a huge benefit. It’s just that this scenario has very little in common with how OH operates.
Others can read the official documentation and compare it to your interpretation, and I think they would see the differences. I don’t believe your tone is very constructive. I would ask you to consider a more collaborative and open dialogue with those who just want to improve openHAB. Thank you.
Anybody can read the documentation, but since you obviously read something very different there than I do, I don’t understand why choose to just insinuate “something” instead of pointing out what you think it is that I misinterpret. But, that’s ultimately your choice.
Constructive? Towards what goal?
I’m only presenting my analysis of the matter, I don’t have any say over what decisions are made here, others can draw their own conclusions.
I’m actually one of very few that tries to get something to change regarding the restrictions that make things slow. I don’t have neither the intent nor the power to deny anybody to contribute anything, so I don’t really know what you’re implying, other than that you seem to disagree with me in some vague, undefined way.
I will do what I can to try to lift the restrictions. I won’t work towards using virtual threads until I believe that they would actually solve a problem.
Mischaracterising what one says with false quotes and hyperbole is some of what I referred to as to a lack of constructive tone. This is very clear.
Here is a concise description of how virtual threads interact with locks in Java 21 and later versions. It is at odds with your understanding in posts above so I share it in hopes of curtailing the spread of misunderstanding.
Of course there is real work for some skilled and interested volunteer to bring the real advantages of virtual threads to openHAB, and hopefully someone will take it on despite the occasional emotional dismissive replies.
I don’t know enough of either solution, but it is clear openHAB can benefit when ‘thread control’ is more orchestrated. I think this is a discussion that should take place in the openHAB core repository.
Maybe we can take intermediate steps and i assume when this is more aligned accross the repositories (atleast core and addons) it would be way simpler to switch between either solution.
I guess we need an overall design, and from there seperate smaller tasks.
Sure, in general, but I don’t know what you’re referring to here. I don’t think I’ve quoted anybody at all. Quotes (") are used for more than literal quotes.
I don’t see what makes this so concise, it’s quite generic and goes into little detail. That said, it’s not “at odds” with my understanding at all - it’s repeating what I’ve been saying. How Java 24/25 is behaving isn’t of interest as I see it at this time, what I’ve been looking into is how it works in Java 21.
I’ve already said that it they have indeed solved the pinning, it might change the overall picture. But, it seems that they haven’t completely “solved” it. You must still for example take care to not call native code from virtual threads (which can be challenging because some libraries do this without your knowledge), and there are some other “yet to be resolved” situations mentioned at the end of JEP 491.
However, getting stuck at pinning is missing the bigger picture. There is no way to utilize virtual threads without abandoning the whole approach OH has towards thread management. OH makes an effort to keep the number of threads low, at the cost of things taking more time. There is also the fact that most thread pools in OH imlements ScheduledExecutorService, which lets you schedule a thread to run at a scheduled time in the future. Virtual threads don’t solve/allow this.
There are lots and lots of things that would have to be rewritten to use virtual threads in any meaningful way. Tasks that needs scheduling would have to be separated from tasks that don’t. This alone would require a huge amount of refactoring. Keeping track of potential native calls might be more or less impossible. The way you acquire virtual threads are also very different, you basically create a “thread” each time you need one. That’s almost never done in OH, instead threads are “loaned” from thread pools managed by Core. This would also mean a lot of refactoring to change.
This also ignores the fact that using virtual threads also has a cost. It’s not “free”, implementing thread scheduling in the JVM instead of leaving it to the OS is less efficient. It’s for this reason that Oracle has given the estimate of “more than 10000 threads to benefit”. Given how few threads in OH that actually waits for blocking I/O, I’m skeptical if there would be any net benefit at all from virtual threads. But, that’s just an estimate - to know, one would have to go through all the above refactoring, in the hope that it might pay off.
