Ah, that makes a lot more sense.
So how important is it to you that the lights be processed in order? My idea of using Timers to run each iteration of the loop in parallel might get you to your goal, or at least close to it.
NOTE: I’m just typing this in, there is almost certainly an error but the concept should be valid.
while (steps > 0) {
for( key : keys) {
createTimer(now.plusMillis(1), [| // should trigger the code in a separate thread almost immediately, if not you may need to make it 5 or 10
var data = fade_data.get(key)
...
]
}
}
...
step_sleep = // this needs to be long enough for the loops to finish. With the changes already made 55 or 60 might be a good number here actually
Theoretically what should happen above is that the calculation of the new values and sendCommand for each light set (i.e. key) should process in parallel so it should only take slightly longer than it takes for the longest time through the loop for ALL the lights to finish processing. And depending on timing you may not have to wait until all the timers complete before moving to the next step as the old Timers will probably have completed before you get to sending the command on the next time through the loop.
One major drawback to this is that you are not always guaranteed that the lights are sent a command in the same order for each step. Also, if for some reason one of the threads has a hickup (e.g. garbage collection) a couple steps could process out of order. Because we are not using locks or anything like that (which we could use if we have to simulate a fork/join type behavior) you will have to come up with how long you have to sleep at the end of the while loop to ensure reliable behavior. But with an average of 50 msec per key you might be able to get away with a timeout of 50 msec with the code as is.
I’d give it a try and see what happens. It is simple enough to test out. And this could get you to your goal, or at least really close to it.
If it does work, you can then think about ways to make it more reliable/robust, though anything you add to it at this point will make it take longer. Things you can do include adding a lock for each Timer and and requiring all of the locks to be released before the next iteration of the for loop (fork/join), holding on to your Timer objects in case you need to cancel or reschedule them, etc.