I was able to find out my EPANID. I just started the packet capture in Ubiqua and completeley removed one of my lights from the soket. After reconnecting the EPANID was shown in Ubiqua.
I was also able to use the coordinator directly using console app on my Rasperri Pi. I just unplugged the coordinator, plugged it back in and waited until the green led did shut off. After that the console app started up and began recognizing my lamps. I tried the same with the OH addon but still got the "Dongle reset failed. Assuming bootloader is running and sending magic byte 0x01." error. There must be something different in the addon as in the console app.
The "cdc_acm 1-1.4.2:1.0: failed to set dtr/rts" did appear again in dmesg, but not directly on the start of the addon. It appeared a few seconds later. So I still don't know if this is related to the problem.
Any ideas what could cause this problem?