After a lot of struggling, I finally seem to have resolved all my problem with frequent reconnects from my Sonoff-S20 sockets.
My biggest mistake (thanks Rich Koshak for pointing that out) was inconsistency in my configuration method. All other bindings I had defined in the addons.cfg, but the MQTT binding I added just using paper ui. This worked temporarily, but on next restart of openHAB it was dropped. So file based config is a kind of all-or-nothing.
Still after that I suffered problems with very unstable connections.
It seems that in my local Wifi I have additional problems because it is organized as a mesh (FritzBox), where there is roaming between access points. These share the same SSID (i.e. readable name), but different BSSId (i.e. MAC addresses).
Most of my devices (e.g. mobile phones) manage to roam between these Wifi cells without losing IP-level connectivity and TCP connections generally stay alive. But the devices running latest tasmota 6.4.1 regularly check the connection, and if they find another access point with a stronger signal, they hop over.
FritzBox is not known for the most stable Wifi implementation, and Mesh even raises the complexity. So I am not really sure who to blame (tasmota or FritzBox), but the Sonoff devices periodically loose TCP connectivity and do a full reconnect to the MQTT server, losing around 5-10 sec of connectivity. The openHAB commands are not buffered, so the devices might fail to receive some relevant on/off commands. Their state still reflects this, but the MQTT binding does not care (in my case) and does not resend the command; maybe with some polling rules, this might be improved.
I cannot 100% prove my above explanation. Now, what was the solution? I simply switched to tasmota 6.2.1. From that time on the frequent reconnects totally vanished.
As a side-effect of that downgrade I needed to make a few minor adaptions to the JSONpath transformation in my thing definition ("$.Wifi.BSSId" needed to become “$.Wifi.APMac”), but apart from that, I am not missing features. The LWT mechanism still works fine, so openHAB shows more or less reliably when a network socket was totally unplugged and thus is not reachable. The APMac tells me to which access point a switch is connected.
Devices I have successfully connected to openHAB 2.4.0 MQTT binding to the embedded broker:
4 x Sonoff S20
1 x Sonoff Basic
1 x Shelly 1
All are running Tasmota 6.2.1. Currently, I am only using MQTT (as opposed to MQTTS), but that will be my next goal.
Hope this might help somebody else,