i use the new binding:
binding-mqtt - 2.5.1
Then why did you post a 1.x version example??? 
Because I didn’t know it was for version 1.
I found it in the community.
How can I solve it with the new 2.5.1 version?
Use PaperUI to create the Thing and you can set the transformation there in the configs.
Is it also possible to program it via *.item *.sitemap *.rules?
i tried this one, but i get a error message in my log
How can i send this outgoing mqtt message?
"{\"message\":\"{\"systemcode\":1,\"programcode\":4,\"on\":1},\"protocol\":\"rsl366\"}"
i try it with this thing but without luck 
Type switch : Subwooferr "Subwoofer" [ stateTopic="home/OpenMQTTGateway/commands/MQTTtoPilight", commandTopic="home/OpenMQTTGateway/commands/MQTTtoPilight", on="on", off="off" ]
someone can help me to send not only “on” or “off”.
i have to send:
"{"message":"{\"systemcode\":1,\"programcode\":4,\"on\":1},"protocol":"rsl366"}"
on=“ON”, off=“OFF” might work but what is the mqtt topic?
Looks like 1 is ON and 0 is OFF so try changing the configuration channel, Custom ON/Open Value to 1 and Custom Off/Closed Value to 0, via PaperUI.
If that does not work then remove MQTTtoPilight and try the topic without it.
If you still have issues then you need to read and learn how mqtt works. Not just mqtt on OH but mqtt in general. After that you shouldn’t need help but if so you’ll have a good idea of what info is needed to help with the problem.