I just found out that there is even a better way to change the path.
After updating to the new version of OH (2.5.4-1) I had to redo the changes. The currently used version of systemd allows an easy overriding of configs without having to change the original file.
sudo systemctl edit openhab2.service
with this content
[Service]
Environment=OPENHAB_LOGDIR=/media/usb-drive/
also makes the change - but future upgrades will not affect that patch anymore.