Monitor the network traffic of Z-wave/ZigBee

Both Zwave and Zigbee are mesh networks. You can monitor what the controller/coordinator sees by putting the bindings into debug or trace level logging (see Taming openHAB 2 Logging), which you already indicate you have done.

But realize that a value like RSSI isn’t going to be all that useful to you because it will reflect the RSSI between the device that sent the message to the controller which may not and probably is not the device that sent the original message. That is kind of the point of a mesh network. Even if a device can’t see the controller/coordinator it can reliably send/receive messages by routing the message through the other devices on the network.

What you see in the logs is what is available in OH. You might be able to see more with some third party tool like OZT or Zensys tools (and whatever the equivalent would be for Zigbee. I also think there are makers of sniffing devices you can use that might tell you something about the network. I don’t know.