Which rules language?
- Rules DSL: Rules | openHAB (be sure to follow the links in that doc too)
- Blockly: Blockly Reference (to be added to the official docs in the near future)
- ECMAScript 5.1: Oracle Nashorn: A Next-Generation JavaScript Engine for the JVM, openHAB specific stuff documented at JSR223 Scripting | openHAB though Helper Libraries for openHAB Scripted Automation — openHAB Helper Libraries documentation is probably a little better, but you’ll have to install the helper libraries.
Those are the languages that come with OH. If you install an Automation add-on, see that add-on’s readme (and often there will be a corresponding thread here on the forum) for details. Though the syntax will primarily be what ever the language is (e.g. JS Scripting is standard ECMAScript 2021) so the docs focus mostly on interacting with openHAB.