Root cause should be the same for STARTTLS.
While with STARTTLS the encryption negotiation is started on demand for TLS it’s done by default when the session is initialized.
In both cases you need to make the binding / OH / Java aware of the public certificate of your signing CA.
See e.g. Icalendar binding with self signed certificate: SunCertPathBuilderException which is a different use case but also an own CA is being used.