IpCamera: Fixes and optimizations to Onvif connection

I’m creating this topic to discuss various changes I made to the IpCamera addon I earlier discussed with @matt1 in the following topic: https://community.openhab.org/t/problems-with-reolink-e1-outdoor/158358/41

At first I had problems that my Reolink camera always restarted when Openhab connected via Onvif. I later found out that this is a firmware problem of the Reolink camera that will be fixed soon. Basically the camera did restart the Onvif service when events where pulled one at a time with a MessageLimit of 1.

So I decided to take a look at the code and make changes so that multiple events can be processed in a single message. During testing I found various other issues so I began to rework how events are processed in the addon and I would like to discuss those changes here.

I uploaded my current changes here on GitHub.

One of the main changes is that the addon now relies on the camera correctly reporting which event types are supported in the GetCapabilities response. I decided to use this instead of the GetServiceCapabilities response because my old Reolink C1-Pro did not have the GetServiceCapabilities method implemented. But it correctly reports the event capabilities via GetCapabilities.

Also the addon now falls back to the WSBaseNotification if the camera does not support PullMessages. The WSBaseNotification also requries support for multiple events in a single message because there is no MessageLimit setting for this notification type. Before the WSBaseNotification was subscribed in parallel to the PullMessages subscription and was newer refreshed, so it only worked for a few seconds.

Optionally I could also implement a thing setting where the user could force which type of event subscription should be used if there are cameras that not reliably report the event capabilities.

This will have to be tested thoroughly with various cameras. Sadly I only have my old Reolink C1-Pro and the newer RLC-820A.

@matt1 Can you please take a look at my changes here? I also made various comments on GitHub.

I get these errors that stop the build

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] C:\Users\Matt\git\openhab-addons-netcamera\bundles\org.openhab.binding.ipcamera\src\main\java\org\openhab\binding\ipcamera\internal\onvif\OnvifConnection.java:[326,64] Null type mismatch (type annotations): required 'java.lang.@NonNull String' but this expression has type 'java.lang.@Nullable String'
[ERROR] C:\Users\Matt\git\openhab-addons-netcamera\bundles\org.openhab.binding.ipcamera\src\main\java\org\openhab\binding\ipcamera\internal\onvif\OnvifConnection.java:[391,64] Null type mismatch (type annotations): required 'java.lang.@NonNull String' but this expression has type 'java.lang.@Nullable String'
[ERROR] C:\Users\Matt\git\openhab-addons-netcamera\bundles\org.openhab.binding.ipcamera\src\main\java\org\openhab\binding\ipcamera\internal\onvif\OnvifConnection.java:[491,53] Null type mismatch (type annotations): required 'java.lang.@NonNull String' but this expression has type 'java.lang.@Nullable String'
[ERROR] C:\Users\Matt\git\openhab-addons-netcamera\bundles\org.openhab.binding.ipcamera\src\main\java\org\openhab\binding\ipcamera\internal\onvif\OnvifConnection.java:[1163,63] Null type mismatch (type annotations): required 'java.lang.@NonNull String' but this expression has type 'java.lang.@Nullable String'
[INFO] 4 errors

@TheNetStriker
EDIT: I changed the null checks to .isEmpty and used an empty string instead of null. Compiled and tested the following, will update as I find the time…

Hikvision and Dahua:
Multiple events do work but both are getting this error.

2024-11-07 22:52:52.021 [ERROR] [amera.internal.onvif.OnvifConnection] - Error processing PullMessages error:

java.lang.NullPointerException: Cannot invoke "org.w3c.dom.Element.getElementsByTagName(String)" because "sourceElement" is null 

@matt1 Ok strange, I don’t have this error. It is a while ago I developed with Java. Maybe it depends on the Java version.

I uploaded a new commit where the subscriptionXAddr is not nullable and is initialized as an empty string: Various Onvif fixes for IpCamera · TheNetStriker/openhab-addons@20e832a · GitHub

One of my Instar cameras now does not work with PullMessages and also WSBaseNotification, due to it replying false to supporting both of those, even thou it does work. As a suggestion what do you think about making it a user choice and then the binding just try’s to use the selection? Then people can switch between them if a camera does both.

Did you configure the Instar camera as a generic Onvif camera? I did not see that the Instar Thing does use ONVIF for events.

I just uploaded two new commits:

In this commit I added some code on the GetCapabilities response. If the camera does not report ONVIF event support in GetCapabilities the addon now also calls GetServiceCapabilities to check if event support is reported there.

In this commit I also added a Thing setting for generic ONVIF and Reolink cameras where you can force the ONVIF event type to be used. I also added the new packageAlarm channel that will be releases sometime in the future by Reolink.

Please let me know if this worked for your camera.

Correct it does not use it when setup normally as an Instar thing type, so I set it up to be an ONVIF thing type for testing your changes. We need to test changes with as many different cameras as we can to see the results. I did the same thing for Hikvision and Dahua which I put the results in my last reply that they generated the same ERROR as each other.

java.lang.NullPointerException: Cannot invoke “org.w3c.dom.Element.getElementsByTagName(String)” because “sourceElement” is null

example message is

<?xml version="1.0" encoding="UTF-8"?>

<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope" xmlns:soapenc="http://www.w3.org/2003/05/soap-encoding" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:tt="http://www.onvif.org/ver10/schema" xmlns:tds="http://www.onvif.org/ver10/device/wsdl" xmlns:trt="http://www.onvif.org/ver10/media/wsdl" xmlns:timg="http://www.onvif.org/ver20/imaging/wsdl" xmlns:tev="http://www.onvif.org/ver10/events/wsdl" xmlns:tptz="http://www.onvif.org/ver20/ptz/wsdl" xmlns:tan="http://www.onvif.org/ver20/analytics/wsdl" xmlns:tst="http://www.onvif.org/ver10/storage/wsdl" xmlns:ter="http://www.onvif.org/ver10/error" xmlns:dn="http://www.onvif.org/ver10/network/wsdl" xmlns:tns1="http://www.onvif.org/ver10/topics" xmlns:tmd="http://www.onvif.org/ver10/deviceIO/wsdl" xmlns:wsdl="http://schemas.xmlsoap.org/wsdl" xmlns:wsoap12="http://schemas.xmlsoap.org/wsdl/soap12" xmlns:http="http://schemas.xmlsoap.org/wsdl/http" xmlns:d="http://schemas.xmlsoap.org/ws/2005/04/discovery" xmlns:wsadis="http://schemas.xmlsoap.org/ws/2004/08/addressing" xmlns:wsnt="http://docs.oasis-open.org/wsn/b-2" xmlns:wsa="http://www.w3.org/2005/08/addressing" xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" xmlns:wsrf-bf="http://docs.oasis-open.org/wsrf/bf-2" xmlns:wsntw="http://docs.oasis-open.org/wsn/bw-2" xmlns:wsrf-rw="http://docs.oasis-open.org/wsrf/rw-2" xmlns:wsaw="http://www.w3.org/2006/05/addressing/wsdl" xmlns:wsrf-r="http://docs.oasis-open.org/wsrf/r-2" xmlns:trc="http://www.onvif.org/ver10/recording/wsdl" xmlns:tse="http://www.onvif.org/ver10/search/wsdl" xmlns:trp="http://www.onvif.org/ver10/replay/wsdl" xmlns:tnshik="http://www.hikvision.com/2011/event/topics" xmlns:hikwsd="http://www.onvifext.com/onvif/ext/ver10/wsdl" xmlns:hikxsd="http://www.onvifext.com/onvif/ext/ver10/schema" xmlns:tas="http://www.onvif.org/ver10/advancedsecurity/wsdl" xmlns:tr2="http://www.onvif.org/ver20/media/wsdl" xmlns:axt="http://www.onvif.org/ver20/analytics"><env:Header><wsa:Action>http://www.onvif.org/ver10/events/wsdl/PullPointSubscription/PullMessagesResponse</wsa:Action>

</env:Header>

<env:Body><tev:PullMessagesResponse><tev:CurrentTime>2024-11-09T00:07:29Z</tev:CurrentTime>

<tev:TerminationTime>2024-11-09T00:17:29Z</tev:TerminationTime>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:VideoSource/MotionAlarm</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="Source" Value="VideoSource_1"/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="State" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:VideoSource/GlobalSceneChange/ImagingService</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="Source" Value="VideoSource_1"/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="State" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:RuleEngine/CellMotionDetector/Motion</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="VideoSourceConfigurationToken" Value="VideoSourceToken"/>

<tt:SimpleItem Name="VideoAnalyticsConfigurationToken" Value="VideoAnalyticsToken"/>

<tt:SimpleItem Name="Rule" Value="MyMotionDetectorRule"/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="IsMotion" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:RuleEngine/TamperDetector/Tamper</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="VideoSourceConfigurationToken" Value="VideoSourceToken"/>

<tt:SimpleItem Name="VideoAnalyticsConfigurationToken" Value="VideoAnalyticsToken"/>

<tt:SimpleItem Name="Rule" Value="MyTamperDetectorRule"/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="IsTamper" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:VideoSource/ImageTooDark/ImagingService</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="Source" Value="VideoSourceToken"/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="State" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:RuleEngine/FieldDetector/ObjectsInside</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="VideoSourceConfigurationToken" Value="VideoSourceToken"/>

<tt:SimpleItem Name="VideoAnalyticsConfigurationToken" Value="VideoAnalyticsToken"/>

<tt:SimpleItem Name="Rule" Value="MyFieldDetector1"/>

</tt:Source>

<tt:Key><tt:SimpleItem Name="ObjectId" Value="0"/>

</tt:Key>

<tt:Data><tt:SimpleItem Name="IsInside" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:Monitoring/OperatingTime/LastReset</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Data><tt:SimpleItem Name="Status" Value="2024-11-03T00:59:26Z"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:Monitoring/OperatingTime/LastReboot</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Data><tt:SimpleItem Name="Status" Value="2024-11-03T00:59:26Z"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:Monitoring/OperatingTime/LastClockSynchronization</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Data><tt:SimpleItem Name="Status" Value="2024-11-03T00:59:26Z"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

<wsnt:NotificationMessage><wsnt:Topic Dialect="http://www.onvif.org/ver10/tev/topicExpression/ConcreteSet">tns1:Device/HardwareFailure/StorageFailure</wsnt:Topic>

<wsnt:Message><tt:Message UtcTime="2024-11-09T00:07:29Z" PropertyOperation="Initialized"><tt:Source><tt:SimpleItem Name="Token" Value=""/>

</tt:Source>

<tt:Data><tt:SimpleItem Name="Failed" Value="false"/>

</tt:Data>

</tt:Message>

</wsnt:Message>

</wsnt:NotificationMessage>

</tev:PullMessagesResponse>

</env:Body>

</env:Envelope>

Great. Whats your thoughts on using the channel itemLeft so a duplicate channel is not added? If using packageAlarm you will need to add it to the readme.
Does Reolink currently have an entry in the reply from /api.cgi?cmd=GetAbility that tells us if the camera does or does not have this feature so the channel can be removed from cameras that do not have the support?

I’m pretty sure some of the cameras do not bother implementing GetServiceCapabilities I would need to re-test to be sure, but thats why I added the requests and did not implement anything for the replies. I saw a lot of logs back from users on what their cameras were replying with… This is a common thing with ONVIF, that brand X do not bother implementing stuff, and sometimes methods work that their cameras are saying they do not support. The cameras that are going to get used by a generic ONVIF type are likely to have a poor implementation, as all the popular models on the market tend to already have API support in the binding. The exception would be Axis cameras that have no API support, and would follow the spec very strictly, but due to their price they will not be a camera that is as commonly used as the cheaper end brands.

By taking the stance we can choose to force one method I think that works well, so I like your changes that allow forcing a method and the default of auto detection.

I just uploaded a new version with all my changes in a single commit.

I fixed the XML problem you reported. Some events don’t have a subject element and some also don’t have a data element. The subject is only used for the log entry, so I simply ignore this element when it is null. When the data element is missing I just skip to the next event.

The /api.cgi?cmd=GetAbility api does not yet report the packageAlarm ability. I did not implement this event this time because it’s not working at the moment anyway. We can always add it later if the feature is offically released.

You can find the new version here: Various Onvif fixes for IpCamera · TheNetStriker/openhab-addons@000d640 · GitHub

I can confirm they are fixed. The multiple events in the one reply is working well and being able to force which type of event method you use is also working and a great addition.

You will need to do these and create a PR so this can get merged:

  1. Add the new config to the readme.
  2. run spotless to fix the code formatting.

Lastly I found the wording of the config not clear for someone without knowledge of how ONVIF works. Suggest changing ONVIF service type to be ONVIF event method.

Like your changes and look forward to seeing a PR and this being merged.

I just updated the readme and changed the text as you described, but I’m not sure how to run spotless correctly. I’m using eclipse with the official installation method described here: Eclipse IDE | openHAB

In the coding guidelines I found the line “Official openHAB Eclipse IDE setup is preconfigured”. Does this mean that formatting was done automatically already or do I still have to trigger this manually?

Please also review the changes I made to the readme: Various Onvif fixes for IpCamera · TheNetStriker/openhab-addons@45c7e00 · GitHub

If everything is ok i will create a PR for this.

Go to your command line and type mvn spotless:apply this will fix the formatting of the files.

See Coding Guidelines | openHAB

I managed to get the spotless command line to work. I had set a wrong JAVA_HOME environment variable and this caused an error. There where no issues with the code formatting.

I just uploaded the PR here: [ipcamera] ONVIF fixes for IpCamera by TheNetStriker · Pull Request #17732 · openhab/openhab-addons · GitHub