Homematic Binding: "Channel not found for datapoint" errors for definitely existing channels

Add-ons Bindings
, ,
102

I also have two CCUs, one is a CCU2 (stock) and one is raspberry matic (CCUpi). The second one is running smoothly but the CCU2 started getting the above errors, e.g. Channel not found for datapoint '0000DA498CXXYY:1#STATE'.
I had similar errors even before I first introduced the CCUpi to the setup which was in November 2020. But I can’t remember that data not correctly shown in openhab.
Bundle restart via console did not help, even openhab2.service restart did nothing. CCUpi is always coming up again, CCU2 fails to send data to openhab or openhab fails to receive data properly.

Edit: just restarted the binding via console. After that, my items are added again and openhab.log prints:
Channel not found for datapoint '000C9A49A7XXYY:0#CONFIG_PENDING'
for all channels for this HmIP-BWTH
Afterwards (2min later) it commes for the next device, which is a HmIP-SRH, next HMIP-SWDO. And it doesn’t matter whether they are connected to CCUpi or CCU2.
Just found out during analyzing: some devices were not listed, but come up when the state in the CCU changed. So I opened one window and the HMIP-SWDO there, just came up in the logs.

Some other devices report correct values, they don’t occur in the logs. No consistency the the CCU they are connected to, the device type or anything else I can see. I realized it for HmIP-BWTH, which seem to report every few minutes (between 2 and 6 in the last log file)

openHAB 2.5.12.
Ubuntu 18.04.5 LTS
openjdk version “1.8.0_282”
OpenJDK Runtime Environment (Zulu 8.52.0.23-CA-linux64) (build 1.8.0_282-b08)
OpenJDK 64-Bit Server VM (Zulu 8.52.0.23-CA-linux64) (build 25.282-b08, mixed mode)

Edit 2:
Just installed OH3 3.0.1 locally (desktop)
Ubuntu 21.04 (dev branch)
OpenJDK Runtime Environment Zulu11.45+27-CA (build 11.0.10+9-LTS)
OpenJDK 64-Bit Server VM Zulu11.45+27-CA (build 11.0.10+9-LTS, mixed mode)

Imported the backup (opencli backup/restore) from my server… Everything is fine. Every device gets the correct data.

Edit 3, 10 days later:
After lots of “restart, configure, restart binding, load items, …” I’ve removed my CCU2 and use a HmIP Access Point as LAN Gateway. Let’s see, how it is working…

103

I also have 2 CCUs (1x RasperryMatic acting as a CCU3 & 1x CCU2) and running this setup for a year now using 2.5.X as well as 3.0 now. If I remember I had the datapoint issue in the past, but currently I dont have them.
Happy to help in case of testing/comparison, etc.
Best Sebastian

104

No more problems afterwards… So for me it seems like 2 CCUs are causing problems.

105

I have just seen this issue on a single Homegear system aswell. So seems like it is not related to multiple CCUs.

106

I just spent some time investigating this issue as I can reliably reproduce it at the moment. The channels are not there but they are added in the HomematicTypeGeneratorImpl’s generate() method. The groupType is also correctly generated and then it is also correctly passed to createThingType(), where the groupType also still contains the channels, I did this there to check that:

            String id = usPos == -1 ? groupType.getUID().getId() : groupType.getUID().getId().substring(usPos + 1);
            logger.debug("Adding group type '{}', '{}', '{}'", id, groupType.getUID(),
                    groupType.getChannelDefinitions());
            groupDefinitions.add(new ChannelGroupDefinition(id, groupType.getUID()));

Now I don’t know where to continue tracing this. The only thing that comes to my mind is that for some reason the groupType can’t be found later on for some reason, so even though the groupDefinitions contain the proper ChannelGroupDefinition it can’t find the ChannelGroupType that it is referring to. Unfortunately I don’t know how to check that. There is the channelGroupTypeProvider which should know about this ChannelGroupType.

I have just added debug code to the ChannelGroupTypeProvider and ChannelTypeProvider to show what is found and returned. Upon using the Web UI to check out the thing the channel group type and then the channels are returned but not shown in the webinterface, so I would assume that the addon is returning the channels properly and the issue is somewhere in the core?

107

I have done additional checks: Requesting /rest/thing-types/homematic:HG-BC-TC-C-WM-4 does properly show a list of channels: {"channels":[],"channelGroups":[{"id":"-1","label":"Bc-tc-c-wm-4","channels":[]} - Pastebin.com

However, once I look at the thing at /settings/things/homematic:HG-BC-TC-C-WM-4:gateway:NEQXXXXXXX the channels are not shown in the channels tab. So the thing type is properly populated but the actual thing isn’t for some reason. Is it possible, that the thing doesn’t get updated if the thing type is changed afterwards for some reason?

Edit: It seems to be a classic race condition here: If the thing type is not available yet when the thing is created then the thing will be missing the channels afterwards. Unfortunately I don’t yet know what to do in that case, a changeThingType if no channels found is a dirty hack but it ends in a MISSING_HANDLER status (but with the channels populated properly).

108

Feels like a strange coincidence, today my one CCU failed with two devices and now I am stuck with restarting the CCU, bundle:stop/start the homematic binding and disable/enable the things, reloading the files with items to get it working again to no avail. Spam of Channel not found for datapoint all over openhab.log.

109

Can you give us some more information?

  • OH version used?
  • Where is OH runinng?
  • Is Homegear running on the same system?

Can you also please enable the TRACE log mode, try to reproduce the problem and attach the relevant part of openhab.log. Especially in the case of a race condition, the log file can give some hints.

110

OH version is 3.2.0.M1, it is running on a odroid c2, homegear is running on the same system. I am using text-based config, that might be relevant aswell.

TRACE doesn’t provide any additional information, I guess I would have to enable some core logging there to see when the thing is created based on the thing-type, then we can see that the thing-type wasn’t populated at that point in time most likely. The issue is that OH only creates the channels from the thing-type when the thing is created. If the thing-type is not created/populated properly at that time you end up with an empty thing, and that’s what’s going on here.

My patch for this is to create the channels when they don’t exist, works pretty well:

diff --git a/bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/handler/HomematicThingHandler.java b/bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/handler/HomematicThingHandler.java
index 39899a5d65..43d9324460 100644
--- a/bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/handler/HomematicThingHandler.java
+++ b/bundles/org.openhab.binding.homematic/src/main/java/org/openhab/binding/homematic/internal/handler/HomematicThingHandler.java
@@ -150,6 +150,42 @@ public class HomematicThingHandler extends BaseThingHandler {
         // update thing channel list for reconfigurable channels (relies on the new value of the
         // CHANNEL_FUNCTION datapoint fetched during configuration update)
         List<Channel> thingChannels = new ArrayList<>(getThing().getChannels());
+
+        if (thingChannels.size() == 0) {
+            for (HmChannel channel : device.getChannels()) {
+                for (HmDatapoint dp : channel.getDatapoints()) {
+                    if (HomematicTypeGeneratorImpl.isIgnoredDatapoint(dp)
+                            || dp.getParamsetType() != HmParamsetType.VALUES) {
+                        continue;
+                    }
+                    ChannelUID channelUID = UidUtils.generateChannelUID(dp, getThing().getUID());
+                    if (containsChannel(thingChannels, channelUID)) {
+                        // Channel is already present
+                        continue;
+                    }
+
+                    ChannelTypeUID channelTypeUID = UidUtils.generateChannelTypeUID(dp);
+                    ChannelType channelType = channelTypeProvider.getInternalChannelType(channelTypeUID);
+                    if (channelType == null) {
+                        channelType = HomematicTypeGeneratorImpl.createChannelType(dp, channelTypeUID);
+                        channelTypeProvider.addChannelType(channelType);
+                    }
+
+                    Channel thingChannel = ChannelBuilder.create(channelUID, MetadataUtils.getItemType(dp))
+                            .withLabel(MetadataUtils.getLabel(dp))
+                            .withDescription(MetadataUtils.getDatapointDescription(dp)).withType(channelType.getUID())
+                            .build();
+                    thingChannels.add(thingChannel);
+
+                    logger.debug(
+                            "Updated value datapoints for channel {} of device '{}' (function {}), now has {} datapoints",
+                            channel, channel.getDevice().getAddress(), channel.getCurrentFunction(),
+                            channel.getDatapoints().size());
+                }
+            }
+            updateThing(editThing().withChannels(thingChannels).build());
+        }
+
         if (updateDynamicChannelList(device, thingChannels)) {
             updateThing(editThing().withChannels(thingChannels).build());
         }

I have created an issue for this: https://github.com/openhab/openhab-addons/issues/11214

111

On which OH version are you?

112

It could really have something to do with that. I mean that something has changed there in 3.1. The Homematic binding works a bit different than most other bindings.

I haven’t worked with text files since version 3, and I never had this problem.

I have also seen the issue. Do you make a pull request with your changes?

113

I am wondering if this is indeed a proper fix though. I mean what I am doing is essentially hoping that the thing type is working properly, checking if it is and then creating the channels, then checking if the function of the device is matching the channel config and then updating the channels again. Why not remove the channels from the thing type and create them directly and only for the thing?

114

As far as I understand the class model, the ThingType contains a list of ChannelDefinitions and the Thing contains Channels. Without the ChannelDefinitios I would assume that a configuration in Main UI would no longer work.

But maybe the whole problem is caused by a change in OH Core. I have never seen such a problem in 2.5 where I used text files for the definitions of my items.
With 3.x I had never problems with definitions that I made in Main UI (I am no longer using text files because of the enhanced model functionality).
So it seems that the timing has changed in 3.x but only if the definitions are stored in text files. Maybe a core developer can help.

115

I talked to @marcel_verpaalen about this as he is maintaining the miIO Binding which uses dynamic channels aswell. If I understood him correctly he doesn’t use ChannelDefinitions in the Thing Type at all and only loads the channels into the thing at initialization of the thing. If it work’s there then it should work here aswell.

116

The binding seems to be much simpler (maybe because of the limited number of different devices).

I am not sure what really would happen if no ChannelDefinitions would be created. But I would expect an increased memory consumption. If you have a lot of devices of the same type and you add a complete channel definition to each device (= thing) you a lot of duplicated definitions in memory. And there are devices with a large number od data points (= channels).

But I am not an expert in this part of binding development. For me the question is: why does this problem only seems to happen with definitions in text files? Or does it only happen in combination with Homegear?

117

You mean the miIO binding? That binding supports a whole lot of devices, if I had to guess more than 50-100 at least. They all come down to 2 (if I am not mistaken) thingTypes though.

I think nobody really knows why this is happening, some thought it is related to multiple CCUs, some thought it is related to Homegear but so far nobody was able to find the real cause.

I am not sure if what you are saying about the memory consumption is a real issue: The channelType is still “cached” and only created if it doesn’t exist (that’s why there the == null check for the channelType), otherwise that is used so I think there is no additional definition created for each channel? At least my code doesn’t build any ChannelDefinition each time?

118

Hi,
no my OpenHab 3 are suffering the same problem. (2 CCUs, both are RaspMatic). Homegear isn’t involved in my environment. I’m on text based config too.

Kind regards,
Fabu

119

Sorry, I meant the smaller number of ThingTypes.

I was just a bit guessing here.

If you think, you have a working solution, you can create a jar to test the implementation. I can offer to execute some regression tests with definitions made in Main UI.

120

I know of installations with two CCUs that are working correctly. Same is true for installations using Homegear.

But it seems to have something to do with the text definitions and the Homematic binding in general. But I have no idea why there is this problem in OH 3.x.

121

I have a working jar based on the diff I posted above. I just wasn’t sure if I should remove the complete logic that adds the channels to the thing types or just do it this way.