Create a diagramm of openHAB

Michdo93 · July 22, 2022, 11:19am

Hi,

I keep seeing that there are misunderstandings about how openHAB actually works exactly. At the end of the day, everyone has an individual setup. This is also due to the different items. My idea would be that I openHAB therefore represented in a block diagram. I’ll explain various things and ask a few questions.

Often the event bus is represented as follows.

What I like about this representation is that you can see that different protocols are used in how the binding communicates with the device.

What I personally miss is that I have to create one or more Things from the Binding first. These Things are needed to create items. The communication between items and things takes place via channels.

There is another similar representation:

What I like is that you can see what is physical and what is virtual. Also good is the representation of item registry.

What I don’t like is that it says Hue, Samsung and Sound at the bottom.

And something else bothers me. For example, the REST API. Can it now directly trigger a command on the event bus or does this go through the item registry? Do changes to items through rules or through the UI then only have the way via the item registry? I think the arrow from REST API with commands to the Event Bus is too much, because the REST API is bidirectional to the Item Registry.

Also, the arrow with status updates to the Persistence Service bothers me. In my opinion, this should also be done via the item registry. What also bothers me here is that there is no arrow in the opposite direction. I mean something like restoreOnStartup.

Perhaps one would have to write with some arrow to the fact that this would be done only with certain configurations or in special cases so.

I have one more representation:

Granted, this is still for openHAB 2.x. What is again good is the representation that a binding refers to hardware or network service. In this representation, Rules appear to have direct influence on the Event Bus, and so do the UI and REST API. What I like is that there are multiple UIs that would be displayed. Since some fall out for openHAB 3.x.

The important question remains. Does everyone now communicate via the item registry and this results in changes to the event bus or vice versa, does one communicate with the event bus and this in turn results in the item registry adapting?

I have one more representation:

I think here you can ignore a large part. If you look at the first two graphics shown and now look at this one, it is confusing what is meant by openHAB Repository. You might think from the first to the second figure that this would be the Item Registry, REST API and Persistence.

Let’s finally get to the last graphic I brought:

I would almost be inclined to make the Item Registry larger. Just draw in some items there. These of course have an arrow coming from a Thing. So quasi: The item registry has several items. I can also draw an Unbound Item (Virtual Item) there as an example.

So, that I expand the second figure.

Also I would have to create one or more Things from the Binding first. And also here the correct representation would be as a rule that I would have to represent different Things. Let’s go to the Philips Hue Binding as an example. I would have a Thing for the Philips Hue Bridge. My Hue lamp would have a reference to the Bridge. And this Thing would have more precisely several references via Channels to different Items, which would be stored in the Item Registry. On the hardware side, I would also have to draw in the bridge first and behind the bridge are different lamps accordingly.

For Automation Rules, I could of course also subdivide into DSL and JSR223. There also exemplarily a .rules, a .py, a .js file etc. represent.

For User Interfaces I could also list different User Interfaces. According to my understanding, MainUI, BasicUI, HABPanel and HABot should be listed there.

Now I see openHAB as a program running on a server/computer. So I would draw openHAB in a big box and around it the computer. For various reasons. Because bindings communicate with locally installed programs or via different interfaces of the computer.

An example would be the MQTT binding. For this to run, an MQTT broker can be installed on the computer (or on another computer). Definitely, however, an MQTT client must be installed. The binding communicates with another program (client) and this in turn communicates with another (broker). And from there everything goes to the network interface.

Another example would be the Exec Binding. With this I execute commands locally on the computer outside of openHAB. I would therefore represent the console as a block. Over it a Python, C, C++, Java or whatever program can be executed. This would remain local. Or I execute ssh over it, which would be another block and this then goes out over the network interface and possibly to another computer, where I draw in that something is executed there.

Then I have both Zigbee and Z-Wave, soon also LIRC and Bluetooth. There the binding should go to the USB interface of the computer. So I would have to draw in that, for example, a Zigbee device then communicates through it and the binding accesses one of several USB ports.

Philips Hue also goes out via the network interface. I’ll take HomeMatic as another example. Up to the gateway (CCU3) the communication from the binding also goes over my classic network. From here, of course, I must then also draw in that the HomeMatic gateway to a HomeMatic device communicates via a radio standard.

I still use HABApp, which I would have to draw in as another program. This accesses the REST API.

A difficult use case for me will still be the representation of the openHAB Cloud. And the connection to Amazon Alexa.

I took a closer look at the diagram and I think something is wrong. Or rather, it is presented in a very idealized way. It also feels like you are promised more than really works in my experience. But it may be that if I create my own server with the cloud software, that would actually add more than just the IFTTT integration.

I am irritated by the Device Connector. I think the devices are not connected via the cloud but via openHAB. And other inconsistencies.

I don’t want to digress too much here. The question would rather be the representation of the two bindings. The cloud connection takes place via a Thing. A secret is used here and so on. The only thing that happens in my opinion is that the cloud is then allowed access to the REST API and can query all items, things, sitemaps, etc. and thus display them. If I understand the Amazon Echo Control Binding correctly, then the binding goes directly to the Amazon Cloud also via login data, but the data flow comes from the openHAB Cloud. So for this I would have to make an arrow from the Amazon Echo Control Binding to the openHAB Cloud, which would be a kind of gateway and from there just continue to the Amazon Cloud? The login data would ultimately also have to be passed on via the REST API. In other words. The openHAB Cloud mirrors my REST API.

I am unfortunately ill, otherwise I would have already sketched a first draft and uploaded it. But I’m collecting ideas for possible representations. Maybe you can think of other things that should be considered for a correct and detailed presentation.

Many thanks in advance.

rlkoshak · July 22, 2022, 3:23pm

Thanks for doing this. I agree it would be great to have a diagram like this in the Concepts section of the docs. Of course the text would need to be added to.

Some comments follow.

Looking at the first drawing:

I’m not sure what is meant by “openHAB Repository”
Many of those arrows need to be double headed. Automation Logic can both send and receive both commands and updates. The Console can send commands and updates. The bindings can send and receive commands.
Where’s persistence?
I’m not certain that having logging and the console on a high level diagram like this makes sense. Most average users will not ever use the console and there’s a lot more that goes into the logging than commands and updates.
There are more events that get put on the event bus beyond just Item events. Though it’s questionable whether those should be on a drawing of this level.
I’d like to see Things and Channels represented.

Not necessarily. You can have unlinked Items.

Second drawing.

Hurray! Persistence.
The User Interfaces strictly go through the REST API so probably should be shown that way or the REST API not shown at all. Also, the REST API is bidirectional and supports updates, commands, and other events (Thing status, rule status, etc.).

I don’t think it’s bad to have concrete examples.

Good question. We’d have to trace through the code to understand that. What parts of OH actually generate the events? Item changed events clearly come from the Item/IR but do updates come from the Item or do the Items react to an event placed on the event bus by something else, or both?

The UI strictly go through the REST API.

I’m pretty sure, given how updates and commands are issued in some of the rules languages, that rules put the event directly on the event bus and the IR picks it up from there.

In fact, the REST API touches just about everything which makes it challenging to show on a diagram like this.

It’s driven by the event bus. It might be indirectly driven by the IR. How that all works depends on the results of tracing the code.

But that arrow should be commands, updates, changes going to it and historic data coming back from it. How best to show the coming back part is a question. We have restoreOnStartup (arrow going to the IR?), charting (arrow going to the UIs, perhaps through the REST API), and rules.

Drawing 3.

I think this one might be the most correct over all but I’m not certain it gives enough detail to be as useful as the others. I’d probably merge the multiple UIs into one block so the drawing doesn’t become dated (the only thing that makes this 2.5 are the words “Classic UI” and “Paper UI”).

I don’t think everything goes through the IR. Given there are a number of events that are not Item related it would be weird if that were the case.

But again, in some rules languages you actually can only postUpdate and sendCommand using the EventBus actions. Behind the scenes, I think that even in languages like Rules DSL, when you call sendCommand or postUpdate on the Item, behind the scenes it’s actually going straight to the event bus. Looking at the JavaDoc for the ItemRegistry: ItemRegistry (openHAB Core 3.4.0-SNAPSHOT API), there is nothing to indicate it processes events.

I think it’s this.

Drawing 4:

This looks similar to a diagram that used to be in the docs. It’s a very helpful diagram for a software architect. I think it’s less useful to an end user. It doesn’t really show the flow of the events which I think is the hardest part for end users to understand.

Drawing 5:

I think this one has the same problems as Drawing 4. It is useful for devs, less so to end users. All it really shows is the plug-in architecture. It doesn’t show data flows at all.

Don’t do this. For one, the whole “JSR223” term is out of date. Not all the supported automation add-ons use JSR223 and I think JSR223 is a deprecated term anyway and that mechanism is called something else now.

I’d recommend simply not making much of a distinction. Just show that several languages are supported and ignore any Rules DSL/JSR223 type distinctions. Rules DSL is very likely to become a separately installed add-on instead built into the core and it may not even come installed by default in the future. So Blockly, Rules DSL, JS Scrtipting, Jython, jRuby, etc are all at the same level. Whether they are built into core (Rules DSL), use JSR223 (Jython), or something else (JS Scripting uses GraalVM) doesn’t matter. And in fact, the specific languages shown on the diagram is less important than just indicating that multiple languages are supported in the first place.

NOTE: I’m working with @florian-h05 (to be honest he’s done most of the work thus far) on a complete reworking and restructuring of the Rules docs.

Just do enough to show that multiple UIs can be supported, similar to the automation languages.

I’m wondering if a single drawing will work. When doing systems architecting one often uses several different views (usually drawings) to describe the system from different perspectives. For example, you might have one view showing the physical parts of the system and how they connect. Another view might show the functional view and how the data flows between functional parts. And you’d have yet another end user view showing how the user interacts with the system.

Maybe trying to fit it all into one diagram is beyond what can be done.

At this level though, don’t get hung up about specifics of different bindings. The individual’s binding docs are there for that stuff. I would not go beyond a single double headed arrow showing interaction between the binding and the end device. I fear the details will become too much for a diagram that’s primary job is to explain openHAB internal concepts and data flows.

Don’t show that. That’s a different drawing. For this drawing you could just show it those the same as any other add-on.

It takes some doing but yes, if you stand up your own cloud server, you can get it connected to Alexa or Google Assistant.

I think this is showing internals to the app. These are functional proxies that represent those capabilities in your openHAB core instance.

When you do draw something, use Draw.io. That’s the project standard and it ensures that future maintainers can modify the drawing at a later date since it saves as XML.
Consider what it is you want to show in the diagram and stick to that. I think how the data flows between the core concepts in OH (External, Bindings, Things/Channels, Items, Automation, Persistence, UIs) are the most important parts. Don’t get bogged down in trying to be complete. Save those sorts of details for a different special purpose drawing.
Ignore the cloud stuff. It doesn’t really add anything at this level. It’s really it’s own topic worthy of it’s own drawing.
Definitely show the UIs going through the REST API, if the REST API is shown at all. Also show other stuff going through the REST API, like HABApp. I don’t know if it makes sense to list it by name since it’s an external project, but it’s important to show that the REST API is used by more than just UIs.
Something like the first drawing (with the missing stuff added) should fit the bill I think, but I also think one drawing might not do it. It might take three or four if you really want to show everything in every way.

PtrO · July 22, 2022, 8:35pm

Thanks for doing this. For a couple of years I’m pulling my hair to understand openHAB’s intimit(ten)dating internal effects and how the core is interactive with bundles, add-ons, refreshments, bindings and the user interfaces. As time progresses, I learnt that each one of these are independent modules an interface with the openHA…Bus.

I’m familiar with all of your pictures which try to show without really explaining what the relations are. The most explanatory diagrams area is those with the eventbus which on its own is vague as it’s (to me) unclear how that to detail that is (actually) functioning. Is it a message queue, an exchange point, a continuous flow, is it a acknowledged c/s of what is it…
I would like to understand the process flow of the core… at what state en step, is the event bus handled and are the attached modules serviced…

By he way, just thinking of it, perhaps you or others other are able to shine a light… is there a debug routine that can monitor the event bus like a kind of wireshark ?

rlkoshak · July 22, 2022, 8:54pm

Modify /var/lib/openhab/etc/log4j2.xml and change the logging level for all the entries that start with openhab.event to INFO. That will add all the events that occur on the event bus to events.log.

Or in MainUI you can open the Developer Sidebar and start the event stream (second tab) without any filters and see the event bus in real time.

But there really isn’t anything to an Event Bus. All it does is announce “Hey! Something happened!” with details about what happened. Clients subscribe to the bus for messages of a certain type for further processing.

For example, a “MyItem commanded to ON” might be an event published on the Bus. If MyItem is linked to a Channel, the binding will see that message on the bus and process it, commanding the end device to turn on. At the same time, autoupdate may see that command and predict the Item will change to ON and then issue an “item updated to ON” event. The Item will see that event and change it’s internal state to ON. Since it was a change, the Item will publish “MyItem changed to ON” event on the bus. Persistence may care about that so it can save it and will receive the event and process it. Rules may care about any of these too.

PtrO · July 23, 2022, 2:06pm

Thanks though I’m not using MainUi nor have sidebar or a log4j2.xml ^{(Iḿ using R2.4.0)} but i do get your hint to log/trace-all which not necessarily cover all activities (only those where a programmer has set a preceding trace/debug/info log operation).

I’m looking for the source module(s) - if any - that physically interact which reading, writing and deleting events on the BUS. The internal program-structure of openHAB is quite complex and though I’m able to understand things to language, it’s very complex (to say the least) to comprehend. The absence of comments aside something was done or stacked for a reason due to PR does not help to get an understanding.

For the evenBUS, I can imagine it’s a simple pathway, like my frontdoor. However the contents and moreover who’s using it, for what purpose, when in time en where (to go) with it; is what I’m looking for. In other words, are there any message/process flows. what exactly is executed. I distinguish 3 levels: process karaf/osgi/bundles, openHAB config settings (rules/items etc) and ‘data’ due to changes in item/things status or values.

Nonetheless, it brings me to an idea to investigate if I can set hooks in the source to optionally display with time, origin en target of every read/write/delete status of every eventbus change. Challenge is how to identify the “evenbus” in terms of program-lines.

Updating myself, as I still run 2.4.0, the file is named as org.ops4j.pax.logging.cfg and the log-items mentioned are not openhab.event , but prefixed as smarthome.event (all due to “old” release, as 3x cannot run on my 32bit/qnap box).
The same can also be accomplished by setting these on the openhab command console by for example
openhab> log:set INFO smarthome.event.ItemChangedEvent (and reset back to the the normal state DEFAULT
However, monitoring the event file does not really tell where and in what path on openHAB things are happening, just they are happening.

rossko57 · July 23, 2022, 2:37pm

That would be a lot of work and require updating any time something else was plugged into the bus.

A better approach might be … just subscribe to the bus and record the messages. Like the logger already does, as Rich was explaining. Like wireshark does, withoout invading every device attached to your network.

Udo_Hartmann · July 24, 2022, 7:30am

Hm. There is a version openHAB 2.5.12, and although there are still some references to smarthome and eclipse, I’m pretty sure it’s the better option over openHAB 2.4.0.

As of “can’t run on 32bit QNAP”: Please be aware that openHAB 3 is limited to at least Java 11 (maybe Java 17 does also work? Not sure about additional steps…).
openHAB itself is capable to run under 32 Bit perfectly well (Raspberry recommendation is 32 Bit anyway).

PtrO · July 24, 2022, 5:01pm

That seems valid but Iḿ still looking and fighting to migrate my good en old mqtt (item) stuff to the newly (for me oldskool “uncomprehending”) who’s approach is based on things-driven.

Yeah the Java11 was killing my ambitions to follow that 3.x.x route.
Nonetheless, in time I wil eventually reach 2.,5.12 but first I was busy to set-up things as preparation. Finally I’ve managed to create a qnap-APP package for relase 2.4.0 (as I prefer to avoid a manual installation on the QNAP.)
I now can run two (or more) instances on tmy same QNAP for openHAB (which lacks container/docker services.). Thanks to the capability of running multiple instances and full control on where data resides, migrating will be a lot easier then continuous up/downgrading when things go wrong. Now, i can simply run Active-Active and move items/things/rules one by one forward or back.

Coming winter, I likely will (slowly) activate 2.5.0 & thereafter the latest one.
^{FYI, I only migrate based on full compile of all my sources, which is & remains an interesting challenge, including the step-stone to leave exchange the smarthome-code with openhab. As I do this approach, i had in mind to check/see if my set-up can di 3.x.x. However, for this I must also migrate my IDE set-up wich is now based on java8…</sup)}

rlkoshak · July 24, 2022, 9:49pm

You absolutely do have a log4j2.xml. If you didn’t there would be no logs at all.

In this case, what I recommended will cause all events in the event bus to be logged to events.log. Not just select events

It’s kind of like a CAN bus in a car. When ever anything has an event of note, they publish it to the bus. If something cares about certain events, it subscribes to those events from the bus.

That’s all there is to it.

The rules engine subscribes to events based on the triggers. Persistence subscribes based on the persistence strategies. Items subscribe based on the Items. The uis get all the events through an SSE steam and filters based on what’s currently being shown.

There really isn’t message flows in the way you are looking I think. There is just publish and subscribe. The same individual event may be “consumed” by multiple parts of the system.

First, there is no delete. Delete is pointless as events are ephemeral.

You don’t need the timestamp if you turn up the logging on the event bus as I described.

Origin is going to be harder because in an event bus architecture, by design, the origin of the event doesn’t matter. Currently that’s information that’s thrown away. So you’d have to find every place that publishes to the event bus which I’m sure could be done, but you probably won’t learn too much because, again, the origin of the events do not really matter. What matters more is what events a subsystem can produce and what events a subsystem consumes.

I highly recommend not trying to create this drawing if you are not up on how OH 3 works. Not only had a lot of the code moved around, lots of other little stuff has changed too, even between 2.4 and 2.5 let alone 3.3 which is the latest.

To have the must utility, also an diagram should present everything in a way that the users can easily recognize based on what they actually see in OH. That’s going to mean having at least a passing familiarity with MainUI.

PtrO · July 25, 2022, 12:00am

Thanks for all your information and endless patience for the better and the good.

I’ve already discovered my log-file control (in R240) is named org.ops4j.pax.logging.cfg and log settings can also be managed via de log:set command in the karaf-console.
The tip for debug the (changed)events was very useful to get a bit more understanding.

Me, myself en I, like to do what’s not perhaps so much of a challenge for others as they just binary upgrade to recent versions. For the time being, I like crunching source-code and slowly getting an idea how events (a)crossystem are flowing and threaded in isolation. Understanding science removes the magic.
Basically it learns me that openHAB is (of course) an event driven system which are asynchronously processed. Good but also as bad, as some things are difficult to trackdown. Anyway, i’ve identified some artifact that interact with the event bus (f.e. post(Event event) using payload/topic etc.etc.).

The more I look into it, the more respect I get for what has architecturally been accomplished and their founders who are continuously pursue improvements by guiding users.

Youŕe fully right that creating a data flow-diagram based on 2.x is basically a waste of time as OH3 is leading. Also as there’s no serialized data-flow concept active (as youǘe correctly stated).

Nonetheless, the internals are intriguing and will in part, follow the same design-principals. for OH.3 which remains a Bus event driven system.

system · September 4, 2022, 4:01pm

This topic was automatically closed 41 days after the last reply. New replies are no longer allowed.