"new" tool to analyze openhab.log file for zwave messages

Add-ons Bindings
1

Hi,

I made a command line version of the zwave-analyzer from the http://opensmarthouse.org website from @chris . Would that be interesting to you? Here a screenshot:

Note: these are not the raw Z-wave messages, but the messages which go from OpenHab towards de Zwave hardware controller (and back). Routing information can not be extracted this way.

afbeelding
afbeelding737Ă—459 70.2 KB

Why I created this
I was using the Z-wave message analyzer on the site of @chris ( Z-Wave Log Viewer ) quite frequently. However, I always had to copy the log file from my Rasperry Pi towards my laptop, and then upload it to Opensmarthouse. It has splendid output, but it required some steps to get there. So, why not convert this towards a command line tool, that can be started just from the Linux shell?

Conversion from browser Javascript to node.js
That was easy said, but required some effort. First of all, browser based Javascript is different from running Javascript from the Bash commandline. It requires node.js as a runtime, and on the RPi, with Node.js 14, it has a different dialect/generation of the javascript language. And of course, the webbrowser interface had to be converted to a command line interface. I never used much Javascript before, but I managed to create a quite preliminary and early, yet useful version.

Questions

  • Would you use such a tool? I could put in on Github
  • Can anyone help creating a single codebase (a Javascript library) which can be used in the Opensmarthouse website and in the Bash commandline ? (This was actually a good suggestion of @chris)

By the way, I discovered quite some Zwave issues in my network due to the fact that I look more frequently right now.

Please let me know!

2 Likes
2

Just to clarify this point - no data is ever uploaded to the opensmarthouse website - all processing is performed locally on your computer and not on remote servers.

As we discussed on email, I personally am happy if someone wants to create a command line version. My only concern is that it gets out of sync with the online tool, and this would really be a nightmare IMHO. Would I personally use it - no - I prefer the online tool as there’s nothing to download or keep up to date, but others will have different views on using online tools (again though - data and logs are always private and NOT uploaded to the remote site).

1 Like
3

I’ve never needed to use the analyzer, so forgive me if this is a dumb question: is it possible/desirable to make the online version available in MainUI, enabling it to directly pull the log and do the analysis from within the GUI? I feel like that would be most helpful to new users trying to sort out Z-Wave problems, but have no idea what it would take to make it work (or if it even would).

Ignore me if this has already been discussed before. :wink:

1 Like
4

Probably - anything is “possible”, but I suspect it would be quite a bit of work I think.

What issue is this trying to solve though? Currently using the viewer is as easy as clicking on a link from the documentation. Is adding this into the main ui really going to solve a problem or really make it easier for anyone?

Log processing is always a bit of a manual task - you can’t just select a massive log and try and process it - it needs to be a bit selective, and you need to enable the logging etc, etc. This is described in the docs, and the viewer is linked from the docs - personally I feel this is a good approach.

The software is available - it’s open source - so if anyone wanted to add it to the main UI, this can be done - just like creating a standalone processor could be done. As with any fork, it then requires change management, which is a pain…

1 Like
5

I hear you on that.

Like I said, I’ve never needed it, so I’m not a good person to answer that question. It’d only be worthwhile if you felt that the additional work would significantly reduce the amount of handholding that’s required in the community. Of course, if that was the case then you probably would have done it already. :wink:

6

Agreed - and I don’t think integrating the viewer into the main UI will help this. We also have the Zigbee viewer, and maybe other bindings do a similar thing (I don’t know) and I don’t think trying to add everything into the main UI is a good idea unless it’s done as a plugin so it could be developed separately. This was raised during the initial development of the main UI, but was considered not to be worth the effort to have such a plugin architecture.

1 Like
7

As a bit of a compromise, would it be possible to have a link to opensmarthouse/the log tool and/or the OH documentation from the serial controller Thing, maybe as another link under the Z-Wave section where View Network Map and the other options are? I don’t know really anything about binding development so I don’t know if it’s possible but that seems like a good place to have such a link in addition to on the binding info page where it currently is

8

Anything is possible I guess, but what does this really achieve? Linking from the UI to the binding documentation is presumably possible - if you think this is useful, then I guess you can propose this as an enhancement in the UI - it could be done for all bindings - I don’t see why ZWave should be different.

The documentation already has links to the log viewer, but by itself adding this link to the UI is (IMHO) a bit of a waste of time, since the user really needs to read the documentation to understand how to enable logging. If they don’t do this, then the log will not be very useful, and the log viewer will not display anything at all.

9

Zwave networks have issues, as all wireless networks have. So you need to debug that sometimes. For those who do not recognize that, be happy! But that’s not always the case.
And than debugging is the first step. In many cases of Zwave issues on this forum, one of the first answer is usually: please send a log file processed by http://opensmarthouse.org.

Yes, you are right, I know that no data is going to any server, the Javascript is just executed within the browser, locally. But that’s not my point.

My point is, this requires a few steps, especially when you are remote:

  • copy data from (remote) pi to local laptop (if your pi is in your network, you can directly use the Samba mount of course)

  • Feed the log file to https://opensmarthouse.org

If you only want to have a quick view on Zwave issues that is a few steps. But Website, GUI or command line, either way you have to tell your average users how to do that.

I totally agree on that.

Just some other idea, and I do not know how much work that is:
Would it be possible to create a nice line in the openhab.log, formatted similar as in opensmarthouse?

So instead of: (or on top of)

10:30:37.150 - Receive Message = 01 07 00 13 B0 00 00 7D 26
10:30:37.152 - processReceiveMessage input 0<>128 : Message: class=SendData[19], type=Request[0], dest=0, k=176, payload=B0 00 00 7D
10:30:37.153 - Received msg (0): Message: class=SendData[19], type=Request[0], dest=0, callback=176, payload=B0 00 00
10:30:37.154 - lastTransaction TID 6934: [WAIT_REQUEST] priority=Immediate, requiresResponse=true, callback: 176
10:30:37.155 - Checking outstanding transactions: 1
10:30:37.156 - Last transaction: TID 6934: [WAIT_REQUEST] priority=Immediate, requiresResponse=true, callback: 176
10:30:37.157 - Checking TID 6934: (Callback 176)
10:30:37.158 - Callback match!
10:30:37.159 - Correlated to TID 6934: callback 176
10:30:37.161 - Incoming Message: Message: class=SendData[19], type=Request[0], dest=0, callback=176, payload=B0 00 00
10:30:37.162 - NODE 5: SendData Request. CallBack ID = 176, Status = Transmission complete and ACK received(0)
10:30:37.163 - NODE 5: resetResendCount initComplete=true isDead=false
10:30:37.164 - Receive Message = 01 09 00 04 00 05 03 25 03 00 D2
10:30:37.164 - TID 6934: Advanced to WAIT_DATA
10:30:37.165 - NODE 5: TID 6934: Transaction not completed
10:30:37.166 - ZWaveReceiveThread queue empty
10:30:37.166 - processReceiveMessage input 0<>128 : Message: class=ApplicationCommandHandler[4], type=Request[0],  callback=0, payload=0
10:30:37.167 - Transaction SendNextMessage 1 out at start. Holdoff false.
10:30:37.168 - Received msg (0): Message: class=ApplicationCommandHandler[4], type=Request[0], dest=5, callback=0, =00 05 03 25 03 00
10:30:37.168 - lastTransaction null

a line like this would be nice:

10:30:37.150  - NODE 5: RX REQ SendData 176 ACK RECEIVED from device in 1260ms 0 /128

If that would be easily extractable by grep, that would be a nice step forward.
(of course, the line @23.470 resembles @23.483…). But, I know, easily said, but perhaps complicated to build…

10

I use a remote system, but I map the drive, so there isn’t really any steps. I just open the web page, select the file, and display the log.

However, I appreciate people do things differently - to me, this is not difficult or time consuming, and having it on the web means it is always up to date.

The online system then allows me to filter logs too only show certain nodes.

The debug logs are primarily there for debugging, and the simplified view seen in the log viewer is used as a quick look. If I then want to see what is happening in the binding, I still need to look at the details in the log, so the information provided at the moment is needed - it shows all the different steps required to decode a packet.

11

Yes, seems like a tool that I can use!
I would enjoy to take a closer look. :slight_smile:

12

For me this kind of tool would be very usefull in some cases.

There could be even better solution. Why not embed java code that is on http://opensmarthouse.org into the openHAB itself?
You would go to z-wave binding page “tick” debug logging ON and immediately could see well formated human readable debug log.

I don’t know if technically it is possible but could be useful :slight_smile:

13

I already answered this a few posts above. I don’t really see that this makes things much more easier - is it really that hard to click on the link to go to the log viewer? It’s linked directly from the documentation, and IMHO we cant just keep piling things like this into the UI.

Do we also add the zigbee one there as well? How do we maintain compatability between different versions (ie the online version, and the one in the UI)? It’s a nightmare - this is why I put this online - so it’s always up to date, and available to anyone who wants to use it. These viewers are not just used by OH users, but also other systems, so I would always need to keep the online version, and it’s then a pain to manage the changes.

14

Which is at least one step away from where the others have requested it be - the openHAB UI itself.

This, in my opinion, is the right reason to keep it at a separate location.

1 Like
15

I agree. That’s something I hadn’t considered when I initially asked the question.

16

Hey tonus, could I get a copy of your CLI analyzer please - much better for me than a web page version. Unless I’m mistaken I didn’t see it linked anywhere yet …