How to watch and look through logging

Tutorials & Examples
4

Right, but the example will only pass lines with MyItem1 to the last grep, so in effect only lines with MyItem1 AND MyItem2 will be shown.

Also, using tail -F instead of tail -f will work even after a log file rotates. Handy when DEBUG is enabled and output is massive.

5

Indeed, that was a dumb mistake.

I’m not too concerned about following a log rotate but that is a good point.

6

Good tutorial, that’s useful stuff! Thanks!

7

Very helpful - thumbs up!

8

Added back a version of the OR case that will work with regular grep. The trick was finding the -E flag.

1 Like
9

@rlkoshak
I am wondering what linux command I can use if I don’t want to analyze the events.log in realtime using tail -f? What am I up to? Well, let’s say I want to filter for a specific item and its related events during the last 72 hours. How am I supposed to do that?

10

Just drop the tail -f part and add the name of the file to grep

grep MyItem $OH_EVENTS
11

:+1:

Follow up question: How can I filter all the events that happened between 22:59h and 23:01 for all devices?

12

I know of no way to do that.

13

Depending on how long ago it was, you can use the -n option for tail to start the tail showing some number of previous lines, then scroll to the time you need. This will start the tail and show the previous 3000 lines:

tail -F -n3000 $OH_EVENTS

14

Is there a limit how many lines will be shown? Even if I use -n100000, I do see the same amounts of lines as with -n3000. It will show me approximately the last six hours.

15

I don’t know… but if you need to go that far back, just open the file.

16

Hehe, that’s true :wink:

17

I thought of another trick that might work for you… use the -A option and filter on the start time. So something like:

grep -E -A100 "2017-11-07 22:59" $OH_EVENTS

or try this:

grep -E "2017-11-07 22:59|2017-11-07 23:0[0-1]" $OH_EVENTS

18

There is no limit but realize that the log file gets rotated out whenever it gets above a certain size and periodically so unless you are unlikely to ever have a file with 100000 lines in it to begin with.

19

Very helpful tutorial.

My idea is to use it to exclude log entries in Frontail output, which works perfect on a manual call, but ignores “| grep -v MyItem” when I add it to the Systemd Unit file.

Does anyone have an idea on how I can get that working?

20

Use the “Filter” field in the upper right. That takes regular expressions.

21

Hey there, hoping someone might be able to shed some light on what is going on with my system with respect to logging. I am a relative beginner to this stuff. Hope this is the right place for the post. About 2-3 months ago with some system upgrade my events.log went from including only item state changes to including all item state updates. The vast majority of the time updating the item with the exact same value which it was previously - for example, the following repeats at least several thousand times through the day, every day and this is just one Item!

2023-06-08 21:18:01.366 [INFO ] [openhab.event.ItemStateUpdatedEvent ] - Item 'MarkPhone' updated to ON

With almost 1k items my log has become almost unusable for finding meaningful information about state changes as I used to be able to do. I used to be able to see the last 30 minutes or so of logging when I open openhabian:8080 in a browser but now it is only the last ~2 minutes.

I have tried using the above and the info here to change the logger “openhab.event.ItemStateEvent” to WARN or DEBUG instead of INFO as it is currently (I did this with log:set from the console) but the system continues to log all state updates instead of just state changes as it used to do. I have also explored the file log4j2.xml to see if what I am trying to add to the file is making it in there, and it is indeed:

name="org.openhab.core.items.events.ItemStateUpdatedEvent"/>
		<Logger level="WARN" 
name="org.openhab.core.items.events.GroupStateUpdatedEvent"/>
		<Logger level="WARN"

I am obviously misunderstanding something - any help would be greatly appreciated. I’m on a RP4B using Openhabian and OH 4.0.0.M3. The problem has persisted through at least two milestone updates, if that is relevant. Thanks so much in advance for any guidance :+1: I am on the verge of starting from scratch with an Openhabian backup and fresh install/restore (but I worry the problem would persist b/c of the restore) :thinking:

22

No, the right place would be a new thread in the Setup, Configuration and Use category.

You are running OH 4 and missed checking the announcements and/or monitoring the discussion thread. There are two new update events which are not suppressed by default. If you keep your existing log4j2.xml file during the upgrade, you won’t get the changes to that file that suppress the logging of these new events.

That syntax is incorrect. For one, assuming it’s not a copy and paste mistake it’s not even correct XML.

In the Logger section of the file you should have

...
                <!-- openHAB specific logger configuration -->

                <Logger level="INFO" name="org.openhab"/>

                <Logger level="ERROR" name="openhab.event.ItemStateEvent"/>
                <Logger level="ERROR" name="openhab.event.ItemStateUpdatedEvent"/>
                <Logger level="ERROR" name="openhab.event.GroupStateUpdatedEvent"/>
                <Logger level="ERROR" name="openhab.event.ItemAddedEvent"/>
...
1 Like
23

Thanks for the response.

Noted. In light of us both having already irrevocably posted in this oh so inappropriate thread…

Exactly what I needed thank you! Yes you are correct, monitoring threads is not my forte/priority (358 posts at present). I did read the announcements as those are more of a priority and are more succinct - but obviously not thoroughly enough or perhaps I just did not understand what I was reading secondary to being a relative beginner as I mentioned. Appreciate your patience. I found the reference you mentioned and was able to fix my problem :+1:

That actually is the syntax in my log4j2 file. Tbh I just don’t understand it (as I mentioned previously I am a relative beginner and do not speak XML). I was confused by the way that WordPad opened the file with strange spacing which I now realize was stupidly due to text wrapping. This strangely carried over into what I copied/pasted, as you suspected, so I unknowingly copied half of a statement. Anyway turns out when I did my log:set I had added “org.openhab.core.items.events.GroupStateUpdatedEvent” instead of what you posted above and what is mentioned in the thread you referenced. What I had was close to correct - I just added the “org.” and the “.core.items.” parts in error, trying to match the formatting of the other entries I guess, and thus accomplished nothing even though I had relatively blindly stumbled on roughly the correct sequence of steps.

Thanks again for the help and understanding, it has been a learning experience.

1 Like