openHAB - Filtering event logs

Simple string filtering on the event log. Useful for filtering out high change items that don’t need constant log visibility.

Re-posting this as it’s own thread on the recommendation of other members. I also recommend reading Taming openHAB 2 Logging and Log4j2 Sample Config which this was based on.

Log configuration data using openhabian-pi is stored in the following file. Location may vary under other installation methods.

/srv/openhab2-userdata/etc/org.ops4j.pax.logging.cfg

openHAB using log4j edit the file to add filter rules using the following methodology.

# Filtering Rules
log4j.appender.event.filter.<your-filter-name>=org.apache.log4j.varia.StringMatchFilter
log4j.appender.event.filter.<your-filter-name>.StringToMatch=<your_filter_string>
log4j.appender.event.filter.<your-filter-name>.AcceptOnMatch=false

openHAB 2.2 uses log4j2 with a different syntax.

# Filtering Rules
log4j2.appender.event.filter.<your-filter-name>.type = RegexFilter
log4j2.appender.event.filter.<your-filter-name>.regex = .*(<your_filter_string1>|<string2>|<string3>|<etc>).*
log4j2.appender.event.filter.<your-filter-name>.onMatch = DENY
log4j2.appender.event.filter.<your-filter-name>.onMisMatch = ACCEPT

Replacing <your-filter-name> with a suitable filter name, and similarly <your_filter_string> with the string you want to filter out. In my case I have a Smappee power meter which updates household power consumption on a second by second basis via an MQTT feed. The event data for this just fills the log up, so since upgrading to OH 2.2 have now updated my filter rule to the following.
Shout out to @kintrupf for the regex (OR) example shown in the posts further down and now edited into the example above.

# Jp - filter out stuff
log4j2.appender.event.filter.smappee.type = RegexFilter
log4j2.appender.event.filter.smappee.regex = .*power_smappee.*
log4j2.appender.event.filter.smappee.onMatch = DENY
log4j2.appender.event.filter.smappee.onMisMatch = ACCEPT

power_smappee being the name of my item in my set up. I haven’t tried it yet, but I don’t believe there should be any reason you can’t create multiple filters to filter out different sets of data, similar to the sun, moon astro examples in the linked pages.

DISCLAIMER: I don’t profess to be an expert on regex or log4j, this is just based on my own research, and once set up works well for me. :slight_smile: If it helps you, excellent!, if it doesn’t work, I probably can’t tell you why it didn’t, sorry.

EDIT: The above will prevent events from being written to the log. So you won’t have any option to review them after if you change your mind. If you’re just looking for a way to restrict what you view ‘in’ the log - you can pipe the output through various OS tools to only show you the key lines you’re looking for. Details in this post: How to watch and look through logging

Jp.

28 Likes

Can someone provide an example of how I can hide the Astro events in the logs? Not all the Astro log entries contain the phrase “astro.” I thought I needed to provide the module name in order to filter out those events. Currently, I have them filtered out by creating three filter entries; one for astro, Sun, and Moon.

Update: I spoke too soon. it did not work

from my testing you only get one filter, and if you have two, the last one in the file is the one that works.

1 Like

I’m no expert, but it seems you can use a more elaborate regular expression to filter on more than one event :slight_smile:
This works for me to filter out all regular events from the ntp, astro and WeatherUnderground bindings.
The ntp bindung defines the item “CurrentDate” and all my WeatherUnderground items start with “Weather”.

# custom filter 
log4j2.appender.event.filter.uselessjunk.type = RegexFilter
log4j2.appender.event.filter.uselessjunk.regex = .*(CurrentDate|astro|Weather).*
log4j2.appender.event.filter.uselessjunk.onMatch = DENY
log4j2.appender.event.filter.uselessjunk.onMisMatch = ACCEPT

4 Likes

that seems to have done it. thank you!

Yeah, regex isn’t my forte. ;p Hadn’t even thought to look into using or clauses in the syntax.

Fantastic, this was exactly what I was looking for!!
The structure for the filter expression.
.(anything|anything2|anything3|anything4).
Now I can filter all astro junk like this:
.(Elevation changed from|Azimuth changed from|Diffuse_Radiation changed from|Total_Radiation changed from).
While still not filtering any other things that might have astro in their expression.
Thank you! :smiley:

I was trying to stop the logreader binding flooding the eventlog also the network binding with

# chod - less flood on the logging - network
log4j2.appender.event.filter.network.type = RegexFilter
log4j2.appender.event.filter.network.regex = .*network.*
log4j2.appender.event.filter.network.onMatch = DENY
log4j2.appender.event.filter.network.onMisMatch = ACCEPT

# chod - less flood on the logging - logreader
log4j2.appender.event.filter.logreaderx.type = RegexFilter
log4j2.appender.event.filter.logreaderx.regex = .*logreader_reader.*
log4j2.appender.event.filter.logreaderx.onMatch = DENY
log4j2.appender.event.filter.logreaderx.onMisMatch = ACCEPT

it looked like the logging for the matching texts stopped but later it continued

i added the lines to

/var/lib/openhab2/etc/org.ops4j.pax.logging.cfg 

Try this :slight_smile:

#  chod - less flood on the logging
log4j2.appender.event.filter.floodstop.type = RegexFilter
log4j2.appender.event.filter.floodstop.regex = .*(network|logreader_reader).*
log4j2.appender.event.filter.floodstop.onMatch = DENY
log4j2.appender.event.filter.floodstop.onMisMatch = ACCEPT
1 Like

Does anyone have a clue on how to set RegexFilter for the karaf console?
I’d like to apply the filter on that too, not only to the Event log appender…

did you try to grep stuff? Not sure if you can use a full RegexFilter on the console. Maybe if you modify the log4j2 config for the console settings?

e.g. log:display |grep … or log:tail |grep …

https://karaf.apache.org/manual/latest-2.x/commands/shell-grep.html

1 Like

Thanks for the grep idea… I knew about it, but I didn’t think of it.
log:display | grep -v... works, but log:tail | grep -v ... does not… Weird…
But even if it works, looks kinda tedious to enter the command every time I’m watching the console.

I tried setting the console logger with the RegexFilter, but it’s not working - Console appender not used by default (see log4j2.rootLogger.appenderRefs)

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = STDOUT
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = %d{HH:mm:ss.SSS} [%-5.5p] [%-36.36c] - %m%n

log4j2.appender.console.filter.ignore.type = RegexFilter
log4j2.appender.console.filter.ignore.regex = .*(NAS|SystemStatus).*
log4j2.appender.console.filter.ignore.onMatch = DENY
log4j2.appender.console.filter.ignore.onMisMatch = ACCEPT
1 Like

After changing the file org.ops4j.pax.logging.cfg

What is the fastest way to activate it?
It’s obviously not read automatically like rules and such!?

EDIT: Never mind. openhab DOES reload it obviously automatically.
My cfg file was somehow messed up the first time I tried it…

Does anyone know how to filter openhab.log?

Would you just swap out system for event in the examples above??

Tried that. Doesn’t work.

OpenHAB - Filtering event logs shows for events.log but could be adapted to openhab.log. oops, that’s this thread. I guess I can’t help except to point out alternatives.

Log4j2 Sample Config shows how to shunt all the logs from a certain binding or part of oh to a separate file.

How to watch and look through logging shows both posix (Linux and Mac) as well as powershell ways to tail logs and filter what gets printed.

3 Likes

I thought I had a fix by using NEUTRAL instead of ACCEPT for the first filter, which seemed to work initially but that didn’t work out either. So back to using Marvin_Autonomous’ solution.

Did you ever figure this out for openhab.log ? I can’t seem to get this to work for that log file…

Kind of, what do you want to do?