Archive logs to a separate folder

Tutorials & Examples
, ,
1

There are other posts about separating out logs for specific bindings, etc. (the logger below will do that). This post is to show how to send logs rolled over from an appender to a separate directory. This would be useful, for example, to archive zwave log files in one place for log term storage and analysis, without bloating the OH directory. You’ll be modifying the /userdata/etc/org.ops4j.pax.logging.cfg file (location with depend on installation type).

The .filePattern is what rolls the logs to another directory. I couldn’t find a way to roll them after the strategy.max was reached, so there will only be one log file in the OH directory structure. Make sure strategy.max is removed or commented out so that files aren’t removed from your archive directory. If you’re moving the files to another file system, you will get an error in Karaf (‘Invalid cross-device link’), but the logs are still being moved to the archive directory. Using a symlink did not prevent this. Still investigating.

log4j2.logger.ZWave.name = org.openhab.binding.zwave
log4j2.logger.ZWave.level = DEBUG
log4j2.logger.ZWave.additivity = false
log4j2.logger.ZWave.appenderRefs = ZWave
log4j2.logger.ZWave.appenderRef.ZWave.ref = ZWAVE

log4j2.appender.ZWave.name = ZWAVE
log4j2.appender.ZWave.type = RollingRandomAccessFile
log4j2.appender.ZWave.fileName = /opt/openhab2/userdata/logs/zwave/zwave.log
log4j2.appender.ZWave.filePattern = /opt/openhab2-backup/_zwavelogs/zwave.log.%d{yyyyMMddHHmmss}
log4j2.appender.ZWave.immediateFlush = false
log4j2.appender.ZWave.append = true
log4j2.appender.ZWave.layout.type = PatternLayout
log4j2.appender.ZWave.layout.pattern = %d{yyyy-MM-dd HH:mm:ss.SSS} [%-5.5p] [%-50.50c] - %m%n
log4j2.appender.ZWave.policies.type = Policies
log4j2.appender.ZWave.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.ZWave.policies.size.size = 10MB
log4j2.appender.ZWave.strategy.type = DefaultRolloverStrategy
#log4j2.appender.ZWave.strategy.max = 500

Also note the naming convention I used in the .filePattern. I set this up a while back so that when grepping the archived logs, the results are chronological. The .immediateFlush set to false will hopefully give my SSD a little more life span, but I set it back to true if I want to tail the log file in real-time. Also, most changes to org.ops4j.pax.logging.cfg take affect immediately, but the .filePattern requires an OH restart. I’m sure there are some more goodies to find in here.

2 Likes
2

That’s what I’m finding too, but haven’t found a way to change it, or the source code where this is done.

3

I think I know why a hardlink is used. My guess is that log4j creates a hardlink to the log archive before a new log file is opened for write. That is a relatively inexpensive operation. That way there is no interruption to ongoing logging up until log4j reopens its current log file using something along the lines of freopen().

Have you searched for a call to link(oldpath, newpath) (C function, I don’t have a clue whether or not java provides an equivalent)?

1 Like
4

Rather than trying to modify log4j to work around the hardlink issue, what about creating a cron job that moves log archives to your archive volume, say once a day at 03:00, or whatever interval seems right for your circumstances.

5

That is another solution, but this was easiest! :wink: The files are being copied properly, so I think I can live with the errors for now.

6

Ahhh, so log4j fields the error and moves the file for you, nice!

7

Right, and the errors seem to be from OH or Karaf…

2018-07-21 12:43:16,501 ZWaveReceiveThread ERROR Unable to move file /opt/openhab2/userdata/logs/zwave/zwave.log to /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124316: java.nio.file.AtomicMoveNotSupportedException /opt/openhab2/userdata/logs/zwave/zwave.log -> /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124316: Invalid cross-device link
2018-07-21 12:43:23,399 Thread-99 ERROR Unable to move file /opt/openhab2/userdata/logs/zwave/zwave.log to /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124323: java.nio.file.AtomicMoveNotSupportedException /opt/openhab2/userdata/logs/zwave/zwave.log -> /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124323: Invalid cross-device link
2018-07-21 12:43:38,324 Thread-99 ERROR Unable to move file /opt/openhab2/userdata/logs/zwave/zwave.log to /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124338: java.nio.file.AtomicMoveNotSupportedException /opt/openhab2/userdata/logs/zwave/zwave.log -> /opt/openhab2-backup/archive/_zwavelogs/zwave.log.20180721124338: Invalid cross-device link