LogReader binding

Ok. It might somehow be related to log rotation. But as soon as you get home you can test updated version. I’ve fixed a few bugs here and there and hopefully I’ll get it finished today.

1 Like

The jar in post #1 seems to be from 24th Nov. Did you made an update the last 3 days?

Just few seconds ago I uploaded new version from today. I’ll post a changelog soon.

Ok, I was to fast. Now I got the new one. :slight_smile:

ps. Thank you for that binding. I tried a few days earlier to get this done with exec binding. But I failed. :wink:

1 Like

Updated binding available

You can find the download link in post #1

Changes and new features:

  • Removed configuration option File location. Log file path is now automatically picked up using system properties.
  • Detection and handling of log rotation changed.
  • 3 New channels.
    • lastLine - DateTime of last log line
    • lastWarningLine - Last Warning as String
    • lastErrorLine - Last Error as String

Attention: Check your items. Channel names converted to camelCase.

Hey @binderth, there’s now channels for you to write up a rule to check if logging has stopped.

1 Like

You’re welcome and good to hear that this binding is useful.

One request though. Can you provide some info (platform and OH version) and TRACE logs?

/Miika

I wonder if adding support for journalctl is out of the scope of this addon? I’m using just that instead of files.

Background for this: I have read-only filesystem setup and journalctl serves as volatile log handler.

Best,
Sami

I think nothing is out of the scope. As this binding reads and parses logs why not add more ThingTypes one being journalctl.

I’m not really familiar with journald/journalctl so can you tell me more about your setup? Is all openhab logging directed to same place with journald? Only openhab.log or events too? Everything? :smile:

Journalctl is only used in Linux systems that use the systemd init system and have a service file enabled. In reality, this is only used for people using the apt or yum package in the latest operating systems.

That is true and and if I’m correct journalctl is the tool to view logs made by journald witch are in binary format. I do know something about it since I’m running debian but I’m also curious to see how Sami has it all configured.

Thanks for the quick response!

I have Raspberry Pi 3 with rasbian (Jessie at the moment, which is a bit old now I see). For now I’m using openHAB1, but planning moving on openHAB2 – your binding caught my eye as interesting.

I have configured logging such that events are not logged at all, but everything else basically. The logback configuration is here: https://pastebin.com/rLZ5Yk53 . Would probably have something similar with openHAB2 as well. In addition, I have configured reduced jetty to use STDOUT in logging.

That’s the main thing! As you probably know, stdout goes to systemd journal automatically. Naturally this is just my config… but for me main thing would be just to see the error counts / last error line.

Indeed I’m using journalctl as the command currently to view the logs. On my PC, sudo is required. However, with proper setup sudo does not ask for password, or sudo is not even required (1).

I wonder what would be the right approach, tailing the journalctl? Btw, there are different formats for the log, e.g. --json (one line json), or --json-pretty (same as json but more readable for humans) – might ease getting the actual log message from openHAB. For example,

$ sudo journalctl -u openhab --output=json-pretty
{
        "__CURSOR" : "s=a4db7d1973f448bb9bd923bc54b91281;i=1d4680;b=dfa1d437ea9a44b6b7c4ad5a79a59062;m=a5b379c8f0;t=55f0ca40bea06;x=e8136efa794152b4",
        "__REALTIME_TIMESTAMP" : "1511882780043782",
        "__MONOTONIC_TIMESTAMP" : "711680706800",
        "_BOOT_ID" : "dfa1d437ea9a44b6b7c4ad5a79a59062",
        "_TRANSPORT" : "stdout",
        "PRIORITY" : "6",
        "SYSLOG_FACILITY" : "3",
        "SYSLOG_IDENTIFIER" : "openhab.sh",
        "MESSAGE" : "2017-11-28 17:26:20.025 [INFO ] [.o.c.j.i.engine.TimeTriggerJob] - TimeTrigger for rule: org.openhab.core.jsr223.internal.shared.Rule$$NashornJavaAdapte
        "_PID" : "1037",
        "_UID" : "0",
        "_GID" : "0",
        "_COMM" : "openhab.sh",
        "_EXE" : "/bin/dash",
        "_CMDLINE" : "/bin/sh /usr/share/openhab/bin/openhab.sh -o",
        "_CAP_EFFECTIVE" : "0",
        "_SYSTEMD_CGROUP" : "/system.slice/openhab.service",
        "_SYSTEMD_UNIT" : "openhab.service",
        "_SYSTEMD_SLICE" : "system.slice",
        "_MACHINE_ID" : "2c9b9d45656b45afb58b1f0a4682f91a",
        "_HOSTNAME" : "lerbacka-raspi"
}
{
...
}

(1) Allowing a certain command to called without password using sudo: openhab ALL=(ALL) NOPASSWD: /bin/journalctl -f -u openhab. Or simply allowing user to access (all!) logs without sudo: sudo usermod -a -G systemd-journal openhab

Best,
Sami

1 Like

This looks intresting. Imho, tailing is the best option and as you know now the logreader reads the whole log file once a minute. Reading 100,000+ lines so often just to get those few new lines from the end is pointless so this will be also changed to tailing.

Output in json is a good option. I’ll do some more research. Wondering how it handles multiline stack traces.

I also read about giving user openhab rights to use journalctl. In some OS the group to access journalctl is root so with your usermod command it would ad user openhab to group root. This is imho not the proper way and worth investigating.

1 Like

Good point regarding journalctl group, did not know that. Allowing only certain commands by the user in sudoers file is at least transparent.

Nice to hear that the file reader would tail as well, perhaps there is synergies in implementation and the actual systemd part would be small…

I would expect that new lines are escaped in json?

Also found out this: https://github.com/bwaldvogel/log4j-systemd-journal-appender/blob/master/README.md

See notes about filtering messages with journalctl based on metadata. I can’t come up with practical use case in openhab context though :slight_smile:

Example also shows how java can access the c api but not sure if it is really worth it.

I was thinking… Would it be bad to allow user to give the journalctl command? I guess the default could be “journalctl -b -f --json -u openhab”, but user could even follow some other service / log, or filter messages using journalctl filters. It would be also transparent what access rights are required (sudoers). Food for thought.

Best
Sami

1 Like

tailing is the best option and as you know now the logreader reads the whole log file once a minute.

There are several java tail implementations available, e.g. apache commons io tailer. Another possibility could be to listening Java console rather than files or configure custom log2j appender for monitoring purposes, e.g. JMS appender and binding then listening that queue. Custom appenders could also have different log level (e.g DEBUG) level than console or log files appenders use.

Thanks for the ideas. I’m looking at apaches tailer and it looks intresting.

Hi,

Liking the good work and this has been something Ive been wanting to add for that extra level of monitoring the entire OH2 service.

Ran into an issue this morning as I upgraded the jar to the latest version so that I could include the LastWarning and LastError output lines and now I can see those but the original 5 options “logRotated|lastRead|lastLine|warningLines|errorLines” now all show no data?

Heres my config based on this binding post, I have also restarted OH2 a couple of times but that hasn’t helped, not tried reversion to previous jar but may do that later.

//Log Reader items
DateTime logreaderLogRotated "Last Log Rotation [%1$tY.%1$tm.%1$te %1$tR]"  <time>  { channel="logreader:reader:reader1:logrotated" } 
DateTime logreaderLastRead   "Last Read [%1$tY.%1$tm.%1$te %1$tR]"          <time>  { channel="logreader:reader:reader1:lastread" }
Number   logreaderWarnings   "Warning lines [%d]"                           <alarm> { channel="logreader:reader:reader1:warninglines" }
Number   logreaderErrors     "Error lines [%d]"                             <alarm> { channel="logreader:reader:reader1:errorlines" }
String   logreaderLastWarningLine  "LastWarningLine [%s]"                           { channel="logreader:reader:reader1:lastWarningLine" }
String   logreaderLastErrorLine  "LastErrorLine [%s]"                               { channel="logreader:reader:reader1:lastErrorLine" }
Frame label="Logging" {
				Text item=logreaderLogRotated
				Text item=logreaderLastRead
				Text item=logreaderWarnings
				Text item=logreaderErrors
				Text item=logreaderLastWarningLine
				Text item=logreaderLastErrorLine
			}

logreader.things

logreader:reader:reader1[ refreshrate="60" ]

From habconsole

openhab> bundle:list|grep Log
229 | Active    |  80 | 2.2.0.201711271855     | LogReader Binding

Ive clearly messed up somewhere but cant spot the obvious.

Cheers

Kevin

First post including example items is edited to show current layout. You can copy from there.

This should do the trick.

Hi

Thanks I just re-read one of the earlier posts and it hadn’t clicked what you meant now updated and working fine.

Thanks for the quick response.

Kevin

1 Like

I did, had some time in between :grinning:

It’s working now, thx. the only thing I can’t change is the refresh time, it is stucked to 60 seconds.
This is what I tried:

//logreader:reader:reader1 [ refreshrate="300" ]
logreader:reader:reader1 [ refreshrate=300 ]

You need some camel humps here like Kevin :smile:

Try refreshRate instead of refreshrate.