Syslog and Gray Log - working, but would like some refinements

Setup, Configuration and Use Runtime
1

So I played with graylog today, and thoguht it’d be nice to have openhab going in there.
I’ve found a couple of threads and have now managed to get both my events log and openhab log going to graylog. I put in a couple of streams and it can now split them back out again…not bad.

image
image838×140 8.31 KB

So ignoring the times, these are about the same messages, We can tell the ones from the shelly binding as they appear under bundle name and id.
image
image1142×66 5.52 KB

But on script entries, we have the data in the square brackets with the name of the Script, eg the UPSCheckRule that I run.

Does anyone know how I might get that script name to be passed to syslog/graylog?
The logging config:

# Root logger
log4j2.rootLogger.level = WARN
log4j2.rootLogger.appenderRefs = out, osgi, syslog
log4j2.rootLogger.appenderRef.out.ref = LOGFILE
log4j2.rootLogger.appenderRef.osgi.ref = OSGI
log4j2.rootLogger.appenderRef.syslog.ref = SYSLOG

log4j2.appender.syslog.type = Syslog
log4j2.appender.syslog.name = SYSLOG
log4j2.appender.syslog.appName = OpenHAB
log4j2.appender.syslog.port = 514
log4j2.appender.syslog.host = graylog
log4j2.appender.syslog.protocol = UDP
log4J2.appender.syslog.facility=local0
log4j2.appender.syslog.Header=true
log4j2.appender.syslog.format = RFC5424

I was able to switch it to a patterned format, but that seems to bust all the bundle id etc. How does the RFC5424 format know about bundle.id and bundle.name? and would be possible to get a bundle.script or something?

Here’s a raw tcpdump:

5:58:34.010003 IP (tos 0x0, ttl 63, id 46621, offset 0, flags [DF], proto UDP (17), length 351)
    192.168.10.27.53894 > 172.26.0.2.514: SYSLOG, length: 323
        Facility local0 (16), Severity info (6)
        Msg: 1 2021-04-21T15:58:34.007+12:00 melkor OpenHAB 22 - [mdc@18060 bundle.id="149" bundle.name="org.openhab.core.io.monitor" bundle.version="2.5.0"] logreader:reader:openhablog:newWarningEvent triggered 2021-04-21 15:58:33.311 [WARN ] [e.model.script.DP-DAY-CentralHeating] - (3.1) Keeping the heating in the current state

Any way we can fiddle with those mdc values bundle.name etc? and add a new one?

2

I don’t think you can because RCF5424 doesn’t provide a field for that information. The only way to add that to the log statements will be to create your own format and experiment until you can add that information in a way that doesn’t violate RFC5424. Maybe you can somehow make it a part of the MSG.

3

and I assume the bundle.id etc are set in code with the logging statements of openhab

4

The logger name (stuff between the [ ]) is standard Java logging. The “name” of the logger is almost always the name of the Class doing the logging. And the name of a Class consists of a hierarchy of package names. for example the class that generated the second line in your screen grab above’s full name is probably “org.openhab.binding.shelly.internal.handler.ShellyBaseHandler”. This wasn’t an openHAB choice, it’s all but hard coded in all Java logging libraries.

The “class name” for rules (in OH 2) will be “org.eclipse.smarthome.model.script.<name chosen in log statement>”.

You can choose to show it and choose how much to show it in a log statement through the format config in the logger’s config.

Because it’s mainly a Java thing, I’d have no expectation that syslog would know anything about it nor would it know what to do with it as a separate field in a log statement. So if you want that included as part of a syslog log statement you’ll have to figure out a log format that includes it as part of a field that syslog does allow. The RFC5424 format is going to leave it out because it’s not something supported by RCF5424.

5

Hi…after much playing I think I managed to get it…

log4j2.appender.syslog.type = Syslog
log4j2.appender.syslog.name = SYSLOG
log4j2.appender.syslog.appName = OpenHAB
log4j2.appender.syslog.port = 514
log4j2.appender.syslog.host = localhost
log4j2.appender.syslog.protocol = UDP
log4J2.appender.syslog.facility=local0
#log4j2.appender.syslog.newLine = true
#log4j2.appender.syslog.protocol = TCP
#log4j2.appender.syslog.facilityPrinting=false
log4j2.appender.syslog.Header=true
log4j2.appender.syslog.format = RFC5424
log4j2.appender.syslog.mdcId = mdc
log4j2.appender.syslog.layout.type = loggerFields
log4j2.appender.syslog.layout.pairs.type = KeyValuePair
log4j2.appender.syslog.layout.pairs.key=bundle.script
log4j2.appender.syslog.layout.pairs.value=%c{1}

Adding the syslog.mdcId = mdc and below, has meant that it’s added a new field with the name bundle.script and %c{1} is the last part of the script name, normally the binding name…or for the case of a rule, the rule name…

Quite pleased with myself.

image
image1014×231 16.9 KB

2 Likes
6

Here’s an update log4j2.xml config for OH 3+. Instead of sending OH logs to syslog, you can senbd it to Graylog itself.

Create the following Socket appender.

                <!-- Gelf appender -->
                <!-- https://logging.apache.org/log4j/2.x/manual/layouts.html#GELFLayout -->

                <Socket name="GRAYLOG" host="10.10.1.111" port="12201" protocol="tcp" immediateFail="true">
                        <GelfLayout host="argus" compressionType="OFF" includeNullDelimiter="true" includeStacktrace="true">
                                <!-- <KeyValuePair key="additionalField1" value="constant value"/>
                                     <KeyValuePair key="additionalField2" value="${ctx:key}"/> -->
                        </GelfLayout>
                </Socket>

Then add GRAYLOG to each logger you want to forward to GrayLog. For example:

                <!-- Karaf Shell logger -->
                <Logger level="OFF" name="org.apache.karaf.shell.support">
                        <AppenderRef ref="STDOUT"/>
                        <AppenderRef ref="GRAYLOG"/>
                </Logger>

If you prefer UDP see the URL in the comment above the appender.