Starting centralised logfile analysing with ELK stack (elasticsearch, logstash, kibana)

Setup, Configuration and Use Runtime
,
4

I revisited this again and did the following (on OH 4.2.1 running on openHABian / Pi4):

  1. install “LOG4J2 Extra” from the marketplace (LOG4J2 Extra)
  2. setup ELK in docker
  3. configure JSON for logstash
  4. edit log4j2.xml to send the openhab.log to logstash and subsequently to elasticsearch/kibana

ad 1)
simple install:

ad 2)
using the following docker compose:

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.15.0
    container_name: elasticsearch
    environment:
      - xpack.security.enabled=false
      - discovery.type=single-node
    ports:
      - "9200:9200"

  kibana:
    image: docker.elastic.co/kibana/kibana:8.15.0
    container_name: kibana
    ports:
      - "5601:5601"
    depends_on:
      - elasticsearch

  logstash:
    image: docker.elastic.co/logstash/logstash:8.15.0
    container_name: logstash
    volumes:
      - /YOUR-PATH-TO/ELK/logstash/config:/usr/share/logstash/config
    ports:
      - "5000:5000"
    command: logstash -f /usr/share/logstash/config/logstash.conf
    links:
      - elasticsearch
    depends_on:
      - elasticsearch

caveat: this configures ELK without security, if you’re not alone on your local network, please change accordingly, for example:

ad 3)
my logstash.conf

input {
  tcp {
    port => 5000
    codec => json
  }
}
output {
  elasticsearch {
    hosts => ["elasticsearch:9200"]
  }
}

change, if your docker or physical install doesn’t count for the hostname “elasticsearch”

ad 4)
in my environment it’s /var/lib/openhab/etc/log4j2.xml to edit:

  1. <Sockets…> is added als last entry in <Appenders..>
  2. the reference to that is added in <Root level=“WARN” …>
<?xml version="1.0" encoding="UTF-8" standalone="no"?><Configuration monitorInterval="10">
	<Appenders>
...
		<!-- logstash appender -->
		<Socket name="JSON" protocol="tcp" host="192.168.78.20" port="5000">
			<JSONLayout compact="true" complete="false" eventEol="true" objectMessageAsJsonObject="true" />
		</Socket>
...
		<!-- Root logger configuration -->
		<Root level="WARN">
			<AppenderRef ref="LOGFILE"/>
			<AppenderRef ref="OSGI"/>
			<AppenderRef ref="JSON"/> <!-- added this -->
		</Root>
...

you could also send events.log or others, I don’t need the events in kibana visualized, so I only want the “real” logs! :wink:

that’s it. Now openHAB sends the openhab.log entries also to logstash, which then populates elasticsearch with it.

Now I have to find out, how to insert ALERTs or a decent enough monitoring in kibana. Let’s say for “ERRORs” or some “WARNs”.

1 Like