Welcome to openHAB!
As you are on RPi 4 I recommend to replace your SD card by an USB-SSD and uninstall ZRAM.
I can’t answer the first part but the second part is yes. Otherwise OH might be actively writing while the contents of zram are being dumped and you might end up with corrupted files in the dump.
Unless something has changed in openHABian I’m unaware of, the developer of openHABian actively discourages automatically dumping the zram like this so I wonder if this comes from openHABian in the first place.
The syslog is saying that it’s a systemd cron job. Try looking for a script in /etc/cron.daily or cat /etc/crontab /etc/cron.d/* to see all the jobs scheduled through systemd.
Given that the time isn’t consistent I’m going to guess it’s a script in the cron.daily folder.
Is there a simple migration from SD to USB, or does it require a clean install?
Note, running openHABian on ssd is not supported. It will work for sure but the support you will get from the developers and here on the forum will be limited.
There are two topics which have been discussed in this community: Installing Pi OS on an SSD boot disk and using Pi OS 64bit. Some have still the opinion that it is still advisable to install PiOS 32bit on SD card and activating ZRAM.
As 32bit is the right option for Raspberries with 1GB of RAM, I recommend to install PiOS/bullseye on a SSD and deactivate ZRAM.
The world moved on and more and more users here go for a Pi4 with at least 2 or 4 GB. Then it makes sense to install 64bit PiOS as the first run of a rule takes up to 20 seconds on 32bit and just 2 seconds on 64bit.
Migration is fairly easy. Simply backup oh conf and userdata folder and restore it after a fresh installation (does not apply to apps like mosquitto, persistency databases, etc).
With openhabian it is even easier as you throw the backup zip file into the boot directory and during first-boot it gets automatically restored. Having said this, in the context of openhabian I am not allowed to recommend usage of SSD as boot disk, otherwise this post gets deleted.
1 Like
It’s unclear what you did to your system. Did you reinstall the openhab package only ? Or the OS ?
Syncing ZRAM no longer requires to restart OH so openHABian no longer does this in quite some time. As it obviously does on your system I believe you’re still running an ancient OS setup.
It’s /etc/systemd/system/zsync.service usually calling /usr/local/sbin/zram-config.
You can replace the latter with the latest version from https://github.com/ecdye/zram-config.
Well or you can install everything from scratch on SSD as recommended by some lonely shouters here. But then you will no longer receive help from myself or others on this forum as it’s no longer a supported openHABian setup so if you do and when you’re in need for help then for support please turn to those that recommended doing so.
1 Like
I did a clean reinstall of openhabian OS via RPI imager. From what I see image is from 2023-08-12, so seems recent to me. I did restore openhab configuration which was used on earlier openhabian OS version, but OS was clean.
When I SSH into openhabian, it says:
openHABian
openHAB 4.0.4 - Release Build
I lost power yesterday, and subsequently lost all historic data since I booted the Raspberry Pi.
Do I understand the statement I quoted correctly, that zram is syncing nightly to disk?
I just ran sudo /usr/local/sbin/zram-config sync which caused OH to restart. This implies to me running the command is not advisable when OH is running.
So, is there a way to sync the zram data to disk? Ideally automatically.
[edit1] Reading on Git: When running zram on a directory that has services accessing it, they will need to be stopped before starting or stopping zram. For example, in the log zram device zram-config stops the services that run by default in the /var/log directory before starting or stopping. If your system has other services that write to /var/log that are not stopped zram may fail to properly sync files and remove the zram device when stopping, and will probably outright fail to start when initializing a zram device. This issue is not limited to logs, if you are running zram on another directory that is written to by a service you will run into the same issue.
It seems zram cannot be synced to disk while OH is running.
It can but zram-config doesn’t do by default.
I don’t know if you have the most recent version (of zram) installed, but if so, try this:
SERVICE=1 /usr/local/sbin/zram-config sync
Put it into a cron job or service timer.
1 Like
I’ve just added that to openHABian, please test.
Only in main branch for now.
I didn’t find it after updating openhabian to 1.9(main).
Is it in crontab or a service timer? Or do I have to select something in openhabian-config?
A service timer, it only gets installed on zram install during fresh openHABian installs.
You could uninstall reinstall zram via menu but don’t do on production systems.
1 Like
Hello fellow Openhab users and maintainers.
I have a (fairly) fresh openhabian install:
Version: 4.1.2 (Build)
User: openhab (Active Process 739)
User Groups: openhab tty dialout audio bluetooth gpio
Directories: Folder Name | Path | User:Group
----------- | ---- | ----------
OPENHAB_HOME | /usr/share/openhab | openhab:openhab
OPENHAB_RUNTIME | /usr/share/openhab/runtime | openhab:openhab
OPENHAB_USERDATA | /var/lib/openhab | openhab:openhab
OPENHAB_CONF | /etc/openhab | openhab:openhab
OPENHAB_LOGDIR | /var/log/openhab | openhab:openhabian
OPENHAB_BACKUPS | /var/lib/openhab/backups | openhab:openhab
ZRAM is running its nigthly sync.
I’ve found that openhab.log and syslog freezes at the time the sync is done.
for openhab.log is helps to change some logging setting i.e. set log level from info to warn in the web UI.
Syslog can be restarted by a
killall -HUP rsyslogd
(there is no PID file to use with kill)
killall -HUP openhab
Doesn’t work as openhab is not a know process, the process is Java.
I could find out the PID number via PS (or openhab-cli info), and use that for
kill -HUP [PID]
Can we add some post deploy scripts to /usr/local/sbin/zram-sync
to have syslog and openhab.log reopened after the ZRAM sync?
If any more info is needed please let me know.
Thanks in advance for any help.
Regards André
That is not the best way to restart a service, assuming you are on an Debian based OS.
systemctl restart rsyslog
systemctl restart openhab
You can add these two commands to the same cron job that has the zram sync command.
Hi Rich,
Thanks for your rapid reply.
I’ve mistyped the part that you quoted.
I don’t actualy want to restart the services (with all the included logrotation and start-up actions in openhab).
I just want to get the logging up and running again after the sync.
(I have the latest version of openhabian, the sync runs via the systemd timer).
The rest of openhab keeps on running nice at/after the sync, including rules and persistence.
so I don’t think there is a need to restart the services.
Am I missing some thing here?
Just for reference, seems to be the same as reported here:
I found the same issue with openhab.log, but did not notice syslog. As @mstormi pointed out in the other thread, the openhab.log issue might need a fix in OH core. It’s a bit strange that events.log doesn’t suffer from the same issue, so maybe differences between the handling of the two could be identified and a fix could be created. But - if the same issue exists for syslog, it won’t be enough to “fix” OH.
It doesn’t surprise me that the sync causes problems. A zram sync is a pretty disruptive operation from a file system perspective. I would not be surprised if the sync causes OH to lose the file handle.
Probably just a change to log4j2xml in the openhab-distro repo. openhab.log uses a RollingFile appender but events.log uses a RollingRandomAccessFile appender. The difference is consistent with the observed behavior. A RollingFile appender likely opens the file once and just keeps appending to that open file. However on a sync the open file gets swapped out from under the logger and the logger never knows and continues blindly writing to the old file that is no longer there.
A RollingRandomAccessFile appender, since it allows the file IO to read and write to any part of the file instead of just blindly appending to the end, needs to be a bit more active in monitoring and following the file as it gets swapped out and back during the ZRAM sync.
That’s a theory not based on actually looking at the code. I could be way off base. But the reason matters less than whether changing the appender type for openhab.log to a RollingRandomAccesssFile works or not. And that’s an easy test to perform.
As for syslog/rsyslog, that’s probably going to be a problem best left for the zram community. It’s probably a similar root cause but I don’t think we have the knowledge to address that here.
1 Like
In an other thread a user posted this screenshot ( see first four rows )
Are similar messages also shown in your cases of problems ?
Hi all,
I’v changed the time setting in the .timer just to be sure that is wasn’t the Amanda backup causing the freeze of both log files.
below are the last lines of syslog before it freezes, next morning i wake it up by HUP’ing it.
May 14 02:05:01 domoberry CRON[13933]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
May 14 02:05:01 domoberry systemd[1]: Starting Perform nightly zram sync to persistent storage...
May 14 02:05:01 domoberry zram-sync[13943]: zram-config sync 2024-05-14-02:05:01-CEST
May 14 02:05:01 domoberry systemd[6276]: var-log.mount: Succeeded.
May 14 02:05:01 domoberry systemd[1]: var-log.mount: Succeeded.
To check the openhab logging i’ve made a test rule writing a logInfo line ever minute from 2:00
Below are the last line’s before if freezes
2024-05-14 02:00:01.125 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:00:01.123678815+02:00[Europe/Berlin]
2024-05-14 02:01:00.851 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:01:00.850710519+02:00[Europe/Berlin]
2024-05-14 02:02:00.852 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:02:00.851513460+02:00[Europe/Berlin]
2024-05-14 02:03:00.851 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:03:00.850359501+02:00[Europe/Berlin]
2024-05-14 02:04:00.851 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:04:00.850190971+02:00[Europe/Berlin]
2024-05-14 02:05:00.850 [INFO ] [org.openhab.core.model.script.test ] - de tijd is nu 2024-05-14T02:05:00.849583835+02:00[Europe/Berlin]
And yes indeed events.log continues to be writen as expected.