NTP service allways changes to inactive

Yes. I don’t know how you came to use sudo -u, i.e. start as some ordinary user.
In a default openhabian systemd config (/lib/systemd/system/systemd-timesyncd.service), it’s started as root.

Thanks for the hint. I can’t imagine how the user would have been changed. I didn’t know where the configuration was until you told me.

Regardless, I changed User=systemd-timesync to User=root in the config file and started the unit. It appears to be running as expected.

I received a warning that the file had been changed and I should run the daemon reload, which I did.

openhabian@openhab:/lib/systemd/system $ systemctl start systemd-timesyncd
==== AUTHENTICATING FOR org.freedesktop.systemd1.manage-units ===
Authentication is required to start 'systemd-timesyncd.service'.
Authenticating as: ,,, (openhabian)
Password: 
==== AUTHENTICATION COMPLETE ===
Warning: The unit file, source configuration file or drop-ins of systemd-timesyncd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
openhabian@openhab:/lib/systemd/system $ systemctl status systemd-timesyncd
Warning: The unit file, source configuration file or drop-ins of systemd-timesyncd.service changed on disk. Run 'systemctl daemon-reload' to reload units.
● systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
           └─disable-with-time-daemon.conf
   Active: active (running) since Sat 2022-12-03 15:32:09 CST; 5s ago
     Docs: man:systemd-timesyncd.service(8)
 Main PID: 6542 (systemd-timesyn)
   Status: "Synchronized to time server for the first time 204.93.207.11:123 (0.debian.pool.ntp.org)."
    Tasks: 2 (limit: 4915)
   CGroup: /system.slice/systemd-timesyncd.service
           └─6542 /lib/systemd/systemd-timesyncd

Dec 03 15:32:09 openhab systemd[1]: Starting Network Time Synchronization...
Dec 03 15:32:09 openhab systemd[1]: Started Network Time Synchronization.
Dec 03 15:32:09 openhab systemd-timesyncd[6542]: Synchronized to time server for the first time 204.93.207.11:123 (0.debian.pool.ntp.org).
openhabian@openhab:/lib/systemd/system $ systemctl status systemd-timesyncd
openhabian@openhab:/lib/systemd/system $ systemctl daemon-reload
==== AUTHENTICATING FOR org.freedesktop.systemd1.reload-daemon ===
Authentication is required to reload the systemd state.
Authenticating as: ,,, (openhabian)
Password: 
==== AUTHENTICATION COMPLETE ===
openhabian@openhab:/lib/systemd/system $ B

Sorry I overlooked the User= line it’s indeed correct to start as systemd-timesyncd, but you’ll have to find and fix where it fails due to permissions.
Eventually use sudo -u systemd-timesyncd strace/lib/systemd/systemd-timesyncd ....|& grep .... to find out.

1 Like

Thanks for the additional information. I will do as you suggest and report back.

It wasn’t happy with the changed to root:

Dec 03 15:41:23 openhab systemd-timesyncd[7214]: Failed to chmod or chown /var/lib/systemd/timesync/clock, ignoring: Operation not permitted
Dec 03 15:41:23 openhab systemd-timesyncd[7214]: Failed to change group ID: Operation not permitted
Dec 03 15:41:23 openhab systemd-timesyncd[7214]: Failed to drop privileges: Operation not permitted
Dec 03 15:41:23 openhab systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 15:41:23 openhab systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Dec 03 15:41:23 openhab systemd[1]: Failed to start Network Time Synchronization.
Dec 03 15:41:23 openhab systemd[1]: systemd-timesyncd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
Dec 03 15:41:23 openhab systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 1.
Dec 03 15:41:23 openhab systemd[1]: Stopped Network Time Synchronization.
Dec 03 15:41:23 openhab systemd[1]: Starting Network Time Synchronization...
Dec 03 15:41:24 openhab systemd-timesyncd[7217]: Failed to chmod or chown /var/lib/systemd/timesync/clock, ignoring: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7217]: Failed to change group ID: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7217]: Failed to drop privileges: Operation not permitted
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Dec 03 15:41:24 openhab systemd[1]: Failed to start Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 2.
Dec 03 15:41:24 openhab systemd[1]: Stopped Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: Starting Network Time Synchronization...
Dec 03 15:41:24 openhab systemd-timesyncd[7220]: Failed to chmod or chown /var/lib/systemd/timesync/clock, ignoring: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7220]: Failed to change group ID: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7220]: Failed to drop privileges: Operation not permitted
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Dec 03 15:41:24 openhab systemd[1]: Failed to start Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 3.
Dec 03 15:41:24 openhab systemd[1]: Stopped Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: Starting Network Time Synchronization...
Dec 03 15:41:24 openhab systemd-timesyncd[7223]: Failed to chmod or chown /var/lib/systemd/timesync/clock, ignoring: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7223]: Failed to change group ID: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7223]: Failed to drop privileges: Operation not permitted
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
Dec 03 15:41:24 openhab systemd[1]: Failed to start Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Service has no hold-off time (RestartSec=0), scheduling restart.
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Scheduled restart job, restart counter is at 4.
Dec 03 15:41:24 openhab systemd[1]: Stopped Network Time Synchronization.
Dec 03 15:41:24 openhab systemd[1]: Starting Network Time Synchronization...
Dec 03 15:41:24 openhab systemd-timesyncd[7226]: Failed to chmod or chown /var/lib/systemd/timesync/clock, ignoring: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7226]: Failed to change group ID: Operation not permitted
Dec 03 15:41:24 openhab systemd-timesyncd[7226]: Failed to drop privileges: Operation not permitted
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Main process exited, code=exited, status=1/FAILURE
Dec 03 15:41:24 openhab systemd[1]: systemd-timesyncd.service: Failed with result 'exit-code'.
lines 158-230
openhabian@openhab:/lib/systemd/system $ sudo -u systemd-timesync strace /lib/systemd/system/systemd-timesyncd.service
execve("/lib/systemd/system/systemd-timesyncd.service", ["/lib/systemd/system/systemd-time"...], 0xbedcb730 /* 17 vars */) = -1 EACCES (Permission denied)
strace: exec: Permission denied
+++ exited with 1 +++
openhabian@openhab:/lib/systemd/system $ 

Hopefully this helps

What are the permissions of

here it is

-rw-r--r-- 1 root root 1433 Aug  6  2021 /lib/systemd/system/systemd-timesyncd.service

Thanks for the help.

openhabian@openhab:~ $ ls -l /lib/systemd/system/systemd-timesyncd.service
-rw-r--r-- 1 root root 1433 Dec  3 16:07 /lib/systemd/system/systemd-timesyncd.service
openhabian@openhab:~ $ 

To my untrained eye, it looks the same as yours.

1 Like

Openhabian on Pi 4 4GB openHAB 3.4.0 Build #3029

Hi @BigGeorgeTx just found this thread as I am having the exact same issue.

In my case I have found NTP runs fine for awhile, then stops a few days later:

sudo systemctl status systemd-timesyncd.service
[sudo] password for openhabian: 
● systemd-timesyncd.service - Network Time Synchronization
     Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
     Active: failed (Result: start-limit-hit) since Tue 2022-12-13 03:23:15 PST; 1 day 13h ago
       Docs: man:systemd-timesyncd.service(8)
    Process: 18118 ExecStart=/lib/systemd/systemd-timesyncd (code=exited, status=0/SUCCESS)
   Main PID: 18118 (code=exited, status=0/SUCCESS)
     Status: "Idle."
        CPU: 438ms

Dec 13 03:23:15 openhabian systemd[1]: Starting Network Time Synchronization...
Dec 13 03:23:15 openhabian systemd[1]: Started Network Time Synchronization.
Dec 13 03:23:15 openhabian systemd[1]: Stopping Network Time Synchronization...
Dec 13 03:23:15 openhabian systemd[1]: systemd-timesyncd.service: Succeeded.
Dec 13 03:23:15 openhabian systemd[1]: Stopped Network Time Synchronization.
Dec 13 03:23:15 openhabian systemd[1]: systemd-timesyncd.service: Start request repeated too quickly.
Dec 13 03:23:15 openhabian systemd[1]: systemd-timesyncd.service: Failed with result 'start-limit-hit'.
Dec 13 03:23:15 openhabian systemd[1]: Failed to start Network Time Synchronization.

Then I restart the service and it comes back …

openhabian@openhabian:~ $ sudo systemctl start systemd-timesyncd.service
openhabian@openhabian:~ $ sudo systemctl status systemd-timesyncd.service
● systemd-timesyncd.service - Network Time Synchronization
     Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
     Active: active (running) since Wed 2022-12-14 17:07:26 PST; 5s left
       Docs: man:systemd-timesyncd.service(8)
   Main PID: 9336 (systemd-timesyn)
     Status: "Initial synchronization to time server 38.229.60.9:123 (0.debian.pool.ntp.org)."
      Tasks: 2 (limit: 4915)
        CPU: 433ms
     CGroup: /system.slice/systemd-timesyncd.service
             └─9336 /lib/systemd/systemd-timesyncd

Dec 14 17:07:17 openhabian systemd-timesyncd[9336]: Initial synchronization to time server 38.229.60.9:123 (0.debian.pool.ntp.org).

Much of my system has the same output as from the steps you did with Markus.

The next time it fails I’ll gather similar info as you did to compare.

Not sure when this problem started happening, I haven’t rebooted my raspberry for 80 days, although I have restarted Openhab 3-4 times in that period. I often use the openhabian-config tool and it updates various things (not sure what it updates but stuff from Markus) … I assume those are the config tool updates but maybe system updates too?

What command are you running here so I can replicate it?

Glad to have some company trying to troubleshoot this issue.

The information you are referring to comes from the openHAB UI, not from a command on the openhabian Pi. You can find it at “Help & About” / “Technical Information” / “View Details” / “Copy”.

Ah interesting … never noticed that :slight_smile: thanks!

runtimeInfo:
  version: 3.4.0
  buildString: "Build #3029"
locale: en-US
systemInfo:
  configFolder: /etc/openhab
  userdataFolder: /var/lib/openhab
  logFolder: /var/log/openhab
  javaVersion: 11.0.16
  javaVendor: Raspbian
  osName: Linux
  osVersion: 5.15.32-v7l+
  osArchitecture: arm
  availableProcessors: 4
  freeMemory: 310686536
  totalMemory: 1021313024
  startLevel: 70
bindings:
  - androiddebugbridge
  - astro
  - chromecast
  - harmonyhub
  - http
  - hue
  - network
  - ntp
  - telegram
  - tplinksmarthome
  - zigbee
  - zwave
clientInfo:
  device:
    ios: false
    android: false
    androidChrome: false
    desktop: true
    iphone: false
    ipod: false
    ipad: false
    edge: false
    ie: false
    firefox: false
    macos: true
    windows: false
    cordova: false
    phonegap: false
    electron: false
    nwjs: false
    webView: false
    webview: false
    standalone: false
    os: macos
    pixelRatio: 2
    prefersColorScheme: light
  isSecureContext: false
  locationbarVisible: true
  menubarVisible: true
  navigator:
    cookieEnabled: true
    deviceMemory: N/A
    hardwareConcurrency: 12
    language: en-US
    languages:
      - en-US
      - en
    onLine: true
    platform: MacIntel
  screen:
    width: 2560
    height: 1440
    colorDepth: 30
  support:
    touch: false
    pointerEvents: true
    observer: true
    passiveListener: true
    gestures: false
    intersectionObserver: true
  themeOptions:
    dark: light
    filled: true
    pageTransitionAnimation: default
    bars: filled
    homeNavbar: default
    homeBackground: default
    expandableCardAnimation: default
  userAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36
    (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36
timestamp: 2022-12-15T03:33:56.763Z

The problem happened again about 12 hours after I restarted it … same symptoms and status.

sudo systemctl status systemd-timesyncd.service
[sudo] password for openhabian: 
● systemd-timesyncd.service - Network Time Synchronization
     Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; vendor preset: enabled)
     Active: failed (Result: start-limit-hit) since Thu 2022-12-15 03:09:45 PST; 1 day 10h ago
       Docs: man:systemd-timesyncd.service(8)
    Process: 13081 ExecStart=/lib/systemd/systemd-timesyncd (code=exited, status=0/SUCCESS)
   Main PID: 13081 (code=exited, status=0/SUCCESS)
     Status: "Idle."
        CPU: 436ms

Dec 15 03:09:44 openhabian systemd[1]: Starting Network Time Synchronization...
Dec 15 03:09:45 openhabian systemd[1]: Started Network Time Synchronization.
Dec 15 03:09:45 openhabian systemd[1]: Stopping Network Time Synchronization...
Dec 15 03:09:45 openhabian systemd[1]: systemd-timesyncd.service: Succeeded.
Dec 15 03:09:45 openhabian systemd[1]: Stopped Network Time Synchronization.
Dec 15 03:09:45 openhabian systemd[1]: systemd-timesyncd.service: Start request repeated too quickly.
Dec 15 03:09:45 openhabian systemd[1]: systemd-timesyncd.service: Failed with result 'start-limit-hit'.
Dec 15 03:09:45 openhabian systemd[1]: Failed to start Network Time Synchronization.

Not sure what information maybe needed … seems to be exactly the same as what George is reporting. interesting that it is failing at roughly the same time (~3am) … probably just a coincidence.

Still having this problem and I have no idea what todo next to troubleshoot.
@Mr-JR does logrotate also fail for you.


openhabian@openhab:~ $ systemctl -a --failed
  UNIT                      LOAD   ACTIVE SUB    DESCRIPTION                 
● logrotate.service         loaded failed failed Rotate log files            
● systemd-timesyncd.service loaded failed failed Network Time Synchronization

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.

2 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.
openhabian@openhab:~ $

Happens all the time for me, sometime after systems-time synced fails.


openhabian@openhab:~ $ systemctl status logrotate.service
● logrotate.service - Rotate log files
   Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2022-12-20 00:00:01 CST; 21h ago
     Docs: man:logrotate(8)
           man:logrotate.conf(5)
  Process: 26596 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
 Main PID: 26596 (code=exited, status=1/FAILURE)

Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.
openhabian@openhab:~ $ 

Yes, it is also failing. I had just restarted the timesync before running this command.

FYI - I noticed I had installed the NTP Binding. I do not use this and I dont believe it is needed. My NTP was always fine until recently, its possible it happened after installing the NTP binding - not sure. Anyway, I just removed the NTP Binding and will watch to see if NTP still fails.

openhabian@openhabian:~ $ systemctl -a --failed
  UNIT               LOAD   ACTIVE SUB    DESCRIPTION
● ifup@wlan0.service loaded failed failed ifup for wlan0
● logrotate.service  loaded failed failed Rotate log files

LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
2 loaded units listed.
To show all installed unit files use 'systemctl list-unit-files'.

Here is the status of logrotate … not sure what the effect of this crashing is.

openhabian@openhabian:~ $ sudo systemctl status logrotate.service
● logrotate.service - Rotate log files
     Loaded: loaded (/lib/systemd/system/logrotate.service; static)
     Active: failed (Result: exit-code) since Tue 2022-12-20 00:00:03 PST; 19h ago
TriggeredBy: ● logrotate.timer
       Docs: man:logrotate(8)
             man:logrotate.conf(5)
    Process: 12208 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
   Main PID: 12208 (code=exited, status=1/FAILURE)
        CPU: 375ms

Dec 20 00:00:03 openhabian systemd[1]: Starting Rotate log files...
Dec 20 00:00:03 openhabian logrotate[12208]: error: skipping "/var/log/exim4/mainlog" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group shoul>
Dec 20 00:00:03 openhabian logrotate[12208]: error: skipping "/var/log/exim4/rejectlog" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group sho>
Dec 20 00:00:03 openhabian logrotate[12208]: error: skipping "/var/log/exim4/paniclog" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group shou>
Dec 20 00:00:03 openhabian systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Dec 20 00:00:03 openhabian systemd[1]: logrotate.service: Failed with result 'exit-code'.
Dec 20 00:00:03 openhabian systemd[1]: Failed to start Rotate log files.
openhabian@openhabian:~ $ 

Did removing the NTP binding make a difference?

I have it on my system. I have a vague recollection of having used it for a rule when using OpenHAB. I need to search my rules and see if I’m still using it. (I do that by using the REST API to generate output of all the rules, copying that to a text editor and then searching for an item name. There is probably a better way, but it gets the job done).

Hi George, sorry forgot to follow up … no it didn’t make any difference.

I think what I’m going to do is restart the service manually nightly. Doesn’t seem there is much interest out there to solve this :slight_smile:

Doesn’t seem you get the point here… it’s not a systemic software error, it’s your system to have the problem, others don’t have it so cannot analyze/resolve it. It’s up to you to solve it.
And even if it was systemic, there’s no vendor or other anonymous magic that would need to do that for you. openHAB is a community effort.

I have spent hours searching on various websites, even posting on some, trying to address this problem. I’m not asking someone else to solve it, but I would appreciate it if someone could suggest some areas to explore in trying to troubleshoot it. I’m willing to put in the effort, but openHAB/openHABian is my first and only Linux adventure, and I have run out of ideas.

My two setups are both running on Pi 4Bs, on different networks 3,000 km apart. I started with the openhabian distribution on the official Raspberry Pi site, and the only things running on the Pi are openHAB and related applications loaded with openhabian (such as Tailscale).

Any suggestions would be greatly appreciated.

1 Like

Sorry - I didn’t make my point clear. I was not directing this comment to the Openhab development team as I do understand that it is not an Openhab issue. I meant to imply that there aren’t others beside George and myself who are seeing this and that there isn’t much interest to offer more ideas of how we might narrow down the issue.

In my case, this issue came up in the last few months as it was never an issue before. I have been running the exact same Raspberry Pi image other that what gets updated when I use the Openhab Config tool.

Still trying to troubleshoot this. Using the command suggested with my own guess for what to grep for, I get:

openhabian@openhab:~ $ sudo -u systemd-timesync strace /lib/systemd/systemd-timesyncd |& grep clock_adjtime
[sudo] password for openhabian: 
clock_adjtime(CLOCK_REALTIME, 0xbeee92a0) = -1 EPERM (Operation not permitted)
sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="PRIORITY=3\nSYSLOG_FACILITY=9\nCOD"..., iov_len=160}, {iov_base="MESSAGE=", iov_len=8}, {iov_base="Failed to call clock_adjtime(): "..., iov_len=55}, {iov_base="\n", iov_len=1}], msg_iovlen=4, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 224
clock_adjtime(CLOCK_REALTIME, 0xbeee92a0) = -1 EPERM (Operation not permitted)
sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="PRIORITY=3\nSYSLOG_FACILITY=9\nCOD"..., iov_len=160}, {iov_base="MESSAGE=", iov_len=8}, {iov_base="Failed to call clock_adjtime(): "..., iov_len=55}, {iov_base="\n", iov_len=1}], msg_iovlen=4, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 224
clock_adjtime(CLOCK_REALTIME, 0xbeee92a0) = -1 EPERM (Operation not permitted)
sendmsg(3, {msg_name=NULL, msg_namelen=0, msg_iov=[{iov_base="PRIORITY=3\nSYSLOG_FACILITY=9\nCOD"..., iov_len=160}, {iov_base="MESSAGE=", iov_len=8}, {iov_base="Failed to call clock_adjtime(): "..., iov_len=55}, {iov_base="\n", iov_len=1}], msg_iovlen=4, msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 224

Searching the internet, I found:
EPERM
buf.modes is neither 0 nor ADJ_OFFSET_SS_READ, and the caller does not have sufficient privilege. Under Linux, the CAP_SYS_TIME capability is required.
on clock_adjtime(2) — Arch manual pages, which I think confirms that it is a permission issue. But I’m not sure exactly what it is that needs its permission changed.