Long-running daemon does not reflect updated /etc/localtime after timezone change
0 reputation · 31 May 2023, 12:33 UTC
0 reputation · 31 May 2023, 12:33 UTC
Goal: Determine how a long‑running Linux service can detect and apply a new system timezone (e.g., after timedatectl set-timezone) without being restarted, while preserving correct date‑time conversions across all its threads.
Uncertainty: glibc caches the timezone data from the first call to localtime(), mktime() or strftime(). Editing /etc/localtime or changing the TZ environment variable does not update that cache until tzset() is invoked or the process restarts. In a multi‑threaded program, calling tzset() concurrently with timezone‑conversion functions may lead to torn or inconsistent state, and there is no documented portable signal or hook that automatically triggers a cache refresh.
Questions:
tzset() from a signal handler (e.g., SIGUSR1) in a multi‑threaded daemon?35275 reputation · 31 May 2023, 16:41 UTC
Is it safe to call tzset() from a signal handler in a multi‑threaded daemon? No. tzset() is not async‑signal‑safe; invoking it directly from a signal handler can corrupt glibc’s shared timezone data or deadlock a mutex used by other threads.
Does glibc provide a portable way to invalidate its timezone cache? No. The cache is refreshed only by a subsequent call to tzset() (or a process restart). There is no signal or kernel event that automatically triggers a refresh.
Recommended pattern for a safe reload:
SIGUSR1 (or another user‑defined signal) that sets a volatile sig_atomic_t flag.tzset() once, then clear the flag.inotify to watch /etc/localtime for IN_MODIFY or IN_ATTRIB events and trigger the same flag‑driven tzset() call.localtime(), strftime(), or mktime() while tzset() is running. In practice, calling tzset() from the main thread is safe because the glibc cache is protected by a mutex that other threads will block on.Below is a compact code sketch that demonstrates the pattern:
#include <signal.h>
#include <time.h>
#include <stdatomic.h>
volatile sig_atomic_t tz_reload_flag = 0;
void sigusr1_handler(int signo) {
tz_reload_flag = 1; /* set flag only */
}
int main() {
struct sigaction sa = { .sa_handler = sigusr1_handler };
sigaction(SIGUSR1, &sa, NULL);
while (1) {
if (tz_reload_flag) {
tzset(); /* safe here, single thread */
tz_reload_flag = 0;
}
/* normal work: localtime(), strftime(), etc. */
sleep(1); /* placeholder for real work */
}
}
In glibc, tzset() manipulates global data structures protected by a mutex. Signal handlers run asynchronously and may interrupt any thread, including one holding that mutex. If tzset() is invoked while the mutex is locked, the handler will block until the mutex is released. If the handler runs in the same thread that holds the mutex, it will deadlock. Moreover, because signal handlers can be invoked at any time, other threads may concurrently read partially updated timezone data, leading to torn or inconsistent results.
For daemons that cannot rely on a user‑defined signal, inotify provides a reliable way to detect changes to /etc/localtime. A small watcher thread can block on the inotify descriptor, and when an IN_MODIFY event arrives, it sets the same tz_reload_flag and exits. The main thread then performs tzset() safely.
Does your daemon use glibc or another C library such as musl? The behavior of tzset() and the caching mechanism differ between implementations, so the exact pattern may need adjustment if musl or a custom time library is in use.
timedatectl set-timezone.SIGUSR1 to the process: kill -SIGUSR1 <pid>.localtime() calls return times in the new zone.strace -e trace=localtime,strftime,mktime,tzset to confirm that tzset() is only invoked from the main thread.To keep a long‑running daemon in sync with a timezone change, avoid calling tzset() from a signal handler. Instead, use a flag set by a minimal handler or an inotify watcher, and invoke tzset() in a controlled, single‑threaded context. Verify the library used by the daemon before applying this pattern.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.