Signal Handling

Unix signals are how processes get notified of events from the outside: the user pressed Ctrl-C, the system is shutting down, the parent wants you to reload your config, you tried to write to a closed pipe. A daemon needs to handle signals carefully — they're the standard way to control its lifecycle.

The signals worth knowing

By number and name:

Default action: most signals terminate the process if not handled. SIGKILL and SIGSTOP can never be trapped — the kernel handles them directly so you can always force-kill or pause anything.

What a well-behaved daemon does

Daemon startup:
  - Trap SIGTERM, SIGINT, SIGQUIT  → start graceful shutdown
  - Trap SIGHUP                    → reload config
  - Trap SIGUSR1                   → rotate logs (or whatever)
  - Trap SIGCHLD                   → reap dead children
  - Ignore SIGPIPE                 → handle write errors via return values, not signals
  - Don't trap SIGKILL/SIGSTOP     → impossible

Daemon graceful shutdown:
  1. Stop accepting new connections / requests
  2. Finish or cancel in-flight work (with a deadline)
  3. Persist state to disk
  4. Close sockets, files, child processes
  5. Remove PID file
  6. Exit cleanly (status 0)

The deadline matters. SIGTERM is "please exit cleanly"; init systems typically wait 30 seconds, then send SIGKILL. Your shutdown handler must complete in that window or get killed mid-flight.

Tokio signal handling

In Rust + Tokio:

use tokio::signal::unix::{signal, SignalKind};

async fn shutdown_signal() {
    let mut sigterm = signalterminate()).expect("setup SIGTERM";
    let mut sigint  = signalinterrupt()).expect("setup SIGINT";
    tokio::select! {
        _ = sigterm.recv() => tracing::info!("SIGTERM"),
        _ = sigint.recv()  => tracing::info!("SIGINT"),
    }
}

// In main loop:
tokio::select! {
    _ = main_work() => {}
    _ = shutdown_signal() => {
        graceful_shutdown().await;
    }
}

The tokio::select! over signals + work is the standard pattern. Signals interrupt the select and trigger shutdown.

SIGHUP — the reload signal

SIGHUP's traditional use is "reload config without restarting." Conventions:

• nginx: SIGHUP re-reads config, gracefully replaces workers
• syslogd: SIGHUP reopens log files (after rotation)
• sshd: SIGHUP re-reads sshd_config

If your daemon supports config reload, SIGHUP is the conventional trigger. If it doesn't, treat SIGHUP as another shutdown signal (or ignore it; some daemons do).

SIGCHLD — reaping zombies

When a child process exits, the kernel keeps a "zombie" entry until the parent calls wait() or waitpid() to read the exit status. If the parent doesn't, zombies pile up; eventually the system runs out of PID slots.

A daemon that spawns children (e.g., Mxr spawning a sync worker process, Lazydap spawning codelldb) needs to:

• Trap SIGCHLD
• Loop calling waitpid(-1, NULL, WNOHANG) until it returns 0 (no more zombies)

Or: use Tokio's Child::wait() which handles reaping internally. kill_on_drop(true) plus Child::wait().await is the modern pattern; you rarely write SIGCHLD handlers manually in Rust.

SIGPIPE — the silent killer

When you write to a pipe whose read end was closed, the kernel sends SIGPIPE to the writer. The default action is to terminate. If you write to stdout and the user pipes through head, your process can die mysteriously when head closes the pipe.

Standard fix: ignore SIGPIPE. Your write() call will return EPIPE, which you handle as a normal error.

unsafe {
    libc::signalSIGPIPE, libc::SIG_IGN;
}

Or in tokio, just check write errors normally.

Async-signal-safe — the sharp edge

Signal handlers run in arbitrary contexts, including in the middle of malloc. Calling non-async-signal-safe functions from a handler can deadlock or corrupt state. The list of safe functions is short (man 7 signal-safety).

Modern practice: don't do work in the handler. Set a flag (atomic boolean) and let the main loop notice. Or use signalfd / Tokio's signal stream so signals are delivered as regular events.

What mxr and lazydap do

Both:

• Trap SIGTERM and SIGINT for graceful shutdown.
• Don't currently use SIGHUP for reload (config changes require restart; future enhancement).
• Use Tokio's tokio::signal::unix for async signal delivery — no manual handlers.
• Reap children via Child::wait() rather than SIGCHLD handlers.
• Ignore SIGPIPE implicitly (Rust doesn't kill on it by default in async contexts).

Mxr's graceful shutdown sequence: stop accepting IPC connections → cancel pending sync tasks → flush state to SQLite → close adapters (which kills child processes via kill_on_drop) → remove PID file → exit.

Lazydap similarly: stop accepting IPC → DAP disconnect to active adapters with 1s timeout → kill adapter processes → flush state.toml → exit.

Common pitfalls

• Doing too much in the handler. Stick to setting a flag.
• Forgetting to ignore SIGPIPE in CLI tools. Mysterious "broken pipe" deaths in piped commands.
• Not handling SIGCHLD when spawning many children. Zombies accumulate.
• Not having a deadline on graceful shutdown. init systems will SIGKILL you eventually; better to exit cleanly under your own control.
• Using signal() instead of sigaction() in C — signal() has portability gotchas. Tokio abstracts both away.

See also

• Daemons — what handles signals
• PID Files — what tells you which PID to signal
• How Daemons Work — synthesis
• man 7 signal and man 7 signal-safety