Daemons

A daemon is a long-lived background process. No controlling terminal. Detached from the user's session. Runs forever (or until told to stop), serving requests, doing work, holding state.

The trailing d is convention: cron, sshd, httpd, inetd, udevd, dbus-daemon, mxr-daemon, lazydap-daemon. Pronounced "demon" (the Greek δαίμων, helpful spirit). MIT's CTSS hackers in the 1960s named them; the term predates Unix.

Why daemons exist

Two problems daemons solve:

1. Persistent state across short-lived client invocations. A CLI command runs and exits in milliseconds. If every invocation had to load 100MB of email cache from disk, parse it, do its thing, then exit, it would be unusably slow. A daemon keeps that state in RAM between invocations.
2. Background work. Periodic syncing, listening for incoming connections, watching files for changes — none of these can happen if there's no process running.

A daemon is the answer to "how do I have stateful, always-on behaviour on a system that's organised around short-lived processes."

Properties of a well-behaved daemon

Conventions accumulated over five decades of Unix:

• Detached from the controlling terminal. Closing the terminal that started it doesn't kill it. The classic technique: fork → setsid → fork → chdir("/") → close 0/1/2 → reopen to /dev/null.
• Has a PID file at a known location so other tools can find/control it.
• Logs to a file, not stdout/stderr. Or to syslog. Or to a structured log shipper. Stdout/stderr have nowhere to go after detachment.
• Handles signals gracefully (see Signal Handling): SIGTERM for clean shutdown, SIGHUP for reload, SIGINT for Ctrl-C-style stop, SIGQUIT for crash dump.
• Single-instance per scope (typically). Two sshd instances on port 22 don't make sense.
• Doesn't drift state silently. If it crashes, the next start should rebuild from durable storage.

How daemons get started

1. System-level (init / systemd / launchd)

The OS starts the daemon at boot, supervises it, restarts on crash. systemd unit files, launchd plists, OpenRC scripts. The daemon is "always there."

Examples: sshd, cron, dbus-daemon, nginx in production.

2. User-level (systemd --user, launchctl)

Same idea but per-user, started at login. The daemon runs as the user, not root.

Examples: dbus-daemon per-user, pulseaudio, IDE language servers.

3. Auto-spawned by clients (the mxr/lazydap pattern)

The daemon doesn't run constantly. The first CLI command that needs it forks the daemon, then connects. See The Local Daemon Pattern.

Trade-off: cold-start latency on first invocation (~50ms typical) in exchange for not consuming resources when idle.

4. Manually started

Old-school: user runs daemon & and walks away. Common in development.

What's NOT a daemon

• Background tabs/threads in a UI app — not separate processes; die with the app.
• A cron job — short-lived, exits when done. cron itself is the daemon; the jobs aren't.
• A foreground server (python -m http.server) — has a terminal; isn't detached. Often useful in dev but not a "daemon" properly.

Daemon vs service vs server

The terms blur. Rough Unix distinctions:

• Daemon — Unix-specific term. Implies the detachment ritual (fork/setsid/etc.).
• Service — broader; macOS, Windows, and modern systemd usage. Less prescriptive about how it runs.
• Server — the role: something that handles requests. A daemon may be a server (sshd) or not (cron). A server may be a daemon or not (a CGI script isn't).

In modern usage these are mostly synonymous, but if you say "daemon" Unix people expect the detachment behaviours.

What mxr and lazydap do

Both Mxr and Lazydap use the auto-spawning daemon pattern: the user runs mxr search ... and the binary checks for a running daemon (via socket probe), forks one if absent, drops a PID file, binds a Unix socket, and the CLI client connects.

The daemon owns:

• The expensive in-memory state (sessions, caches, watch handles)
• Background work (sync loops, snooze timers, idle adapters)
• The IPC server (Unix socket, JSON requests)
• Persistence to disk (debounced writes)

Clients (CLI subcommands, TUI, agent skill) are stateless. They open the socket, send a request, read the response, close. The daemon survives between invocations.

This is a deliberate inversion of "everything is a one-shot CLI" — a hybrid that keeps the CLI's UX (composable shell commands) with the daemon's performance (no cold-start cost per command).

Common pitfalls

• Detachment leaks — closing all file descriptors is critical; missing one means the parent shell can't exit cleanly.
• PID file races — two clients trying to spawn the daemon simultaneously. Atomic write + lock file is the fix.
• Socket permissions — Unix socket should be 0700 (owner-only). Otherwise other users on the machine can connect.
• Zombie processes — if the daemon spawns children (codelldb, debugpy), it must reap them on signal/exit. kill_on_drop(true) in tokio handles this.
• Stale state — if the daemon crashes and restarts, it shouldn't trust the file system being in a consistent state. Recover or refuse to start.

See also

• How Daemons Work — synthesis: lifecycle from spawn to shutdown
• The Local Daemon Pattern — the specific architecture mxr/lazydap use
• PID Files — process tracking convention
• Signal Handling — graceful shutdown, reload
• Headless Core + Multiple Clients — the broader pattern daemons enable
• Mxr · Lazydap — concrete implementations