Inter-Process Communication (IPC)

How separate processes exchange data. Processes have isolated address spaces by default — process A's memory is invisible to process B. IPC is the family of mechanisms that lets them cooperate anyway.

The Unix kernel provides several primitives; modern application protocols (HTTP, gRPC, JSON-RPC) layer on top. Choosing well matters: the wrong IPC mechanism is a performance ceiling, a security hole, or both.

The menu

A process can talk to another process through:

• Pipes (anonymous) — pipe(2) syscall. Unidirectional byte stream between parent and child. The thing behind shell |.
• Named pipes / FIFOs — mkfifo(2). Like pipes but accessible by filesystem path; works between unrelated processes.
• Unix domain sockets — bidirectional, file-system-namespaced or abstract (Linux). The default IPC for modern local daemons. See Unix Domain Sockets.
• TCP / UDP sockets — work over network or loopback. Used when "local" might become "remote." Heavier than Unix sockets.
• Shared memory — shm_open + mmap or System V shmget. Two processes map the same memory region. Fastest IPC; needs explicit synchronisation.
• Message queues — mq_open, System V msgget. Fixed-size message queues in kernel. Rare in modern code.
• Semaphores — synchronisation primitive, not data transport. Used alongside shared memory.
• Signals — covered in Signal Handling. Notification, not data.
• Files — write to disk, other process reads. Crude but works. Lock files use this pattern.
• Memory-mapped files — mmap a file; multiple processes share the mapping. Crosses the line between shared memory and storage.

That's the kernel-level menu. Above this layer, application protocols give shape to the bytes:

• JSON-RPC — request/response over any transport.
• gRPC — request/response with protobuf, typically over HTTP/2.
• HTTP — request/response, header-and-body framed, hugely versatile.
• Custom binary protocols — when performance matters more than tooling.
• D-Bus — session-bus IPC standard on Linux desktop. Method calls, signals, properties.

Choosing the right one

Decision tree:

1. Same machine only?
   • Yes → Unix sockets, named pipes, or shared memory (depending on perf needs)
   • No → TCP (with a real protocol)
2. High throughput / low latency (millions of messages/sec, sub-microsecond)?
   • Yes → shared memory + ring buffer + semaphores
   • No → Unix sockets are plenty fast (~1M msgs/sec)
3. Need request/response semantics?
   • Yes → layer JSON-RPC, gRPC, or HTTP on top
   • No (push only) → raw socket / named pipe is fine
4. Multiple concurrent clients?
   • Yes → daemon listening on a socket, accept loop
   • No → simpler IPC suffices
5. Cross-machine? → see Local IPC vs HTTP
6. Cross-language?
   • Yes → JSON-RPC or gRPC (both have many language bindings); avoid bincode/custom binary
   • No → anything that works for your language

For Mxr and Lazydap: same machine, request/response, multiple concurrent clients, cross-language friendliness wanted (so agents in any language can drive). Answer: Unix sockets + length-prefixed JSON. See Unix Domain Sockets and Length-Prefixed Framing.

Why processes don't share memory by default

Two reasons, both mostly safety:

1. Isolation. A bug in one process can't corrupt another's memory. A security exploit can't read another process's secrets. The OS enforces this via virtual memory; the MMU translates each process's "addresses" into different physical pages.
2. Independent lifetimes. Process A can crash and restart; process B doesn't notice. With shared memory, they're entangled.

IPC mechanisms re-introduce communication while preserving isolation. The trade-off is overhead — every byte exchanged crosses a kernel boundary, gets copied, gets parsed.

What "kernel boundary" costs

A typical Unix socket message:

1. App A writes to socket buffer (one user → kernel copy)
2. Kernel notifies app B (epoll/kqueue event)
3. App B reads from socket buffer (one kernel → user copy)

Two memory copies per message. Each read/write syscall is ~1µs of overhead. For most applications, irrelevant. For high-frequency trading or video transcoding, it matters; that's when shared memory becomes worth the synchronisation pain.

Common application-level protocols

For local daemons with multiple thin clients, JSON-RPC over Unix socket dominates because it's easy to implement, easy to debug (open the socket with socat and read), and works everywhere.

See also

• How Processes Talk to Each Other — synthesis
• Unix Domain Sockets — the dominant local IPC
• Length-Prefixed Framing — common framing technique
• JSON-RPC — common request/response on top
• Local IPC vs HTTP — when to leave the local layer
• TCP vs Unix Sockets vs Named Pipes vs Shared Memory — comparative
• Headless Core + Multiple Clients — what IPC enables architecturally