Enter your search text in the box above

Select a result to preview

Mxr

#projects #projects/mxr #resources/programming

Mxr

A daemon-backed, terminal-first email client. CLI core, one length-delimited JSON protocol, several transports, multiple frontends. The architectural blueprint that I later applied to Lazydap.

Project lives at: ~/code/planetaryescape/mxr/
Repository: https://github.com/planetaryescape/mxr
Extracted crates: Mxr Extracted Crates
Subprojects: Mxr Web App, Mxr Docs Site

What mxr is, in one paragraph

A single binary mxr with subcommands. Bare mxr enters a TUI if interactive; mxr search, mxr cat, mxr compose, and the other CLI commands connect to the daemon through one length-delimited JSON protocol. Unix sockets are the default. The CLI can also use token-authenticated loopback TCP or a child command such as SSH plus mxr daemon dial-stdio. The daemon owns SQLite (canonical state), Tantivy (search index, rebuildable from SQLite), and provider adapters for Gmail, Outlook, IMAP, SMTP, and the fake demo provider. Every operation flows through one IPC contract: the TUI does not get privileged access, and neither do the agent skill or MCP server. The CLI is the canonical surface; everything else is one of N possible clients.

Philosophy

1. Local-first, daemon-backed, scriptable

Email lives on the daemon host. SQLite is the source of truth. Tantivy can be rebuilt from SQLite if it is lost. No phone-home. No telemetry. The default client path is local over an owner-only Unix socket. Explicit TCP and command transports let the CLI reach that same daemon contract without turning mxr into a hosted service.

This is the Local-First Software principle applied to communication tooling — your email shouldn't disappear when your network does, and you shouldn't have to ask Google for permission to grep your own inbox.

2. CLI-first, scriptable from any language

The CLI is the product. The TUI is one client. The agent skill is another. The MCP server is another. Anything anyone wants to build (Electron app, web dashboard, vim plugin, Slack bot, custom analytics) is just another client speaking the same JSON protocol over an allowed transport. No client gets special access.

This is the porcelain on plumbing pattern: SQLite plus Gmail, Outlook, IMAP, and SMTP are the plumbing; mxr is the porcelain. Any tool consuming mxr is a porcelain on top of mxr's porcelain. Wrappers all the way down. (See Yes It's a Wrapper thinking applied here too.)

The single rule, enforced by Cargo's dependency graph rather than convention: the TUI cannot depend on the daemon-internal crates. It cannot reach past the protocol. If you want a feature, it has to flow through the IPC contract.

3. Provider-agnostic internal model

mxr's internal types (Email Internal Model) are provider-agnostic. Gmail's labels, IMAP's folders, and Microsoft's both-of-them all collapse to mxr's Label enum with LabelKind: System | Folder | User. The provider quirks live in adapter crates (crates/provider-gmail, crates/provider-outlook, crates/provider-imap, crates/provider-smtp); the daemon talks to stable sync and send traits.

This is the adapter pattern applied to email. Same shape lazydap uses for debug adapters. Generalisable: when you wrap N backends, define a common trait, push backend quirks into adapter crates, keep the core clean.

4. JSON output is a product feature

Every command supports --format json. Auto-detected from TTY: if stdout is a terminal, default to table; if piped, default to json. Schemas are stable. Breaking the JSON shape is a major version bump. Documented in docs/blueprint/15-decision-log.md under "Non-negotiables."

This is what makes mxr agent-friendly without being agent-only. An LLM, a shell pipeline, and a web app all consume the same JSON happily.

5. Test with real systems, not mocks

mxr's testing discipline: a FakeProvider exists for fast unit tests, but the canonical integration tests run real IMAP via a Dovecot fixture and real Gmail via OAuth-mocked transport. Mocks of provider behaviour silently mask real-world bugs (label reorderings, IDLE disconnects, OAuth refresh races). Real systems catch them.

6. Destructive and bulk mutations are dry-runnable

Mailbox changes that can affect many messages support --dry-run. The selection logic that previews what would change is the same code path as the real mutation; you cannot have a preview that selects one set and an execution that changes another. Setup and authorization flows are different: mxr accounts reauth opens an OAuth session and has no --dry-run flag.

This becomes Lazydap's rule too. It came from mxr's pain: a bulk-archive bug where preview said "1,200 messages" and execution archived 12,000 because the queries diverged. After that, framework rule: same code, same selection, no exceptions.

Architecture summary

   Agent skill     MCP server        TUI       Web app (SPA)     Custom script
       │              │              │              │                  │
       │              │              │              ▼                  │
       │              │              │         Web bridge (HTTP+WS)    │
       │              │              │              │                  │
       └──────────────┴──────────────┴──────────────┴──────────────────┘
                              │
                              │  mxr protocol
                              │  (length-delimited JSON over Unix, TCP, or cmd)
                              ▼
                         mxr-daemon
                              │
            ┌─────────────────┼─────────────────┐
            ▼                 ▼                 ▼
        SQLite           Tantivy           Provider adapters
        (canonical)      (search,          (Gmail/Outlook/IMAP/SMTP/Fake)
                          rebuildable)

The Mxr Web App SPA is the GUI surface. It speaks HTTP and WebSocket to the web bridge (crates/web/), which translates to the daemon protocol. The SPA dist is embedded in the daemon binary via include_dir!() and served at / - one artifact ships both. Launched via mxr web. See Embedded SPA in Daemon Binary for the distribution pattern.

There is no native desktop client in the repo today. The earlier Electron shell is gone; the GUI story is the embedded web app served by mxr web, installable as a PWA. That keeps the product to one shipped binary and one daemon/client contract instead of adding a second packaging lane around the same local runtime.

Crate dependency rule (enforced by Cargo, not convention):

If a feature wants to break these rules, the architecture is wrong, not the rule. Violations are caught at build time.

Public crates extracted from mxr

Most mxr workspace crates are private implementation boundaries. Four small email contracts have been extracted and are now consumed back from crates.io:

This is not a plan to publish all of mxr. The lesson is narrower: publish bounded contracts when they clear a real ecosystem gap, have their own conformance story, and make mxr's boundary sharper.

IPC contract — four buckets

Every IPC request must classify into one. New buckets require a decision-log entry.

Bucket Purpose Examples
CoreMail Read/write email content Search, Cat, Thread, Compose, Send, Archive, Trash, Label
MxrPlatform Per-account state Accounts::List/Add, Rules::*, SavedSearches::*, Semantic::*
AdminMaintenance Daemon health Status, Logs, Doctor, BugReport, LocalReset
ClientSpecific Pane state, focus, scroll (TUI/web only — daemon doesn't see these)

This bucketing inspired Lazydap's own four-bucket protocol (Session / Project / Diagnostics / ClientSpecific).

Key technical decisions (the load-bearing ones)

Files agents/contributors should read first

Why mxr exists

Cards on the table: I wrote mxr because I wanted the email client I couldn't find. Specifically: one that lives in the terminal, doesn't require Electron, doesn't require an IDE, doesn't ship analytics, lets me grep my own data, lets agents drive it via shell, and treats SQLite as the canonical store.

The closest existing tools — mutt, notmuch, aerc, meli — are good but each has limits I kept hitting. None had agent-native CLI surface. None auto-spawned a daemon for snappy per-command response. None made gh-style JSON output a first-class feature. None imported Gmail labels with proper semantics.

mxr fills that gap, and along the way it's become the architectural pattern I now apply to other domains. Lazydap is "mxr's pattern, but for debugging." Future tools probably will be too.

What mxr is NOT

Connections

Vault clusters this project touches

If you're reading this in a year and want to traverse, mxr connects to these synthesis notes (each opens its own cluster):

The atomic notes inside each cluster cite mxr's specific implementation files as concrete grounding.

Lessons learned (worth carrying to lazydap and beyond)

  1. The crate dependency graph is the architecture. Not the diagram, not the docs — the Cargo.toml files. Once enforced there, violations stop at build time and discussions get short.
  2. Test with real systems first, mocks second. Real IMAP, real Gmail OAuth, real SMTP. Mocks lie. The bugs that bite are the ones mocks paper over.
  3. Stable JSON is a product feature. Once one client depends on a schema, all clients do. Treat the protocol the way you'd treat a public API.
  4. Auto-spawning daemons remove a class of UX bugs. "Daemon not running" is never a user-facing error if the CLI handles it.
  5. $EDITOR for compose is more powerful than custom editors. Markdown with YAML frontmatter is the format; the editor is whatever the user already loves.
  6. Do not promise one-click OAuth until the app is verified. Bundled Gmail credentials are useful as a fallback, but an unverified shared client is not a durable v1 onboarding plan. BYOC can be the honest launch path if the docs make the Google setup boring.
  7. --dry-run and --yes are non-negotiable for mutations. Same code path. No exceptions. See Same-Code-Path Preview.
  8. Stop using new bucket categories. Once you have N buckets in your IPC, every new request must justify why it doesn't fit. Adding a bucket is a decision.
  9. Name the layer that has parity. Search taught this one. Parser parity, AST parity, execution parity, and product parity are different promises. If a local index accepts Gmail syntax but stores only boolean stars, say so. See Parser Parity Is Not Execution Parity.
  10. One registry beats three hardcoded lists. A web app with palette + keymap + help dialog will drift unless all three derive from a single action registry. But the registry is for durable product actions, not every page-local motion key. See Shared Action Registry and Keyboard Parity Is a Product Contract.
  11. Funnel all mutations through one optimistic hook. Per-mutation hooks copy-paste snapshot/restore logic and diverge. One funnel, per-action projection functions. See Optimistic Mutation Funnel.
  12. Embed the SPA in the daemon binary. One artifact ships server and UI; impossible to mismatch versions. See Embedded SPA in Daemon Binary.
  13. A client is not launched until its first run works. The removed desktop shell taught the uncomfortable version: having a working UI shell and bridge is not enough if a clean user lands on an empty mailbox, raw JSON account setup, or an OAuth flow hidden behind a headless daemon. See First Run Is the Launch Surface.
  14. Artifacts are product promises. mxr now ships the CLI/TUI/web binary and Homebrew formula, not a separate desktop artifact. That is healthier than publishing a fresh-looking wrapper whose signing, smoke tests, and first-run flow are not ready. See Release Artifacts Are Product Promises.
  15. Release gates should encode the launch contract. Deterministic build/test failures block release. Missing optional Apple signing secrets warn and ship unsigned macOS archives because that is the current distribution contract. Missing live Gmail test credentials skip the live smoke. Missing both bundled Gmail build credentials can warn because BYOC is the official v1 path; setting only one bundled value is a hard error because it creates a broken artifact. See Release Gates Encode Product Contracts.
  16. Lock the decisions that get relitigated. A "do not relitigate" section in the doc saves every reviewer the same conversation. See Locked Decisions in Docs.
  17. Runtime identity must cover credentials too. Separate data dirs and sockets are not enough. If dev and prod both read mxr/work-imap from the OS keychain, the boundary leaks. See Runtime Identity Is a Data Boundary.
  18. A demo seed is a product fixture, not decoration. The useful demo is the one that exercises real threads, repeat contacts, labels, spam, promotions, attachments, links, images, rules, analytics, semantic search, and LLM summaries through the same daemon paths users hit. See Demo Data Is a Product Fixture.
  19. Prewarm reusable truth, not client screens. The demo warms analytics, Wrapped, LLM-backed surfaces, and active-profile semantic vectors through IPC. It does not bake a web or TUI inbox payload. See Warm Reusable Truth, Not Screens.
  20. Audit the substrate before building AI features. The AI email plan only became safe once contacts, reply pairs, draft intent, safety placement, citation shape, and client surfaces were real in code. See Substrate Audits Before AI Features.
  21. Retry optimistic intent before reconciling it away. If a row disappeared because the user archived it, a transient pool timeout should not bring it back immediately. Keep the optimistic projection during bounded retries; reconcile only after the failure is terminal or exhausted. See Optimistic Retries Preserve User Intent.
  22. Local surfaces are still capability surfaces. A Unix socket, loopback bridge, API docs route, or attachment save path can all drive real authority. Make the boundary explicit in code, not just in deployment assumptions. See Local Capability Surfaces Need Defense in Depth.
  23. Attachment writes are not boring. Filename sanitation, destination allowlists, symlink rejection, file mode, and remote body caps all belong to the same boundary. See Attachment Writes Are a Trust Boundary.
  24. Audit closure needs evidence. Code fixes, tests, CI gates, docs, and tracked advisory exceptions are all different kinds of closure. See Audit Evidence Is Part of the Fix.
  25. Hot reads should stay hot. If sync promised bodies are eager, opening a message should be a SQLite read, not a provider repair attempt hiding behind "Loading...". Keep cache reads and repair paths separate. See Hot Reads Should Not Repair Cold State.
  26. Nonblocking work still needs a place to show up. Moving LLM summaries off the hot path fixed the stall, but the user still needed a visible Summary block above the message body. Background work without an obvious result surface feels broken even when the scheduler is healthy. See Nonblocking Work Needs a Visible Home.
  27. Extract contracts, not folders. A standalone package needs a public promise, not just reusable code. The README, conformance corpus, coverage matrix, and semver policy are part of the product. See Package Extraction Is Trust Building.
  28. Portable fixtures beat translated tests. If Rust and JS packages may exist, the shared JSON corpus is the contract. Tests in each language are just harnesses around that corpus. See Conformance Corpus as Contract.
  29. Keep app policy at the boundary. list-unsubscribe parses standard headers; mxr adds BodyLink. mail-query parses syntax; mxr executes local semantics. That adapter layer is healthier than exporting product-shaped internals. See Public Boundary Before Internal Reuse.
  30. Handle embedded protocol objects in their host workflow first. Calendar invites arrive as email, so the first useful product is mail-derived invite handling: parse, show, search, dry-run, and reply over iMIP. Full calendar sync is a separate platform expansion. See Email Invites Are Mail State Before Calendar State.
  31. Keep always-on agent context as a router. The 2026-05-28 cleanup cut AGENTS.md down to the universal guardrails and moved source-work detail into an mxr-development skill. The general lesson is Always-On Agent Context Is a Router: root context should tell the agent where to load detail, not carry every detail itself.
  32. Agent surfaces need product gates, not just examples. The skill teaches the CLI, and MCP gives typed tools, but both are only first-class because the daemon sees agent and mcp as real origins with profiles, account scope, and send/destructive gates.
  33. Launch proof should exercise public surfaces. The useful proof starts the binary, configures a fake account, syncs, reads, drafts, blocks unsafe agent/MCP actions, and runs a gated mutation path. Unit tests alone cannot prove a launch surface.
  34. Generated docs can drift at metadata level. site/public/openapi.json lagged at 0.5.58 while the repo had moved to 0.5.61. The surface still generated, but the published contract said "older product." Regenerate, then ask why the release PR did not own that derived state.
  35. A local identity registry does not grant remote authority. mxr can reject an unowned From and show the exact alias in a dry-run. Zoho, Gmail, or the SMTP server still decides whether the authenticated account may send as it. See Local Identity Registration Is Not Provider Authorization.
  36. Account scope is not sender identity. The transport account owns authority and provider state; a registered alias chooses the visible From inside that boundary. Validate ownership in the daemon, and use the same resolution path for dry-run and send. See Account scope is not sender identity.
  37. Prompt-capable credentials should not gate daemon startup. Resolve a password when its account connects, keep other accounts and diagnostics alive, and make repair work without the daemon. Disk-first storage is mxr's chosen tradeoff, not a universal answer; failure isolation is the reusable rule. See Credential Prompts Are Side Effects.
  38. Untrusted input cannot grant authority. Mail can contain a request, a fake system message, or a plausible instruction from a known sender. It still cannot change the agent's task, recipients, permissions, or tool access. Keep mail-derived text inside an explicit data boundary and enforce authority outside it. See Untrusted input cannot grant authority.
  39. Rendered mail is not source mail. Body views, snippets, exported markdown, HTML, attachments, and provider payloads are separate evidence surfaces. A no-attachment result can be true while a body renderer is still stale or placeholder-only. See Rendered Mail Is Not Source Mail.
  40. Check the exact version surface before choosing a workflow. Draft editing changed between the July receipt workflow and mxr 0.6.19. The right move now is mxr drafts edit, not creating a replacement draft and deleting the old one. See Version Truth Has Surfaces.
  41. Deadlines bound stalls, not work. A client-side wall-clock timeout on a data-sized operation encodes the author's hardware. Long calls stream progress and the client bails only when nothing observable moves. Issue 179 was exactly this, on the README's front-door command. See Deadlines Bound Stalls, Not Work.
  42. Settle deferred work on the completion edge. Backfills can end on an empty page; anything accumulated across pages must fire on the has_more true-to-false transition, not on a page that carried mail. This bit twice in one wave. See The Last Page Can Be Empty.
  43. Faster code breaks coarse clocks. Second-granularity "changed since" checks went blind the moment the seed became sub-second, and the failure mode was a hang, not an error. See Faster Code Breaks Coarse Clocks.
  44. Bound every event payload. One unbounded Vec in a daemon event crossed the 16 MiB frame cap at demo scale and tore connections down silently. See Every Event Payload Needs a Bound.
  45. Process identity checks need provenance-matched trust defaults. The pid file the daemon wrote is ours unless disproven; a ps scan match is foreign unless proven. Applying the strict check to the wrong strand deleted live daemons' pid files. See Trust Defaults Follow Evidence Provenance.
  46. Mail is historical evidence, not live state. The travel workflow proved that search, source reads, attachment extraction, dry-run, and compose can build a useful evidence pack. The agent must still distinguish a booked state from current provider state and retain conflicts instead of resolving them silently. Expense work adds another boundary: Mxr can collect receipts and invoices, but the bank or card ledger owns posted transactions, and several records can describe one purchase. See Consolidated records need a contradiction ledger, Reconcile economic events, not transaction rows, and Notifications are leads, not source of truth.

Local build contention — 1 October 2026

Queue heavy local commands with agent-work run -- <command> [args...], or
use the dotfiles Cargo proxy. The host uses Pueue's two-job queue with
four-worker defaults, nice priority 10 and a macOS memory-pressure gate.
~/.dotfiles/docs/agent-work.md records setup, cancellation and the limits.

The runner fix removes global Cargo process killing. It is installed locally
and prepared on fix/shared-build-queue; it has not been published to the
remote. The handover is docs/issues/local-build-contention.md. This is a
resource ownership lesson for Parallel Agent Work Needs a Closing Ledger:
a worktree is source isolation, not a separate CPU budget.