IMAP
IMAP
Internet Message Access Protocol. The protocol email clients use to read mail from a server. Server-stored, so the same mailbox is accessible from multiple clients consistently. Defined in RFC 9051 (current; predecessors 3501, 2060). Dates to 1986.
IMAP is stateful and far more complex than SMTP: folders, flags, search, partial fetching, push notifications, delta sync. Replacing it (with JMAP, etc.) has been an ongoing project for two decades; IMAP keeps winning by being entrenched.
Core concepts
- Mailbox (often called a "folder"): a container of messages. INBOX is mandatory; user-defined mailboxes nested by separator (
/or.per server). - UID (Unique Identifier): a per-mailbox monotonic integer. Never reused unless the mailbox is recreated. The thing you fetch by.
- UIDVALIDITY: a per-mailbox cookie. If it changes, the mailbox was recreated and all UIDs are invalid; client must resync from scratch.
- Flags: per-message mutable state.
\Seen,\Flagged,\Deleted,\Draft,\Answered, plus user-defined keywords. - Special-use folders (RFC 6154): server tells you which folder is INBOX, Sent, Drafts, Trash, Spam, Archive — via
\Inbox,\Sent, etc. attributes.
Wire format
Line-based ASCII (or UTF-8). Tagged commands with asynchronous responses. Each command tagged with a unique short string so responses can be correlated.
C: A001 LOGIN alice "p@ssw0rd"
S: A001 OK LOGIN completed
C: A002 SELECT INBOX
S: * 42 EXISTS
S: * 0 RECENT
S: * OK [UIDVALIDITY 1234567890]
S: * OK [UIDNEXT 43]
S: A002 OK SELECT completed
C: A003 UID FETCH 42 (FLAGS BODY.PEEK[] RFC822.SIZE)
S: * 42 FETCH (UID 42 FLAGS (\Seen) RFC822.SIZE 1234 BODY[] {1234}
S: <1234 bytes of RFC 5322 message>)
S: A003 OK FETCH completed
BODY.PEEK[] fetches the full message without marking it as \Seen. Critical for non-destructive sync.
Delta sync — UIDPLUS, CONDSTORE, QRESYNC
Naive sync ("list every UID") is O(n). IMAP grew extensions for incremental sync:
- UIDPLUS (RFC 4315) — ranges and cursors.
- CONDSTORE (RFC 7162) — message modseqs (mod-sequence numbers). Each message has a per-mailbox modseq that bumps when flags change.
FETCH ... CHANGEDSINCE <modseq>returns only messages with newer modseqs. - QRESYNC (RFC 7162) —
SELECT <mailbox> (QRESYNC (<UIDVALIDITY> <highestmodseq> <known-uids>))returns: changed messages since the modseq, plus a list of vanished UIDs (deletions). One-shot delta sync.
Modern sync engines prefer QRESYNC, fall back to CONDSTORE, fall back to full UID-range sync.
CONDSTORE is only a partial delta story. It can find flag/body changes since a modseq, but it does not by itself return vanished UIDs. A client that falls back from QRESYNC to CONDSTORE still needs a delete-detection path, usually a UID set diff or periodic reconciliation. Otherwise deleted mail lingers locally.
IDLE — push notifications
IDLE (RFC 2177) lets the client say "tell me when something changes." Server holds the connection open and emits untagged responses (* 43 EXISTS, * 1 EXPUNGE) when state changes. The client DONEs out to issue commands, then re-enters IDLE.
Keeps connections open; fragile across NATs and proxies; widely supported.
Folders, labels, and the Gmail-IMAP impedance mismatch
Gmail uses labels, not folders — a message can have multiple labels. IMAP models folders — a message lives in exactly one. Gmail-over-IMAP is a leaky abstraction:
- Each label appears as a folder.
- A message with N labels appears N times across N folders (with the same Message-ID).
- "Move to label X" via IMAP doesn't work the way you'd expect — it removes from the source folder.
- Native Gmail features (priority inbox, smart labels, snooze) aren't visible.
Because of this, Mxr uses Gmail's REST API directly for Gmail accounts, IMAP only for non-Gmail. The internal Email Internal Model reconciles labels-vs-folders with a LabelKind: System | Folder | User enum.
What mxr does
Per crates/provider-imap/src/lib.rs:
- Library:
async-imapoverasync_native_tls. - Folder listing filters out
\All(Gmail's "All Mail" pseudo-folder), defaults to INBOX. - Special-use mapping (RFC 6154):
\Inbox,\Sent,\Drafts, etc. get internalLabelKind::System. - Per-mailbox cursor:
ImapMailboxCursor { uid_next, uid_validity, highest_modseq }. - Sync uses QRESYNC if advertised; falls back to CONDSTORE; falls back to full sync.
- Message fetch query:
(FLAGS BODY.PEEK[] RFC822.SIZE). Full body, not envelope-only. - Auth probe order: PREAUTH (already authenticated session) → LOGIN → ANONYMOUS → unauthenticated LIST probe.
2026-05-31 reassessment: this shape is still directionally right, but the CONDSTORE fallback is the risky edge to keep testing. QRESYNC gives changed messages plus vanished UIDs. CONDSTORE gives changed messages, so mxr needs explicit delete reconciliation before the IMAP adapter can be treated as a boring v1 surface.
2026-08-24 compatibility update: older HCL Domino servers can report the wrong byte count for a BODY[] literal. A strict parser then consumes part of the next response and the connection loses framing. One bad message used to abort a mailbox-wide UID FETCH.
mxr now fetches non-Gmail UID ranges in batches of 100. If a batch returns a malformed response, it drops the corrupted connection and bisects the batch on fresh selected sessions until it finds the bad UID. That UID is skipped; later messages still sync. Each IMAP account also has a max_connections setting for servers with a low session limit. Raw async_imap trace events stay out of mxr logs because they include LOGIN commands and full response buffers. See mxr issue 216.
Common pitfalls
- UIDVALIDITY reset — the mailbox was recreated server-side. Wipe local cache for that mailbox, resync from scratch.
- CONDSTORE without vanished UIDs — do not mistake "changed since modseq" for "complete mailbox delta." Deletions need their own reconciliation.
- Flag changes are bidirectional — marking
\Seenlocally must be pushed to the server; flag changes on the server (other client) must be pulled. - Server-specific quirks — Dovecot vs Cyrus vs Exchange all behave slightly differently. Real-world testing with multiple servers is the only way to find them.
- Connection limits — many servers cap concurrent IMAP connections. IDLE eats one connection per mailbox you're watching.
IMAP vs alternatives
- POP3 — older (1984), simpler. Downloads + deletes by default. Not for "same mail accessed from multiple clients."
- JMAP (RFC 8620) — JSON-RPC over HTTP, designed by Fastmail. Cleaner semantics, not widely adopted outside Fastmail.
- EAS / ActiveSync — Microsoft's proprietary protocol for Exchange. Mobile-friendly. Closed.
- Proprietary REST APIs — Gmail, Microsoft Graph. The escape hatch when IMAP isn't enough.
See also
- SMTP — the send counterpart
- Email Internal Model — how mxr reconciles IMAP-vs-Gmail differences
- Email Threading — runs on top of fetched messages
- How Email Actually Works — synthesis
- Mxr — concrete implementation
- RFC 9051 (IMAP4rev2): https://datatracker.ietf.org/doc/html/rfc9051