Enter your search text in the box above

Select a result to preview

How Email Actually Works

#resources #resources/programming #resources/programming/email

How Email Actually Works

Synthesis. Stitches together the layers between "I press send" and "Bob's client shows the message in his inbox, threaded with the previous reply."

The full lifecycle

   [ Compose ]
   Alice writes a message in her client.
        │
        │  Client constructs RFC 5322 + MIME message.
        │  Headers: From, To, Subject, Date, Message-ID,
        │           In-Reply-To, References (if reply).
        │  Body: text/plain + text/html (multipart/alternative).
        │  Attachments: multipart/mixed wrapper.
        ▼
   [ Submit ]
   Client connects to its outbound mail server (Gmail SMTP, mxr's SMTP provider).
        │
        │  SMTP over TLS (port 587 STARTTLS or 465 implicit).
        │  Auth: OAuth2 XOAUTH2 (modern) or PLAIN/LOGIN (legacy).
        │
        ▼
   [ Outbound MTA ]
   Alice's mail server (Gmail) accepts the message.
        │
        │  Looks up Bob's domain's MX records via DNS.
        │  example.com → mail.example.com (priority 10).
        │
        ▼
   [ MTA-to-MTA relay ]
   Gmail's relay opens SMTP to mail.example.com:25.
        │
        │  STARTTLS upgrade.
        │  HELO, MAIL FROM, RCPT TO, DATA.
        │  Authentication via SPF, DKIM, DMARC checks.
        │
        ▼
   [ Inbound MTA ]
   Bob's mail server accepts and stores the message.
        │
        │  Possibly: spam filter, virus scan, content rewriting.
        │  Stored in Bob's mailbox (Maildir, mbox, Gmail's storage, etc.).
        │
        ▼
   [ Push notification (optional) ]
   Bob's client receives a push notification (IMAP IDLE or Gmail Pub/Sub or APNs).
        │
        ▼
   [ Fetch ]
   Bob's client connects to read mail.
        │
        │  IMAP (port 993): LOGIN, SELECT INBOX, UID FETCH.
        │  OR Gmail API: messages.list, messages.batchGet.
        │  OAuth2 access token in either case.
        │
        ▼
   [ Parse ]
   Bob's client parses the RFC 5322 + MIME message.
        │
        │  mail-parser library handles:
        │    - Header decoding (encoded-word for non-ASCII subjects)
        │    - Multipart traversal
        │    - Content-Transfer-Encoding (base64, quoted-printable)
        │    - Address parsing
        │    - Date parsing (with fallback to delivery time)
        │
        ▼
   [ Threading ]
   Client groups the message with related messages.
        │
        │  Provider-supplied (Gmail's threadId) OR
        │  JWZ algorithm based on Message-ID, In-Reply-To, References.
        │
        ▼
   [ Index + Render ]
   Stored locally (SQLite + Tantivy in mxr's case), rendered in UI.
        │
        ▼
   Bob sees Alice's message in his inbox, threaded under the previous reply.

Per-layer responsibilities

Layer Owns Atomic note
Compose Building the RFC 5322 + MIME message MIME
Submit (client → outbound MTA) SMTP submission, auth SMTP · OAuth for Email
MTA-to-MTA relay DNS lookup, SMTP between servers, deliverability (SPF/DKIM/DMARC) SMTP
Inbound MTA Storing, filtering, indexing (server-side; no atomic note)
Push IMAP IDLE or Pub/Sub or APNs IMAP
Fetch IMAP or REST API to read IMAP · Gmail API
Parse RFC 5322 + MIME parsing MIME
Threading Grouping by Message-ID/References Email Threading
Index + Render Local storage, search, UI Email Internal Model · Mxr

Key insights worth remembering

1. SMTP and IMAP are completely separate protocols

The wire formats, the ports, the auth mechanisms — different. Sending and reading are wholly distinct concerns. mxr models this with two separate traits (MailSendProvider, MailSyncProvider) and supports cross-provider mixes (Gmail API for read, SMTP for send).

2. Gmail-over-IMAP is a leaky abstraction

Gmail's labels don't fit IMAP's folder model. For real Gmail support you need the Gmail API directly. For everything else, IMAP is fine.

3. Threading is reconstructed, not native

Unless the server supplies thread IDs (Gmail does, most don't), the client runs the JWZ algorithm on Message-ID/In-Reply-To/References headers. Thirty-year-old algorithm; still the standard.

4. MIME is what makes email rich

Without MIME, email is plain ASCII. Multipart, base64, quoted-printable, encoded-word — these are what give you HTML email, attachments, multi-language subjects. RFC 2045–2049 (1996), still load-bearing.

5. OAuth replaced passwords

Gmail and Microsoft 365 won't accept passwords for new clients. OAuth2 with the loopback redirect is now standard. Refresh tokens are sensitive; treat them like passwords.

6. The internal model matters more than the wire protocols

Most of the engineering in an email client is figuring out the right provider-agnostic shape. Get the seam right and adding a new provider is mechanical; get it wrong and every UI feature has N branches.

How Mxr embodies this

mxr is one specific implementation of the lifecycle above:

This is a complete, modern email client in a single Rust workspace. The atomic notes above describe each stage.

See also (atomic notes)

Other syntheses (sibling MOCs in the vault)

The other "How X actually works" synthesis notes — each is the entry point to its own cluster:

Further reading