Mail that lives in Ceph.
No file system. No database.

ShardPost

The mailbox and processing layer: IMAP, POP3, JMAP, LMTP and ManageSieve on one store, with Sieve during delivery and IMAPSieve on mailbox events. Several identical replicas share the same pools. Connections, parsers and caches live in the process; the durable mail state lives in RADOS, so a client that lands on another replica needs nothing but a new connection.

Background work (maintenance jobs, learning, delayed sending, expiries) runs in the servers or in job runner pods of their own. JMAP push workers share 64 partitions through RADOS leases.

Ceph RADOS

ShardPost writes through librados straight into RADOS objects, with no file system and no database in between. The index pool holds each account's mailboxes, message entries, quota and full-text index as omap and must be replicated. The data pool holds the message blobs, which are written once and never changed, so it may be erasure coded. An optional system pool keeps the small, hot store-wide state (admin store, account directory, queues and locks) on fast OSDs.

Every account is one RADOS namespace in the shared pools; there is no pool per user. Read the storage guide →

Nauthilus

Nauthilus decides every login and resolves an address to the canonical account whose namespace holds the mail. ShardPost reaches it over gRPC AuthService (or its HTTP JSON API) and passes the client address, protocol and TLS state along for Nauthilus' brute-force and policy checks. Bearer tokens are checked by OAuth 2.0 token introspection (RFC 7662).

A short-lived auth cache in front of Nauthilus answers repeated logins, and shardpostctl auth flush empties it on every server at once. For JMapper, Nauthilus is also the OpenID Connect provider. nauthilus.org →

nauthilus-director

In the reference deployment the protocol traffic enters through nauthilus-director, a mail protocol director and transparent proxy. It authenticates each session through Nauthilus, picks a healthy backend by routing facts such as the mail shard, and forwards the connection to ShardPost with the PROXY protocol, so ShardPost sees the real client address. JMAP requests take the same path.

ShardPost accepts PROXY headers only from trusted networks. nauthilus-director on GitHub →

JMapper Mail

A native mail app for iPhone, iPad and Mac from the same author, built for JMAP and OpenID Connect. It registers itself with the identity provider, signs in through the system browser with PKCE and keeps only the issued tokens.

ShardPost serves it over JMAP (RFC 8620, RFC 8621) with a bearer token that Nauthilus verifies, including shared folders, Sieve scripts, quotas and delayed sending, and, on iPhone and iPad, wakes it with Web Push when new mail arrives. jmapper.de →

IMAP, POP3 and ManageSieve clients

Any standards client works: IMAP4rev2 and IMAP4rev1 with CONDSTORE, QRESYNC, IDLE, NOTIFY, COMPRESS=DEFLATE, METADATA, ACL, SORT and THREAD; POP3 for the INBOX; ManageSieve for filter scripts. Logins use a password, PLAIN, XOAUTH2 or OAUTHBEARER, over STARTTLS or implicit TLS.

The protocol reference lists every capability, command and deviation.

Inbound mail

ShardPost does not accept internet SMTP and has no submission listener. An MTA such as Postfix receives and queues the mail and hands it over with LMTP (RFC 2033). ShardPost asks for the canonical recipient, runs the user's Sieve script and stores the message; a storage error is answered with a temporary failure, so the MTA tries again.

For local delivery there is also shardpostctl deliver, with sysexits exit codes.

Web Push

When a JMAP client registers a push subscription, the separate shardpost push worker sends its notifications to the client's push service, encrypted with RFC 8291 and signed with VAPID (RFC 8292). JMapper's push service is a gateway that forwards them to Apple's push notification service. Clients that keep a connection open can use the JMAP EventSource stream instead.

Outbound mail

Mail the server creates (Sieve redirect, vacation and notifications, and JMAP EmailSubmission) goes to a configured SMTP relay, never directly to the internet. JMAP submissions can name the user with Postfix XCLIENT, and can be held back with FUTURERELEASE (RFC 4865) for delayed sending that the client can still cancel.

Observability

Prometheus metrics on /metrics, /healthz and /readyz (which checks the store and the drain state), OpenTelemetry traces over OTLP and structured JSON logs with a session ID and the real client address. An IMAP protocol trace can be switched on for a single account.

Administration

shardpostctl talks to the admin API, an OpenAPI contract, over a Unix socket in the pod or over mutual TLS. Roles such as diagnose, provision, repair, content and export keep metadata apart from message content. It shows accounts and quotas, runs maintenance jobs, traces the IMAP protocol of an account, exports and restores accounts, and imports them from another IMAP server with their UIDs and MODSEQ kept.