Design OutlineOne of these days, I'll write up a spec for the mail access protocol of my dreams. I'm planning to call it the "reimagined Mail Access Protocol" (short: reMAP) which is handy because reMail already owns the domain name.
I would aim for a RESTful design with the following properties:
- All communication over HTTP / HTTPS: Pure TCP connections are great, but for transferring large amounts of email, HTTP is the way to go. Problems like security, parallel downloads, persistent connections, caching, compression, download continuation via ranges, and so on have already been solved. There is no reason to solve them again.
- Stateless: There's no reason to introduce state like IMAP does with its selected mailboxes. All you need is a HTTP session cookie used for authentication purposes. Session cookies also allow for things like OAuth. OAuth would let third parties get your permission to access your email without having to give them your username and password.
- JSON and UTF-8: All data that's ever sent to or received from the server would be in JSON format. JSON is much more human-readable than XML. UTF-8 would be the only encoding allowed, since it is able to represent any character in the Unicode standard.
- Conversations as first-order objects: Gmail, the iPhone SMS app, and Facebook's messaging system have shown the value of viewing messages not individually but in the context of conversations. In reMAP, the server would be responsible for grouping together messages. While you could still access individual emails, the first-order unit of data would be a conversation.
- Labels, not folders: Labels are much for flexible than folders. Each conversation should have multiple labels, and the labels would be included when you request the message, rather than having to scan all folders for the message via IMAP.
- Stable and unique IDs: IMAP has a UID for each message, but it changes the moment you move the message into a different folder. An IMAP server can also declare all UIDs to be invalid at any moment throughout the session. No more! reMAP would have stable and unique IDs for all conversations, emails, and attachments.
- The beginning of the end for MIME: Yes, you could still get the MIME representation of each message that is sent. But MIME is a messy and complex beast. Instead of requesting the MIME-encoded message parts, you could just ask the server to give you the message as represented in plain text or HTML. Attachments can be downloaded in separate HTTP calls.
- Push built in: The two prevailing methods for implementing push email are the IMAP IDLE command (not widely available in IMAP servers) and Microsoft's ActiveSync, which requires developers to purchase a license from Microsoft. In reMap, clients could just call an HTTP endpoint on the server which returns as soon as new messages are arrive.
- Full-text index on the server: reMAP servers would need to maintain a full-text index of the contents of all messages. There's no reason clients should be required to download and index everything in order to do an exhaustive full-text search of your email.