Tuesday, July 29, 2008

Five Open Questions in Email

The email space is heating up after a long cool-off: The modern email desktop application was invented 15 years ago, and webmail is now 12 years old. With few exceptions, little about how users interact with email has changed since then. Yet, email clients are under attack from all sides: People under 25 use Facebook, not email. Overloaded office workers hate the email flood so much that companies are introducing email-free Fridays.

Email is being pulled in all sorts of directions. Here are places in which there will be lots of movement in the next years. They're ranked them by how much they'll affect you as a user.

  1. Email vs. Facebook: Until recently, all my electronic communication took place in email. Now it's moving away, being replaced by more casual media, like IM, your Facebook wall, Twitter, or SMS. In the future, email could be where snail mail is today: A medium to receive advertising and official stuff.

    But there's an alternate world in which your email client could once again becomes your hub, by pulling in data from all these sources and displaying it in a sensible fashion.

  2. Folders vs. Tasks vs. People: 10 years from now, will you still be organizing your email in folders that you manually create? Or will your email be automatically organized around tasks, or even better, around people who are important to you and their profiles? It's clear that the "move to folders, sort by arrival time" system is broken - what will step into its place?

  3. Is search good enough? With the arrival of fast full-text indexes, email search sucks a little bit less than it used to. But there's still no PageRank-type magic to help you find those emails you're looking for. Hopefully, there will be smarter email search that learns from my search patterns (e.g. you might search twice a week for the same email that contains a password you can't remember) or understands the meaning behind your queries.

  4. Does email need Artificial Intelligence? In the AI / machine learning / natural language community, there's been tons of research on making email smarter. We've looked into automatically summarizing email, predicting whether it needs a reply, generating a reply automatically, and clustering or automatically foldering emails to your heart's desire. Most of this stuff already works reasonably well, but no one has bothered to implement it in a consumer product. AI-based spam filtering is the norm today. Five years from now, are we going to be using intelligent email agents? Or are consumers not going to trust these helpers?

  5. Little guys vs. the big guys: Email is hot again. Companies like Xobni, Xoopit, Zenbe, or Orgoo are being started, some with serious funding. On the other hand, established players like Microsoft, Google, and Yahoo are launching new features and opening up access to their data. Big players have traditionally been making slow incremental progress. It will be interesting to see if they can evolve their products more quickly and bring them to users faster than the little guys ever could.

This entry was inspired by discussions at the AAAI Enhanced Messaging Workshop in Chicago.


Jav said...

Re #1: Those <25 kids will all be using email once they get a job. :-)

F said...

essentially, facebook ist just email with a bad UI. But I still don't understand why people prefer badly programmed webmail to a dedicated client even at home. (I think, it's probably due to the difficulty of setting it up and due to the fact that until shortly IMAP was seldomly found in freemailers. Also, the IMAP support in email clients is atrocious)
I just don't understand why people would want to check half a dozen webpages just to get all their messages. Especially if it's technically trivial to just deliver them to your inbox.

On AI in email: I would probably never completely trust such a system. I would never let it answer my emails without a check from me and I'd fear an important message would get lost. It might be useful for prioritizing messages though

Patrick said...

Regarding machine learning in an email client. I think there exist at least two opportunities for machine learning in the email market (you mention both):

1. ranking emails according their relevance to a given search query.

2. order/archive/show related emails/tags etc. These methods would be some kind of clustering/tagging of your whole email database and would basically need to "understand" the messages (whatever that means). There are obviously interesting techniques for this, e.g. pLSI, Latent Dirichlet Allocation, n-gram models, whole PCFGs etc.

These two fields in my opinion have to overcome two quite different problems to be useful for the mailer-world.

1. Ranking is still a very active research topic, but there have been few proposals that are practical on a large-scale. No doubt that we will see some nifty new methods in this area in the near future. However, once such a highly efficient ranking algorithm exists, it should be straightforward to integrate it in an email program.

2. I think the biggest problem with the clustering/tagging/understanding-the-email-by-NLP techniques is not so much in the technical nature (some of the methods that I mentioned above are pretty efficient), but I think nobody has really demonstrated it's usefulness in a mail client. Sure one can use it for improving the ranking in 1.) or integrate them in the search (this is definitely useful); but I think the real promise could be that it allows you for a new way to browse through your emails. This browsing should however allow for the machine learning algorithms to make errors, which it will, there's no doubt about this (see f's comment).

Definitely two very interesting problems...

Brent said...

To build on Patrick's problem #2 I think one of the most effective uses of clustering may be to understand (read: organize) a conversation. Viewing a thread as a conversation seems a bit dated. Just tonight I had to sort through half a dozen threads to freshen up on the context of a single discussion with a single person. Along those lines... allowing a third party to get a summary of that conversation (~10 threads) would be a nightmare. These seem like low hanging fruit.

This also may or may not tie nicely in with the second para of Gabor's 1st idea. How do I organize a single conversation I've had with someone over several different mediums in one location.

ff0066 said...

I'm under 25 and I hate the Facebook Inbox, Wall, and fbIM. Normal AIM, MSN, Yahoo or even IRC are a hassle too. It's a task just to check whether somebody is trying to get in contact with me because I have to go to 10+ sources. This takes time.

I give people my email and ask them not to use Facebook, but I still have to keep Facebook and IM around for the people that don't write emails yet.

The annoying web 2.0 fad is the heart of the problem now. Once people realize what a hassle it is to have so many separate web identities there will start to be solutions.

As for email inundation at the workplace, I hope you guys fix it before I get there. It's a broken paradigm.

Tony said...

I thought of one AI-related feature that would really help me - suggested deletions. If a message came up saying: "Would you like to delete all messages sent to you before 2004 with the term 'Pikesville High School' included in the body? We noticed you have not referenced these in 3 years, and they have no attachments." THAT WOULD BE AWESOME. It would be an easy way to keep your inbox cleaner, which could also contribute to better search.

In response to "f", I think you are right on in saying that it is not a simple task for tech-tards to set up a desktop email client to read from all of their email sources. Trusting that one program may be tough for some people as well.

Emily said...

Re 1: It's been interesting to watch the effect that Facebook and MySpace messaging have had on email usage, especially for that <25 set, and it will be exciting to see social networking sites continue to challenge mail providers.

We built Fuser based on a recognition of the potential conflict between email and social network management. While social networks are continuing to grow, most people are reluctant to abandon email altogether for the trend. With Fuser we are trying alleviate the pain of trying to manage all of your separate Email, Facebook, MySpace and Twitter accounts by merging your accounts into one inbox.

This space is growing quickly and changing rapidly, and we're excited to be a part of it.

Tony said...

Fuser looks pretty cool...Have you considered putting up a demo video? I know I would like to see one before registering.

Uldis Bojars said...

Brent says: "How do I organize a single conversation I've had with someone over several different mediums in one location."

Answer to this can have 2 parts:

1) there can tools or services that provide a solution for keeping track of conversations over different mediums and sites (such as comment tracking services). but there services can also become a "walled garden" that locks your content in the same was as Facebook does.

2) what can help avoid this is data formats and best practices for publishing data in an interoperable way. think of this as basic data infrastructure that helps you connect conversation from all over the web.

A good solution would probably combine both. Take a look at there for some ideas:

XTech 2008: Data Portability with SIOC and FOAF

Social Network and Data Portability using Semantic Web Technologies [PDF]

Uldis Bojars said...

correction in part 1): but these services can also become a "walled garden" that locks your content in the same way as Facebook does.

Felipe Costa said...


thank you for bring these 5 very interesting questions to the blogsphere.

During the next 5 months I will be doing my master's final thesis on e-mail interface and conceptualization, so I will work over this question, hoping to bring some contribution to all community.

I will be following your blog and will be glad to read more on this topic...

Felipe Avila da Costa

Anonymous said...


I may be a spoilsport, etc., but I have some very simple observations. Please do prove me wrong - it's not like I like what I'm writing here. It's just the way reality presents itself to me.

about #1:
Like many other people have commented already: The main feature of email is that it is universal. Got Twitter? - you must have email (to register). And so on. So it is safe to assume that everybody I know has email, and that I can dump whatever I want there. "For your information"? "Have a good laugh"? "Please do XYZ?" --> email inbox, there it has a higher chance to get attention than any other single place.

With a bit of configuration, I can have everything in my inbox: Mail, blog entries, chat, even voicemail, fotos and video. I can replace a file system - but not email.

Email was the true founding father of the internet, all the various systems have converged - it's here to stay.

About #2+#3: Whatever we do - it needs to have a decent ratio of effort for classification to frequency of retrieval. For me, only desktop search works. (~200 mails/day)

The trick behind PageRank is that hyperlinks are evaluated - kind of a collaborative tagging mechanism. Unless somebody does something like that for my inbox, I'll have to live with my brains to compose near-unique search combos.

About #4: One might think that the easiest classification is SPAM/HAM. So far, AI is delivering meagre results on that. And within the HAM, I (--> human intelligence with tons of background information) still have enough of a job to do to figure out what people want from me. Not always, but more often than I like to.

Lastly: Uldis: We have what you ask for: It's Email (for receiving) and the Web (for publishing). There may be requests for more convenience, but it's hard to ask for more enablement.

Paramendra Bhagat said...

A simple suggestion to Gmail, since you are now back with Gmail: Create about five Inboxes.

Inbox 1: Emails only from people who you have saved on your contacts list.

Inbox 2: Emails from people on your contacts list that have also been sent out to more than you.

Inbox 3: Emails from mailing lists you might subscribe to.

Inbox 3: Emails from businesses, say if I signed up for a service.

Inbox 4: Emails from strangers sent only to me. Fan mail, hate mail.

Inbox 5: Emails from strangers sent to more than me.

The best part about Facebook email is I only get email from people on my friend list.

My suggestion is simple and doable.

As for search and artificial intelligence, impress me! I am all ears.