Friday, April 24, 2009

Email Reply Behaviors

I found a great paper by Thomas Karagiannis and Milan Vojnovic at Microsoft Research Cambridge in my email this morning and thought I'd share some of the highlights with you.

In the paper, they surveyed email behavior in a large company, and analyzed how likely it was that emails will be replied to, and how long the replies would take. They looked at all the various factors - length of the email, number of recipients, time of day, and so on. I thought I'd paste in a couple of the highlights.

First of all, let's look at the relationship between the size of an email and average reply time. Anecdotally, I reply to short emails quickly, but if you send me a long email, you're going to have to wait for a reply. This is true in general:

Let's look at queue size vs. reply time. When I get lots of emails, some of them get pushed further and further down my inbox, and it will probably take me a while to get to them. This is a beautiful visualization of that effect:

Do replies get sent at particular times of the day? Heck yeah! The graph below shows time of day vs. replies sent, and you can see how they follow a clear awake / asleep pattern:

But a main takeaway for me is the following graph. I've always assumed that you'd reply faster to people above you in the company hierarchy - your manager would get replies faster than people at your level or below. You'd think you reply to Steve Ballmer faster than to Joe Intern. This does not hold true. On the graph below, dots to the left are "replies higher-ups", dots to the right are "replies to lower-downs". As you can see, more important people do not receive faster replies.

Here's the paper for people interested in more detail:

Behavioral Profiles for Advanced Email Features, Thomas Karagiannis and Milan Vojnovic, Microsoft Research, WWW-2009, Madrid, Spain [PDF]


Valio said...

Hi Gabor, this is interesting article -

Anonymous said...

Thanks for drawing my attention to the article... you just got them a citation!

With regard to reply time w.r.t. "level-distance" perhaps the behaviour of favouring communications with your boss is offset by the greater proportion of emails which are actioning work, rather than just request information? With lots of these trends you can only start understanding them by digging into the content of the emails.

I found the "probability of reply conditional on the email size" interesting... I suspect this will be very specific to particular organisations or types of work.

I would have liked to see plots of response time and probability of response based on the geographical distance between sender and receiver. They don't seem to make much use of their "org-structure dataset".

Gabor said...

craigloftus - you're welcome!

Can you elaborate on one of the thoughts you had? You said you suspect that the probability of reply conditional on the email size would be organization-dependent ... why would this be the case? I'd imagine that would be pretty universal.

Anonymous said...

The suspicion was just based on a thought that the particular sized of an email does not seem like a good indicator of importance or value.

On an anecdotal level, in the corpora I have experience with large emails are usually associated with the distribution of documents rather than discussions or conversations.

Without comparing across many organisations I think that a particular behaviour associated with a very specific characteristic (such as size) is more likely to be associated with a particular internal process or activity.

As an example, the sector which I work in is generally manufacturing and assembly in the UK. The organisations in this area, almost without exception, have internal project management systems which handle internal file exchange, and these are well used. An analysis of a corpus from one organisation indicates that a significant proportion of emails with attachments are with external parties - if those emails had a high reply rate, it would be dubious to assume that it is because they are large...

I wonder what the numbers say about posting long comments on Friday afternoons? ;)