1. Subject Lines Matter
2. Use Your Company Email Address
3. Remind Him of Context
4. Limit Your Entire Email to 5 sentences or Less
5. Make Your Ask Explicit
6. Respond Immediately
7. Include a Short, Professional Signature

Need to remember that the next time I email.

Using “Show original” on Gmail gives away more clues on why this email is 5 days late. Here’s an excerpt of the email header:

Received: from [124.108.111.128] by nm1.bullet.mail.hk2.yahoo.com with NNFMP; 23 Nov 2010 04:50:34 -0000
Received: from [124.108.111.126] by tm1.bullet.mail.hk2.yahoo.com with NNFMP; 23 Nov 2010 04:50:33 -0000
Received: from [127.0.0.1] by omp1001.mail.hk2.yahoo.com with NNFMP; 18 Nov 2010 11:49:46 -0000
X-Yahoo-Newman-Id: 501708.68722.bm@omp1001.mail.hk2.yahoo.com
Received: (qmail 8748 invoked from network); 18 Nov 2010 11:49:46 -0000

So somehow this email was sitting on a Yahoo mail server for bloody 5 days! The hostname also resolves to [127.0.0.1] which might be the culprit there. Looks like a major fault to me. I only missed a meeting — but imagine missing a much more important message!

The challenge for me though, is to move all my past emails from my email server (running Postfix + Dovecot) to Gmail. Although I am usually a “deleter” (rather than an archiver), I still kept some of my emails all the way from 1997. Over the years I have over 10,000+ emails sitting in Maildir format on my server, that somehow I need to move them to Gmail.

So I tried to connect to my Gmail account using Thunderbird + IMAP, and then manually drag all the emails over. That was a disaster. For example if I highlighted 100 emails, then drag ‘n’ drop them into “All Mails” under Gmail, and then the operation failed half way through (happens all the time) — I might end up having 50 random emails inside Gmail, but those were not deleted from my old account. That means I have to manually figure out which exact emails have been copied over — and that’s quite a tedious process. To make it safer, you just drag smaller batches over (say 10 at a time). Not a good idea if you have 10,000+ emails waiting to be moved.

Being a lazy programmer I thought the easiest way would be writing a small program that automates this. It would do one email at a time. If the operation failed, it will also know where to resume. The end result? A small Python script that’s conveniently named as maildir2gmail.py.

Download

• Download: maildir2gmail.py (4.6k)

Usage

This script basically works through all files in a directory, working out which ones are RFC822 email messages, and then push those files up to Gmail via an IMAP connection. It also remembers the file names that it had worked through so if the program somehow died (due to a bug for example), just restart it again. Well, this script is provided “as is” with no warranty. It works for me that migrated all my 10,000+ of old emails to Gmail, but YMMV.

To run it:

Usage: maildir2gmail.py [options] [maildirs]

Upload email messages from a list of Maildir to Google Mail.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -f FOLDER, --folder=FOLDER
                        Folder to store the emails. [default: All Mail]
  -p PASSWORD, --password=PASSWORD
                        Password to log into Gmail
  -u USERNAME, --username=USERNAME
                        Username to log into Gmail

For example moving all my inbox to Gmail’s “All Mails”, and all my sent mails to Gmail’s “Sent Mails”:

$ python maildir2gmail.py -u username@gmail.com -p password ~/.maildir/cur
$ python maildir2gmail.py -u username@gmail.com -p password -f "Sent Mail" ~/.maildir/.Sent/cur

It will then print out which message it is working on. Go to sleep, and hopefully all messages will be migrated when you wake up in the morning :) On my old home server (AMD Duron 1GHz in Sydney), it took around 1-2 seconds per message. On a 64MB VPS I had with RapidXen in Fremont CA, it was doing around 2-3 messages per second.

Hopefully it would be helpful to some.

Yes, it’s Google Mail. Or to be more specific, Gmail part of Google Apps, and it is now hosting my (and Vivian’s) emails on this domain (yang.id.au).

I have been running my own email server for the last 10 years, hosting my own domains. It has been sendmail at the beginning, then qmail, and then postfix for the better part of last 10 years. I have tried various IMAP solutions. The ultra-insecure wu-ftpd first, then Courier-IMAP, and for the last 2 years Dovecot. The server has been running on various Linux distributions — RedHat, Mandrake and Gentoo. And in the recent years I also have to deal with email spams/virus. I have tried various combinations of Spam Assassin, ClamAV, Gld for greylisting and dspam.

Yes. Running a mail server — even for just two people — is a lot of work. Moreover I have not yet mention about hardware related issues that have been plaguing my home server for the last couple of weeks. What a pain!

So I am at the cross-road of choices again. Should I bow down to the Google god and submit all my private emails into the cloud in return of piece of mind that they will be in safe hands? Or should I continue to keep my ageing server running, and train that stupid bayesian filter that still can’t seem to figure out the difference between spam and ham…

Not the first time I had this internal battle. I always picked the former in the past years. However this time, under the “encouraging” words of josephb (yeah, I am just finding someone to blame in case it did not work out :), I picked the cloud. I think I made the right choice (so far).

Signing up and modifying the DNS records are easy. It’s also quite straight forward to set up IMAP4 so Vivian can still use Thunderbird to access her emails. Now I have fully integrated email solution that can be accessed from anywhere…

• Web-based access via Google Mail
• IMAP4-based access via Thunderbird
• Mobile-based access via Gmail J2ME app

So far so good. No more software upgrade, or ringing home from work “hey Vivian can you reboot the server?” Although I am no longer able to grep -r foobar ~/.maildir/ to scan through my mails, I can now search my emails like how I search on Google.

Copying all my existing emails (from the last 12 years) to Google did require a little bit more work. I will write about it in my next post.

Because “inboxes are metaphors for our lives,” Dr. Greenfield says, there’s no cure-all solution to inbox management. We’re all too different. But he believes an awareness of our inbox behavior can help us better understand other areas of our lives.

“If you have 1,000 emails in your inbox, it may mean you don’t want to miss an opportunity, but there are things you can’t pull the trigger on,” Dr. Greenfield says. “If you have only 10 emails in your inbox, you may be pulling the trigger too fast and missing the richness of life.”

So how do you usually organise your inbox? Are you an archiving person? Or an deleting person? Or just letting your mails piling up there?

Not sure about you, but my habit of getting my inbox organised has few big changes over the last year. I think the article Aristotle Pagaltzis organised his mailboxes was very helpful. I have basically implemented a similar strategy, and it worked well.

First of all, my main inbox is on my mail server at home, and I usually access it from (1) IMAP via Thunderbird, or (2) local Maildir via Mutt. One is a great open source GUI mail user agent that renders HTML emails beautifully, and another is a fast terminal-based mailer that “suck the least”.

Now, the strategies I’ve implemented are:

1. Your sent folder is your inbox.
2. Archive, but don’t spend time categorise.

Sent Folder is Inbox

First of all, after using Gmail and its conversation view of email correspondents, I do not see why all email clients want to have a Sent folder! I know — it kept all the emails that you have sent, but what about the emails that you have received? Oh, they get archived elsewhere. Or stayed in the inbox. Or maybe just get deleted. Now, when you scan through your Sent folder, how do you know the context that these emails were replying to?

Both Thunderbird and Mutt have great threaded view, where emails of similar subjects or related headers are grouped together. Instead of just grouping received emails, why not add all your sent mails into the pile, to form a real conversation view similar to what Gmail is having?

For example, here’s a screenshot of part of my inbox:

The threaded view of my inbox shows not only the emails that I have received, but also the emails that I have sent, and grouped them nicely together. I can now click on an email that I have sent (where Sender is “Scott Yang”), and see which message I was actually replying to.

To achieve this, you simply need to set your inbox as your Sent folder, so all the emails sent will be saved in your inbox first. For example in Thurnderbird, under Account Settings → Copies & Folders, set “When sending messages, automatically” “Place a copy in” → “Other: Inbox”.

In Mutt, add the following to your ~/.muttrc:

set record="="
set sort=threads

It’ll set the FCC to inbox (or whatever folder is set to) by default, and display mailboxes using threaded view.

Another advantage of saving sent mails to inbox is, when you are archiving and deleting a “conversation”, the operation involves your sent mails as well. For example if I wish to move the whole conversation into another folder, the mails I sent will be moved together (if you selected the whole thread in Thunderbird), and they’ll stay as a conversation in that folder.

Archive, Not Categorise

When I had just way too many emails, I started to archive them by putting them into different folders. However, the ways I categorised them are not always consistent. Sometimes you just don’t know what label you wish to this mail under. And unlike labelling in Gmail, you cannot put the same email into multiple folders, if you think it belongs to multiple categories.

At the end I gave up categorising them. Just dump all the conversations that I wish to keep into another folder — yes all of them into one folder. No labelling. No sub-folders. It has everything.

The point is, why do you categorise when you can search?

There are many email indexing products. Google Desktop, for example, can index emails in your Thunderbird mailboxes so you can easily find the emails you need with a few keywords. It is much faster than manually locating the category/folder, and scan through the emails. If you wish to use the “search” feature of your email client, you might as well save yourself sometime by not categorising them in the first place!

So far it worked well for me. If there’s an email conversation I wish to keep, I’ll just move the entire thread to my one and only archive folder, without thinking which folder.

So, it is how I organise my inbox these days. How do you do yours?

But it might still fail. Will we still “email” in 2025? Will they still talk in RFC821, with messages packed in RFC822 envelops? Will there be an email client available for download? Will there be “downloads”? Would Internet still exist in the current form in 20 years time?

Will I forget to check that long expired account? Will I still be alive reading my own emails? Will Christ come back to bring in the Judgement Day prior to that?

Against these improbable (or, maybe very probable) odds, what will you write in such email?

G’day Scott!

How’s going mate? So you are almost 50 I presume. Coping with mid-life crisis well? How is it like getting closer to the end of the tunnel? Was that a smooth or roller-coaster ride? So, Jesus didn’t come over the last 2 decades right? But you know he is just taking his time, don’t you? Stand firm mate!

I am all good. Feeling a bit sick in a stormy afternoon in Sydney Australia. Just wondering what has been happening.

Cheers! See you in 20 years.

Wasn’t that an pathetic attempt?! But I cannot imagine what will I think when I receive it in 20 years time. I do archive emails — only all the way back to 1996. When I read through some of them they are just either (1) out right funny (2) idiotic. How would this email be perceived in 20 years time?

Then I saw this when I logged in today:

Yes, 1295Mb! That’s an unannounced new feature which Google has just given me 30% more space! Even though I’ve only used 3Mb so far (so pathetic), but that extra 295Mb has just made myself so confident that I will never burst my inbox!

Or was that an April Fool?

Update: Thanks to the comment by Joey. I checked the login page of Gmail and now everything is clear.

… starting today, we’re beginning the roll-out of our new and top secret Infinity+1 storage plan.

And then there is a funky chart comparing the storage between different webmail providers, theoretical limit, and Gmail’s infinity + 1.

Click on “Full Headers” reveals what Yahoo SpamGuard has done:

Sounds like my IP address has just been black listed, and it affects every single email originated from this IP – including personal emails sent by myself! In fact, I don’t recall myself sending any unsolicited commercial email lately, nor have I infected by virus – the mail server is guarded well with firewalls and anti-virii software. Who knows how my IP address ended up over there?! Anyway, if you are using Yahoo, and you are expecting an email from me (or from one of the FOCUS mailing lists), then please check your bulk mail folder. You might find surprises there…

What steps should I take? I might write to Yahoo complaining about the situation. If you have my emails (or other FOCUS related emails) in the bulk folder, please select them and mark them explicitly as “Not Spams” – maybe this can somehow re-train the filter.

Hopefully it is only temporary.

Updated 11 Feb 2005 @ 9:35am: Instead of asking Postfix to send emails directly to the destination, I am having it relaying to Exetel’s SMTP box. It seems to be working now, and all mails appear in Inbox!

Gmail has already been criticised with its contact management, especially when it is compared with Yahoo! and MSN/Hotmail. I see it is still miles away from Y! or MSN’s integrated fully featured address book, even though it is sufficient (for me) with my daily emailing needs. But it is not a fair game anyway – both Yahoo! and MSN/Hotmail have integration with their instant messenger clients. Unless of course, that Google releases its own IM network. More rumours :)

According to other sources that Google is also adding Atom feed and automatic forwarding to selected accounts. Not on mine though. It is certainly on its way to become one of the best web-mail client, if not already is.

However, today I have received a notification email from AventureMail, saying that they have again reactivated the service to allow registration – but with a catch. It says,

Dear User,

AventureMail.com Version 2 is now live! All previous accounts/email etc has been removed to make way to the final release of AventureMail. After several months of intense beta testing to resolve issues such as the massive influx of virus email and spam, we have been able to make a drastic amount of changes…

Basically AventureMail is now opening its business again with improved spam and virus detection. What then is the catch? They have introduced a $2 once off registration fee to combat against service abuse. My guess would be that previously they have received too many abuses regarding to their 2Gb account, so they now only want legitimate users who are serious about getting an email address from them.

Once off USD$2 for a 2Gb email account with improved spam and virus detection – it is actually quite a good deal, provided that they don’t oversubscribe themselves again which produces crappy responses. However, I do have an issue with the way they dealt with existing beta testing customers.

First of all, when I first signed up, I never knew that it was a beta testing program. I always thought that they were an Internet company offering big free email accounts, instead of someone looking for beta testers. Secondly, why delete accounts of existing users, especially when there might be legitimate users who depend on this mail service everyday (I doubt due to previous experiences with their speed and responsiveness)? With a track record like this, how can you know that they will not suddenly terminate your account, and ask you to sign up again (with a fee), when they have finally released the new version of AventureMail? For this reason alone, I would not sign up to it, even though it is cheap.

And I cannot imagine if Google does that to all its Gmail users, when Gmail has finally got out of beta (if ever)…