How to Migrate Emails from Maildir to Gmail

Migration As previously reported, I have moved my main email from my own mail server to Gmail hosted on Google Apps. Signing up was easy. Moving DNS records was pretty straight-forward (a few clicks if your domain is with DreamHost). Getting IMAP up and running requires one simple setting inside Gmail, plus reconfiguring your MUA (Thunderbird, Outlook, etc).

The challenge for me though, is to move all my past emails from my email server (running Postfix + Dovecot) to Gmail. Although I am usually a "deleter" (rather than an archiver), I still kept some of my emails all the way from 1997. Over the years I have over 10,000+ emails sitting in Maildir format on my server, that somehow I need to move them to Gmail.

So I tried to connect to my Gmail account using Thunderbird + IMAP, and then manually drag all the emails over. That was a disaster. For example if I highlighted 100 emails, then drag 'n' drop them into "All Mails" under Gmail, and then the operation failed half way through (happens all the time) -- I might end up having 50 random emails inside Gmail, but those were not deleted from my old account. That means I have to manually figure out which exact emails have been copied over -- and that's quite a tedious process. To make it safer, you just drag smaller batches over (say 10 at a time). Not a good idea if you have 10,000+ emails waiting to be moved.

Being a lazy programmer I thought the easiest way would be writing a small program that automates this. It would do one email at a time. If the operation failed, it will also know where to resume. The end result? A small Python script that's conveniently named as maildir2gmail.py.

Download

Usage

This script basically works through all files in a directory, working out which ones are RFC822 email messages, and then push those files up to Gmail via an IMAP connection. It also remembers the file names that it had worked through so if the program somehow died (due to a bug for example), just restart it again. Well, this script is provided "as is" with no warranty. It works for me that migrated all my 10,000+ of old emails to Gmail, but YMMV.

To run it:

Usage: maildir2gmail.py [options] [maildirs]

Upload email messages from a list of Maildir to Google Mail.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -f FOLDER, --folder=FOLDER
                        Folder to store the emails. [default: All Mail]
  -p PASSWORD, --password=PASSWORD
                        Password to log into Gmail
  -u USERNAME, --username=USERNAME
                        Username to log into Gmail

For example moving all my inbox to Gmail's "All Mails", and all my sent mails to Gmail's "Sent Mails":

$ python maildir2gmail.py -u username@gmail.com -p password ~/.maildir/cur
$ python maildir2gmail.py -u username@gmail.com -p password -f "Sent Mail" ~/.maildir/.Sent/cur 

It will then print out which message it is working on. Go to sleep, and hopefully all messages will be migrated when you wake up in the morning :) On my old home server (AMD Duron 1GHz in Sydney), it took around 1-2 seconds per message. On a 64MB VPS I had with RapidXen in Fremont CA, it was doing around 2-3 messages per second.

Hopefully it would be helpful to some.