Pixelastic

My development environment is set up thanks to EasyPHP. I've used this app since the first days that I started to write PHP and it followed me since.

I guess I have a better understanding of what it does under the scenes now and could probably work with a clean and custom Apache/MySQL/PHP install. But I like simplicity.

Anyway, I just wanted to run my imap script from the CLI but ran into "Fatal error : imap_open not defined".

The same script was perfectly working when accessing it from my browser but failed when launched from the CLI.

That when I remember that I needed to enabled the php_imap.dll extensions in both EasyPHP PHP options (for web access) and C:\windows\php.ini (for cli access)

I have previously blogged on how I migrate mails from one IMAP server to another. Today I had to migrate some mails again.

My client's staff wasn't happy about the GMail webmail we gave them. The lack of a true "Draft" feature (one tha could be used and reused) had them drop a lot of productivity, so we decided to get back to Squirel Mail.

I started by manually recreating all the email adresses on the back-end on the Dreamhost server. Once at least one address is created, Dreamhost automatically creates the matching A and MX records.

I had to update my zone file on my registrar to update the previously MX and A records that were set to point to Google and updated to point to Dreamhost. I have to use an intermediate zone file on my registrar because I'm dealing with a limitation of the .fr TLD here.

Once the zone were updated and changes repercuted accross the network, I connected to the new Squirel mail. My inbox was empty.

So I ran imapsync to copy all mails from the google hosted service to my new server.

The syntax to be used is a little different than the one I mentioned on my previous post, so here it is :

imapsync --host1 imap.gmail.com --ssl1 --authmech1 LOGIN --user1 foo@domain.fr --passfile1 /path/to/pass1 --host2 208.97.*.* --user2 foo@domain.fr --passfile2 /path/to/pass2 --useheader="X-GMail-Received" --useheader 'Message-Id' --noauthmd5

After that, I reloaded my Squirel Mail inbox, and it was populated. I just had to do that for every account.

I'll finally have to migrate mails from one IMAP server to another because of a change in the hosting provider of one of my clients.

Let me first do a quick review of my current config :

I have an IMAP server on a not-so-reliable host. I need to move all the mailboxes to a brand new Dreamhost server.

The current host use horde/imp as its webmail, but Dreamhost is running under Squirel Mail.

How to do that ?

When I first contacted Dreamhost to ask them the better way to copy the content of my old IMAP server to their new one, they told me that IMAP to IMAP transfer was unfortunatly not possible.

After some digging I found a little linux tool called imapsync that was developped to synchronize two IMAP servers. As I only wanted to copy content from one server to an empty one, I guess it should work.

And it does.

Installing the tool

First of all, you really need to have a Linux machine. I'm sorry for all Windows and Mac users out there, but this solution will only work with Linux.

Obviously, you'll first have to download and install it.

I'm a newbie linux user, and don't use it at its full potential. It means that I rely a lot on the UI, I don't know most of the more basics commands.

So, I don't know the correct procedure to install something, I always use the Synaptic Packager Manager UI. I just have to type imapsync, check its checkbox and start the install.

I guess you, more experienced linux user, would know how to do that anyway.

Running the command

Once it's installed, you'll just have to run one command. But before typing it, you have make sure that you have all the required informations.

You'll need the server name, login and pass of both the source and destination account.

In my case I had previously reset all the password to newly generated one on the source server, then created the same email (with same password) on the destination server.

For the source server name, you shouldn't have any problem finding it. It usually is something like mail.domain.com.

But for the destination server, that's a little more tricky. As I haven't yet changed DNS, I have a Dreamhost server but no domain name pointing to it. So I can't use mail.domain.com there.

I had to go my Dreamhost DNS panel (clicking on DNS under the name of my domain in Manage domain) and check for the IP address associated with the mail A entry.

Once you have all this, create a file (name it pass) which contain the password of your account (or pass1 and pass2 if source and destination password aren't the same). This is done so you won't type in clear text the password of your accounts, or they could be found in the history files.

Once it's done, just run the following command

imapsync --host1 mail.domain.com --user1 contact@domain.com --passfile1 /path/to/pass1 --host2 208.97.XXX.XXX --user2 contact@domain.com --passfile2 /path/to/pass2 --noauthmd5

It is not an instant process, actually in can be quite long. One of the mailbox I had to move had more than 20.000 mails in the sent folder and about 40.000 in the received one. It took me almost 18h to complete.

One of my clients wanted to change its hosting provider because of almost constant downtime. I suggested that we move the whole site and the associated email addresses to Dreamhost.

He is heavily using its webmail, with a dozen of different accounts, all with custom address book and drafts to reuse.

I had already moved a previous site of the same client to a GMail powered webmail and he wasn't so satisfied of it because of the lack of a "draft" feature that would allow him to reuse the same message templates.

Its current webmail is powered by Imp (Horde) and the default Dreamhost webmail is Squirel Mail.

So now I'm faced with a challenge. Which webmail should I choose and how will I initiate the migration ?

Contestant 1 : Horde

The easiest way would have been to reuse Horde. Horde has a nifty export/import function. It means that I could have exported the settings on the current host and imported them back in the new host.

Unfortunatly, Horde is not pre-installed on Dreamhost. It means I would have to install it and I don't really want to do it. This is not my area of expertise and if something goes wrong it will take me ages to fix.

Additionnaly, the default php5 install on Dreamhost lacks the IMAP extension needed and SSL connection requires an additional monthly fee.

That would be the better solution for the client, but I don't feel capable of doing it.

Contestant 2 : Squirel Mail

Squirel Mail is the default webmail on Dreamhost. It means that they do handle all the heavy stuff of configuring it and making sure it does not crash.

They assure me that I can have unlimited hosting of my mails, the only limits are a maximum of 100 SMTP sending per hour and no attached filed > 7Mo.

Those are almost no issues at all, we can handle that. I just have to check if the 100 sending per hour is for each mail address or for the entire account.

The only problem will be that I can't export/import the horde feature into Squirel.

What I could do however is synchronize the two IMAP servers, effectively copying all the mail from the current one to the new. Unfortunatly, there is no easy way to do that.

The Dreamhost support even told me that it was impossible. I found a linux command named imapsynch however that maybe could help, but I'll need to know the name of the dreamhost imap server and needed credentials to connect to it. I'll sort this out with them.

Contestant 3 : GMail

I've already set mails up with GMail a couple of times in the past so this won't be too hard. Could take some times, but nothing difficult about that.

I may be able to copy the current mails to the GMail server, I've seen some articles on the subject floating on the network, but I'll definitly won't be able to copy drafts.

Conclusion

I'm still in contact with the Dreamhost support to choose the better option. I don't think I'll install Horde, this will be too much work. I guess I'll go with the second option of migrating mails to Squirel Mail.

Whatever solution I'll pick, I'll be sure to post the whole details here for others to follow.

Update

I finally won't have to migrate mails, the client makes regular POP backup of its mails so he will just do one more right before the transfer and will start over with a brand new mail boxes. Concerning the drafts, he will manually copy-paste them from one webmail to the other and he told me that he wasn't using the address book...

So it appears that I'll just have to change the DNS to point to the new webmail and create a temporary CNAME to still access the old webmail on the old server until the migration is totally over.

Re-update

In fact, it appears that I won't be able to access both webmail at the same time. I'll have DNS issues, and the registrar panel that I have is very basic, I won't be able to fine tuned it correctly.

It seems that I'll have to migrate mails from one server to the other.

Ok, so remember when I was writing about that client of mine who had trouble with its SquirelMail implementation. I thought it was a problem of SquirelMail having trouble dealing with large directories.

It was not exactly that, it more was a question of the IMAP server having issues with disk space. I made some backups of the mails, cleaned it up a little then everything went fine.

But suddenly today, the same SquirrelMail started acting funny, blocking some login attempt telling me that the IMAP server stopped responding, or displaying message list without subject nor owner.

Tired all of this, I decided (my client) to move its whole mail system to GMail, using the Google system, while keeping my clients domain name. To finish the registration process at Gmail, I had to put a googlehostedservice.html file online on the domain to prove that I am the owner and...

452 Transfer aborted.  No space left on device

Here is what the FTP server answered. What ? No more space ? But, but, but, but I just cleaned it all, deleting a whole 1Go log file. Wtf ?

I connect to the server and got the list of the biggest files and directories by doing a :

find  /var -type f -ls | sort -k 7 -r -n | head -10

I found that the maillog was 1.1Go... I deleted it and hurry the Gmail activation process. Problem solved.