I've been using Netscape 4.7x on Windows 2000 (and previously on NT 4.0) for several years now, but I'm somewhat sick of its limitations and bugs, so it's time to move on to other things. I tried transitioning to Netscape 6.2, but it's still decidedly incomplete in many aspects, and doesn't materially improve in others.

So, I'm up in the air about my other options and I realized this is the right (read: geek) crowd to ask for advice.

My requirements:

- Must let me store my mail on an IMAP server
- Must have some kind of support for HTML e-mail (I get several of these things every week and it's nice when they format inline)
- Must be secure against typical annoying e-mail worms that plague Outlook
- Must run on either Linux or Windows (my computer runs Win2000 natively and Linux via VMware, displaying apps with XWin32)

Stuff that would be nice:

- Emacs integration (so I can hack it upside down and backwards)
- Offline support
- Fast searching / indexing support

For the past day, I've been playing with Ximian Evolution 1.0. It works remarkably well for a 1.0 product, although it's definitely a bit rough around the edges. I'm currently hacking up a Perl script to convert my Netscape mail filtering rules to Evolution. Evolution can be extremely slow for some operations, particularly when it's never seen a folder before. But, it's a strong contender.

The tool I'm most intrigued by is Gnus. The manual, particularly how it describes the IMAP integration, is pretty weak. Does anybody here have experience using Gnus? It seems like such a nice idea, although I'm not sure whether I want to shift from my normal "mail reading" paradigm to Gnus's "news reading" paradigm. Years ago, I ran Mew, another Emacs mail reading, MIME-aware, front-end for MH. Mew rocked, but I had to drop it when I transitioned to the IMAP universe. It's not entirely clear whether Mew can work with IMAP or not since all the documentation is in Japanese. There are other Emacs mail readers as well (e.g., Wanderlust, VM, RMAIL, etc.) with which I'm even less familiar. Post about your favorite!
Also, for what it's worth, I use Outlook pretty extensively for my calendar and address book needs. I've considered using Outlook for reading my e-mail, but Outlook 2000, at least, crashes when I point it at my IMAP server. It sees all the folders and subfolders and just seems to go kaboom. Anybody have experience with Outlook XP? I'm not intrinsically opposed to using a Microsoft product, although the security issues with them are definitely bothersome.

I use a paid version of eudora 5.1. It works great and does everything you specified well. For some reason, the only isp I have ever had that used IMAP is the university that I am attending. I find it more useful than POP3 becuase I check my mail from several different places. Why don't other isps use it? Is it due to the extra load placed on the server?

Sean

Why don't other isps use it? Is it due to the extra load placed on the server?

Well... POP3 is older. Standards take forever to get replaced. IMAP is a much newer protocol. It's a little more complicated in its design, and because its encourages you to keep the mail stored on the server, it's likely to take more disk space for a large number of users. I think that might have a little to do with it.

You can check POP mail from several different places too, you just have to tell your clients to leave the mail on the server. By default most clients delete after checking, but it's not required.

Oh, I use Eudora 5.1 as well. I wish it had more than 7 or 8 labels for message categories/colors, but other than that it's great. One of my email accounts supports IMAP but I just use POP3 because it's simpler to manage.

IMAP has been out for a while now, so its not all that new. Its at least 4 years old. One of the cool things that imap offers is shared folders on the server. These are used for the stuff that would have otherwise been sent to all students as spam. I just looked and found the setting for leave mail on server. Does deleting mail require me to change that setting afterwards?

Sean

You're speaking about 4 years as if that's a long time.. As I said, these things take forever. Yeah I know the benefits of IMAP, but there are very few benefits for ISP's to upgrade. IMAP makes a lot of sense for a school or a business, but not much sense for an ISP. That's why you don't see it much.

Yeah the messages eventually have to get deleted. I have my Eudora at home delete after checking, but when I use PINE or Netscape or whatever other email client from work or school, I set those to leave the mail on server. That's the only sensible way to make it work, have one client that does the deleting, and have the rest just download the mail and don't delete it. Otherwise you'll have some messages on this client, some others on the other client, and it'll be a mess.

I like Pegasus Mail (also available here, with lots of useful info). It's IMAP support isn't perfect yet, but it's not bad and improvements are on the way. Is a nice powerful mail program and free!

Huh? IMAP is new? IMAP is ancient in tech terms. Much (MUCH!) older than 4 years. I was using PINE in 1993 with IMAP. It's gone through multiple revisions. I haven't used IMAP in a long time - is IMAP4 still the current implementation? POP3 is newer than "IMAP" as far as I know. I don't know the history of its revisions, so I don't know if its first incarnation is actually older than the first IMAP.

I hadn't even heard of POP3 when I was first using IMAP.

Bruno

THE BAT.

www.ritlabs.com

Plenty of people swear by it. And if you're looking for geek factor it beats down Eudora or anything of that kind. The only reason I didn't switch to it a few years ago was it lacked in its editor. Wasn't standard enough for me. That should have changed by now as it was a promised new feature.

Bruno

I'm a big Gnus fan. It's been my newsreader for about 10 years, and my mailreader almost since it started supporting mail.

I wouldn't recommend it to an Emacs newbie, but you're obviously not one, so that's OK.

It runs anywhere you have Emacs, so that's your cross-platform portability layer taken care of.

I'm currently running 5.8.8 that comes with Debian woody; its IMAP and MIME support is reasonably complete now (and it has useful features like the ability to read text/plain in preference to HTML in multipart/alternative). I've been using the IMAP backend for a couple of years now, on Linux and previously on Windows.

I've no experience of the Gnus Agent (offline support), and I've never had need for anything beyond simple searching in one group - sorry, folder.

Basically, it meets all your requirements and at least your first wish, possibly more.

Like the empeg, Gnus's greatest asset is its active developer and user communities, with a mailing list and newsgroup respectively.

HTH,

Hi.

Oh well, I did a quick lookup and found that IMAP v4 was first defined December 1994 in RFC1730 and later modernized as IMAP v4.1 in RFC2060 during December 1996. Seven (7) years ago. It's predecessor, IMAP2 was first defined in July 1988 as documented in RFC 1064 and later redefined in RFC1176.

Now to be fair, IMAP2 was never widely used and wasn't really well usable at all. So only IMAP4 has put some useful features in there, but nevertheless, IMAP4 is now seven years old, and it's implementation could be considered stable.

BTW: HTTP was first defined in 1989 (and in use by CERN since 1990) and was in network wide use by 1992. Speaking of slow developing standards.

cu,
sven

I was just going through the same RFC searches, you saved me the trouble of posting them all :)

And POP has been around since 1984 (!) and POP3 since like 1991, both as usable standards. I'm not saying IMAP doesn't have its advantages, I'm saying that things take forever to change, and POP3 is ubiquitous. IMAP's advantages are advantages for the end user, not the people who implement it like ISP's...

I use Pine. Unless I'm on my MacOSX, when I use Apple Mail, which is surprisingly nice. Something to note is that most IMAP searches are done on the server, so ``fast searching'' may not be a feature of the client, except in offline mode. You might want to take a look at Mulberry. It's a very nice client, but it's very quirky (that is, it doesn't do things in the same way as other email clients).

IMAP's advantages are advantages for the end user, not the people who implement it like ISP's...

As a former email admin, IMAP is far superior in many respects from my point of view. An ISP has very few requirements to their customers, but working as an admin, I have to deal with the most asinine requests. One of the common ones is ``Can you undelete my mail?''. With POP, this would require backing up every person's workstation in order to comply. With IMAP, I can just back up the server. Much easier. Also, when dealing with your mail from multiple locations, IMAP wins in usability hands down. Imagine downloading all 1000 messages in your POP account over a 56k modem every time you want to check your mail from a new computer. This is why web mail was invented, but I'm sick of having to have all applications run via a web browser (not to mention that most of them use IMAP on the back end anyway). This is easier on the admin because you don't have to listen to your users complain. Of course, there's also the issue of selecting an IMAP server that doesn't stink....

I've boycotted Eudora ever since their CEO (or somesuch) publicly claimed that IMAP was the worst thing ever and that people would complain about how slow it is compared to POP and that they would not be implementing IMAP support. (I'm paraphrasing wildly, of course.) This amazing lack of understanding in a public forum (I wish I could find a copy now, but can't) soured me on their products forever.

Umm I dunno where this quote came from (or when) but Eudora definitely supports IMAP these days.

You're right (I meant to mention that), but that just makes it worse, doesn't it? Now (years ago, really) he's not even got the courage of his convictions.

Tell me more! From reading what passed for the Gnus documentation, it's not clear whether I can stomach moving to its "newsreading" paradigm for e-mail. I'm quite comfortable with seeing a unified inbox of my unread messages with auto-refiling based on a bazillion rules that I've slowly cobbled together over time. Can Gnus be convinced to do all this?

Is there such a thing as a beginner's guide to Gnus? I've done a fair bit of hunting around the Web and I haven't found anything terribly enlightening.

The main difference between a "newsreader" approach and a traditional "mailreader" approach to mail is that (AIUI) traditional mailreaders continue to show you old, read, mail every time you look in that mailbox. Gnus can be configured to work that way if you like, but most people I know prefer to have old mail disappear until it's explicitly requested.

The other newsreaderly feature of Gnus's mail handling is that it can be configured to delete ("expire") read mail after a period of time (this is not the default behaviour; you only get it if you ask for it).

It's not too hard to get started with Gnus, though I expect it's easiest if you set it up as a newsreader first and then add mail sources to it. Start off with reading your mail out of the IMAP "INBOX" folder, then gradually add split rules to relocate all your list traffic etc. The rules can be simple regular expressions (with backref substitutions if you want them) or abritrary lisp forms. Splitting is done at mail fetching time, so if you do this, you see how many new articles are in each group (like on this BBS, but you've chosen the list of groups) and then enter each group in turn. With the default, simply tapping space pages you through each article in turn, moving on to the next group when no articles remain. So you can read all your mail and news without having to wake up and move your hand.

Another cool thing is that most settings can be done either globally, per-group, or by topic (hierarchical arrangement of groups). For example, mail in my "spame" group expires after 3 days, but that in my "girlfriends" group remains forever. And in my "egroups" topic, Gnus is set to strip out those annoying banner adverts.

The lack of good new-users' documentation is probably Gnus's biggest failing; I think some effort is going into this. Did you try http://my.gnus.org/? I'm not in the newsgroup (news:gnu.emacs.gnus) that much these days, but IIRC it's quite supportive of new users.

TBH, I've been using Gnus for too long to be able to tell a good newbie guide from a bad one...

(screenshot attached - of course, it can't tell you what it's like to use, or even what it looks like on a light background or with the default colours, but I thought it might help)

Forgot to mention, Dan - If you go with Gnus, feel free to email me, except when I'm away offline (from Tuesday, for 4 weeks).

If imap is superior, why don't more isps use it? The imap server I get my mail from is cyrus I think. Does cyrus stink?

Sean

Because IMAP can be heavy on the server, and means that users will generally store lots of their mail (not just their current undownloaded inbox, as is more customary with POP3) on the server too.

I've seen IMAP supported by ISPs, but in the POP3 sense; ie, you have a single mailbox (INBOX) and can't create any new folders.

Hugo
random fact: I wrote the of the EPOC ER5 IMAP client (not the UI, just the part that does the IMAP) during the early days of empeg... this is the one in the Psion Series 5MX and the Ericsson R380. Met lots of the IMAP server people including Cyrus at the IMC meeting in '99 which was in the same building as the LinuxExpo :)

Because it requires more resources on their part.

POP is remarkably simple. The paradigm is to store messages until the user requests them, then delete them. This can be modified, but that's the general idea. There are very few POP commands. They amount to ``show me what you've got'', ``send me that message'', and ``delete that message'' (beyond auth commands). There are two optional POP commands -- one that allows the client to see only the first portion of the message (defined in bytes), and one that provides unique identifiers for messages, which can allow for better synchronization.

IMAP is, at least relatively, very complex. In addition to the basic commands that POP has, IMAP allows retrieval of portions of messages defined by MIME sections, multiple folders per user, assignation of flags (even arbitrary user-defined ones) to messages, on-server full-text searches (which is useful because you might not have all of the messages on your client), etc. The paradigm is also quite different. IMAP expects that the user will leave all messages on the server and deal with the messages ``remotely''. As such, good clients will fetch only the bare minimum of the message to show in list mode and get the rest only when needed. In fact, there is a way to get the server to send the message headers for all (or any subset) of the messages. Which is not to say that it's impossible to download everything to your client. It's just not what it's best at. But most IMAP servers have ancillary POP3 servers, as well, that read from the same spool as the IMAP server for those clients that are trapped in POP-world for some reason.

In summary, a quote from the POP3 RFC seems in order: ``In short, the philosophy of this memo is to put intelligence in the part of the POP3 client and not the POP3 server.'' So basically, ISPs can either set up POP, which is a very simple message delivery protocol, or they can set up IMAP, which tends to require more storage space, and much more processing power. And since they have no requirements to meet, other than simply getting the mail to you, it's a no-brainer for them. Interestingly, Apple's iTools free email (@mac.com addresses) are supplied as IMAP accounts, though.

To read more about IMAP vs. POP, read Comparing Two Approaches to Remote Mailbox Access: IMAP vs. POP and Message Access Paradigms and Protocols, or visit your local library.

Oh -- Cyrus is my favorite IMAP server. It has many nice features, including quotas, mailbox sharing via very precise ACLs, direct delivery to folders other than your INBOX, and a lot more.

I've been using Mutt for several years now, after a few years of Pine and a short stint with Elm. Mutt is highly customizable, stable, actively developped and should meet all your requirements.

Borislav

Wow, I have Mutt installed on my box and didn't even know it. I've been using Pine for my remote mail checking needs... I'll have to mess around with this new toy.

You've been using Mutt as in IMAP client? Every time I try to use it as such, it is horrible. Am I missing something, or has something been updated in the last few months?

You've been using Mutt as in IMAP client? Every time I try to use it as such, it is horrible. Am I missing something, or has something been updated in the last few months?

No, I don't use IMAP. The Mutt and IMAP page gives the impression of fairly robust IMAP support, although it does say you should use the development (1.3.x) version if you need IMAP. FWIW, I've been running 1.3.17 with no problems.

Borislav

Yeah. They've given that impression for years. It sucks.

Yeah. They've given that impression for years. It sucks.

You are welcome to your opinion, but simply saying that it sucks doesn't tell anybody anything useful. What problems are you having with it?

Borislav

Sorry. I meant that it's IMAP support sucks, not the program itself. You're right though, regardless. At it's heart, it feels like the basic support for IMAP has been hacked in to make it emulate local spools, which just is not the right way to do it (just like it's not right to make it emulate a POP server, which many older clients did, and do, as well). I'm downloading the latest development snapshot now and recompiling it to remember more precisely. I'll let you know in a while.

Well, it actually seems to have gotten a lot better in the last few months. Currently, though, it seems to be unable to store flags (except for deleted) on the IMAP server. (Or maybe it does, but it only flushes them on exit?) It uninterruptably downloads the headers for every message every time it opens a mailbox. Searching doesn't seem to happen on the IMAP server, making full text searches impossible. It doesn't seem possible to select a mailbox from the folders list that also serves as a folder. That is, I can have the mailbox imap://imap-server/INBOX/read-mail and the mailbox imap://imap-server/INBOX/read-mail/from-bob. Mutt cannot open the parent folder/mailbox from it's list mode. I can type it in directly, though. (Actually, after further research, it looks like I can't open any mailboxes in list mode on servers that support combined mailboxes and folders, as it thinks that they're all folders.)

On the plus side, they implemented IMAP over SSL, which seems to work quite well. Also, they finally implemented understanding external changes to IMAP mailboxes (where another client accessing the same mailbox changes something), but, apparently, since most flags are not stored on the IMAP server, it may get them wrong when messages are deleted externally.

That's all I can find after about 10 minutes of looking. I'm sure that there are still more problems. I just can't remember what they might be right now. All in all, it's just not ready for prime time. But I think it probably will be, eventually. By the way, most of these problems seem to stem from the fact that it was initially written against the UW-IMAP server, which, for years, supported very few advanced IMAP features. As such, ability to use those features seems not to have been thought of when implementing IMAP support for Mutt. UW-IMAP and Mutt seem to slowly be adding those features, but neither one is there yet.

I can't speak for Mutt, but I've been using UW IMAPD and Netscape 4.7x (and its predecessors) as my mail client for years. The way to think of IMAP is as a remote filesystem. It has directories and files, only those are now called folders and messages, for some reason. IMAP daemons (file servers) seem to have two different ways of storing messages. Some, like UW IMAPD, allow you to store your folders as traditional Berkeley mbox-style flat files. These are quite portable, but they're excruciatingly slow to work with when they get large. UW IMAPD and Cyrus support nifty database-like storage that speeds things up immensely, but nixes any backward compatibility that you might otherwise want.

On the client side, based on my research, the clients worth talking about are Microsoft Outlook, Qualcomm Eudora, Netscape/Mozilla Communicator/Messenger, Ximian Evolution, and Emacs Gnus. I'm leaving Mutt off the list because I'm not seeing much in the way of multimedia message support beyond basic MIME attachment handling. Gnus can, at least in theory, integrate with Emacs W3 mode (although I haven't figured out yet how to make it happen, or how to view previously-read messages, either -- the Gnus manuals suck).

Outlook, Eudora, Netscape, and Evolution are all spiritually related. You get a "three-pane" GUI, with your mailboxes down the side, the messages in the current mailbox on top, and the current message everywhere else. HTML content is handled in the obvious way, and MIME attachments can be viewed inline. All of them have filtering and auto-refiling support which isn't wonderful but doesn't suck. Gnus is better in some respects, but worse in others. (Emacs Mew totally rocked for refiling rules, but it doesn't have IMAP support yet.) I've been running Evolution for a few days now, and I'm quite impressed. In many respects, it's much faster than Netscape or Eudora, although it's not as stable yet. Evolution also has some fairly nifty search-engine-like features built into it.

I'm still not totally happy with my setup. Maybe I need to try the latest Outlook from Office XP. Hopefully it's got better IMAP support than Outlook 2000, which consistently crashed before it could finish scanning all my folders.

Thanks, that was very useful. I've poked around the code a little bit (trying to add support for automatic save to multiple folders, but it turned out to be hard). Now I have a list of stuff to look at if I get back to it at some point.

Regards,
Borislav