[ILUG] tool for removing duplicate mails from mbox files?

[Ken Guest]

On Thu, 2001-11-01 at 11:42, John Gaughan wrote:
> On Thu, 01 Nov 2001, Ken Guest wrote:
> > I've tried google and freshmeat, searching for a tool to remove
> > duplicate mails from mbox files to no avail.
> > Does anybody know of such a utility, or do I need to cobble one
> > together?
> 
> Formail (part of procmail) should do it.  With the -D option, formail
> keeps a cache of Message-IDs it has seen, which is used to check for
> duplicate messages.  When used with the -s option (splitting), formail
> won't output duplicate messages.  You need to specify the size and
> filename for the Message-ID cache.  
> 
> For example (with an ID cache called msgid.cache of size 8192):
> 
>     formail -D 8192 msgid.cache -s < oldmboxfile > newmboxfile

thanks. That seemed to do the trick.

k.