mu - e-mail searching and indexing

welcome to mu

For many people, e-mail is the 'flow' in their work-flow. One spends a lot of time searching for old e-mails, digging up some important piece of information. With people having tens of thousands of e-mails (or more), this is becoming harder and harder. How to find that one message in the evergrowing information haystack?

Enter mu. 'mu' is a set of command-line tools for Linux/Unix that enable you to quickly find the e-mails you are looking for. The main requirement is that you store your e-mails in Maildirs. If you have no idea what 'Maildirs' are, you are probably not using them.

how does it work?

First, you need to index your messages. Indexing means filling a database with information about your e-mails; this may take a couple of minutes the first time you do it, but after that it's a lot faster. mu indexes your mail with the index command:

  $ mu index

It tries to pick reasonable defaults, but you can specify your own options as well. You could run mu index periodically to keep your database up-to-date. Or you could trigger it when new mails have arrived.

After building the database, it's easy to search for messages. For example:

get all mails about birds and bees somewhere in the message or headers:
```
    $ mu find birds bees
```
if you want to find message with either one of those, you can do:
```
    $ mu find 'birds OR bees'
```
get all Smith's mails with 'capybara' in the subject line
```
    $ mu find from:smith subject:capybara
```
get the mails in the archive folder where Mary was Cc'd
```
    $ mu find maildir:/archive cc:mary
```

Starting with version 0.9, you can search for date ranges, message flags and message priority as well:

get mails with attachments from the last two weeks:
```
    $ mu find flag:attach date:2w..now
```

get all important mails in March and April 2010:

    $ mu find prio:high date:2010-03..2010-04

Searches are case-insensitive as well as 'accent insensitive' (version 0.9 and up); so angStroM will match Ångström. Often-used queries can be stored in bookmarks file.

The way to express the searches may be a bit cryptic at first, but easy to learn (in the author's biased opinion); the mu manpages discuss syntax and usage. There is also the mu-easy man-page which contains a lot of simple examples to get you going.

NOTE: while searching from the command-line is useful, mu can also easily be integrated with some e-mail clients. The documentation includes examples for integration with mutt and Wanderlust.

mu is Free Software (GPLv3), runs on Unix/Linux-based systems, and uses the Xapian text indexing engine. Important: for mu to work, your mails must be stored in a set of maildirs

mug

Starting with version 0.9.2, there is now two toy UIs for mu, called mug and mug2. The latter has more features, but also adds some dependencies if you want it - both mug/mug2 are optional. The longer-term goal is to have a bit more complete graphical user-interface. Currently, the UIs are not installed, you need to run them from their build directory. If you have defined bookmarks, the are shown in the toolbar on the left side of the window, as can be seen in the screenshot.

news

[2011-02-02] mu version 0.9.2 released, with a few new features, such as size-based matching (e.g,. size:2k..2m to match message between 2kB and 2MB), (experimental) output in XML/JSON/s-expression format, updated toy-UI ('mug' and 'mug2'), support for Solaris and Gentoo. For a complete list of the user-visible changes and some of the changes in command-line parameters, see NEWS.
[2010-12-05] mu version 0.9.1 released; fixes a couple of issues users found with a missing icon, the unit-tests.
[2010-12-04] mu version 0.9 released. Compared to the bèta-release, there were a number of improvements to the documentation and the unit tests. Pre-processing queries is a little bit smarter now, making matching e-mail address more eager. Experimental support for Fedora-14.
[2010-11-27] mu version 0.9-beta released. New features: searching is now accent-insensitive; you can now search for message priority (prio:), time-interval (date:) and message flags (flag:). Also, you can now store ('bookmark') often-used queries. To top it off, there is a simple graphical UI now, called mug. Documentation has been update, and all known bugs have been fixed.
[2010-10-30] mu version 0.8 released, with only some small cosmetic updates compared to 0.8-beta. Hurray!
[2010-10-23] mu version 0.8-beta released. The new version brings mu extract for getting the attachments from messages, improved searching (matching is a bit more 'greedy'), updated and extended documentation, including the mu-easy manpage with simple examples. All known non-enhancement bugs were fixed.
[2010-02-27] mu version 0.7 released. Compared to the beta version, there are few changes. The maildir-matching syntax now contains a starting /, so ~/Maildir/foo/bar/(cur|new)/msg can be matched with m:/foo/bar. The top-level maildir can be matched with m:/. Apart from that, there are so small cosmetic fixes and documentation updates.
[2010-02-11] mu version 0.7-beta released. A lot of changes:
- Automatic database scheme version check, notifies users when an upgrade is needed
- Adds the mu view command, to view mail message files
- Removes the 10K match limit
- Support for unattended upgrades - that is, the database can automatically by upgraded (--autoupgrade). Also, the log file is automatically cleaned when it gets too big (unless you use --nocleanup)
- Search for a certain Maildir using the maildir:,m: search prefixes. For example, you can find all messages located in ~/Maildir/foo/bar/(cur|new)/msg with m:foo/bar. This replaces the search for path/p in 0.6
- Fixes for reported issues #17 and #18
- A test suite with a growing number of unit tests
- Updated documentation
- Many internal refactoring and other changes
This version has been tagged as v0.7-beta in repository, and must be considered a code-complete preview of the upcoming release 0.7. Please report any problems you encounter with it.
[2010-01-23] mu version 0.6 released. Changes compared to the beta version are:
- improved documentation, with more examples and more extensive discussion of various technical aspects
- some fixes, thanks to bug reports from beta-testers
- added search for the the Message-Id field.
There is a separate branch for 0.6 in the git repository (see below), HEAD will become 0.7
[2010-01-16] mu version 0.6-beta. After some hiatus in the release schedule, mu is back now! The new version is a total overhaul of what was there before. It's smaller and faster, and removes quite a bit of the (in the eyes of the author) needless complexity. It adds some new features that users asked for, such as a way to exclude directories from indexing (simply put a file called .noindex in it. The queries are bit less featureful, the more useful parts will slowly be added again in future versions. Note, this is version 0.6-beta, the 'real' 0.6 will follow soon.
[2008-11-01] mu version 0.4 released. This version adds extensions to the query language and the ability to get statistics about the messages that match some queries. mu-index is substantially faster. Also, mu-msginfo has been extended. Small optimizations have been done, and some bugs have been fixed. Finally, some GTK+-widgets have been added; they are to be considered 'experimental'.
[2008-09-25] mu version 0.3 released; this version adds further signicant indexing speedups (esp. on ext3-file systems). Added searching for relative dates ('3w' for 'three weeks ago'); search for message properties such as has-attachment/signed/encrypted; search for message priority; updated documentation; define bookmarks for search queries; more unit-tests.
[2008-09-10] mu version 0.2 released; this version adds supports for message-body searches (Xapian); indexing is much faster; added Maildir++ support; sorting of results; added mu-mkmdir/mu-msginfo. Improved documentation, unit tests.
[2008-08-25] mu version 0.1 released

features

mu find:

search for sender and recipient; subject,message body text
output to stdout, or as maildirs with links to the original messages

mu index:

scan messages in recursive Maildirs, Maildir++ or separate files;
only changed messages are considered when re-indexing;

download and license

You can download mu releases from their download page (Google Code). mu is released under the terms of the GNU General Public License (GPL), version 3 or later.

development

The mu source code is available in Gitorious; get it from there:

  $ git clone git://gitorious.org/mu/mu-ng.git

This is the source code for future versions of mu; there are tags for released versions. If you're not planning on getting involved in the development of mu, it is recommended you use the actual releases. The git version, in particular the 'master' branch, may break at times.

There is now also a mailing list available.

building and dependencies

mu uses GMime 2.4 and Xapian; you'll need to have those installed to build mu. On Debian/Ubuntu, the following should get you all you need:

 # apt-get install libxapian-dev libgmime-2.4-dev

(obviously, you also need the normal build tools; gcc/g++, make and friends).

mu uses autotools, so building follows the normal ./configure/make pattern.

This should work without any problems at least on recent Debian/Ubuntu, for both 32-bit and 64-bit architectures. If it does not work for your distribution, please file a bug with all the error messages, relevant information about your system etc. that you got.

contact

If you think you have found a bug, or you have a good idea for a feature, please put them in the issue list (Google Code).

mu was designed and implemented by me, Dirk-Jan C. Binnema, as a hobby project for my copious free time. It has no relation to my employer.

You can send e-mail to djcb-at-djcbsoftware-dot-nl.