This change makes index cleanup ~4x faster by changing how we
determine whether a file mentioned by the database still exists on
disk. Previously, we'd call access(2) for each file the database
mentioned. Doing so produced a lot of system call overhead. Now, we
read the directory entries of the directories containing the files
whose existence we're checking, build a hash table from what we find,
then do the existence check against this hash table instead of
entering the kernel.
The semantics of the cleanup check do change subtly, however.
Previously, we checked whether the mentioned file was *readable*.
Now we check merely that it exists. Extant but unreadable files in
maildirs should be rare.
BEFORE:
$ time mu index --lazy-check
lazily indexing maildir /home/dancol/Mail -> store /home/dancol/.cache/mu/xapian
/ indexing messages; checked: 0; updated/new: 0; cleaned-up: 0
real 0m19.310s
user 0m1.803s
sys 0m12.999s
AFTER:
$ time mu --debug index --lazy-check
lazily indexing maildir /home/dancol/Mail -> store /home/dancol/.cache/mu/xapian
- indexing messages; checked: 0; updated/new: 0; cleaned-up: 0
real 0m4.584s
user 0m2.433s
sys 0m2.133s
cdb619e4f5