alx/mu4e
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

indexer: make lazy check even lazier

Browse Source 
In lazy-mode, we were skipping directories that did not change; however,
this didn't help for the case were users received new messages in big
maildirs.

So, add another check where we compare the ctime of message files with
the time of the last indexing operation. If it's smaller, ignore the
message-file. This is faster than having to consult the Xapian database
for each message.

Note that this requires in mu4e:
      (setq mu4e-index-lazy-check t)
or
   --lazy-check
as a parameter for 'mu index'.
This commit is contained in:
Dirk-Jan C. Binnema
2024-11-10 13:37:17 +02:00
parent fa59040ebc
commit b0d8d42dd2
3 changed files with 30 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
man/mu-index.1.org
View File

@ -41,7 +41,7 @@ If there is a file called _.noupdate_ in a directory, the contents of that
directory and all of its subdirectories will be ignored. This can be useful to
speed up things you have some maildirs that never change.
_.noupdate_ does not affect already-indexed message: you can still search for
_.noupdate_ does not affect already-indexed messages: you can still search for
them. _.noupdate_ is ignored when you start indexing with an empty database (such
as directly after *mu init*).
@ -58,7 +58,7 @@ the database for which there is no longer a corresponding file in the Maildir.
If you do not want this, you can use *-n*, *--nocleanup*.
When *mu index* catches one of the signals *SIGINT*, *SIGHUP* or *SIGTERM* (e.g., when
you press Ctrl-C during the indexing process), it attempts to shutdown
you press *Ctrl-C* during the indexing process), it attempts to shutdown
gracefully; it tries to save and commit data, and close the database etc. If it
receives another signal (e.g., when pressing Ctrl-C once more), *mu index* will
terminate immediately.
@ -67,12 +67,17 @@ terminate immediately.
** --lazy-check
In lazy-check mode, *mu* does not consider messages for which the time-stamp
(ctime) of the directory they reside in has not changed since the previous
indexing run. This is much faster than the non-lazy check, but won't update
messages that have change (rather than having been added or removed), since
merely editing a message does not update the directory time-stamp. Of course,
you can run *mu-index* occasionally without *--lazy-check*, to pick up such
messages.
(*ctime*) of the directory in which they reside, has not changed since the
previous time this directory was checked.
This is much faster than the non-lazy check, but won't update messages that have
changed (rather than having been added or removed), since merely editing a
message does not update the directory time-stamp. Of course, you can run
*mu-index* occasionally without *--lazy-check*, to pick up such messages.
Furthermore, in lazy-check mode, files which have a *ctime* smaller than the time
the previous indexing operation was completed, are ignored. This helps for the
use-case where new messages can appear in big maildirs.
** --nocleanup
Disable the database cleanup that *mu* does by default after indexing.
@ -172,7 +177,7 @@ The instructions are a little different since we have a proper repeatable
benchmark now. After building,
#+begin_example
$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
$ sudo sh -c 'sync && echo 3 > /proc/sys/vm/drop_caches'
% THREAD_NUM=4 build/lib/tests/bench-indexer -m perf
# random seed: R02Sf5c50e4851ec51adaf301e0e054bd52b
1..1
@ -185,7 +190,7 @@ ok 1 /bench/indexer/4-cores
#+end_example
Things are again a little faster, even though the index does a lot more now
(text-normalizatian, and pre-generating message-sexps). A faster machine helps,
(text-normalization, and pre-generating message-sexps). A faster machine helps,
too!
** recent releases