support xapian ngrams

Xapian supports an "ngrams" option to help with languages/scripts
without explicit wordbreaks, such as Chinese / Japanese / Korean.

Add some plumbing for supporting this in mu as well. Experimental for
now.
This commit is contained in:
Dirk-Jan C. Binnema
2023-09-09 11:57:05 +03:00
parent f6122ecc9e
commit 264bb092f0
20 changed files with 207 additions and 81 deletions

View File

@ -90,6 +90,14 @@ handle_result(const Result<void>& res, const Mu::Options& opts)
int
main(int argc, char* argv[])
{
/*
* We handle this through explicit options
*/
g_unsetenv("XAPIAN_CJK_NGRAM");
/*
* set up locale
*/
setlocale(LC_ALL, "");
/*