b75f9f508b
mu's query parser is the piece of software that turns your queries
into something the Xapian database can understand. So, if you query
"maildir:/inbox and subject:bla" this must be translated into a
Xapian::Query object which will retrieve the sought after messages.
Since mu's beginning, almost a decade ago, this parser was based on
Xapian's default Xapian::QueryParser. It works okay, but wasn't really
designed for the mu use-case, and had a bit of trouble with anything
that's not A..Z (think: spaces, special characters, unicode etc.).
Over the years, mu added quite a bit of pre-processing trickery to
deal with that. Still, there were corner cases and bugs that were
practically unfixable.
The solution to all of this is to have a custom query processor that
replaces Xapian's, and write it from the ground up to deal with the
special characters etc. I wrote one, as part of my "future, post-1.0
mu" reseach project, and I have now backported it to the mu 0.9.19.
From a technical perspective, this is a major cleanup, and allows us
to get rid of much of the fragile preprocessing both for indexing and
querying. From and end-user perspective this (hopefully) means that
many of the little parsing issues are gone, and it opens the way for
some new features.
From an end-user perspective:
- better support for special characters.
- regexp search! yes, you can now search for regular expressions, e.g.
subject:/h.ll?o/
will find subjects with hallo, hello, halo, philosophy, ...
As you can imagine, this can be a _heavy_ operation on the database,
and might take quite a bit longer than a normal query; but it can be
quite useful.
96 lines
2.5 KiB
C++
96 lines
2.5 KiB
C++
/*
|
|
** Copyright (C) 2017 Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>
|
|
**
|
|
** This library is free software; you can redistribute it and/or
|
|
** modify it under the terms of the GNU Lesser General Public License
|
|
** as published by the Free Software Foundation; either version 2.1
|
|
** of the License, or (at your option) any later version.
|
|
**
|
|
** This library is distributed in the hope that it will be useful,
|
|
** but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
** MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
|
|
** Lesser General Public License for more details.
|
|
**
|
|
** You should have received a copy of the GNU Lesser General Public
|
|
** License along with this library; if not, write to the Free
|
|
** Software Foundation, 51 Franklin Street, Fifth Floor, Boston, MA
|
|
** 02110-1301, USA.
|
|
*/
|
|
|
|
#include <vector>
|
|
#include <glib.h>
|
|
|
|
#include <iostream>
|
|
#include <sstream>
|
|
|
|
#include "parser.hh"
|
|
using namespace Mux;
|
|
|
|
struct Case {
|
|
const std::string expr;
|
|
bool is_first;
|
|
const std::string expected;
|
|
};
|
|
using CaseVec = std::vector<Case>;
|
|
using ProcFunc = std::function<std::string(std::string, bool)>;
|
|
|
|
|
|
static void
|
|
test_cases(const CaseVec& cases, ProcFunc proc)
|
|
{
|
|
for (const auto& casus : cases ) {
|
|
|
|
const auto res = proc(casus.expr, casus.is_first);
|
|
if (g_test_verbose()) {
|
|
std::cout << "\n";
|
|
std::cout << casus.expr << ' ' << casus.is_first << std::endl;
|
|
std::cout << "exp:" << casus.expected << std::endl;
|
|
std::cout << "got:" << res << std::endl;
|
|
}
|
|
|
|
g_assert_true (casus.expected == res);
|
|
}
|
|
}
|
|
|
|
static void
|
|
test_date ()
|
|
{
|
|
g_setenv ("TZ", "Europe/Helsinki", TRUE);
|
|
|
|
CaseVec cases = {
|
|
{ "2015-09-18T09:10:23", true, "001442556623" },
|
|
{ "1972-12-14T09:10:23", true, "000093165023" },
|
|
{ "1854-11-18T17:10:23", true, "000000000000" },
|
|
{ "fnorb", true, "000000000000" },
|
|
{ "fnorb", false, "999999999999" },
|
|
{ "", false, "999999999999" },
|
|
{ "", true, "000000000000" }
|
|
};
|
|
|
|
test_cases (cases, [](auto s, auto f){ return date_to_time_t_string(s,f); });
|
|
}
|
|
|
|
static void
|
|
test_size ()
|
|
{
|
|
CaseVec cases = {
|
|
{ "456", true, "0000000456" },
|
|
{ "", false, "9999999999" },
|
|
{ "", true, "0000000000" },
|
|
};
|
|
|
|
test_cases (cases, [](auto s, auto f){ return size_to_string(s,f); });
|
|
}
|
|
|
|
|
|
int
|
|
main (int argc, char *argv[])
|
|
{
|
|
g_test_init (&argc, &argv, NULL);
|
|
|
|
g_test_add_func ("/utils/process-date", test_date);
|
|
g_test_add_func ("/utils/process-size", test_size);
|
|
|
|
return g_test_run ();
|
|
}
|