[Xapian-discuss] patch proposal: omindex library or daemon

[Liam]

On Mon, Oct 17, 2011 at 10:25 PM, Liam <xapian at networkimprov.net> wrote:

> Olly (looking at commit logs, I think this is your dept :-)
>
> For apps which re/index files frequently and need format conversion, I'd
> like to propose a patch for one of...
>
> Omindex library (thread safe):
>
> Omindex::init(options) // struct Omindex::options { ... }
>     initialize mime_map, store default options
> session = new Omindex::Session(db_pathname)
>     user threads use different sessions
> session.index_files(list, options) // list & return value are vector of {
> char * url, * file_path, * file_ext }
>     perform a transaction for all files in list; create & return skip_list
> session.index_directory(url, dir_name, options)
>     perform a transaction for all files in a directory tree; return
> skip_list
>
> main() moves to omindex_main.cc
>     process command line, call Omindex::init(), proceed normally
>
> SWIG & Node.js bindings
>
> Omindex daemon mode:
>
> The initial directory pass is optional.
> Listen on a domain socket; for each connection, start a thread with a
> WritableDatabase and read JSON-formatted messages.
> Perform a transaction for each message.
> Respond with a skipped list.
>
> The library is more flexible, I think... Would you accept a patch for this?
>

Hoping for some feedback on this...

Also realized that the Omindex::Session object could be a subclass of
WritableDatabase

My aim here is to leverage the format conversion and other logic in omindex
without requiring a process invocation and a recursive directory scan.