[Xapian-discuss] How to update DB concurrently?

[Olly Betts]

On Thu, May 18, 2006 at 04:39:36PM -0800, oscaruser at programmer.net wrote:
> <       for (int index = 0; index < 150; index++) {
> <       std::ostringstream s;
> <       s << "/svr/hda1/omega/data/mydb" << std::setfill('0') << std::setw(4) << index
> <         << "/default";
> <       //cout << s.str() << endl;
> <       db.add_database(Xapian::Database(s.str()));
> <       }

There's no context, so I can't see where you're patching that, but it
looks plausible.

However, I don't think you want to search 150 databases at once like
this.  The overhead from opening that many databases on every search is
likely to be noticable, and searching one big database will be more
efficient.

Instead use xapian-compact to merge them all together like so:

xapian-compact -F -m /svr/hda1/omega/data/mydb*/default /path/to/output/db

And then search the merged database '/path/to/output/db'.

You almost certainly want to use -m if you can - it merges in multiple
passes which for 150 databases should be substantially quicker.  It's
probably worth using -F too unless you want to update the merged
database by opening it as a WritableDatabase.

Cheers,
    Olly