[Xapian-discuss] Converting MySQL database to Xapian

[Michael Lewis]

Simon,

Thanks for the reply. I believe I can handle the migration. I am really just trying to figure out how to set up xapian so that I can access multiple remote databases for a single search.

For example, let's say the database is broken up into 100 segments. Each segment containing 200,000 documents. These segments are stored on a single server (although at some point in the future multiple servers will contain the segments as the db grows). Let's call this server: serverX. ServerX will be accessed from the Web Server and ServerY which contains various PHP programs to perform searches.

My first question is: On ServerY and the Web Server how do I set-up Xapian for access. In the PHP code do I have to actually know where the databases are located or is that the function of some kind of stub database? In an ideal world the stub database file would contain a xapian db segment name and a host address and port where the actual segment resides.

My second question is: What do I run on the ServerX listening on the port? I've seen two references xapian-tcpsrv and something else. It would appear that tcpsrv would be the server of choice, but I'm not sure.

Is there anything special that I have to do on the clients to access ServerX?

Thanks,

Michael

-----Original Message-----
From: simon.roe at gmail.com [mailto:simon.roe at gmail.com] On Behalf Of Sym Roe
Sent: Friday, April 26, 2013 9:01 AM
To: Michael Lewis
Cc: xapian-discuss at lists.xapian.org
Subject: Re: [Xapian-discuss] Converting MySQL database to Xapian

On Thu, Apr 25, 2013 at 10:56 PM, Michael Lewis <mal at icginc.com> wrote:
> I would appreciate any help

So I guess it's hard to help with such a large problem here, with little specific questions.

If I were in your position I would ignore the migration phase for the time being, and assume there will be ways to do it later.

Then, with some test data you should recreate what you're trying to do and test it.  I guess you will have questions about mulit-db support and sharding, etc, but searching back through this mailing list is normally really helpful when the docs fail you.

It sounds like you have the sharding problem solved though, as your "sub databases" (I don't really know what that means), are effectively shards already.

Xapian is very flexible, so from what you've said I think it will all be possible.  It's really hard to give any more help without getting involved properly in your project, or without you breaking it down in to smaller questions about any problems you've found.

Sym

--
E: sym.roe at talusdesign.co.uk
M: 07742079314
@symroe