[Xapian-discuss] creating a database that's compatible with omega

[Olly Betts]

On Wed, Mar 09, 2005 at 07:35:44PM -0500, Sig Lange wrote:
> Great, that clear things up quite a bit. It really is quite unclear to
> someone like me, You say values, so I natually think
> xapian.Document.add_value(..), but infact I read it again and again
> after it didn't work.

When I mentioned "values", I was just correcting Sam's terminology
("keys" are now "values").  But this seems to have confused the issue.

"Values" store bits of data that need to be accessible quickly during
matching.  As Sam said, they're used for things like sorting results,
collapsing similar results, etc.

Things like fields used for displaying a result go in the document data.
Xapian then knows it can store this in a more compact way since access
speed isn't so critical.

> I realized I needed .set_data(). I guess the
> only way to clear this up in the docs is to perhaps say what functions
> are used to set these attributes,

The API docs already cover this, and the method names are the obvious
ones (Document::set_data sets the document data, Document::add_value
adds a value to a document).

> or give examples of what *kind* of data is stored in that particular
> space.

Well, there's already the example of what might be stored in the
document data in the omega's overview.txt...

> What really got me was omega
> seemed to do everything it could to fill in the gaps when displaying
> the query results. setting .set_data() actually set sample=<...> where
> as I never really set sample=<...>.  Some documents had titles but
> most were untitled.

If the document data doesn't have the structure omega expects (the
rather crude test is "does a line have an '=' in?"), then it
tries to give a useful sample and title by making the sample the whole
record, and the title the first line.

This is intended to be helpful and allow you to inspect random databases
using Omega.  But none of the matches will have URLs, so perhaps this
might be better as a separate template using a new $data OmegaScript
command.

Cheers,
    Olly