[Xapian-discuss] Emtpy records & unique key...

[Olly Betts]

On Wed, May 11, 2005 at 12:04:59AM -0700, arjan holscher wrote:
> I escape newlines as expected, since the documents
> already in the database already do contain spaces in
> the texts with spaces.

OK.

Looking at the code, we don't currently complain if
we find a line without an '=', which we ought to as that
would almost certainly catch anyone who did make this mistake.

> Could it have anything to do with the fact that I pipe
> the buffer at once to scriptindex? I believe my buffer
> is several Mb's of size. Could it help if I split my
> buffer in pieces? Or isn't there a possiblity that
> this will solve my problem?

I can't see how this would make a difference.  The OS
will buffer some, and once the buffer is full, it'll
block write calls to the input end of the pipe until
data has been read from the output end.

You could try writing the output from the generating
script to a file, then indexing from that, but I can't
see how it can make a difference.

> > And there are 14222 documents in the input?
> > 
> > Might be worth checking (with delve from
> > xapian-examples) how many
> > documents are in the database now.
>
> Delve isn't installed on the server omega is running
> on. However I'll try to install it ;)

It's a very handy tool for probing databases.  As of 0.9.0
it will be including in xapian-core so it'll always be
handy.

> So far, it doesn't work as expected and I hope that
> somebody here is able to work out a working solution.

If you can send me (off-list) a dump file and index script
(which I already have in this case I think) which demonstrate
the problem, I can investigate.  Compress the dump file (with
gzip, bzip2, or zip) please.

Cheers,
    Olly