[Xapian-discuss] Flint failed to deliver indexing performance to Quartz.

[Olly Betts]

On Mon, Jun 18, 2007 at 07:57:45AM +0200, Arjen van der Meijden wrote:
> I haven't looked at the 1.0/1.0.1 release yet, since it doesn't seem to 
> be a drop-in replacement in our situation. Our environment is basically 
> tied to ISO8859-15, at least untill php has native support for utf-8 
> (and php6 is a while away), so it seems necessary to have support for 
> that within omega/xapian.
> Although it is possible to a encoding-layer, tt would be easiest for me 
> if that is controllable using the scriptindex-inputscript and the 
> omega-template.
> 
> Did you (already) have plans to add support for other input/output 
> encodings for omega (and scriptindex)?

See the "Enhancements to Unicode support" bug:

http://www.xapian.org/cgi-bin/bugzilla/show_bug.cgi?id=150

Scriptindex was already there.  I've added a note about Omega.

Perhaps for Omega the neatest solution would be to provide a variant of
$html{] which rewrites all non-ASCII characters as HTML entities, which
then makes the output character set agnostic.  I guess the output would
be rather bulky if you're working with a non-latin script and need to
drop output into an existing page in a different encoding though.

> Anyway, we'll be having a spare database machine shortly on which I can 
> try how much time it takes to reindex using the 0.9-version we use 
> currently and with the latest 1.0-version available at that time.
> And I'll also try and see if I can compile a simple benchmark on those 
> two databases.

That would be useful.

Cheers,
    Olly