[Xapian-discuss] Problem getting Xapian working with Burmese

[Olly Betts]

On Tue, Feb 02, 2010 at 10:44:37AM +0100, emmanuel at engelhart.org wrote:
> Le dim 31/01/10 23:53, "Olly Betts" olly at survex.com a écrit:
> > http://trac.xapian.org/ticket/355
> 
> Indeed, this seems to be the issue.
> I have made a test against the dev. source code and it works better (less
> cuts in the words).

Good, thanks for checking.

> > The second issue in your case is that there are zero-width space
> > characters in there as well, which currently act as word breaks.  These are
> > present to indicate acceptable places to split a word when wrapping text,
> > so we should ideally just strip them out when generating terms.
>
> Ok, so that may explain why they are still cuts in the words (also with the
> dev. code).
> 
> Do I have to open a bugs for that ?
> Do they exist plan to fix that ?

I didn't bother opening a ticket, as it's a quick change.  It's now addressed
on trunk by r13921.  If you could test that and see if it works better for you,
that would be great.

For 1.0.x, I think all we can do is to make these characters phrase generators,
or else we introduce an incompatibility with existing databases.

Cheers,
    Olly