Pull requests: CJK words and Snippet generator

[rsto at paranoia.at]

Hi,

On Fri, Jul 29, 2016, at 13:45, James Aylett wrote:
> On Fri, Jul 29, 2016 at 12:12:25PM +0200, rsto at paranoia.at wrote:
> > The FastMail snippet generator has been written when MSet didn't create
> > snippets. I'll first compare both implementations to see if there is a
> > good reason for them to coexist, or might just as well merge any
> > additional features into MSet.

I created pull request https://github.com/xapian/xapian/pull/117 which
updates MSet to optionally calculate snippet relevances based on query
term coverage. That's similar to how it is done in the latest version of
the FastMail snippet generator. I'd be happy to migrate the remaining
features of the FastMail snippet generator into MSet, but it might be
good to do so feature by feature.

Oddly enough, the pull request causes Travis to break for clang but not
for gcc [1]. That's because the clang build process fails for the test
'querypairwise1' [2], which AFAIK I didn't touch at all.  Is that a
known issue or did I break anything?

[1] https://travis-ci.org/xapian/xapian/builds/149512190
[2] https://travis-ci.org/xapian/xapian/jobs/149512191#L15051

> > I've just opened a pull request for the CJK tokenizer:
> > https://github.com/xapian/xapian/pull/114
> > Unfortunately, Travis breaks since pkg-config can't find libicu on the
> > machine [1]. 
> You should be able to install it; if you add libicu-dev to the
> packages stanza in .travis.yml it will put it in there.
> However you seem to be using pkg-config, which Ubuntu 12.04 LTS (which
> travis currently uses) doesn't provide for libicu. 14.04 LTS does, and
> it's possible to use that as a beta with travis, I think by changing:

I'm a bit a loss here. I can't get it to build on both 12.04 LTS and
14.04 LTS. On 12.04 LTS Travis breaks due to pkg-config missing [3]. On
14.04 LTS the build breaks due to Python missing the sphinx package [4].
That's probably a Python version 2 vs 3 environment issue.

[3] https://travis-ci.org/xapian/xapian/jobs/149516398#L1530
[4] https://travis-ci.org/xapian/xapian/jobs/149516019#L4262

The only way is see is to get rid of the pkg-config checks in
configure.ac and check in configure.ac for libicu headers and libraries.
Or do you plan to use pkg-config in xapian-core and rather make this
work with Travis?

Cheers,
Robert