[Xapian-tickets] [Xapian] #180: Add support for CJK text to queryparser and termgenerator

Xapian nobody at xapian.org
Tue Jul 19 11:34:03 BST 2011


#180: Add support for CJK text to queryparser and termgenerator
-------------------------+--------------------------------------------------
 Reporter:  richard      |        Owner:  richard  
     Type:  enhancement  |       Status:  assigned 
 Priority:  normal       |    Milestone:  1.2.x    
Component:  QueryParser  |      Version:  SVN trunk
 Severity:  normal       |   Resolution:           
 Keywords:               |    Blockedby:           
 Platform:  All          |     Blocking:           
-------------------------+--------------------------------------------------

Comment(by bschaefer):

 I took over kamstrup branch and the main focus I gathered from him and
 your points were:

 - Avoiding any pre-scanning

 - Using the Utf8Iterators for the CJk tokenizing

 - Being able to switch back and forth

 - If all normal text no efficiency cost to having the CJK which the CJK
 checking is constant

 Here my branch with trying to keep all those in mind:
 https://code.launchpad.net/~brandontschaefer/xapian/cjk-support-patch

 Also a patch.

-- 
Ticket URL: <http://trac.xapian.org/ticket/180#comment:16>
Xapian <http://xapian.org/>
Xapian



More information about the Xapian-tickets mailing list