[Xapian-tickets] [Xapian] #180: Add support for CJK text to queryparser and termgenerator

[Xapian]

#180: Add support for CJK text to queryparser and termgenerator
-------------------------+--------------------------------------------------
 Reporter:  richard      |        Owner:  richard  
     Type:  enhancement  |       Status:  assigned 
 Priority:  high         |    Milestone:  1.2.x    
Component:  QueryParser  |      Version:  SVN trunk
 Severity:  normal       |   Resolution:           
 Keywords:               |    Blockedby:           
 Platform:  All          |     Blocking:           
-------------------------+--------------------------------------------------

Comment(by kamstrup):

 I've taken rboulton's branch from github and started to massageit into
 Xapian trunk. To add some some confusion to our VCS party you can find it
 in a bzr repo here https://code.launchpad.net/~kamstrup/xapian/cjk-
 support. As of writing it's still just baby steps - so nothing to see yet.
 When it is ready I'll post a patch here for those with bzr allergies :-)

 I've extracted what I could from Olly's comments into the following
 battleplan:

  * Include the module in a new subdir xapian-code/queryparser/cjk/
 (nothing in include/ until the API is acked)

  * Get rid of the excessive copying by using Utf8Iterator and on-the-fly
 conversions

  * Rely solely on Xapian's unicode handling (iow no new deps)

  * License wise - we talking X11/MIT licensed code. Maybe we can talk
 Richard and Yung-chung Lin into dual licensing under GPL as well as a stop
 gap measure?

-- 
Ticket URL: <http://trac.xapian.org/ticket/180#comment:12>
Xapian <http://xapian.org/>
Xapian