[Xapian-tickets] [Xapian] #346: Python 3 support

[Xapian]

#346: Python 3 support
--------------------------------------+-------------------------------------
 Reporter:  olly                      |       Owner:  richard  
     Type:  defect                    |      Status:  assigned 
 Priority:  highest                   |   Milestone:  1.3.2    
Component:  Xapian-bindings (Python)  |     Version:  SVN trunk
 Severity:  normal                    |    Keywords:           
Blockedby:                            |    Platform:  All      
 Blocking:                            |  
--------------------------------------+-------------------------------------

Comment(by barry):

 Replying to [comment:35 olly]:

 > In response to Barry:
 >
 > >> I guess if you're trying to get everyone onto Python 3 for Ubuntu,
 you've looked at quite a few upstreams already - has a standard pattern
 for resolving such situations already emerged?
 >
 > >> Well, the only upstream I currently have to support is software-
 center, since we're only converting to Python 3 on the standard desktop
 image (for 12.10 anyway). So its use case will be my primary driver. We
 have maybe a dozen reverse depends on python-xapian in total.
 >
 > I meant other upstream Python projects you're needing to get on to
 Python 3, not reverse dependencies of python-xapian (though getting the
 reverse dependencies ported to "python3-xapian" could take significant
 work, especially if we totally throw out compatibility with the current
 python-xapian API).  I was wondering if there was a standard pattern for
 handling wrapping an interface like this.

 Ah, I see what you're getting at.  It's difficult to come up with a one-
 size-fits-all rule because each package has its own unique view of bytes v
 strings.  Some packages have a clear, consistent model, so even if that's
 manifest in a fuzzy way for Python 2, it's not hard to do the right thing
 in Python 3.  Some, like dbus-python have an obvious default once you
 discover it (i.e. I made the wrong choice at first, but it was pretty
 evident by the scope of the changes both to the package and to clients of
 the library that it was the wrong choice; once reverted, it was much
 easier to port things).

 It sounds to me like Xapian has a clear model most of the time, with some
 fuzzy areas, but this information gets lost when mapped to the C++ type,
 so that gets further fuzzy when mapping that single type to its Python
 equivalent.

 Unfortunately, because Python 2 has let us all get away with sloppiness
 for so many years (if you ignore the mystifying UnicodeDecodeErrors in
 non-English domains ;), it gets pretty tricky to clarify these for Python
 3.  Often, you *can't* because things can be either bytes or strings,
 which again, Python 2 let you be sloppy about (not saying that in a
 derogatory way - I'm the biggest culprit of that in my old code :).

 > > One big question is this: what version of Python 2 do you still need
 to support (please tell me, nothing earlier than 2.6 :), and how should we
 handle cases where the API has to change for Python 3?
 >
 > Adding Python 3 support really shouldn't change which Python 2 versions
 are supported.  If the changes are invasive enough that this is really an
 issue, then I suggest we split the Python 3 bindings into a separate
 subdirectory - we already have different versions of all the tests, though
 currently they're mostly the result of 2to3, plus a few tweaks.
 Especially for 1.2.x, we really don't want to be risking breaking Python 2
 support - our general policy is not to break compatibility with a version
 of other software within a Xapian stable release series without a very
 good reason.

 Choosing a minimum Python 2 version does make a difference when
 implementing a single-source bilingual Python layer, which I think works
 best all things considered.  Python 2.6 has *lots* of features for making
 it easier to support both Python 2 and 3 in the same code base.  Python
 2.7 makes it a little bit easier, but usually not enough to recommend it
 over also supporting 2.6.  Supporting 2.5 or earlier makes things
 progressively more difficult with each older version; people have had
 success with that, but mostly for simpler libraries.  I think the Xapian
 bindings are complex enough that it will *probably* be easier to support a
 single codebase with Python   2.6 as the minimum version.  I completely
 understand the constraints on that though.  An example is Twisted, which
 just released 12.1, the last to support anything older than Python 2.6.

 > We've not made a final decision on the versions of things we'll aim to
 support in 1.4.x - 2.6 may well be a sane cut-off there, but that doesn't
 really help since you want this support in 1.2.x.

 We actually have some options here.  Say we targeted a version that was
 not released.  I'd probably take a git/svn snapshot and package that up as
 a separate transitional package.  As I mentioned, my only *requirement*
 right now is software-center, so doing something interim there is fine.
 In fact, even if the API changes once Python 3 support is official,
 updating s-c won't be terribly difficult.  We'll have others that we need
 to support after 12.10, but hopefully by then Xapian's Python 3 API will
 be more stable even if it's not yet released.

 > As for the minimum 3.x to support, I certainly wouldn't worry about 3.0
 - my impression is that the early adopters who actually tried it will have
 quickly moved on to the new cutting edge, while the conservative types
 will have feared the ".0".

 3.0 is an ex-parrot.  I'd say even supporting Python 3.1 is questionable.
 3.2 makes a good baseline and I don't think it will be terribly more
 difficult to support 3.3 once that's released.

-- 
Ticket URL: <http://trac.xapian.org/ticket/346#comment:38>
Xapian <http://xapian.org/>
Xapian