[Xapian-discuss] C++ PostingSource with Python bindings

[Olly Betts]

On Fri, Aug 31, 2012 at 08:23:06PM -0400, Jason Tackaberry wrote:
> I want to use a custom PostingSource to adjust weightings of my search  
> results, but an implementation in Python is proving to be very slow for  
> large result sets.  So I'm looking to write the custom PostingSource in 
> C++.

Ideally there would just be an easy way to wrap extra C++ classes for
Python (and other languages) such that they work as if they were part
of the Xapian API.

It should be possible to wrap classes in this way, but there's not
an easy way to do it currently.  So your approach is probably simpler
to implement right now.

> The basic strategy would be:
>
> q = xapian.Query(...)
> q = mycustommodule.add_posting_source(q)
>
> I can acquire the Query pointer easily enough (it's just int(q.this))  
> and cast that to Xapian::Query, and then construct a new Xapian::Query  
> with the custom posting source mixed in with OP_AND_MAYBE.
>
> The challenge is getting the new C++ query object back into Python  
> space.  SWIG documents say that although int(swigobj) gives you the  
> underlying pointer, the inverse operation of constructing a SWIG object  
> from a pointer isn't possible. [1]  I'm hoping there might be some  
> Xapian-specific way, or that this documented limitation doesn't apply to  
> native C/C++ code.
>
> Is there a solution, or some alternative approach?

I'd try making the way you'd call this from Python:

    q = xapian.Query(...)
    mycustommodule.add_posting_source(q)

And then in the C++ code, construct your new Query object and swap it
with the old one:

    Xapian::Query * q = (Xapian::Query*)(...);
    Xapian::Query q_new(...);
    std::swap(q_new, *q);

Query objects are reference counted handles, so I think that should make
q in the python code refer to the new object.

If not, then you could try (though this isn't part of the public API so
might not work in the future even if it does now):

    q->internal = q_new.internal;

Cheers,
    Olly