[Xapian-discuss] Search::Xapian add_database'd search results are odd?

Tue Dec 21 21:32:13 GMT 2004

Sorry if this is the wrong forum to discuss Search::Xapian issues -- 
this just seems like the best place..

Anyways, I've been testing out using $db->add_database() when searching, 
and it seems like the docids I'm getting out of it are incorrect, almost 
as though they're "double" what they should be (numerically)...

the docids that exist should be around 950,000 and 1000000 not around 
1900000, etc...

$xapiandirbase . '-11'
and
$xapiandirbase . '-10' both exist.

quick example:
===================================================================
my $db = Search::Xapian::Database->new($xapiandirbase . '-11' || 
die("Error.\n"));
my $db->add_database(Search::Xapian::Database->new($ARGV[0] . '-10' || 
die("Error.\n")));

my $query = Search::Xapian::Query->new(OP_AND, 'word', 'word2');
print "Query: " . $query . "\n";
my $enq = $db->enquire($query);

my @matches = $enq->matches(0,1000000);
foreach my $match ( @matches ) {
    print $match->get_docid . ',';
}
===================================================================

If I don't use add_database, or add_database is wrapped in eval and 
fails due to me pointing to a xapian db that doesn't exist, and either 
query xapian db "-10" or "-11", I get docids returned in the 
900000-1000000 range approximately.

If I use add_database() in either order (-10 then -11, or vice versa) 
then I get ids seemingly doubled.

-10 and -11 have unique markers in them (M10 and M11 respectively), so 
if I use one of the markers as the keywords, I get the same number of 
results whether I add on the 2nd database or not.
I just get results that are seemingly approximately doubled?

I'm running Search::Xapian 0.8.4.0 and xapian 0.8.4:
# rpm -qa | grep xapian
xapian-core-0.8.4-1
xapian-core-libs-0.8.4-1
xapian-core-debuginfo-0.8.4-1
xapian-core-devel-0.8.4-1

Thanks for any help you can offer,
Eric
-------------- next part --------------
xapian-read-test.pl in original form, reading domain.com-2004-11, no add_database:

# ./xapian-read-test.pl /data1/xapian/domain.com-2004 M11 warren shannon
Query: Xapian::Query((M11 AND warren AND shannon))
Parsing query 'Xapian::Query((M11 AND warren AND shannon))'
611 results found
{975786,975788,976495,976496,976603,976604,976605,976653,976654,976655,976656,977247,978602,978603,978604,982193,982194,982195,982196,982237,982883,983505,983521,983522,983538,983539,983540,984358,984366,984367,984768,984769,984782,984924,987743,988632,988633,988634,988637,988638,988639,990152,990864,990865,990866,993950,993959,996317,996318,996319,996429,996430,996431,996503,996504,996505,996594,996595,996596,996981,1000204,1000205,1000215,1000286,1000287,1000288,1000340,1000382,1000433,1000434,1000435,1000867,1000868,1000869,977244,977245,977246,990160,975793,982927,982928,982929,994283,994284,994285,983865,983866,983867,988662,988663,988664,983762,983763,983764,977138,994079,977164,977165,977166,977167,984994,984995,984996,1001216,1001217,1001218,996426,984305,984306,984307,984352,984353,984354,996397,1000197,982933,982934,982935,1000211,984361,984362,984363,978573,989961,989962,989963,989322,994093,997031,984341,984342,983550,983551,983552,983553,989326,989327,989328,978580,983565,984826,984827,984828,994825,994826,994827,984990,984991,984992,1001031,983883,983884,983885,997075,997076,997077,984372,984373,984374,978618,996955,996956,996957,975801,975932,978553,978554,978555,982512,984902,984903,1000267,1000268,1000269,976616,976617,983819,983820,983821,989511,989512,989513,996964,996965,996966,984315,984843,984844,984845,987788,987789,987790,989466,989467,989468,977965,1000377,978000,978001,978002,978715,978716,978717,978718,978719,978720,978721,978722,978723,984864,984865,984866,977925,982923,982924,982925,995436,995437,995438,988400,988401,988402,990791,990792,976607,976608,976609,983004,995442,996973,996974,996975,984927,984928,984931,991434,991435,991436,984926,989351,989352,989353,983104,984929,984919,977517,978703,978704,978705,988537,988538,988539,993738,993739,984870,984871,984872,988783,988784,988785,989340,989341,989342,982930,982931,982932,984950,984951,984471,996214,996215,996216,996223,996224,996225,996219,996220,996221,996279,996280,996281,993758,993759,993760,993955,993956,993957,977858,977859,977860,994206,990079,990080,990081,990082,1000912,984887,984888,984889,1000929,983298,984487,984488,984489,989313,989314,989315,977864,977865,977866,1001133,975809,975810,975811,984777,984778,984779,978812,996978,975830,977936,995465,975828,978003,978004,978005,978817,988677,978825,973971,988680,975779,993839,993840,993841,983027,983028,983029,983030,977872,993860,993862,978828,990616,975822,978031,978874,978875,978876,978837,978838,978839,987814,987815,987816,988682,977206,983609,988689,988690,988691,977526,1000880,1000881,1000882,978801,983617,983618,983619,983620,990166,990167,990168,978808,978809,978810,987874,987875,987876,990170,990171,990172,978822,978823,978824,983640,990656,975974,983592,983593,983594,983595,983683,983684,983685,1000233,1000234,1000235,995500,995501,995502,995503,988613,988614,988615,996038,978831,978832,978833,987725,987726,990173,990174,990175,984878,984879,984880,984881,999531,988081,988082,988083,975829,989589,989590,989591,975676,975677,975678,975790,975791,975792,990176,996688,976048,976049,976050,989443,989444,989445,978853,987745,987746,987747,1000221,1000222,1000223,982164,982165,982166,983622,983623,983624,984144,1000650,985013,985014,985015,990408,995177,998011,984632,985018,985019,985020,982167,975883,991023,979094,985021,985022,985023,982187,982188,982189,978305,988266,982213,982214,982215,978007,978008,978009,1001188,1001189,1001190,978527,978528,978529,994592,994593,994594,990562,985184,986102,978532,978533,978534,988549,978624,978625,978626,988563,982299,982300,982301,987888,987889,989400,989401,989402,994539,994540,994541,1000225,1000226,1000227,988672,988673,988674,991809,991810,991811,976096,976097,976098,976103,976104,976105,976107,976108,976109,984382,984383,984384,975910,975911,975912,1000900,975918,975919,975920,975938,975939,975940,977326,990031,990032,990033,1001446,987948,990083,990084,990085,990182,990183,990184,990189,990190,990191,990199,990200,990201,1001255,990207,990208,990209,990090,990091,990092,990194,990195,990196,990692,990216,983571,983572,983573,1000361,1000362,976759,976760,976761,993181,995821,995822,995824,975453,987530,981892,995888,995889,995890,996415,996416,996417,978841,978842,978843,996442,996443,996444,978863,978864,978865,995893,995894,995895,995903,995904,995905}
Elapsed time: 0.0667 seconds.

# vi xapian-read-test.pl
xapian-read-test.pl modified to add-database domain.com-2004-10:

# ./xapian-read-test.pl /data1/xapian/domain.com-2004 M11 warren shannon
Query: Xapian::Query((M11 AND warren AND shannon))
Parsing query 'Xapian::Query((M11 AND warren AND shannon))'
611 results found
{1951571,1951575,1951585,1952989,1952991,1953205,1953207,1953209,1953305,1953307,1953309,1953311,1954487,1954489,1954491,1954493,1957203,1957205,1957207,1964385,1964387,1964389,1964391,1964473,1965765,1965853,1965855,1965857,1967009,1967041,1967043,1967075,1967077,1967079,1968715,1968731,1968733,1969535,1969537,1969563,1969847,1975485,1977263,1977265,1977267,1977273,1977275,1977277,1980303,1980319,1981727,1981729,1981731,1987899,1987917,1992633,1992635,1992637,1992857,1992859,1992861,1993005,1993007,1993009,1993187,1993189,1993191,1993961,2000407,2000409,2000429,2000571,2000573,2000575,2000679,2000763,2000865,2000867,2000869,2001733,2001735,2001737,1988565,1988567,1988569,1967729,1967731,1967733,1977323,1977325,1977327,1967523,1967525,1967527,1954275,1988157,1954327,1954329,1954331,1954333,1969987,1969989,1969991,2002431,2002433,2002435,1992851,1968609,1968611,1968613,1968703,1968705,1968707,1992793,2000393,1965865,1965867,1965869,2000421,1968721,1968723,1968725,1957145,1979921,1979923,1979925,1978643,1988185,1994061,1968681,1968683,1967099,1967101,1967103,1967105,1978651,1978653,1978655,1957159,1967129,1969651,1969653,1969655,1989649,1989651,1989653,1969979,1969981,1969983,2002061,1967765,1967767,1967769,1994149,1994151,1994153,1968743,1968745,1968747,1957235,1993909,1993911,1993913,1951601,1951863,1957105,1957107,1957109,1965023,1969803,1969805,2000533,2000535,2000537,1953231,1953233,1967637,1967639,1967641,1979021,1979023,1979025,1993927,1993929,1993931,1968629,1969685,1969687,1969689,1975575,1975577,1975579,1978931,1978933,1978935,1955929,2000753,1955999,1956001,1956003,1957429,1957431,1957433,1957435,1957437,1957439,1957441,1957443,1957445,1969727,1969729,1969731,1955849,1965845,1965847,1965849,1990871,1990873,1990875,1976799,1976801,1976803,1981581,1981583,1953213,1953215,1953217,1966007,1990883,1993945,1993947,1993949,1969853,1969855,1969861,1982867,1982869,1982871,1969851,1978701,1978703,1978705,1966207,1969857,1969837,1955033,1957405,1957407,1957409,1977073,1977075,1977077,1987475,1987477,1969739,1969741,1969743,1977565,1977567,1977569,1978679,1978681,1978683,1965859,1965861,1965863,1969899,1969901,1968941,1992427,1992429,1992431,1992445,1992447,1992449,1992437,1992439,1992441,1992557,1992559,1992561,1987515,1987517,1987519,1987909,1987911,1987913,1955715,1955717,1955719,1988411,1980157,1980159,1980161,1980163,2001823,1969773,1969775,1969777,2001857,1966595,1968973,1968975,1968977,1978625,1978627,1978629,1955727,1955729,1955731,2002265,1951617,1951619,1951621,1969553,1969555,1969557,1957623,1993955,1951659,1955871,1990929,1951655,1956005,1956007,1956009,1957633,1977353,1957649,1947941,1977359,1951557,1987677,1987679,1987681,1966053,1966055,1966057,1966059,1955743,1987719,1987723,1957655,1981231,1951643,1956061,1957747,1957749,1957751,1957673,1957675,1957677,1975627,1975629,1975631,1977363,1954411,1967217,1977377,1977379,1977381,1955051,2001759,2001761,2001763,1957601,1967233,1967235,1967237,1967239,1980331,1980333,1980335,1957615,1957617,1957619,1975747,1975749,1975751,1980339,1980341,1980343,1957643,1957645,1957647,1967279,1981311,1951947,1967183,1967185,1967187,1967189,1967365,1967367,1967369,2000465,2000467,2000469,1990999,1991001,1991003,1991005,1977225,1977227,1977229,1992075,1957661,1957663,1957665,1975449,1975451,1980345,1980347,1980349,1969755,1969757,1969759,1969761,1999061,1976161,1976163,1976165,1951657,1979177,1979179,1979181,1951351,1951353,1951355,1951579,1951581,1951583,1980351,1993375,1952095,1952097,1952099,1978885,1978887,1978889,1957705,1975489,1975491,1975493,2000441,2000443,2000445,1964327,1964329,1964331,1967243,1967245,1967247,1968287,2001299,1970025,1970027,1970029,1980815,1990353,1996021,1969263,1970035,1970037,1970039,1964333,1951765,1982045,1958187,1970041,1970043,1970045,1964373,1964375,1964377,1956609,1976531,1964425,1964427,1964429,1956013,1956015,1956017,2002375,2002377,2002379,1957053,1957055,1957057,1989183,1989185,1989187,1981123,1970367,1972203,1957063,1957065,1957067,1977097,1957247,1957249,1957251,1977125,1964597,1964599,1964601,1975775,1975777,1978799,1978801,1978803,1989077,1989079,1989081,2000449,2000451,2000453,1977343,1977345,1977347,1983617,1983619,1983621,1952191,1952193,1952195,1952205,1952207,1952209,1952213,1952215,1952217,1968763,1968765,1968767,1951819,1951821,1951823,2001799,1951835,1951837,1951839,1951875,1951877,1951879,1954651,1980061,1980063,1980065,2002891,1975895,1980165,1980167,1980169,1980363,1980365,1980367,1980377,1980379,1980381,1980397,1980399,1980401,2002509,1980413,1980415,1980417,1980179,1980181,1980183,1980387,1980389,1980391,1981383,1980431,1967141,1967143,1967145,2000721,2000723,1953517,1953519,1953521,1986361,1991641,1991643,1991647,1950905,1975059,1963783,1991775,1991777,1991779,1992829,1992831,1992833,1957681,1957683,1957685,1992883,1992885,1992887,1957725,1957727,1957729,1991785,1991787,1991789,1991805,1991807,1991809}
Elapsed time: 0.0847 seconds.