[Xapian-discuss] Emtpy records & unique key...

[arjan holscher]

Hey readers,

Currently I have been working on a indexing systeem
using  scriptindex and a search tool using omega. The
general idea behind the system is that it will index
various sections of the site in one big omega
database. For example, there is a news section and a
review section. These are all to be stored in 1 big
database.

First, my scriptindex script:

internal	:	boolean=Q unique=Q field=internal
titel		: 	unhtml index=XC weight=3 index field=caption
topicstart	:	unhtml index=XA weight=2 index
truncate=200 field=sample
bericht		:	unhtml index
type		:	boolean=XT
url		:	field=url
itemid		:	field=itemid
itemtype	:	field=itemtype
topicstarter	:	boolean=XS
poster		:	boolean=XP

Most of them are pretty easy. Although the internal
field needs a little explenation. I use a section ID
for each section. I multiply this with 1 million and
add the actual database row id to this. This way my
article always has the same unique ID. 

Now for my problems :D There are 2:

- Some of the records in the omega database simply do
not return data. They do not contain document data
although I'm absolutely sure I do not deliver empty
documents to scriptindex. Under what conditions is it
possible for scriptindex to 'discard' a document.

- The second issue is indexing. The first time I index
all the documents I get 14222 added documents. This is
the correct number since it's a new database.

When I want to re-index the database I just throw all
the documents again at the database and I'd expected
to get 14222 updated documents (assuming no documents
are added during the index periods). However
scriptindex returns 2/3th of the total documents as
added and 1/3th of the documents as updated.

However I want ALL the documents to be updated ... not
added. I thought adding the unique field would solve
it. However this is not the case.

If anybody can help me, please do so :>

Thx in advance, 

Arjan Holscher


		
__________________________________ 
Yahoo! Mail Mobile 
Take Yahoo! Mail with you! Check email on your mobile phone. 
http://mobile.yahoo.com/learn/mail