[Xapian-discuss] PHP indexing,
what's the PHP method for indexscript
athlonkmf at yahoo.com
Wed Jan 16 17:58:11 GMT 2008
>Load 5 suggests something's wrong, because dbi2omega and scriptindex
>are both linear processes. Are you running several instances in
>parallel in some way?
it usually starts off fairly low, but then after half an hour of so, it will reach load 5 constantly.
I'm only doing one scriptindex at a time, but it's fairly complicated indexingscript I guess. And of course, the inputfiles are huge, mostly beyond 2GB.
>I believe that right now, none of the supplied Xapian indexing scripts
>or binaries will go significantly above a load of 1, unless you have
>other issues or something else happening on the machine.
The machine is on itself also a webserver, but the normal average load is below 0.25. I can imagine it to get into memory problem when scriptindex tries to parse 2GB worth of data each time.
This is the indexscript i was using.
tid : boolean=Q field=id
pid : unique=Q boolean=Q field=pid
topic_title : unhtml weight=10 field=title index=Z index
forum_id : indexnopos=XFORUMID field=forum_id
postdate : field=startdate
postdateh : field=startdateh date=yyyymmdd value=2
topicStart : field=topicStart
topicStarth : field=topicStarth date=yyyymmdd value=3
author_name: unhtml boolean=XAUTHORNAME field=author
author_id: boolean=XAUTHORID field=authorid
state : field=state
url : field=url
sample : weight=1 index field=sample
Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
More information about the Xapian-discuss