[Xapian-discuss] Re: Big process using Xapian

[Rafael "SDM" Sierra]

On 2/2/07, James Aylett <james-xapian at tartarus.org> wrote:
>
> On Fri, Feb 02, 2007 at 07:17:04AM -0800, Rafael SDM Sierra wrote:
>
> > >[1] - 736M   694M biord  182:27  2.15% python
> >
> > I change from 1000 to 10000 the xapian flush threshold, and the process
> > become bigger oO...
> >
> > 2008M  1504M swread  10:36  0.00% python
> >
> > It's all that I have of memory (2GB), my swap is in use now...
>
> Some systems cannot free main memory from the process back to the
> operating system, even if it is unused within the process.


Yes, to avoid it, I will try close the connection  after X process, and see
what happen ;P

The larger the flush threshold, the more data has to be held in
> memory, so this might explain what you're seeing. You need to keep the
> flush threshold low enough that you have head space above the process
> within core (physical memory),


Closing and opening database periodically maybe "resolve" this problem of
memory

to enable some sort of write-behind
> buffer at the operator system level (although I *believe* that with
> Xapian's access pattern while writing it won't make a huge amount of
> difference with most modern disks, but that's getting into very dicey
> territory trying to claim that with confidence :-).
>
> (This may not actually be your problem in this case, of course...)


Sorry, My english isn't good enough to understand what you say.. xD

-- 
SDM Underlinux
http://stiod.com.br
Membro da equipe UnderLinux
--
PEP-8
There is only 2 kinds of peoples in the world, who know English, and me. oO