[Xapian-discuss] Clearing Database For Tests

Richard Boulton richard at tartarus.org
Tue Nov 3 17:34:38 GMT 2009


2009/11/3 Ryan Bates <ryan at railscasts.com>:
> Thanks Olly,
>
> However, even with Xapian 1.0.16 I'm not seeing much improvement when
> interacting through the Ruby bindings. It's at least 0.35 seconds
> still to generate the database.
>
> rm -rf tmp.db
> time ruby -e "require 'xapian'; Xapian::WritableDatabase.new('tmp.db',
> Xapian::DB_CREATE)"
> # 0.408 total

For the same command, I'm getting around 0.03 seconds.

One possibility which occurs to me is that xapian will perform some
fsync (or fdatasync) calls after creating the database.  On some
filesystems, this will involve syncing a large amount of unrelated
data to the physical disk: I understand ext3 has this problem, for
example, since fsync will sync the entire journal to disk.  If there's
lots of other activity going on on your system, this could be the
problem.

What OS / filesystem are you using?

An easy way to see if this is the problem, without installing lots of
profiling tools, is to do:

rm -rf tmp.db;strace -T -o strace.out ruby -e "require 'xapian';
Xapian::WritableDatabase.new('tmp.db', Xapian::DB_CREATE)"

And then do "grep fsync strace.out".

I get:

fsync(3)                                = 0 <0.004825>
fsync(6)                                = 0 <0.000917>
fsync(7)                                = 0 <0.000795>

(The numbers in angle brackets are the time for each call.)

If that's not the problem, you could look through the strace output
for any other clues.

-- 
Richard



More information about the Xapian-discuss mailing list