[Xapian-discuss] index size

[Olly Betts]

On Mon, Nov 17, 2014 at 03:09:21PM +0000, Mir Siadaty wrote:
> as a sanity checkI understand it is within expectable range to start
> with~9GB data, make chert index of about 19G, where compact –F reduces
> it to about17G.Or does the above case point to some error or
> inefficiency,which might be corrected?Mir

It's larger than I'd expect from the default chert backend in 1.2, but
it's not implausible, especially if your documents are individually on
the small side.

But without knowing how you're indexing each document, it's hard to say
if you're doing something which is causing the database to be larger
than necessary.

If you aren't planning to update the database (which I'm guessing may
be the case because you passed "-F" to xapian-compact, which the help
notes is "not recommended if you plan to update the compacted database")
you can probably delete termlist.* from the database directory - see
http://trac.xapian.org/ticket/181 for details of what features this
will disable.

The development backend on trunk produces smaller databases, but the
format isn't stable yet, so you potentially have to rebuild your
databases after every upgrade.

Cheers,
    Olly