[Xapian-discuss] Searching log files?

[Kevin Duraj]

Moazam,

... and here is an example how Xapian perform searches on 100 millions
records using one physical 1 terabyte hard drive ...

running Xapian search on your name "Moazam Raja"

http://find1friend.com/search?q=Moazam+Raja

Search Result: I see some activities in Meetup.com, iPhone
development, Cloud Computing, SQL Server etc in Palo Alto ...
Xapian might be lot stronger search engine than you need. If you do
not need so much power, you might also consider to use less powerful
search engine like Lucene.


Kevin Duraj
http://find1friend.com/




On Mon, Jan 18, 2010 at 10:36 AM, Kevin Duraj <kevin.softdev at gmail.com> wrote:
> Yes, many people quietly using Xapian for all kinds of things. For
> example I use Xapian to index the whole Facebook.com ... :-)
>
> Check it out http://find1friend.com/
> it's very cool what you can do with Xapian.
>
> Kevin Duraj
> http://find1friend.com/
>
> On Mon, Jan 18, 2010 at 3:21 AM, Olly Betts <olly at survex.com> wrote:
>> On Fri, Jan 15, 2010 at 10:20:33PM -0800, Moazam Raja wrote:
>>> Hi all, has anyone used Xapian/Omega to index and search large amounts
>>> of (Unix) server logs?
>>
>> I'm not aware of anyone doing it, but then a lot of people quietly use
>> Xapian without telling us what for so that doesn't mean much.
>>
>>> I'm looking to create a search application which will allow me to
>>> index and search logs from roughly 20-100 servers but I'm not sure
>>> which engine to use that will provide near real time indexing and good
>>> search performance. Is this something that can be accomplished with
>>> Xapian/Omega?
>>
>> I can't see why not.  You probably want to make the "document" a single logical
>> log entry (i.e. probably a single line in the log, but it might consist of
>> several lines logged at once in some log formats).
>>
>> Cheers,
>>    Olly
>>
>> _______________________________________________
>> Xapian-discuss mailing list
>> Xapian-discuss at lists.xapian.org
>> http://lists.xapian.org/mailman/listinfo/xapian-discuss
>>
>