approach on parsing the query-log file

[David Forrest]

On Tue, 28 Apr 2009, Gregory Hicks wrote:

>
>> From: Jonathan Petersson <jpetersson at garnser.se>
>> Date: Tue, 28 Apr 2009 08:13:25 -0700
>> Subject: Re: approach on parsing the query-log file
>> To: Niall.oReilly at ucd.ie
>> Cc: Bind Mailing <bind-users at lists.isc.org>
>>
>> Yeah I've thought about using tail but I'm not sure how locking would
>> be managed when logrotate kicks in, does anyone know?
>
> I use "tail -f <log-file>"
>
> When the log rotates, the tail is still running against the rotated
> file.  I have to manually change to the current file. ("^C-!!" works)
>
> A better way to do it might be to have the 'logfile' be a pipe and have
> the parsing intelligence on the other side of the pipe.  Have the log
> rotation "smarts" be on the other side of the pipe also.  (At one $JOB,
> I used this technique to separate out different log messages from
> simultaneously running SMTP processes.)
>
> Regards,
> GRegory Hicks
>>
>> On Tue, Apr 28, 2009 at 3:41 AM, Niall O'Reilly <Niall.oReilly at ucd.ie>
> wrote:
>>> On Mon, 2009-04-27 at 22:26 -0700, Jonathan Petersson wrote:
>>>> The obvious question that occurs is; What would be what's the best
>>>> approach to do this?
>>>
>>>        I've not used it, but a colleague is very keen on File::Tail
>>>        (http://search.cpan.org/~mgrabnar/File-Tail-0.99.3/Tail.pm).
>>>        Apparently, it looks after log-file roll-over and 'just
> works'.
>>>
>>>        /Niall
>>>
>>>
>>>

I use tail --follow=name <logfile> as the tail then switches to the new 
inode.  An alternative is to to put the copytruncate directive in 
/etc/logrotate.conf as the possible loss of one or two queries is 
usually not significant to statistical analysis.  Using inotail (which is 
supposedly less processor intensive) requires the second approach as it 
does not include the --follow=name option.

Dave
-- 
David Forrest                   e-mail   drf @ maplepark.com
Maple Park Development Corporation  http://xen.maplepark.com
St. Louis, Missouri