Bind Queries log file format

[Timothe Litt]

On 04-Feb-17 04:27, Phil Mayers wrote:
> On 03/02/17 16:45, Mukund Sivaraman wrote:
>
>> The query log is getting more fields at the end of it such as
>> CLIENT-SUBNET logging.
>
> Although it would be super-disruptive, has any thought been given to
> moving to an entirely new log format, for example k/v or JSON? They're
> a lot more extendable going forward and most SIEM/ML systems will read
> them with no additional configuration.
>
> Adding the query log hex/ptr thing just inconvenienced me. Strangely,
> changing the entire format to k/v would have massively helped me. This
> applies across all logs (RPZ in particular).
>
> Obviously one sample isn't enough but it's maybe something to consider?
I'm not sure whether I'm in favor of this approach, but it's not
necessarily very disruptive.

It would be trivial to script a converter from JSON to the current log
format - or even one that took a format string to select whatever fields
in random order.  Pipe a new log file though it to existing log readers,
and you're done. 

For almost complete transparency, embed in a daemon that continuously
reads the JSON log & appends to the traditional log; the existing log
readers can read the old format in near real-time...

Then when a support issue (or other requirement) comes up, the enhanced
data is in the JSON log.

When your old log processor is upgraded to use a new field, just add it
to the converter (format).

New processors would preferably read the JSON/native format directly.

The only annoyance is having to manage 2 log files (and some disk space).

FWIW


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4577 bytes
Desc: S/MIME Cryptographic Signature
URL: <https://lists.isc.org/pipermail/bind-users/attachments/20170204/e37319d3/attachment.bin>