Choosing max-journal-size

[Michael Graff]

On Nov 30, 2011, at 4:09 AM, Matus UHLAR - fantomas wrote:

>> On 11/29/2011 11:33 PM, Chris Thompson wrote:
>> I wonder if an external tool to "trim" the journal would be an option? You'd need a timestamp on records (relying on the RRSIGs mean it only works for signed). Not sure about the locking implications.

In general, BIND should handle trimming.

> I think this is something BIND should take care about.
> 
> Does BIND veridy the journal not to exceed usefull size?

There are three issues that I see in our journal files:

(1)  The default size is unlimited.
(2)  To shrink the journal, we copy the more recent half (or some part anyway) to a new file.  For large journals, this is significant time and I/O.
(3)  Because of (2) and other reasons, even if you set a max journal size, we don't always respect it.

(1) is fixable easily.  We could even estimate based on sizes internally to BIND.  We may get the guess wrong for some, but I would submit that unlimited is ALWAYS wrong.

(2) is harder to fix.  I once proposed we used SQLite for storage, so we could expire records very quickly without re-writing the journal files.  I also once proposed that we used two files, each of which was 50% of the max size, and would just delete the older half when needed.  Either fix is reasonable.

(3) is an admin expectations problem.  If you run 9.4 or earlier still, you are aware that our cache size also did not respect the administrator set maximum.  9.5 and later fixed that, and this is one more case where complete correctness of operation interferes with expectations.

We have had journal file issues like this on our road map for some time now.  However, there always seems to be a more pressing issue.  Perhaps it would be possible for some contributed solutions?  If so, contact me directly.  I'm sure someone has an intern or programmer to spare for a bit :)

--Michael