BIND Performance with Huge RPZ

[Vernon Schryver]

> From: Steven Carr <sjcarr at gmail.com>

> It's very difficult to predict the impact on performance in general.

Yes, the reasonable tactic is to build a representative list of
queries from your query logs and use queryperf to hit a test server
with those 800K policy zone labels.

>                                                                      But
> every query will hit the RPZ zone to be checked against, so make sure you
> have enough RAM to hold all 800k records in memory then that will
> significantly speed things up.

RAM for all 800K records would be sufficient, but might not unnecessary.
Names in policy zones are kept in the same kind of red/black trees
that are used for ordinary DNS zones, because policy zones are ordinary
BIND9 DNS zones.  Policy zone IP addresses are kept in radix or patricia
trees.  Sufficent RAM is the size of the working set.  For example,
if you have 800K labels below obscure.example.com and almost never
resolve any of them, then keeping them in RAM would not help your
server's average performance.


> The guidance figures that I've seen banded around by a BIND based DNS
> appliance is that it will have ~30% impact on query performance per RPZ
> feed that has to be looked up against.

RPZ performance with zones with NSIP or NSDNAME policies is significantly
improved in versions of the RPZ code with the new min-ns-dots parameter
with a default value 1.  This turns off checking the many server names
and IP addresses of TLDs.

The version after that has min-ns-dots and also no longer hits every
policy zone with every IP address and name in every response, but
instead hits a summary red/black tree of names or radix tree of IP
addresses  summarizing all policy zones once for each address and name.
This radically improves RPZ performance with multiple policy zones.

The BIND9 tests in bin/tests/system/rpz include a quick and dirty
performance test to ensure that nothing is not unexpectedly broken.
This is typical output from that test:
    I:checking performance with RPZ
    I:checking performance without RPZ
    I:17758 qps with RPZ is 75% of 23596 qps without RPZ
Previous versions of that test did not turn off some very expensive 
run time checks and so produced much smaller numbers for both cases.

Patches for both of those versions of RPZ speed improvements for some
BIND9 releases can be with the BIND RRL patches by following the link
labeled "Patch files for BIND9" on http://www.redbarn.org/dns/ratelimits

Both of those versions are or will be in official BIND releases.
I've lost track of which releases have or will have which of those
two RPZ sets of performance improvements.


Vernon Schryver    vjs at rhyolite.com