BIND 9 Performance History

ISC Performance Lab

We have been monitoring BIND 9 performance using the ISC Performance Lab, described in this 2016 blog post. We realized we really needed something like this when users reported to us a significant performance drop when upgrading from version 9.9.5 to either 9.9.6 or 9.10.0. We now review performance changes every month and when we see a negative change, we can identify the commit that caused the change and investigate.

An Index, Not a Benchmark

These measurements are designed to identify changes in performance from one release to the next, not to measure typical or maximum performance in actual production use. Although we show the absolute number of queries per second on the charts below, we have not made any effort to tune the configuration for our hardware, and these results are not necessarily predictive of performance in another configuration or on other hardware. Many features can have a significant performance impact (e.g. query logging), and our tests don’t measure other important factors, such as the speed of adding and removing zones.

Use these results to help estimate the change in performance you might see when migrating from one release or branch to another.

First, identify which of these ’example applications’ is most like your deployment:

1. 1 Million Delegations (e.g. a TLD)
2. 1 Million Zones (e.g. a large domain hoster)
3. Recursive (For recursive tests we send queries that target a separate authoritative server that holds both a single zone with 1M delegations and each of the 1M small zones referred to therein.)

Then, find the result for the release you are running today, and compare with the release you are considering migrating to.

The data points below each represent the average of 3 test runs, with each test run consisting of 30 measurements. Changes of ~20KQPS in either direction are not significant given our test variability and sample size.

Our recently posted set of BIND maintenance releases should not introduce any noticeable performance changes for anyone upgrading on the same branch.

What caused the drop in performance between 9.9.6 and 9.9.9 / 9.10.4?

We introduced a change in 9.9.6 and 9.10.0 that was intended to improve performance, that caused a huge drop in KQPS. The drop in KQPS was due to reducing the number of UDP listeners to NUM_PROCESSORS / 2. We made this change because of a UDP locking bug in older versions of RedHat EPL, that led us to believe that this would improve performance.

3751. [tuning] The default setting for the -U option (setting
the number of UDP listeners per interface) has
been adjusted to improve performance. [RT #35417]

The change DID improve performance in several tests on our hardware, but the only 8-core machine we had was 5 years past EOL at the time, and in retrospect, not a good basis for making a performance adjustment.

After several other users reported a performance regression on their hardware, and a vigorous internal debate, we made a second change in 9.9.9 and 9.10.4 which brought performance back to the earlier level.

4236. [func] On machines with 4 or more processors (CPU), the
default value for the number of UDP listeners
has been changed to the number of detected
processors minus one. [RT #40761]

It was after this performance regression that we purchased a modern multi-core machine for performance testing, and Ray Bellis began developing the ISC Performance Lab.

We advise testing on your own hardware to get the optimal setting in this Knowledgebase article on ‘Choosing the right value for -U when starting named.’

9.11 Branch Performance

Another, smaller drop in performance from 9.10 to 9.11 can be seen by comparing the 9.11 and 9.10 charts. While addressing a bug reported by a user, we introduced another performance regression in 9.11. BIND was incorrectly preserving case-sensitivity from the zone_name in a zone statement over what is defined in the zone itself. We(1) have made subsequent changes that mitigated the performance impact of case preservation, but some performance penalty is inherent in careful preservation of the case in the authority. (For those who are interested, the fix is one of the changes in commit# 4605, currently committed to the 9.12 branch, which we will be backporting to 9.11.3.)

Using our Performance Lab, we have been able to validate a number of changes which will come out in our next major branch, BIND 9.12, that will improve performance significantly. We will provide an update on that when we release the BIND 9.12.0 alpha.

(1) Both of the performance regressions were diagnosed and fixed by BIND core developer Mukund Sivaraman, who has had a special focus on BIND performance improvement.